[READY] perf improvements for strftime#51298

Open

smarie wants to merge 154 commits intopandas-dev:mainfrom

smarie:feature/44764_perf_issue_new

Commits on Feb 6, 2023

`period_format` now has a faster default formatter leveraging python string formatting
Sylvain MARIE
committed
class `_Period`: new method `fast_strftime`
Sylvain MARIE
committed
class `Timestamp`: new method `fast_strftime`
Sylvain MARIE
committed
New module in tslibs: `strftime.py`. New function in this module: `convert_strftime_format` (raises `UnsupportedStrFmtDirective`). This function converts a `strftime` date format string into a nati…
Sylvain MARIE
committed
`format_array_from_datetime`: new boolean argument `fast_strftime` to use faster datetime formatting.
Sylvain MARIE
committed
datetimelike `strftime`: new boolean argument `fast_strftime` to use faster datetime formatting. `_format_native_types` modified with this new argument too. Subclasses modified to support it (`Date…
Sylvain MARIE
committed
`DatetimeIndexOpsMixin.format` and `_format_with_header`: new boolean argument `fast_strftime` to use faster datetime formatting.
Sylvain MARIE
committed
`NDFrame.to_csv` and `DataFrameRenderer.to_csv` and `CSVFormatter.__init__`: new boolean argument `fast_strftime` to use faster datetime formatting.
Sylvain MARIE
committed

Commits on Feb 7, 2023

Added tests for the `to_csv` dataframe method to cover the new fast_strftime feature
Sylvain MARIE
committed
`TestCategoricalRepr`: added a test for dates without time, with timezone
Sylvain MARIE
committed
Fixed `test_nat` and `test_api` with the new symbols added
Sylvain MARIE
committed
New `test_strftime` module to cover the `strftime.py` module in tslib.
Sylvain MARIE
committed
`convert_strftime_format` argument `target` is now mandatory to avoid mistakes
Sylvain MARIE
committed
`convert_strftime_format`: Completed unsupported directives for datetimes
Sylvain MARIE
committed
Fixed bug in tslib `format_array_from_datetime`
Sylvain MARIE
committed
Fixed issue in `format_array_from_datetime` when tz was not None
Sylvain MARIE
committed
Added 2 todos
Sylvain MARIE
committed
`test_format`: Added various tests for the new feature
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed

Commits on Feb 8, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed Datetime64TZFormatter issue due to arg renaming in recent commits
Sylvain MARIE
committed
Added 2 asv benchs for strftime with iso8601 format, and a variant for tz-aware
Sylvain MARIE
committed
blackened, flake8, and removed asv main
Sylvain MARIE
committed

Commits on Feb 10, 2023

Minor improvement
Sylvain MARIE
committed
Added new ASVs for strftime
Sylvain MARIE
committed
Added asvs for period
Sylvain MARIE
committed

Commits on Feb 14, 2023

Added asvs for datetime and period indexes .format
Sylvain MARIE
committed
`convert_strftime_format` is now part of the public API as it is required by `Period.fast_strftime` and `Timestamp.fast_strftime`
Sylvain MARIE
committed

Commits on Feb 16, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Updated whats new 1.5.4
Sylvain MARIE
committed

Commits on May 9, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed issues following the merge from latest main. Introduced new function `fast_period_format`
Sylvain MARIE
committed
Improved ASV bench slightly (added datetime index formatting tests)
Sylvain MARIE
committed
Merged whatsnew
Sylvain MARIE
committed
pre-commit and docstring checks
Sylvain MARIE
committed

Commits on May 10, 2023

blackened
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Added two asvs
Sylvain MARIE
committed
Fixed mypy error
Sylvain MARIE
committed
Fixed ASV bench
Sylvain MARIE
committed
Improved ASV benchs
Sylvain MARIE
committed

Commits on May 11, 2023

Fixed RST format in whatsnew
Sylvain MARIE
committed
Hopefully fixed the ASV bench for the case when the format is the default one but provided in an explicit way
Sylvain MARIE
committed
Fixed issue in period.pyx
Sylvain MARIE
committed
Made hooks happy
Sylvain MARIE
committed
Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored
Fixed variables used before initialization
Sylvain MARIE
committed

Commits on May 12, 2023

pre-commit hook upgrade
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed error in period.pyx
Sylvain MARIE
committed

Commits on May 14, 2023

Fixed issue: there was no need to encode to bytes to apply string formatting
Sylvain MARIE
committed
Fixed issue: there was no need to encode to bytes to apply string formatting
Sylvain MARIE
committed
Merge remote-tracking branch 'origin/feature/44764_perf_issue_new' into feature/44764_perf_issue_new
Sylvain MARIE
committed
Revert "Fixed issue: there was no need to encode to bytes to apply string formatting"
Sylvain MARIE
committed
Fixed issue: there was no need to encode to bytes to apply string formatting
Sylvain MARIE
committed
Removed todo in timedeltas.pyx
Sylvain MARIE
committed
Fixed asv bench
Sylvain MARIE
committed
Fixed ASV
Sylvain MARIE
committed
Fixed indentation issue
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Doc fix whatsnew entries sorted
Sylvain MARIE
committed

Commits on May 15, 2023

Improved speed of csv formatting of datetimeindex
Sylvain MARIE
committed
pre-commit fixes
Sylvain MARIE
committed

Commits on May 16, 2023

Removed invalid comment
smarie
authored

Commits on May 31, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed typo
Sylvain MARIE
committed

Commits on Jun 1, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed whatsnew
Sylvain MARIE
committed

Commits on Jun 5, 2023

Fixed changelog indentation error: now a one-liner
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed mypy error
Sylvain MARIE
committed
Fixed mypy error 2
Sylvain MARIE
committed
Fixed whatsnew
Sylvain MARIE
committed

Commits on Jun 6, 2023

Fixed whatsnew ?
Sylvain MARIE
committed
Fixed whatsnew ?
Sylvain MARIE
committed
Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Jun 13, 2023

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Nov 11, 2023

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Moved whatsnew items
Sylvain MARIE
committed
Re applied mods to `TestPeriodIndexFormat`, since it had moved somewhere else
Sylvain MARIE
committed
Re applied mods to `TestDatetimeIndexFormat`, since it had moved somewhere else and was renamed since #55603
Sylvain MARIE
committed
Re applied mods from `TestFastStrfTimeScalars` in the right places
Sylvain MARIE
committed
Merge remote-tracking branch 'origin/feature/44764_perf_issue_new' into feature/44764_perf_issue_new
Sylvain MARIE
committed

Commits on Nov 16, 2023

Ruff: Fixed invalid character in comment
Sylvain MARIE
committed
black+isort+fixed tests
Sylvain MARIE
committed
Implemented conservative fallback as suggested per code review
Sylvain MARIE
committed
black
Sylvain MARIE
committed
Implemented hypothesis tests as suggested per code review
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Removed the `fast_strftime` argument everywhere as suggested per code review
Sylvain MARIE
committed

Commits on Nov 18, 2023

Fixed docstring
Sylvain MARIE
committed
Trying to have meson understand that there is a py file in the tslib
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed ruff error
Sylvain MARIE
committed
Fixed tests
Sylvain MARIE
committed
Fixed test
Sylvain MARIE
committed
Fixed test
Sylvain MARIE
committed
Fixed test
Sylvain MARIE
committed
Fixed test for musl linux
Sylvain MARIE
committed
Fixed mypy errors
Sylvain MARIE
committed
Fixed mypy error
Sylvain MARIE
committed

Commits on Nov 19, 2023

Fixed mypy error
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed

Commits on Dec 15, 2023

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Dec 20, 2023

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Dec 31, 2023

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Jan 9, 2024

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Mar 3, 2024

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
As per code review: instance level `fast_strftime` are now private `_fast_strftime`
Sylvain MARIE
committed
Added maintenance comment
Sylvain MARIE
committed
black+isort
Sylvain MARIE
committed
Moved changelog to 3.0.0 and improved it slightly
Sylvain MARIE
committed
Merge branch 'feature/44764_perf_issue_new' of https://github.com/smarie/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed test: fast_strftime not part of api anymore (private)
Sylvain MARIE
committed

Commits on Mar 8, 2024

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
formatting
Sylvain MARIE
committed
Fixed ASV benchmark: format method disappeared from DateTimeIndex
Sylvain MARIE
committed
Improved what's new
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed ASV error
Sylvain MARIE
committed

Commits on Mar 25, 2024

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored

Commits on Apr 2, 2024

Code review: renamed `loc_s` into `locale_dt_strings`
Sylvain MARIE
committed
Code review: moved whats new entry to perf
Sylvain MARIE
committed
Code review: removed commented out code
Sylvain MARIE
committed
Code review: added support for negative and small years and added corresponding tests.
Sylvain MARIE
committed
Merge remote-tracking branch 'origin/feature/44764_perf_issue_new' into feature/44764_perf_issue_new
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Improved test for small years
Sylvain MARIE
committed

Commits on Apr 3, 2024

Fixed failing test
Sylvain MARIE
committed
Fixed failing test
Sylvain MARIE
committed
Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored
Fixed test on linux
Sylvain MARIE
committed
Trying to debug datetime.strftime on linux
Sylvain MARIE
committed
Fixed tests on linux and windows
Sylvain MARIE
committed
Removed useless comment
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed

Commits on Apr 4, 2024

Fixed tests on linux musl
Sylvain MARIE
committed
Attempt to fix on MUSL
Sylvain MARIE
committed
Fixed test for linux musl
Sylvain MARIE
committed

Commits on Jun 29, 2024

Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed

Commits on Jun 30, 2024

Simplified `convert_strftime_format` as per code review
Sylvain MARIE
committed
Renamed `fast_strftime` with `strftime_pystr` as per code review
Sylvain MARIE
committed
Merge branch 'main' of https://github.com/pandas-dev/pandas into feature/44764_perf_issue_new
Sylvain MARIE
committed
Fixed test on WASM/Pyodide
Sylvain MARIE
committed

Commits on Jul 1, 2024

Renamed internal argument ``strftime_pystr`` into ``_use_pystr_engine`` to be more explicit.
Sylvain MARIE
committed
Fixed test on pyodide
Sylvain MARIE
committed
Fixed cython-lint error
Sylvain MARIE
committed

Commits on Jul 5, 2024

Merge branch 'main' into feature/44764_perf_issue_new
smarie
authored