PERF: Slow Series.plot() for DatetimeIndex for pandas 2.2.0 compared to 2.1.4

[rhkarls]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this issue exists on the latest version of pandas.

• I have confirmed this issue exists on the main branch of pandas.

Reproducible Example

Using pandas 2.2.0 I'm experiencing very slow plotting of pd.Series objects with DatetimeIndex. I'm comparing below the time to plot using 2.2.0 and 2.1.4, as well as using matplotlib ax.plot().

Could this be related to the date formatting that pandas applies to the matplotlib axis?
Using an integer index (pd.RangeIndex(start=0, stop=500_000)) does not result in slow Series.plot() with pandas 2.2.0

Related, but not to the possible 2.2.0 regression: #50355

import time
import pandas as pd
import matplotlib.pyplot as plt

ts_index = pd.date_range('2020-01-01 00:00', periods=500_000, freq='10min')

s_a = pd.Series(data=1, index=ts_index)
s_b = pd.Series(data=2, index=ts_index)

t_s1a = time.time()
s_a.plot()
print(f"Time elapsed Series.plot() for s_a: {(time.time()-t_s1a):.2f} seconds")
t_s1b = time.time()
s_b.plot()
print(f"Time elapsed Series.plot() for s_b: {(time.time()-t_s1b):.2f} seconds")

fig, ax = plt.subplots()
t_s2 = time.time()
ax.plot(s_a.index, s_a.to_numpy())
ax.plot(s_b.index, s_b.to_numpy())
print(f"Time elapsed matplotlib, both series: {(time.time()-t_s2):.2f} seconds")

t_s3 = time.time()
s_b.plot(ax=ax) # this is now fast
print("Time elapsed Series.plot() on existing plt axis with matplotlib line plot: "
      f"{(time.time()-t_s3):.2f} seconds")


fig, ax = plt.subplots()
t_s4 = time.time()
s_a.plot(ax=ax)
print("Time elapsed Series.plot() on existing plt axis without matplotlib line plot: "
      f"{(time.time()-t_s4):.2f} seconds")

Outputs for pandas 2.2.0:

Time elapsed Series.plot() for s_a: 13.28 seconds
Time elapsed Series.plot() for s_b: 17.23 seconds
Time elapsed matplotlib, both series: 0.09 seconds
Time elapsed Series.plot() on existing plt axis with matplotlib line plot: 0.63 seconds
Time elapsed Series.plot() on existing plt axis without matplotlib line plot: 21.64 seconds

Installed Versions

Environment with pandas 2.2.0

INSTALLED VERSIONS

commit : f538741
python : 3.11.7.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : Intel64 Family 6 Model 186 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_Europe.1252

pandas : 2.2.0
numpy : 1.26.3
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 69.0.3
pip : 23.3.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.8.2
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.4
qtpy : None
pyqt5 : None

Environment with pandas 2.1.4

INSTALLED VERSIONS ------------------ commit : a671b5a python : 3.11.7.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19045 machine : AMD64 processor : Intel64 Family 6 Model 186 Stepping 3, GenuineIntel byteorder : little LC_ALL : None LANG : None LOCALE : English_Europe.1252

pandas : 2.1.4
numpy : 1.26.3
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 69.0.3
pip : 23.3.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
bs4 : None
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.8.2
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.4
qtpy : None
pyqt5 : None

Prior Performance

Output for pandas 2.1.4:

Time elapsed Series.plot() for s_a: 1.77 seconds
Time elapsed Series.plot() for s_b: 1.77 seconds
Time elapsed matplotlib, both series: 0.05 seconds
Time elapsed Series.plot() on existing plt axis with matplotlib line plot: 0.57 seconds
Time elapsed Series.plot() on existing plt axis without matplotlib line plot: 1.69 seconds

[ba05]

I'm seeing the same thing. Plotting a column with DatetimeIndex index is slow vs plotting with integer index. Can verify by dropping the DatetimeIndex before plot via .reset_index(drop=True)

[ba05]

Here is another example...about 33x slower on my machine (pandas 2.1.4 vs 2.2.1):

import time
import pandas as pd
import matplotlib.pyplot as plt

index = pd.date_range('2020-01-01 00:00', periods=500_000, freq='10min')
s1 = pd.Series(data=1, index=index)

t1a = time.time()
s1.plot()
plt.show()
t1z = time.time()
print(f"Time elapsed Series.plot() for pandas {pd.__version__} datetime index: {(t1z-t1a):.2f} seconds")

s2 = s1.reset_index(drop=True)
t2a = time.time()
s2.plot()
plt.show()
t2z = time.time()
print(f"Time elapsed Series.plot() for pandas {pd.__version__} integer index: {(t2z-t2a):.2f} seconds")

Output with pandas 2.1.4:

Time elapsed Series.plot() for pandas 2.1.4 datetime index: 5.38 seconds
Time elapsed Series.plot() for pandas 2.1.4 integer index: 0.12 seconds

Output with pandas 2.2.1:

Time elapsed Series.plot() for pandas 2.2.1 datetime index: 178.90 seconds
Time elapsed Series.plot() for pandas 2.2.1 integer index: 0.12 seconds

[ba05]

I've tried to further track this down via profiling, but no luck. Anyone have any ideas? Something changed between 2.1.4 and 2.2.0.

[ba05]

I built an environment to find which specific commit causes this issue. The commit that appears to causes the issue is 434d84d from #55944