Skip to content

BUG: Bad shift_forward ceiling around DST change #51501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
epizut opened this issue Feb 20, 2023 · 6 comments · Fixed by #56017
Closed
2 of 3 tasks

BUG: Bad shift_forward ceiling around DST change #51501

epizut opened this issue Feb 20, 2023 · 6 comments · Fixed by #56017
Labels
Bug Timestamp pd.Timestamp and associated methods Timezones Timezone data dtype

Comments

@epizut
Copy link

epizut commented Feb 20, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

just_before_dst = pd.to_datetime('2023-03-26 00:45:00').tz_localize('Europe/London')
just_before_dst.ceil('30min', nonexistent='shift_forward')
# Result:    2023-03-26 03:00:00
# Excpected: 2023-03-26 02:00:00

Issue Description

On March 26th 2023 London will switch to its DST time and goes directly from 00:59.99 to 02:00:00 skipping 1h of local time.
Unfortunately, ceiling 00:45:00 to its upper hour results into 03:00:00.

Expected Behavior

2023-03-26 02:00:00

Installed Versions

INSTALLED VERSIONS ------------------ commit : 2e218d1 python : 3.9.12.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19042 machine : AMD64 processor : Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder : little LC_ALL : None LANG : None LOCALE : English_United States.1252

pandas : 1.5.3
numpy : 1.23.5
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 65.3.0
pip : 22.2.2
Cython : 0.29.30
pytest : 7.2.1
hypothesis : None
sphinx : 4.5.0
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.0
html5lib : None
pymysql : None
psycopg2 : 2.9.3
jinja2 : 3.1.1
IPython : 8.2.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : None
brotli : None
fastparquet : None
fsspec : 2022.3.0
gcsfs : None
matplotlib : 3.5.1
numba : 0.56.4
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : 7.0.0
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.8.1
snappy : None
sqlalchemy : 1.4.41
tables : None
tabulate : 0.8.9
xarray : 2022.3.0
xlrd : 2.0.1
xlwt : None
zstandard : None
tzdata : None

@epizut epizut added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 20, 2023
@DeaMariaLeon DeaMariaLeon added the Timestamp pd.Timestamp and associated methods label Feb 23, 2023
@ivyrxr
Copy link

ivyrxr commented Mar 29, 2023

take

@ivyrxr
Copy link

ivyrxr commented Mar 29, 2023

The result "2023-03-26 03:00:00" is expected because when you shift 00:45:00 forwards by 30 mins, it becomes 01:15:00 which is actually 02:15:00 in London DST time. Ceiling it will return 03:00:00. Hope it helps or please correct me if I am wrong.

@epizut
Copy link
Author

epizut commented Mar 29, 2023

I don't get it, even if we forget the DST:
pd.to_datetime('2023-03-26 02:15:00').ceil('30min') != pd.to_datetime('2023-03-26 03:00:00')

If the correct answer is 2023-03-26 03:00:00 London then it would be 1h 15minutes after the original time 2023-03-26 00:45:00 London, but I don't expect ceil('30min') to shift by more than 29:59.999999

@MarcoGorelli MarcoGorelli added Timezones Timezone data dtype and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 29, 2023
@MarcoGorelli
Copy link
Member

thanks for the report - I think I would also have expected 2023-03-26 02:00:00

@ivyrxr
Copy link

ivyrxr commented Mar 29, 2023

Thank you both for the clarification. After I read more closely to the documentation, I think you are right that the expected result should be 2023-03-26 02:00:00. I can help investigate the code for a fix.

@MarcoGorelli
Copy link
Member

that would be great, thank you! feel free to ping if anything trips you up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timestamp pd.Timestamp and associated methods Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants