Skip to content

BUG: pd.to_datetime() throws if caching is on with Null-like arguments #26078

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 17, 2019

Conversation

anmyachev
Copy link
Contributor

@anmyachev anmyachev commented Apr 13, 2019

The problem was in contradictory behavior between pd.Index([pd.NaT, None]).is_unique that returns False and algorithms.unique([pd.NaT, None]) that returns array([NaT, None], dtype=object) equivalent to the object being unique.

We are only using Index(arg).unique() now.

The test written by @realead was added to verify changes.

@codecov
Copy link

codecov bot commented Apr 13, 2019

Codecov Report

Merging #26078 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #26078      +/-   ##
==========================================
- Coverage   91.89%   91.89%   -0.01%     
==========================================
  Files         175      175              
  Lines       52509    52514       +5     
==========================================
+ Hits        48255    48256       +1     
- Misses       4254     4258       +4
Flag Coverage Δ
#multiple 90.45% <100%> (ø) ⬆️
#single 40.74% <55.55%> (-0.14%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/base.py 96.78% <100%> (-0.01%) ⬇️
pandas/core/base.py 97.81% <100%> (+0.03%) ⬆️
pandas/io/gbq.py 75% <0%> (-12.5%) ⬇️
pandas/core/frame.py 96.79% <0%> (-0.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a15069...d0ec00c. Read the comment docs.

@codecov
Copy link

codecov bot commented Apr 13, 2019

Codecov Report

Merging #26078 into master will increase coverage by 0.08%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #26078      +/-   ##
==========================================
+ Coverage   91.89%   91.98%   +0.08%     
==========================================
  Files         175      175              
  Lines       52509    52384     -125     
==========================================
- Hits        48255    48184      -71     
+ Misses       4254     4200      -54
Flag Coverage Δ
#multiple 90.53% <100%> (+0.08%) ⬆️
#single 40.74% <0%> (-0.15%) ⬇️
Impacted Files Coverage Δ
pandas/core/tools/datetimes.py 84.59% <100%> (ø) ⬆️
pandas/io/gbq.py 75% <0%> (-12.5%) ⬇️
pandas/util/_doctools.py 12% <0%> (-0.88%) ⬇️
pandas/core/panel.py 35.43% <0%> (-0.1%) ⬇️
pandas/core/computation/ops.py 95.62% <0%> (-0.04%) ⬇️
pandas/core/computation/align.py 97.8% <0%> (-0.03%) ⬇️
pandas/core/reshape/melt.py 97.47% <0%> (-0.03%) ⬇️
pandas/core/dtypes/concat.py 96.56% <0%> (-0.02%) ⬇️
pandas/plotting/_core.py 83.76% <0%> (-0.02%) ⬇️
pandas/core/nanops.py 93.82% <0%> (-0.02%) ⬇️
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a15069...a3c66b3. Read the comment docs.

@gfyoung gfyoung added Bug Datetime Datetime data dtype labels Apr 14, 2019
@jreback jreback changed the title BUG: Workaround for gh 22305 BUG: pd.to_datetime() throws if caching is on with Null-like arguments Apr 15, 2019
@pep8speaks
Copy link

pep8speaks commented Apr 16, 2019

Hello @anmyachev! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-04-17 10:56:38 UTC

@jreback jreback added this to the 0.25.0 milestone Apr 16, 2019
@jreback jreback merged commit df1fb03 into pandas-dev:master Apr 17, 2019
@jreback
Copy link
Contributor

jreback commented Apr 17, 2019

thanks @anmyachev

@sjvdm
Copy link

sjvdm commented Aug 19, 2021

This issue persists in pandas 1.2.4. Doesn't seem like this was ever merged to prod - are there perhaps any plans?

@gecko17
Copy link

gecko17 commented Feb 23, 2022

I can confirm this issue occurs in pandas 1.2.4. However, it was not reproducible in pandas 1.3.4.
Can anyone confirm this was fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pd.to_datetime() throws if caching is on with Null-like arguments
9 participants