Skip to content

min(axis = 1) propagates NAN when encountering missing timestamps #31214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
randomgambit opened this issue Jan 22, 2020 · 1 comment
Closed

Comments

@randomgambit
Copy link

Hello there,
Consider this:

dftest = pd.DataFrame({'time1' : ['2019-01-01- 10:10:10.123456Z',
                                  '2019-01-01- 10:10:10.123456Z'],
                       'time2' : ['ha',
                                 'pandas!!!'],
                       'time3' :['here we go',
                                'again']})

dftest
Out[179]: 
                          time1      time2       time3
0  2019-01-01- 10:10:10.123456Z         ha  here we go
1  2019-01-01- 10:10:10.123456Z  pandas!!!       again

I am trying to get the minimum timestamp between time1, time2 and time3. Of course, in this example time2 and time3 are not timestamps so (after proper conversion to NaTs) I expect the minimum to be time1 all the time.

Surprisingly, this fails entirely.


dftest['time1'] = pd.to_datetime(dftest.time1, errors = 'coerce')
dftest['time2'] = pd.to_datetime(dftest.time2, errors = 'coerce')    
dftest['time3'] = pd.to_datetime(dftest.time3, errors = 'coerce') 
dftest
Out[181]: 
                             time1 time2 time3
0 2019-01-01 10:10:10.123456+00:00   NaT   NaT
1 2019-01-01 10:10:10.123456+00:00   NaT   NaT

And now:

dftest.loc[:, ['time1', 'time2', 'time3']].min(axis = 1)
Out[182]: 
0   nan
1   nan
dtype: float64

This looks like a bug. It seems that the errors is coming from the timezone component. What do you think?

Thanks!

@mroeschke
Copy link
Member

Thanks for the report. This looks like a duplicate of #27794

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants