Skip to content

Append with two DateTimeIndex returns Index and not DateTimeIndex #14795

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rsheftel opened this issue Dec 3, 2016 · 1 comment
Closed

Append with two DateTimeIndex returns Index and not DateTimeIndex #14795

rsheftel opened this issue Dec 3, 2016 · 1 comment
Labels
Timezones Timezone data dtype Usage Question

Comments

@rsheftel
Copy link

rsheftel commented Dec 3, 2016

In 0.19.1 appending a DateTimeIndex to another DateTimeIndex where they have different time zones now returns an Index object. In 0.18.1 it would return a DateTimeIndex with a UTC time zone.

In 0.19.1 (and in 0.18.1) if the two DateTimeIndex have the same time zone a DateTimeIndex is returned. I believe the behavior in 0.18.1 of returning a DateTimeIndex with time zone UTC should be the correct behavior.

--- Example of the problem ---

import pandas as pd

df1 = pd.DataFrame({'a': [1, 2, 3]}, index=pd.date_range('1990-02-01 09:00:00', periods=3, freq='1min', tz='America/New_York'))
df2 = pd.DataFrame({'a': [4, 5, 6]}, index=pd.date_range('1990-02-01 08:03:00', periods=3, freq='1min', tz='America/Chicago'))

print(df1.index.tzinfo)  # America/New_York
print(df2.index.tzinfo)  # America/Chicago

result = df1.index.append(df2.index)

print(type(new_index))
print(type(result))

>> pandas 0.18.1 returns <class 'pandas.tseries.index.DatetimeIndex'>
>> pandas 0.19.1 returns <class 'pandas.indexes.base.Index'>
@rsheftel rsheftel changed the title Append with two DateTimeIndex converst to Index and not DateTimeIndex Append with two DateTimeIndex returns Index and not DateTimeIndex Dec 3, 2016
@jreback
Copy link
Contributor

jreback commented Dec 4, 2016

Appending different timezones and coercing to UTC loses information. It is up to the user to explicitly coerce to the same timezone if you don't want this behavior. Having mixed timezones within a single object is only supported with boxed objects, hence the object dtype.

This what fixed in #13360 / xref issue was #7795

In [16]: result.tolist()
Out[16]: 
[Timestamp('1990-02-01 09:00:00-0500', tz='America/New_York', freq='T'),
 Timestamp('1990-02-01 09:01:00-0500', tz='America/New_York', freq='T'),
 Timestamp('1990-02-01 09:02:00-0500', tz='America/New_York', freq='T'),
 Timestamp('1990-02-01 08:03:00-0600', tz='America/Chicago', freq='T'),
 Timestamp('1990-02-01 08:04:00-0600', tz='America/Chicago', freq='T'),
 Timestamp('1990-02-01 08:05:00-0600', tz='America/Chicago', freq='T')]

from above.

@jreback jreback closed this as completed Dec 4, 2016
@jreback jreback added Timezones Timezone data dtype Usage Question labels Dec 4, 2016
@jreback jreback added this to the No action milestone Dec 4, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Timezones Timezone data dtype Usage Question
Projects
None yet
Development

No branches or pull requests

2 participants