Timezone-aware times are inconsistent using object or datetime64

[rockg]

I just upgraded to 0.14 and am seeing an inconsistent behavior when converting datetimes to objects. This used to be consistent in 0.12.1 and now it loses timezone information when converting to datetime64. Is this expected?

import pytz
import datetime
from pandas import DataFrame
tz = pytz.timezone('US/Eastern')
dt = tz.localize(datetime.datetime(2012, 6, 1))
dt
Out[31]: datetime.datetime(2012, 6, 1, 0, 0, tzinfo=<DstTzInfo 'US/Eastern' EDT-1 day, 20:00:00 DST>)
df = DataFrame([{'End Date': dt}])
df
Out[26]: 
                    End Date
0  2012-06-01 00:00:00-04:00
df.dtypes
Out[27]: 
End Date    object
dtype: object
df_c = df.convert_objects(convert_dates=True)
df_c
Out[29]: 
             End Date
0 2012-06-01 04:00:00
df_c.dtypes
Out[30]: 
End Date    datetime64[ns]
dtype: object

[jreback]

this has always been the case iirc
datetimes with tz are object and w/tz are datetime64[ns](before 0.10 most things were simply object)

their is no capacity to attach a timezone to a series (though u can do it with an index) and have it remain datetime64[ns]. it is simply a collection of timestamps that are object dtyped

convert_objects forcibly converts things

what are you trying to do?

[rockg]

Here's the real difference. I have columns that need to be timezone-aware which used to be possible. Now it coerces into a datetime64 (but only if you pass an index rather than constructing with an array of dicts).

0.12:

>>> import pytz
>>> from datetime import datetime
>>> from pandas import DataFrame
>>> tz = pytz.timezone('US/Eastern')
>>> dt = tz.localize(datetime(2012, 1, 1))
>>> df = DataFrame({'End Date': dt}, index=[0])
>>> df
                    End Date
0  2012-01-01 00:00:00-05:00
>>> df.dtypes
End Date    object
dtype: object
>>> 

0.14:

>>> import pytz
>>> from datetime import datetime
>>> from pandas import DataFrame
>>> tz = pytz.timezone('US/Eastern')
>>> dt = tz.localize(datetime(2012, 1, 1))
>>> df = DataFrame({'End Date': dt}, index=[0])
>>> df
             End Date
0 2012-01-01 05:00:00
>>> df1 = DataFrame([{'End Date': dt}])
>>> df1
                    End Date
0  2012-01-01 00:00:00-05:00
>>> df1.dtypes
End Date    object
dtype: object
>>> 

[jreback]

ahh I see, ok that could be a bug. In general we try to coerce anything datelike to datetime64[ns] (excepting strings which require explict conversion). This should not force tz-aware though.

[jreback]

@rockg this seems to work in master, so I put up some tests. I think fixed somewhere along the line

cc @sinhrks

[rockg]

Thanks....he has been fixing a lot of these issues which is great.