datetime not properly converted from Series; timezone dropped

[benjschiller]

This bug also happens if you use the .convert_objects() method of DataFrame or Series.

To reproduce:

In [83]: import pytz

In [85]: d = datetime.datetime(2014, 9, 3, 2, 40, 16, tzinfo=pytz.utc)

In [88]: pd.to_datetime(pd.Series([d]))[0]
Out[88]: Timestamp('2014-09-03 02:40:16', tz=None)

In [89]: pd.to_datetime([d])[0]
Out[89]: Timestamp('2014-09-03 02:40:16+0000', tz='UTC')

In [91]: pd.__version__
Out[91]: '0.13.1'

[jreback]

By definition to_datetime will do this. no need to use it at all.

In [10]: pd.Series([d])
Out[10]: 
0    2014-09-03 02:40:16+00:00
dtype: object

[jreback]

@benjschiller by definition a datetime that is tz-aware is object dtype. So convert_objects is also as expected.

[benjschiller]

I'm not sure I understand your comments. If you had a list of timezone-aware datetimes, how would you conveniently convert that into a time-zone aware pandas Series?

[jreback]

Series([date1,date2,date3])

[benjschiller]

It seems much more desirable to have an actual TimeSeries representation. a datetime64 ndarray is fine but for some reason pandas doesn't support the tz_localize and tz_convert methods of DateTimeIndex with this type (it's not clear to me why this is only desirable for an index). Basically, it should be possible to manipulate the time series as a whole (tz_localize, tz_convert, etc) without individually manipulating every element

Also, this mucks up the convert_to_r_dataframe function, because it's expecting a datetime64 series as the correspondence to a POSX time vector in R, which is what everything in R wants for representing time. "object" Series are converted to strings. Of course, one could coerce the data, but it would be really nice if this stuff all worked according to a common design.

[jreback]

hmm, not really sure what you are trying to do, but 0.14.1 and 0.15.0 (coming soon) have more support for series-like indexes. Of course if you would like to contribute a patch to better support tz-aware Series would be welcome. It is however a failry big implementation.

I have no idea about the r conversion.

[jreback]

Here's an issue: #8260

[benjschiller]

Alright, thanks. I'll see what I can whip together in the next couple weeks. For now I'll just use the fact that the datetime64 series is UTC to do the conversion in R

[jreback]

gr8! timezones are tricky, but the DatetimeIndex handles them real well, its just a matter of having the block record the tz and wrap/unwrap it.