BUG: DataFrame.to_records() bug in converting datetime64 index with timezone

[johngu]

Fix

in to_records(), use pandas.core.common.is_datetime64_any_dtype instead of pandas.core.common.is_datetime64_dtype to check to see if the index is in fact of the datetime64 type.

Code Sample, a copy-pastable example if possible

import datetime
import pandas as pd
import pytz

data = [datetime.datetime.now(pytz.timezone('UTC')) for _ in range(10)]
df = pd.DataFrame({'datetime': data})
df.set_index('datetime', inplace=True)
df.to_records()

Expected Output

data

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 20.3
Cython: 0.23.4
numpy: 1.10.4
scipy: 0.17.0
statsmodels: 0.6.1
xarray: None
IPython: 4.1.2
sphinx: 1.3.5
patsy: 0.4.0
dateutil: 2.5.1
pytz: 2016.2
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.5
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.6.0
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0
pandas_datareader: None

[jreback]

note that this should return an object array with tz-aware objects (as numpy has no clu about non-naive datetimes).

e.g. similar to this (though this is tz-naive)

In [35]: df
Out[35]: 
Empty DataFrame
Columns: []
Index: [2016-08-08 15:50:08.058674, 2016-08-08 15:50:08.058699, 2016-08-08 15:50:08.058713, 2016-08-08 15:50:08.058716, 2016-08-08 15:50:08.058719, 2016-08-08 15:50:08.058722, 2016-08-08 15:50:08.058725, 2016-08-08 15:50:08.058728, 2016-08-08 15:50:08.058732, 2016-08-08 15:50:08.058735]

In [36]: df.to_records()
Out[36]: 
rec.array([(datetime.datetime(2016, 8, 8, 15, 50, 8, 58674),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58699),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58713),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58716),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58719),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58722),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58725),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58728),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58732),),
 (datetime.datetime(2016, 8, 8, 15, 50, 8, 58735),)], 
          dtype=[('datetime', 'O')])

pull-requests welcome. I don't think this is highly tested. Most people don't really use numpy arrays directly once they need something (e.g. tz) that they don't provide.

[jreback]

note this code has been changed quite a in master. But the fix is essentially the same .