Type error on using datetime64's microseconds in datetime.timedelta

[miraculixx]

Using Pandas' datetime64 dtype to convert to a datetime.timedelta object results in this type error:

TypeError: unsupported type for timedelta microseconds component: numpy.int32

Reproduce:

# set up a dataframe with a datetime value in it
from datetime import datetime
import pandas as pd
df = pd.DataFrame({'foo' : [datetime.now()] })
print df.dtypes
# =>
foo    datetime64[ns]
dtype: object
# let's convert to timedelta
df.foo.apply(lambda r : timedelta(hours = r.hour, minutes = r.minute, seconds = r.second, microseconds = r.microseconds))
=> TypeError: unsupported type for timedelta microseconds component: numpy.int32

Expected:

# no excepetions, whatever the correct value
df.foo.apply(lambda r : timedelta(hours = r.hour, minutes = r.minute, seconds = r.second, microseconds = r.microseconds))
# => 
0   16:08:25.347571
Name: foo, dtype: timedelta64[ns]

Work around:

# if you can't control the timedelta conversion (i.e. it is somewhere embedded, e.g. in a library)
df.foo = df.foo.astype(datetime)
# if you can
df.foo.apply(lambda r : timedelta(hours = r.hour, minutes = r.minute, seconds = r.second, microseconds = int(r.microseconds))

Version information

> pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-24-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.16.0
nose: None
Cython: None
numpy: 1.9.2
scipy: None
statsmodels: None
IPython: 3.0.0
sphinx: 1.2.2
patsy: None
dateutil: 2.3
pytz: 2014.10
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: 2.1.4
xlrd: None
xlwt: None
xlsxwriter: None
lxml: 3.4.2
bs4: 4.3.2
html5lib: 0.999
httplib2: 0.8
apiclient: 1.2
sqlalchemy: None
pymysql: None
psycopg2: None

[sinhrks]

Thanks for the report.

Some Timestamp properties return np.int32, and if these are passed to a logic like isinstance(x, int) should fail. Every properties should return int, @jreback?

ts = pd.Timestamp('2011-01-01')
type(ts.hour)
# <type 'int'>
type(ts.hour)
# <type 'int'>
type(ts.minute)
# <type 'int'>
type(ts.second)
# <type 'int'>
type(ts.nanosecond)
# <type 'int'>
type(ts.dayofweek)
# <type 'int'>

type(ts.microsecond)
# <type 'numpy.int32'>
type(ts.quarter)
# <type 'numpy.int32'>
type(ts.dayofyear)
# <type 'numpy.int32'>
type(ts.week)
# <type 'numpy.int32'>
type(ts.daysinmonth)
# <type 'numpy.int32'>

[jreback]

yeh, these should all be a python int.

@miraculixx should should simply do this in any event (once .normalize is part of .dt this will be even simpler)

In [22]: df.foo-pd.DatetimeIndex(df.foo).normalize().values
Out[22]: 
0   06:29:07.091568
Name: foo, dtype: timedelta64[ns]