Creating DataFrame Changes Time Unit of datetime64 Series

[cancan101]

On pandas v0.11.0, For example:

foo = pd.Series(np.array([np.datetime64(datetime.date(2012, 1, 1), 'M')]))

In [64]: foo.dtype
Out[64]: dtype('<M8[M]')

but then:

In [65]: pd.DataFrame({"date":foo,}).date.dtype
Out[65]: dtype('<M8[ns]')

[jreback]

this is not allowed in 0.12 as np.datetime64 is broken in numpy < 1.7.1 (and buggy in 0.11)

what are u trying to do ?

[cancan101]

What do you mean by "this" when you say "this is not allowed"?

I have a time series of data that has a monthly frequency. I am trying to create a DataFrame from this data.

[jreback]

I mean no need to use np.datetime64 directly

pd.Series([datetime(2012,1,1)]) works

there is no concept of monthly in a series

an index is probably what you want

pd.date_range('20120101',periods=5,freq='M')

np.datetime64[freq] is just needs to be converted; but it is broken in numpy and causes all kinds of issues,
so not allowing it, you need to convert your data from np.datetime64; there other ways which work, see pd.to_datetime, you can construct timestamps, use datetimes, etc

[cancan101]

I was going off the ideas here: http://docs.scipy.org/doc/numpy-dev/reference/arrays.datetime.html#datetime-units

It does look like something is actively preventing me from using any timeunit other than ns:

bar = pd.DataFrame({"date":foo,})

In [134]: bar.date.dtype
Out[134]: dtype('<M8[ns]')

bar.date = bar.date.astype("M8[D]")
In [131]: bar.date.dtype
Out[131]: dtype('<M8[ns]')

[jreback]

Use pandas features see here:
http://pandas.pydata.org/pandas-docs/dev/timeseries.html

datetimes are stored as ns but that is an implementation detail

you should be thinking about using DatetimeIndex or PeriodIndex

lots of goodies and neat ways to represent data

[cancan101]

I would argue that even using those feature, there is still some oddities here:

In [144]: pd.date_range(np.datetime64(datetime.date(2012,1,1), 'M'), periods=2)
Out[144]: 
<class 'pandas.tseries.index.DatetimeIndex'>
[2012-01-01 00:00:00, 2012-01-02 00:00:00]
Length: 2, Freq: D, Timezone: None

Theoretically the freq of the date range can be inferred from the datetime units provided.

[cancan101]

In other words, I should not have to say:

pd.date_range(np.datetime64(datetime.date(2012,1,1), 'M'), periods=2, freq='M')

[jreback]

pd.date_range((datetime.datetime(2012,1,1),periods=2,freq='M)
pd.date_range('20120101',periods=2,freq='M')

explicity using np.dtetime64 is not necessary

[jreback]

I suppose you example above, passing a np.datetime64 does allow DatetimeIndex to infer a frequency

[jreback]

created #4341 to track the issue.

[cancan101]

I think you mean #4341

[jreback]

I stand corrected

[cancan101]

@jreback I wasn't trying to be a jerk, I just wanted to refer any future visitors (my self included) to the thread to the updated issue.