API/ENH: stata export should better support np.datetime64 based times

[wuan]

The following code exporting a dataframe with a timeseries index with np.datetime64 type

import numpy as np
import pandas as pd

timestamps = pd.date_range(np.datetime64('2013-01-01 11:00:00.123456'), periods=5, freq='123U')
dataframe = pd.DataFrame(np.random.randn(len(timestamps)), index=timestamps)
dataframe.to_stata('out')

throws a

ValueError: Data type datetime64[ns] not currently understood. Please report an error to the developers.

It would be very nice if np.datetime64 (and thus pd.Timestamp as well) would be supported here.

[jreback]

Here's the example from the tests
I think you have to specify a datetime format as an argument when writing
(and this is the only test, so maybe there are some bugs)

In [44]: original = DataFrame(data=[["string", "object", 1, 1.1,np.datetime64('2003-12-25')]],columns=['string', 'object', 'integer', 'float','datetime'])

In [45]: original["object"] = Series(original["object"], dtype=object)

In [46]: original.index.name = 'index'

In [47]: original
Out[47]: 
       string  object  integer  float            datetime
index                                                    
0      string  object        1    1.1 2003-12-25 00:00:00

In [48]: original.dtypes
Out[48]: 
string              object
object              object
integer              int64
float              float64
datetime    datetime64[ns]
dtype: object
In [50]: original.to_stata('out',{'datetime' : 'tc'})

In [52]: pd.read_stata('out')
Out[52]: 
   index  string  object  integer  float            datetime
0      0  string  object        1    1.1 2003-12-25 00:00:00

In [53]: pd.read_stata('out').dtypes
Out[53]: 
index                int64
string              object
object              object
integer              int64
float              float64
datetime    datetime64[ns]
dtype: object

[wuan]

Thanks for the example. The difference here seems to be, that the datetime64 data is contained as a data column and not as table index.

Unfortunately this

dataframe.to_stata('out', {'index':'tc'})

does not help in my example.

[jreback]

yep I agree....does this cause a failing test for you?

[wuan]

No, I just found this behaviour while looking to reproduce another problem.

[jreback]

ok...great

[bashtage]

This occurs because the column has no name, and so it by default named 0 (number not '0'), which causes issued with the way the date time conversion works.

This works correctly;

timestamps = pd.date_range(np.datetime64('2013-01-01 11:00:00.123456'),
                           periods=5, freq='123U')
dataframe = pd.DataFrame(np.random.randn(len(timestamps)), index=timestamps,
                         columns=['something'])
dataframe.to_stata('out', convert_dates={'index': 'tc'})

[jreback]

is it s problem to have numeric column labels in general?

[bashtage]

Dunno, working through this now - I think the fix will be simple and won't cause an issue.

[bashtage]

PR #6622

No test yet, but too late over here to write a sensible test tonight.