BLD: windows builds failing in pandas.json

[jreback]

cc @Komnomnomnom

was reported by Gagi, but I reproduced this on building on AMD64

This is current master

can you take a look?

thanks

building 'pandas.json' extension
creating build\temp.win-amd64-2.7\Release\pandas\src\ujson
creating build\temp.win-amd64-2.7\Release\pandas\src\ujson\python
creating build\temp.win-amd64-2.7\Release\pandas\src\ujson\lib
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ipandas/src/ujson/python -Ipandas/src/ujson/lib -Ipandas/src/datetime -Ipandas/src/klib -Ipandas/src -IC:\pyth
on27\lib\site-packages\numpy\core\include -IC:\python27\include -IC:\python27\PC /Tcpandas/src/ujson/python/ujson.c /Fobuild\temp.win-amd64-2.7\Release\pandas/src/ujson/python/ujson.obj -D_GNU_SOURCE
ujson.c
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ipandas/src/ujson/python -Ipandas/src/ujson/lib -Ipandas/src/datetime -Ipandas/src/klib -Ipandas/src -IC:\pyth
on27\lib\site-packages\numpy\core\include -IC:\python27\include -IC:\python27\PC /Tcpandas/src/ujson/python/objToJSON.c /Fobuild\temp.win-amd64-2.7\Release\pandas/src/ujson/python/objToJSON.obj -D_GNU_SOURCE
objToJSON.c
c:\python27\lib\site-packages\numpy\core\include\numpy\npy_deprecated_api.h(8) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
pandas/src/ujson/python/objToJSON.c(249) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
pandas/src/ujson/python/objToJSON.c(274) : error C2143: syntax error : missing ';' before 'type'
pandas/src/ujson/python/objToJSON.c(275) : error C2275: 'PyDatetimeScalarObject' : illegal use of this type as an expression
        C:\python27\lib\site-packages\numpy\core\include\numpy/arrayscalars.h(123) : see declaration of 'PyDatetimeScalarObject'
pandas/src/ujson/python/objToJSON.c(275) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2223: left of '->obval' must point to struct/union
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2223: left of '->obmeta' must point to struct/union
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'dts' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : warning C4047: 'function' : 'npy_datetime' differs in levels of indirection from 'int *'
pandas/src/ujson/python/objToJSON.c(277) : warning C4024: 'pandas_datetime_to_datetimestruct' : different types for formal and actual parameter 1
pandas/src/ujson/python/objToJSON.c(277) : error C2198: 'pandas_datetime_to_datetimestruct' : too few arguments for call
pandas/src/ujson/python/objToJSON.c(278) : error C2065: 'dts' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(278) : warning C4133: 'function' : incompatible types - from 'int *' to 'pandas_datetimestruct *'
pandas/src/ujson/python/objToJSON.c(356) : warning C4244: '=' : conversion from 'npy_intp' to 'int', possible loss of data
error: command '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe"' failed with exit status 2
PS H:\jreback\pandas>

[Komnomnomnom]

I haven't been able to test on a windows machine but could you give the PR #4778 a try. I suspect it's the old 'must declare variables before doing anything else' windows malarky which I always fall foul of.

[jreback]

sure ill give a run tomorrow

[jtratner]

would it make sense to revert to a working version of pandas.json (i.e., from v0.12.0) so that windows builds work?

[jreback]

@Komnomnomnom any luck on this?

[Komnomnomnom]

Apologies for letting this slide guys. I'll have some time to look into this today and all going well I'll submit a fix.

[jtratner]

I think we should create a separate windows branch that tracks master, but with pandas.json either reverted to earlier version or with Komnomnomnom's fixes applied, so at the very least we can still build nightly builds.

[jreback]

@Komnomnomnom I pushed this: 7b81ca6

seems to build ok on amd64 now and all tests pass

weird why it didn't work before

let's see if daily builds work now....

[jreback]

@Komnomnomnom but this is now failing on 32-bit builds in linux land (but not travis)....weird

...................................................................................................................................

vagrant@precise32:~/pandas_master$ uname -a
Linux precise32 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012 i686 i686 i386 GNU/Linux

INSTALLED VERSIONS
------------------
Python: 2.7.3.final.0
OS: Linux 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012 i686
byteorder: little
LC_ALL: en_US
LANG: en_US

Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.11.0
statsmodels: Not installed
    patsy: 0.1.0
scikits.timeseries: Not installed
dateutil: 2.1
pytz: 2013b
bottleneck: Not installed
PyTables: 2.4.0
    numexpr: 2.1
matplotlib: Not installed
openpyxl: 1.6.1
xlrd: 0.9.2
xlwt: 0.7.5
sqlalchemy: Not installed
lxml: 3.2.1
bs4: 4.0.2
html5lib: 1.0b1

======================================================================
FAIL: test_date_format_frame (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/pandas_master/pandas/io/tests/test_json/test_pandas.py", line 465, in test_date_format_frame
    test_w_date('20130101 20:43:42.123')
  File "/home/vagrant/pandas_master/pandas/io/tests/test_json/test_pandas.py", line 463, in test_w_date
    assert_frame_equal(result, df)
  File "/home/vagrant/pandas_master/pandas/util/testing.py", line 286, in assert_frame_equal
    assert_index_equal(left.index, right.index)
  File "/home/vagrant/pandas_master/pandas/util/testing.py", line 165, in assert_index_equal
    right.dtype))
AssertionError: [index] left [object Index([u'8589934594000-01-03T00:00:00.000Z', u'8589934594000-01-04T00:00:00.000Z', u'8589934594000-01-05T00:00:00.000Z', u'8589934594000-01-06T00:00:00.000Z', u'8589934594000-01-07T00:00:00.000Z', u'8589934594000-01-10T00:00:00.000Z', u'8589934594000-01-11T00:00:00.000Z', u'8589934594000-01-12T00:00:00.000Z', u'8589934594000-01-13T00:00:00.000Z', u'8589934594000-01-14T00:00:00.000Z', u'8589934594000-01-17T00:00:00.000Z', u'8589934594000-01-18T00:00:00.000Z', u'8589934594000-01-19T00:00:00.000Z', u'8589934594000-01-20T00:00:00.000Z', u'8589934594000-01-21T00:00:00.000Z', u'8589934594000-01-24T00:00:00.000Z', u'8589934594000-01-25T00:00:00.000Z', u'8589934594000-01-26T00:00:00.000Z', u'8589934594000-01-27T00:00:00.000Z', u'8589934594000-01-28T00:00:00.000Z', u'8589934594000-01-31T00:00:00.000Z', u'8589934594000-02-01T00:00:00.000Z', u'8589934594000-02-02T00:00:00.000Z', u'8589934594000-02-03T00:00:00.000Z', u'8589934594000-02-04T00:00:00.000Z', u'8589934594000-02-07T00:00:00.000Z', u'8589934594000-02-08T00:00:00.000Z', u'8589934594000-02-09T00:00:00.000Z', u'8589934594000-02-10T00:00:00.000Z', u'8589934594000-02-11T00:00:00.000Z'], dtype=object)], right [<class 'pandas.tseries.index.DatetimeIndex'>
[2000-01-03 00:00:00, ..., 2000-02-11 00:00:00]
Length: 30, Freq: B, Timezone: None datetime64[ns]]

======================================================================
FAIL: test_date_format_series (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/pandas_master/pandas/io/tests/test_json/test_pandas.py", line 486, in test_date_format_series
    test_w_date('20130101 20:43:42.123')
  File "/home/vagrant/pandas_master/pandas/io/tests/test_json/test_pandas.py", line 484, in test_w_date
    assert_series_equal(result, ts)
  File "/home/vagrant/pandas_master/pandas/util/testing.py", line 260, in assert_series_equal
    assert_index_equal(left.index, right.index)
  File "/home/vagrant/pandas_master/pandas/util/testing.py", line 165, in assert_index_equal
    right.dtype))
AssertionError: [index] left [object Index([u'8589934594000-01-03T00:00:00.000Z', u'8589934594000-01-04T00:00:00.000Z', u'8589934594000-01-05T00:00:00.000Z', u'8589934594000-01-06T00:00:00.000Z', u'8589934594000-01-07T00:00:00.000Z', u'8589934594000-01-10T00:00:00.000Z', u'8589934594000-01-11T00:00:00.000Z', u'8589934594000-01-12T00:00:00.000Z', u'8589934594000-01-13T00:00:00.000Z', u'8589934594000-01-14T00:00:00.000Z', u'8589934594000-01-17T00:00:00.000Z', u'8589934594000-01-18T00:00:00.000Z', u'8589934594000-01-19T00:00:00.000Z', u'8589934594000-01-20T00:00:00.000Z', u'8589934594000-01-21T00:00:00.000Z', u'8589934594000-01-24T00:00:00.000Z', u'8589934594000-01-25T00:00:00.000Z', u'8589934594000-01-26T00:00:00.000Z', u'8589934594000-01-27T00:00:00.000Z', u'8589934594000-01-28T00:00:00.000Z', u'8589934594000-01-31T00:00:00.000Z', u'8589934594000-02-01T00:00:00.000Z', u'8589934594000-02-02T00:00:00.000Z', u'8589934594000-02-03T00:00:00.000Z', u'8589934594000-02-04T00:00:00.000Z', u'8589934594000-02-07T00:00:00.000Z', u'8589934594000-02-08T00:00:00.000Z', u'8589934594000-02-09T00:00:00.000Z', u'8589934594000-02-10T00:00:00.000Z', u'8589934594000-02-11T00:00:00.000Z'], dtype=object)], right [<class 'pandas.tseries.index.DatetimeIndex'>
[2000-01-03 00:00:00, ..., 2000-02-11 00:00:00]
Length: 30, Freq: B, Timezone: None datetime64[ns]]

[Komnomnomnom]

@jreback looks like the linux issue boils down to date encoding

In [2]: pd.json.dumps(pd.Timestamp('2013-01-01'), iso_dates=True)
Out[2]: '"699470488266606557-01-01T00:00:00.000Z"'

opened #4869 to track it.

[jreback]

closed by #4778