powerpc: Test*Parser* fail with AssertionError: 2262-04-10T00:12:45.292707840+0000 != 2262-04-10T00:12:45.292707840+0000

[yarikoptic]

Detected by Debian build farm
https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=powerpc&ver=0.12.0~rc1%2Bgit112-gb79996c-1&stamp=1374189622&file=log

======================================================================
FAIL: test_nat_parse (pandas.io.tests.test_parsers.TestCParserHighMemory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_parsers.py", line 607, in test_nat_parse
    tm.assert_frame_equal(result,df)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 238, in assert_frame_equal
    check_less_precise=check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 197, in assert_series_equal
    assert_almost_equal(left.values, right.values, check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 141, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 172, in assert_almost_equal
    assert a == b, "%s != %s" % (a, b)
AssertionError: 2262-04-10T00:12:45.292707840+0000 != 2262-04-10T00:12:45.292707840+0000

======================================================================
FAIL: test_nat_parse (pandas.io.tests.test_parsers.TestCParserLowMemory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_parsers.py", line 607, in test_nat_parse
    tm.assert_frame_equal(result,df)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 238, in assert_frame_equal
    check_less_precise=check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 197, in assert_series_equal
    assert_almost_equal(left.values, right.values, check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 141, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 172, in assert_almost_equal
    assert a == b, "%s != %s" % (a, b)
AssertionError: 2262-04-10T00:12:45.292707840+0000 != 2262-04-10T00:12:45.292707840+0000

======================================================================
FAIL: test_nat_parse (pandas.io.tests.test_parsers.TestPythonParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_parsers.py", line 607, in test_nat_parse
    tm.assert_frame_equal(result,df)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 238, in assert_frame_equal
    check_less_precise=check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 197, in assert_series_equal
    assert_almost_equal(left.values, right.values, check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 141, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/«BUILDDIR»/pandas-0.12.0~rc1+git112-gb79996c/debian/tmp/usr/lib/python2.7/dist-packages/pandas/util/testing.py", line 172, in assert_almost_equal
    assert a == b, "%s != %s" % (a, b)
AssertionError: 2262-04-10T00:12:45.292707840+0000 != 2262-04-10T00:12:45.292707840+0000

good news -- those seems to be the only failures on powerpc (for python 2.7 at least, python 3 did not have chance to get any testing yet due to 2.7 failing)

[jtratner]

is this current master?

[cpcloud]

@jtratner looks like 2 behind (b79996c), but last 2 are merge + y-p doc commit so should be ok

[yarikoptic]

FWIW on s390 the same test fails too
https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=s390x&ver=0.12.0~rc1%2Bgit112-gb79996c-1&stamp=1374197022&file=log
but then the whole build f.cks up somehow upon test_non_unique_loc_memory_error (yet to troubleshoot) and gets killed... so no details on failures and errors are provided

[jreback]

this is again a byteorder issue....

[jreback]

@yarikoptic

must be something weird going on....on the sparc machine....I can't even build master...it just hangs in the compiler (I turned off quiet)....and goes most of the way thru, but been spinning the cpu for a while (and not out of memory 10% being used)...its the compile step for algos.o....(its pretty large)...maybe that's an issue?
on python 2.7 stock build after I did scroot

any ideas?

[yarikoptic]

well -- that box is quite antique by now (somewhere from 2000 or around), so a bit of extra patience would help ;) I just usually leave it building and come back to it later. also you might like to make use of installed ccache

I see that it now runs python3.2 for you - so I guess there was progress ;)

[jreback]

ok...will let it build (though 20 min still seems very odd to just compile a file)....

[jreback]

So the issue on that wrap around problem is happening
....let me see if I can fix this

(Pdb) p pd.NaT.value
-9223372036854775808L
(Pdb) p df['B'].iloc[3]
Timestamp('2262-04-11 23:47:16.854775807', tz=None)
(Pdb) p df['B'].iloc[3].value
9223372036854775807L

these are supposed to be the same

[jreback]

@yarikoptic pls give a retest....I think I found the problem...

as assignng nan to a M8[ns] numpy array, works fine (it assigns iNaT), the integer version of NaT on little endian. but doesn't work on big endian; so do a pre-convesion to catch this (which should have been done anyhow, as I suspect this is not a documented behavior)