pandas.io.tests.test_json.test_ujson.NumpyJSONTests testOdArray segfaults under python2.7/3.5 in 0.17.0

[jlec]

testOdArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... Segmentation fault (core dumped)

In [2]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Linux
OS-release: 4.1.8-lh
machine: x86_64
processor: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
byteorder: little
LC_ALL: None
LANG: en_GB.utf8

pandas: 0.17.0
nose: 1.3.7
pip: 7.1.2
setuptools: 18.4
Cython: 0.23.3
numpy: 1.9.3
scipy: 0.16.0
statsmodels: 0.6.1
IPython: 4.0.0
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.4.2
pytz: 2015.6
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.4.4
matplotlib: 1.4.3
openpyxl: 2.2.6
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.7.4
lxml: 3.4.4
bs4: 4.4.1
html5lib: 0.9999999
httplib2: 0.9.2
apiclient: 1.3.1
sqlalchemy: 1.0.8
pymysql: 0.6.6.None
psycopg2: 2.6.1 (dt dec pq3 ext lo64)

Abrt report: https://gist.github.com/jlec/6282f0dae0586d6749af

[jreback]

might be a locale issue. again I cannot repro this, can you see what the issue is, propose a fix?

[jlec]

python-3.5 shows also the segf. python-3.4 is fine. aa94ae4 doesn't fix the issue

[jreback]

the commit is pointing to something else

since u r only getting this error (and works on all other platforms / versions)

can u see if u can see where happening

[jlec]

the commit is pointing to something else

That's the one reference

since u r only getting this error (and works on all other platforms / versions)
can u see if u can see where happening

The best I can get is
https://gist.github.com/jlec/6282f0dae0586d6749af

[neirbowj]

I'm seeing this segfault too. It's acting like a Heisenbug. I've been testing with 0.17.1 under python2.7 and python3.4, and the only way I've been able to reliably reproduce it is with python 3.4, pandas compiled with debugging turned off (and optimizations on), and by running the whole NumpyJSONTests class, not just the test0dArray test case.

% nosetests-3.4 -v pandas.io.tests.test_json.test_ujson:NumpyJSONTests.testOdArray
testOdArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
% nosetests-3.4 -v pandas.io.tests.test_json.test_ujson:NumpyJSONTests
testArrayNumpyExcept (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testArrayNumpyLabelled (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testArrays (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testBool (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testBoolArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testFloat (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testFloatArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testFloatMax (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testInt (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testIntArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testIntMax (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... ok
testOdArray (pandas.io.tests.test_json.test_ujson.NumpyJSONTests) ... Segmentation fault (core dumped)

In the following print_versions output, you'll notice that I have exclude as many dependencies as possible.

INSTALLED VERSIONS
------------------
commit: None
python: 3.4.3.final.0
python-bits: 64
OS: FreeBSD
OS-release: 10.1-STABLE
machine: amd64
processor: amd64
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.17.1
nose: 1.3.7
pip: None
setuptools: 17.0
Cython: None
numpy: 1.10.1
scipy: None
statsmodels: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.3
pytz: 2015.7
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
Jinja2: None

Here is a backtrace from the coredump.

Any suggestions for how to proceed?

[jlec]

Interestingly I see this again with 0.19.1. No segfaults but Object_beginTypeContext: Assertion `PyBytes_Check(tmpObj)' failed.