Skip to content

BUG: read_csv segfault #5664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dsm054 opened this issue Dec 8, 2013 · 15 comments · Fixed by #9846
Closed

BUG: read_csv segfault #5664

dsm054 opened this issue Dec 8, 2013 · 15 comments · Fixed by #9846
Labels
Bug IO CSV read_csv, to_csv
Milestone
0.16.1

Comments

@dsm054
Copy link
Contributor

dsm054 commented Dec 8, 2013

Came across this when trying to parse a poorly-formatted csv file and got the number of columns wrong. Mostly it segfaults; sometimes it produces a broken last row.

>>> import pandas as pd
>>> from StringIO import StringIO
>>> pd.__version__
'0.13.0rc1-43-g4f9fefc'
>>> 
>>> f = StringIO('1,1,1,1,0\n'*2 + '\n'*2)
>>> df = pd.read_csv(f,names=list("abcd"))
Segmentation fault (core dumped)

When it doesn't segfault it tends to produce

      a   b   c    d
 1    1   1   1    0
 1    1   1   1    0
NaN NaN NaN NaN  NaN
NaN NaN NaN NaN  [unicode box: 0 0 0 1]
@guyrt
Copy link
Contributor

guyrt commented Jan 5, 2014

This is a C parser problem.

Python parser appears to correctly handle names of lengths 1 and up.
C parser chokes on len(names) in [1, 2] and also 3 or 4 if any names are strings.

@jreback
Copy link
Contributor

jreback commented Jan 5, 2014

gr8

care to submit a PR?

@guyrt
Copy link
Contributor

guyrt commented Jan 6, 2014

Yeah, I'm working on it. Isolated problem (#tokens < expected causing read from out of bounds) and working on fix.

@jreback
Copy link
Contributor

jreback commented Feb 14, 2014

@guyrt progress on this?

@jreback
Copy link
Contributor

jreback commented Apr 6, 2014

@dsm054 you want to do a PR for this?

@guyrt ?

@guyrt
Copy link
Contributor

guyrt commented Apr 18, 2014

My fix hit a road block. I'll try to return to this once I free up some time.

@jreback
Copy link
Contributor

jreback commented Apr 28, 2014

@guyrt ? or cc @mcwitt

@jreback jreback modified the milestones: 0.14.1, 0.14.0 May 5, 2014
@jreback
Copy link
Contributor

jreback commented Jun 17, 2014

cc @mcwitt
cc @guyrt

@jreback jreback modified the milestones: 0.15.0, 0.14.1 Jun 19, 2014
@cstenac
Copy link

cstenac commented Jul 17, 2014

Hi,

Not totally sure, but I think this one is related too:

import pandas as pd, StringIO

pd.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])

Crashes about 50-80% of time, gives some NaN in other cases.

Reproduced on OS X and Ubuntu, Pandas 0.13.1 and 0.14.1

@guyrt
Copy link
Contributor

guyrt commented Jul 17, 2014

Reproduced @cstenac 's example on windows as well. Note first example, where we just loaded in whatever from memory. That's concerning.

 In [3]: pandas.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])
Out[3]:
    a
a   b
c  2♫

In [4]: pandas.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])
<failed>

That does appear to be same issue: the C engine computes an expected size and reads until it fills that size.

@tommyengstrom
Copy link

If there are two implementations of this and one is broken, shouldn't the other one be the default?

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@behzadnouri behzadnouri mentioned this issue Apr 10, 2015
Merged
@jreback jreback modified the milestones: 0.16.1, Next Major Release Apr 11, 2015
@jreback jreback mentioned this issue Sep 8, 2015
Closed
@alessiodore alessiodore mentioned this issue Jan 20, 2016
Closed
@jeromedockes
Copy link

Hello, I am not sure this is the same issue but the following snippet causes a segmentation fault on three machines I have tried (all running ubuntu). I have the same problem whenever I try to load a large csv file. However, if I load the file in chunks (using chunksize=...), there is no problem. Also, this problem doesn't occur if I use the python engine.

I am using pandas 0.20.1

import os

import pandas as pd
import numpy as np

N_WORDS = 1000
DUMP = '/tmp/hello.csv'

CHUNK_SIZE = None
# CHUNK_SIZE = 1000

ENGINE = 'c'
# ENGINE = 'python'

if not os.path.isfile(DUMP):
    text = pd.DataFrame(columns=np.arange(100), index=np.arange(10000))
    text.iloc[:, :] = 'hello ' * N_WORDS
    print(text.shape)

    text.to_csv(DUMP)
    print('wrote {}'.format(DUMP))

if CHUNK_SIZE is None:
    text = pd.read_csv(DUMP, engine=ENGINE)
else:
    chunks = []
    for chunk in pd.read_csv(DUMP, chunksize=CHUNK_SIZE, engine=ENGINE):
        chunks.append(chunk)
    text = pd.concat(chunks)

print('done reading csv')

output:

(10000, 100)
wrote /tmp/hello.csv
*** Error in `python': free(): invalid pointer: 0x00007f41f563173e ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f42085957e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f420859e37a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f42085a253c]
/home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/parsers.cpython-35m-x86_64-linux-gnu.so(parser_cleanup+0x1a)[0x7f41f562815a]
/home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/parsers.cpython-35m-x86_64-linux-gnu.so(+0x1a453)[0x7f41f55ce453]
python(PyEval_EvalFrameEx+0x45c4)[0x51ee74]
python(PyEval_EvalFrameEx+0x49a4)[0x51f254]
python(PyEval_EvalFrameEx+0x49a4)[0x51f254]
python(PyEval_EvalFrameEx+0x49a4)[0x51f254]
python[0x52426f]
python(PyEval_EvalFrameEx+0x508f)[0x51f93f]
python[0x523d29]
python(PyEval_EvalCode+0x1f)[0x5249ef]
python[0x601402]
python(PyRun_FileExFlags+0x9a)[0x60393a]
python(PyRun_SimpleFileExFlags+0x1bc)[0x603b2c]
python(Py_Main+0x456)[0x63dc06]
python(main+0xe1)[0x4cfb91]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f420853e830]
python(_start+0x29)[0x5d53c9]
======= Memory map: ========
00400000-007a7000 r-xp 00000000 08:02 9569867                            /home/jerome/.virtualenvs/py_3/bin/python3
009a7000-009a9000 r--p 003a7000 08:02 9569867                            /home/jerome/.virtualenvs/py_3/bin/python3
009a9000-00a40000 rw-p 003a9000 08:02 9569867                            /home/jerome/.virtualenvs/py_3/bin/python3
00a40000-00a71000 rw-p 00000000 00:00 0
011e9000-02639000 rw-p 00000000 00:00 0                                  [heap]
7f41741b4000-7f41b41b5000 rw-p 00000000 00:00 0
7f41ec000000-7f41ec021000 rw-p 00000000 00:00 0
7f41ec021000-7f41f0000000 ---p 00000000 00:00 0
7f41f37bc000-7f41f3dbd000 rw-p 00000000 00:00 0
7f41f3dbd000-7f41f48a0000 rw-p 00000000 00:00 0
7f41f48a0000-7f41f48b2000 r-xp 00000000 08:02 9585894                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/testing.cpython-35m-x86_64-linux-gnu.so
7f41f48b2000-7f41f4ab1000 ---p 00012000 08:02 9585894                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/testing.cpython-35m-x86_64-linux-gnu.so
7f41f4ab1000-7f41f4ab3000 rw-p 00011000 08:02 9585894                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/testing.cpython-35m-x86_64-linux-gnu.so
7f41f4ab3000-7f41f4c73000 rw-p 00000000 00:00 0
7f41f4c73000-7f41f4c74000 r-xp 00000000 08:02 9585711                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/util/_move.cpython-35m-x86_64-linux-gnu.so
7f41f4c74000-7f41f4e74000 ---p 00001000 08:02 9585711                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/util/_move.cpython-35m-x86_64-linux-gnu.so
7f41f4e74000-7f41f4e75000 rw-p 00001000 08:02 9585711                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/util/_move.cpython-35m-x86_64-linux-gnu.so
7f41f4e75000-7f41f4e8a000 r-xp 00000000 08:02 9585801                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_unpacker.cpython-35m-x86_64-linux-gnu.so
7f41f4e8a000-7f41f5089000 ---p 00015000 08:02 9585801                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_unpacker.cpython-35m-x86_64-linux-gnu.so
7f41f5089000-7f41f508c000 rw-p 00014000 08:02 9585801                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_unpacker.cpython-35m-x86_64-linux-gnu.so
7f41f508c000-7f41f509b000 r-xp 00000000 08:02 9585799                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_packer.cpython-35m-x86_64-linux-gnu.so
7f41f509b000-7f41f529b000 ---p 0000f000 08:02 9585799                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_packer.cpython-35m-x86_64-linux-gnu.so
7f41f529b000-7f41f529d000 rw-p 0000f000 08:02 9585799                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/io/msgpack/_packer.cpython-35m-x86_64-linux-gnu.so
7f41f529d000-7f41f535d000 rw-p 00000000 00:00 0
7f41f535d000-7f41f5373000 r-xp 00000000 08:02 9585888                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/json.cpython-35m-x86_64-linux-gnu.so
7f41f5373000-7f41f5573000 ---p 00016000 08:02 9585888                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/json.cpython-35m-x86_64-linux-gnu.so
7f41f5573000-7f41f5574000 rw-p 00016000 08:02 9585888                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/json.cpython-35m-x86_64-linux-gnu.so
7f41f5574000-7f41f55b4000 rw-p 00000000 00:00 0
7f41f55b4000-7f41f5637000 r-xp 00000000 08:02 9585895                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/parsers.cpython-35m-x86_64-linux-gnu.so
7f41f5637000-7f41f5837000 ---p 00083000 08:02 9585895                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/parsers.cpython-35m-x86_64-linux-gnu.so
7f41f5837000-7f41f583d000 rw-p 00083000 08:02 9585895                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/parsers.cpython-35m-x86_64-linux-gnu.so
7f41f583d000-7f41f58bf000 rw-p 00000000 00:00 0
7f41f58bf000-7f41f58d8000 r-xp 00000000 08:02 9585900                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/reshape.cpython-35m-x86_64-linux-gnu.so
7f41f58d8000-7f41f5ad8000 ---p 00019000 08:02 9585900                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/reshape.cpython-35m-x86_64-linux-gnu.so
7f41f5ad8000-7f41f5adb000 rw-p 00019000 08:02 9585900                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/reshape.cpython-35m-x86_64-linux-gnu.so
7f41f5adb000-7f41f5b1b000 rw-p 00000000 00:00 0
7f41f5b1b000-7f41f5bb0000 r-xp 00000000 08:02 9585891                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/groupby.cpython-35m-x86_64-linux-gnu.so
7f41f5bb0000-7f41f5daf000 ---p 00095000 08:02 9585891                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/groupby.cpython-35m-x86_64-linux-gnu.so
7f41f5daf000-7f41f5db5000 rw-p 00094000 08:02 9585891                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/groupby.cpython-35m-x86_64-linux-gnu.so
7f41f5db5000-7f41f5eb7000 rw-p 00000000 00:00 0
7f41f5eb7000-7f41f5ee4000 r-xp 00000000 08:02 9830674                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/matplotlib/_path.cpython-35m-x86_64-linux-gnu.so
7f41f5ee4000-7f41f60e4000 ---p 0002d000 08:02 9830674                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/matplotlib/_path.cpython-35m-x86_64-linux-gnu.so
7f41f60e4000-7f41f60e5000 rw-p 0002d000 08:02 9830674                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/matplotlib/_path.cpython-35m-x86_64-linux-gnu.so
7f41f60e5000-7f41f62a6000 rw-p 00000000 00:00 0
7f41f62a6000-7f41f6342000 r-xp 00000000 08:02 9585886                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/window.cpython-35m-x86_64-linux-gnu.so
7f41f6342000-7f41f6542000 ---p 0009c000 08:02 9585886                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/window.cpython-35m-x86_64-linux-gnu.so
7f41f6542000-7f41f6549000 rw-p 0009c000 08:02 9585886                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/window.cpython-35m-x86_64-linux-gnu.so
7f41f6549000-7f41f660b000 rw-p 00000000 00:00 0
7f41f660b000-7f41f6610000 r-xp 00000000 08:02 1446109                    /usr/lib/python3.5/lib-dynload/mmap.cpython-35m-x86_64-linux-gnu.so
7f41f6610000-7f41f6810000 ---p 00005000 08:02 1446109                    /usr/lib/python3.5/lib-dynload/mmap.cpython-35m-x86_64-linux-gnu.so
7f41f6810000-7f41f6811000 r--p 00005000 08:02 1446109                    /usr/lib/python3.5/lib-dynload/mmap.cpython-35m-x86_64-linux-gnu.so
7f41f6811000-7f41f6812000 rw-p 00006000 08:02 1446109                    /usr/lib/python3.5/lib-dynload/mmap.cpython-35m-x86_64-linux-gnu.so
7f41f6812000-7f41f6819000 r-xp 00000000 08:02 1446128                    /usr/lib/python3.5/lib-dynload/_csv.cpython-35m-x86_64-linux-gnu.so
7f41f6819000-7f41f6a19000 ---p 00007000 08:02 1446128                    /usr/lib/python3.5/lib-dynload/_csv.cpython-35m-x86_64-linux-gnu.so
7f41f6a19000-7f41f6a1a000 r--p 00007000 08:02 1446128                    /usr/lib/python3.5/lib-dynload/_csv.cpython-35m-x86_64-linux-gnu.so
7f41f6a1a000-7f41f6a1c000 rw-p 00008000 08:02 1446128                    /usr/lib/python3.5/lib-dynload/_csv.cpython-35m-x86_64-linux-gnu.so
7f41f6a1c000-7f41f6a5c000 rw-p 00000000 00:00 0
7f41f6a5c000-7f41f6b41000 r-xp 00000000 08:02 9585896                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/sparse.cpython-35m-x86_64-linux-gnu.so
7f41f6b41000-7f41f6d41000 ---p 000e5000 08:02 9585896                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/sparse.cpython-35m-x86_64-linux-gnu.so
7f41f6d41000-7f41f6d46000 rw-p 000e5000 08:02 9585896                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/sparse.cpython-35m-x86_64-linux-gnu.so
7f41f6d46000-7f41f6dc6000 rw-p 00000000 00:00 0
7f41f6dc6000-7f41f6dd7000 r-xp 00000000 08:02 1446142                    /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f41f6dd7000-7f41f6fd6000 ---p 00011000 08:02 1446142                    /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f41f6fd6000-7f41f6fd7000 r--p 00010000 08:02 1446142                    /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f41f6fd7000-7f41f6fd8000 rw-p 00011000 08:02 1446142                    /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f41f6fd8000-7f41f7018000 rw-p 00000000 00:00 0
7f41f7018000-7f41f707c000 r-xp 00000000 08:02 9585889                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/period.cpython-35m-x86_64-linux-gnu.so
7f41f707c000-7f41f727b000 ---p 00064000 08:02 9585889                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/period.cpython-35m-x86_64-linux-gnu.so
7f41f727b000-7f41f7283000 rw-p 00063000 08:02 9585889                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/period.cpython-35m-x86_64-linux-gnu.so
7f41f7283000-7f41f74c5000 rw-p 00000000 00:00 0
7f41f74c5000-7f41f74c6000 ---p 00000000 00:00 0
7f41f74c6000-7f41f7d06000 rw-p 00000000 00:00 0
7f41f7d06000-7f41f7d1c000 r-xp 00000000 08:02 14422446                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7f41f7d1c000-7f41f7f1b000 ---p 00016000 08:02 14422446                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7f41f7f1b000-7f41f7f1c000 rw-p 00015000 08:02 14422446                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7f41f7f1c000-7f41f808e000 r-xp 00000000 08:02 1183466                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f41f808e000-7f41f828e000 ---p 00172000 08:02 1183466                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f41f828e000-7f41f8298000 r--p 00172000 08:02 1183466                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f41f8298000-7f41f829a000 rw-p 0017c000 08:02 1183466                    /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7f41f829a000-7f41f829e000 rw-p 00000000 00:00 0
7f41f829e000-7f41f82e1000 r-xp 00000000 08:02 9709725                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/numexpr/interpreter.cpython-35m-x86_64-linux-gnu.so
7f41f82e1000-7f41f84e1000 ---p 00043000 08:02 9709725                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/numexpr/interpreter.cpython-35m-x86_64-linux-gnu.so
7f41f84e1000-7f41f84e2000 rw-p 00043000 08:02 9709725                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/numexpr/interpreter.cpython-35m-x86_64-linux-gnu.so
7f41f84e2000-7f41f8586000 rw-p 00000000 00:00 0
7f41f8586000-7f41f87b7000 r-xp 00000000 08:02 9585884                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/join.cpython-35m-x86_64-linux-gnu.so
7f41f87b7000-7f41f89b6000 ---p 00231000 08:02 9585884                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/join.cpython-35m-x86_64-linux-gnu.so
7f41f89b6000-7f41f89bf000 rw-p 00230000 08:02 9585884                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/join.cpython-35m-x86_64-linux-gnu.so
7f41f89bf000-7f41f89c1000 rw-p 00000000 00:00 0
7f41f89c1000-7f41f8a19000 r-xp 00000000 08:02 9585898                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/index.cpython-35m-x86_64-linux-gnu.so
7f41f8a19000-7f41f8c18000 ---p 00058000 08:02 9585898                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/index.cpython-35m-x86_64-linux-gnu.so
7f41f8c18000-7f41f8c1d000 rw-p 00057000 08:02 9585898                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/index.cpython-35m-x86_64-linux-gnu.so
7f41f8c1d000-7f41f8cdf000 rw-p 00000000 00:00 0
7f41f8cdf000-7f41f8cec000 r-xp 00000000 08:02 9585899                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashing.cpython-35m-x86_64-linux-gnu.so
7f41f8cec000-7f41f8eeb000 ---p 0000d000 08:02 9585899                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashing.cpython-35m-x86_64-linux-gnu.so
7f41f8eeb000-7f41f8eed000 rw-p 0000c000 08:02 9585899                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashing.cpython-35m-x86_64-linux-gnu.so
7f41f8eed000-7f41f8f6d000 rw-p 00000000 00:00 0
7f41f8f6d000-7f41f90c9000 r-xp 00000000 08:02 9585885                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/algos.cpython-35m-x86_64-linux-gnu.so
7f41f90c9000-7f41f92c8000 ---p 0015c000 08:02 9585885                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/algos.cpython-35m-x86_64-linux-gnu.so
7f41f92c8000-7f41f92d4000 rw-p 0015b000 08:02 9585885                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/algos.cpython-35m-x86_64-linux-gnu.so
7f41f92d4000-7f41f9458000 rw-p 00000000 00:00 0
7f41f9458000-7f41f948f000 r-xp 00000000 08:02 1188394                    /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7f41f948f000-7f41f968e000 ---p 00037000 08:02 1188394                    /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7f41f968e000-7f41f968f000 r--p 00036000 08:02 1188394                    /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7f41f968f000-7f41f9690000 rw-p 00037000 08:02 1188394                    /usr/lib/x86_64-linux-gnu/libmpdec.so.2.4.2
7f41f9690000-7f41f96b4000 r-xp 00000000 08:02 1446116                    /usr/lib/python3.5/lib-dynload/_decimal.cpython-35m-x86_64-linux-gnu.so
7f41f96b4000-7f41f98b3000 ---p 00024000 08:02 1446116                    /usr/lib/python3.5/lib-dynload/_decimal.cpython-35m-x86_64-linux-gnu.so
7f41f98b3000-7f41f98b4000 r--p 00023000 08:02 1446116                    /usr/lib/python3.5/lib-dynload/_decimal.cpython-35m-x86_64-linux-gnu.so
7f41f98b4000-7f41f98bd000 rw-p 00024000 08:02 1446116                    /usr/lib/python3.5/lib-dynload/_decimal.cpython-35m-x86_64-linux-gnu.so
7f41f98bd000-7f41f9a19000 r-xp 00000000 08:02 9585897                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/interval.cpython-35m-x86_64-linux-gnu.so
7f41f9a19000-7f41f9c18000 ---p 0015c000 08:02 9585897                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/interval.cpython-35m-x86_64-linux-gnu.so
7f41f9c18000-7f41f9c24000 rw-p 0015b000 08:02 9585897                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/interval.cpython-35m-x86_64-linux-gnu.so
7f41f9c24000-7f41f9c26000 rw-p 00000000 00:00 0
7f41f9c26000-7f41f9d24000 r-xp 00000000 08:02 9585893                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/lib.cpython-35m-x86_64-linux-gnu.so
7f41f9d24000-7f41f9f23000 ---p 000fe000 08:02 9585893                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/lib.cpython-35m-x86_64-linux-gnu.so
7f41f9f23000-7f41f9f31000 rw-p 000fd000 08:02 9585893                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/lib.cpython-35m-x86_64-linux-gnu.so
7f41f9f31000-7f41f9f35000 rw-p 00000000 00:00 0
7f41f9f35000-7f41f9fb3000 r-xp 00000000 08:02 9585892                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashtable.cpython-35m-x86_64-linux-gnu.so
7f41f9fb3000-7f41fa1b2000 ---p 0007e000 08:02 9585892                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashtable.cpython-35m-x86_64-linux-gnu.so
7f41fa1b2000-7f41fa1b9000 rw-p 0007d000 08:02 9585892                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/hashtable.cpython-35m-x86_64-linux-gnu.so
7f41fa1b9000-7f41fa1fa000 rw-p 00000000 00:00 0
7f41fa1fa000-7f41fa381000 r-xp 00000000 08:02 9585890                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/tslib.cpython-35m-x86_64-linux-gnu.so
7f41fa381000-7f41fa580000 ---p 00187000 08:02 9585890                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/tslib.cpython-35m-x86_64-linux-gnu.so
7f41fa580000-7f41fa594000 rw-p 00186000 08:02 9585890                    /home/jerome/.virtualenvs/py_3/lib/python3.5/site-packages/pandas/_libs/tslib.cpython-35m-x86_64-linux-gnu.so
7f41fa594000-7f41fa61a000 rw-p 00000000 00:00 0
7f41fa61a000-7f41fa678000 r-xp 00000000 08:02 14420929                   /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f41fa678000-7f41fa878000 ---p 0005e000 08:02 14420929                   /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f41fa878000-7f41fa87c000 r--p 0005e000 08:02 14420929                   /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f41fa87c000-7f41fa883000 rw-p 00062000 08:02 14420929                   /lib/x86_64-linux-gnu/libssl.so.1.0.0
7f41fa883000-7f41fa89a000 r-xp 00000000 08:02 1442967                    /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so
7f41fa89a000-7f41faa9a000 ---p 00017000 08:02 1442967                    /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so
7f41faa9a000-7f41faa9b000 r--p 00017000 08:02 1442967                    /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so
7f41faa9b000-7f41faaa0000 rw-p 00018000 08:02 1442967                    /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-x86_64-linux-gnu.so
7f41faaa0000-7f41fac20000 rw-p 00000000 00:00 0
7f41fac20000-7f41fac21000 r-xp 00000000 08:02 1442966                    /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so
7f41fac21000-7f41fae20000 ---p 00001000 08:02 1442966                    /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so[1]    4490
 abort (core dumped)  python pd_segfault.py

@jreback
Copy link
Contributor

jreback commented Sep 22, 2017

try a newer version 0.20.3 and master

@jeromedockes
Copy link

same issue with 0.20.3 but not on master. Sorry, I should have checked that first

@jreback
Copy link
Contributor

jreback commented Sep 22, 2017

np @jeromedockes great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug IO CSV read_csv, to_csv
Projects
None yet
Milestone
0.16.1
Development

Successfully merging a pull request may close this issue.

BUG: memory access bug in read_csv causing segfault behzadnouri/pandas
6 participants
@cstenac @jreback @dsm054 @guyrt @tommyengstrom @jeromedockes