Problem detecting separator (sep=None) with a DataFrame with a single column

[Leobouloc]

Code Sample

from io import StringIO

import pandas as pd

TESTDATA=StringIO("""col1
    val1
    val2d
    val3
    val4
    """)

df = pd.read_csv(TESTDATA, sep=None, engine='python')
print(df)

returns:

           c  l1
0       val1 NaN
1      val2d NaN
2       val3 NaN
3       val4 NaN

Problem

I would expect the input to be detected as a DataFrame with one column (with no seperator) which is not the case here...

Expected Output

        col1
0       val1
1      val2d
2       val3
3       val4

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-92-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.3
pytest: None
pip: 9.0.1
setuptools: 35.0.2
Cython: 0.23.4
numpy: 1.13.1
scipy: 0.19.0
xarray: None
IPython: 6.0.0
sphinx: 1.5.5
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.4.3
feather: None
matplotlib: 1.5.1
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: 0.7.3
lxml: None
bs4: None
html5lib: 0.999999999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

[gfyoung]

@Leobouloc : Thanks for reporting this! A little odd, especially since you somehow drop an "o" from col1. As a workaround, you should pass in delim_whitespace=True, and you should get the table that you expect. I can't tell at the moment whether this error is stemming from Python's native csv parsing or our own wrapper around it.

Investigation (and PR) are welcome!

[Licht-T]

@gfyoung This seems right behavior.
When sep=None, sniff the first line and check what the separator is.
https://github.com/pandas-dev/pandas/blob/master/pandas/io/parsers.py#L2167

The csv library treats 'o' as separator.
https://docs.python.org/2/library/csv.html#csv.Sniffer

Here is the example.

Input

import csv

sniffed = csv.Sniffer().sniff('col1')
sniffed.delimiter

Output

'o'

[gfyoung]

@Licht-T : Ah, interesting...this issue is beyond our control then since the sniffing behavior is on Python. Do you mind adding this example as a test to the Python engine?

@Leobouloc : We will close this issue after the test is added because the behavior causing this output is outside of our control. Feel free to ping the Python devs about this if you think this is sufficiently bothersome to address.

[gfyoung]

As this issue can be resolved by simply adding a test, I would want to add this addition to the release. If not resolved before 0.21.0, ping, and we'll get it added in time.

[Licht-T]

@gfyoung Okay! I will write this test asap.