Skip to content

TST: Add the default separator test for PythonParser #17822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,8 @@ filepath_or_buffer : various
sep : str, defaults to ``','`` for :func:`read_csv`, ``\t`` for :func:`read_table`
Delimiter to use. If sep is ``None``, the C engine cannot automatically detect
the separator, but the Python parsing engine can, meaning the latter will be
used automatically. In addition, separators longer than 1 character and
used and automatically detect the separator by Python's builtin sniffer tool,
:class:`python:csv.Sniffer`. In addition, separators longer than 1 character and
different from ``'\s+'`` will be interpreted as regular expressions and
will also force the use of the Python parsing engine. Note that regex
delimiters are prone to ignoring quoted data. Regex example: ``'\\r\\t'``.
Expand Down
3 changes: 2 additions & 1 deletion pandas/io/parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,8 @@
_sep_doc = r"""sep : str, default {default}
Delimiter to use. If sep is None, the C engine cannot automatically detect
the separator, but the Python parsing engine can, meaning the latter will
be used automatically. In addition, separators longer than 1 character and
be used and automatically detect the separator by Python's builtin sniffer
tool, ``csv.Sniffer``. In addition, separators longer than 1 character and
different from ``'\s+'`` will be interpreted as regular expressions and
will also force the use of the Python parsing engine. Note that regex
delimiters are prone to ignoring quoted data. Regex example: ``'\r\t'``
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/io/parser/python_parser_only.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@

class PythonParserTests(object):

def test_default_separator(self):
# GH17333
# csv.Sniffer in Python treats 'o' as separator.
text = 'aob\n1o2\n3o4'
expected = DataFrame({'a': [1, 3], 'b': [2, 4]})

result = self.read_csv(StringIO(text), sep=None)

tm.assert_frame_equal(result, expected)

def test_invalid_skipfooter(self):
text = "a\n1\n2"

Expand Down