BUG: read_csv - comment argument does not behave as expected

[marek-caltik]

Code Sample, a copy-pastable example

Giving the sample file:

#empty
a,b,c
1,2,3
2,2,3
3,3,3

I want to skip text line starting with character '3':

pd.read_csv(filepath, sep=',', comment='3', header=1)

Problem description

In the resulting dataframe the line starting with char '3' is skipped, but also other occurrences of '3' are replaced by nan's.

  a b c
0 1 2 nan
1 2 2 nan

Expected Output

I expect only record starting with the comment char is missing.

  a b c
0 1 2 3
1 2 2 3

Output of pd.show_versions()

INSTALLED VERSIONS

commit : f2c8480
python : 3.8.3.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-136-generic
Version : #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.3
numpy : 1.19.0
pytz : 2020.5
dateutil : 2.8.1
pip : 21.0.1
setuptools : 53.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.6 (dt dec pq3 ext lo64)
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.4
numexpr : None
odfpy : None
openpyxl : 3.0.6
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : 1.3.23
tables : None
tabulate : None
xarray : None
xlrd : 2.0.1
xlwt : None
numba : 0.50.0

[asishm-wk]

This is documented behavior -

comment: str, optional
Indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether.

[marek-caltik]

OK thanks, I see now.