BUG: read_csv(index_col=False) -> auto-generated column headers for variable row length get headers of type int

[arekkulczycki]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
from io import StringIO

csv = "1,2\n1,2,3"
io = StringIO(csv)

usecols = lambda header: print(type(header))
pd.read_csv(io, index_col=False, usecols=usecols)

# (..)
# <class 'int'>

Issue Description

When using index_col=False kwarg for read_csv then the number of columns read in is equal to the length of the longest row.

In such scenario, the columns which miss headers get names based on their position, but are assigned integers instead of strings. I think headers should rather be strings in general, but specifically this became a problem when I also used:
usecols=lambda header: header.strip() in headers
and it raised AttributeError.

Expected Behavior

The auto-generated column name/header is a string.

Installed Versions

INSTALLED VERSIONS

commit : 4bfe3d0
python : 3.8.10.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-89-generic
Version : #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : pl_PL.UTF-8
LOCALE : pl_PL.UTF-8

pandas : 1.4.2
numpy : 1.22.3
pytz : 2022.1
dateutil : 2.8.2
pip : 20.0.2
setuptools : 45.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 8.3.0
pandas_datareader: None
bs4 : 4.8.2
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
markupsafe : 1.1.0
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None

[kickthevidit]

take

[simonjayhawkins]

The auto-generated column name/header is a string.

This would be an api-breaking change? I think that this could be quite a disruptive change?

[phofl]

Could you provide a reproducible example, where it shows that the columns are actually added?

csv = "1,2\n1,2,3"
io = StringIO(csv)
pd.read_csv(io, index_col=False)

This raises a warning for me and returns 2 columns

usecols is not meant to format your columns, it is meant to filter them

[arekkulczycki]

Could you provide a reproducible example, where it shows that the columns are actually added?

csv = "a,b\na,b,c"
io = StringIO(csv)
usecols=lambda header: header.strip() in ["a", "b", "c"]
pd.read_csv(io, index_col=False, usecols=usecols)

This raises AttributeError: 'int' object has no attribute 'strip'

edit: changed numbers to chars in csv to not confuse where we get the int from...

[phofl]

Ah got you, thanks.

[phofl]

You are correct, these should be strings, since we interpret all columns as strings anyway