BUG: index_col=False doesn't work for unequal length of data

[Ynjxsjmh]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
from io import StringIO

TESTDATA = StringIO("""
0.5 0.03
0.1  0.2  0.3   2
0.2  0.1     0.1  0.3
0.5 0.03
0.1  0.2   0.3      2
    """)

df = pd.read_csv(TESTDATA, sep=' +', header=None, index_col=False, engine='python')

Issue Description

Current output

print(df)

            0    1
0.5 0.03
0.1 0.20  0.3  2.0
0.2 0.10  0.1  0.3
0.5 0.03  NaN  NaN
0.1 0.20  0.3  2.0

Expected Behavior

Expected output

  0    1    2    3
0.5 0.03  NaN  NaN
0.1 0.20  0.3  2.0
0.2 0.10  0.1  0.3
0.5 0.03  NaN  NaN
0.1 0.20  0.3  2.0

Installed Versions

>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : 4bfe3d07b4858144c219b9346329027024102ab6
python           : 3.10.4.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.17.5-arch1-1
Version          : #1 SMP PREEMPT Wed, 27 Apr 2022 20:56:11 +0000
machine          : x86_64
processor        :
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.4.2
numpy            : 1.21.5
pytz             : 2021.3
dateutil         : 2.8.2
pip              : 21.0
setuptools       : 59.5.0
Cython           : None
pytest           : 6.2.5
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.7.1
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : 3.0.3
IPython          : None
pandas_datareader: 0.10.0
bs4              : 4.10.0
bottleneck       : None
brotli           : None
fastparquet      : None
fsspec           : None
gcsfs            : None
markupsafe       : 2.0.1
matplotlib       : 3.5.1
numba            : None
numexpr          : None
odfpy            : None
openpyxl         : 3.0.9
pandas_gbq       : None
pyarrow          : 7.0.0
pyreadstat       : None
pyxlsb           : None
s3fs             : None
scipy            : 1.8.0
snappy           : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
zstandard        : None

[simonjayhawkins]

Thanks @Ynjxsjmh for the report.

In pandas 1.3.5 this was giving a more appropriate result with a warning about loss of data.

/home/simon/miniconda3/envs/pandas-1.3.5/lib/python3.10/site-packages/pandas/io/parsers/readers.py:586: ParserWarning: Length of header or names does not match length of data. This leads to a loss of data with index_col=False.
  return _read(filepath_or_buffer, kwds)
     0     1
0  0.5  0.03
1  0.1  0.20
2  0.2  0.10
3  0.5  0.03
4  0.1  0.20

labelling as a regression pending further investigation.

[simonjayhawkins]

In pandas 1.3.5 this was giving a more appropriate result with a warning about loss of data.

first bad commit: [94c5ce3] BUG: read_csv not recognizing bad lines with names given (#44646)

cc @phofl

[phofl]

This is a super edge case, thanks for reporting @Ynjxsjmh