Skip to content

BUG: read_csv with both names and parse_date raises 'NoneType' TypeError #33699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
quazgar opened this issue Apr 21, 2020 · 1 comment · Fixed by #38431
Closed

BUG: read_csv with both names and parse_date raises 'NoneType' TypeError #33699

quazgar opened this issue Apr 21, 2020 · 1 comment · Fixed by #38431
Labels
Bug IO CSV read_csv, to_csv
Milestone

Comments

@quazgar
Copy link

quazgar commented Apr 21, 2020

This is probably similar to #14792, but still happens with 0.23.3 (a friend even tested it with 1.0.3).

Testcase

import io
s = """A,B,\n1,2"""
pd.read_csv(io.StringIO(s), parse_dates=["B"], names=["B"])

raises:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-146-082b3d7afa0a> in <module>()
      1 import io
      2 s = """A,B,\n1,2"""
----> 3 pd.read_csv(io.StringIO(s), parse_dates=["B"], names=["B"])

/usr/lib/python3/dist-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
    676                     skip_blank_lines=skip_blank_lines)
    677
--> 678         return _read(filepath_or_buffer, kwds)
    679
    680     parser_f.__name__ = name

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    444
    445     try:
--> 446         data = parser.read(nrows)
    447     finally:
    448         parser.close()

/usr/lib/python3/dist-packages/pandas/io/parsers.py in read(self, nrows)
   1034                 raise ValueError('skipfooter not supported for iteration')
   1035
-> 1036         ret = self._engine.read(nrows)
   1037
   1038         # May alter columns / col_dict

/usr/lib/python3/dist-packages/pandas/io/parsers.py in read(self, nrows)
   1887
   1888                 values = self._maybe_parse_dates(values, i,
-> 1889                                                  try_parse_dates=True)
   1890                 arrays.append(values)
   1891

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _maybe_parse_dates(self, values, index, try_parse_dates)
   1946
   1947     def _maybe_parse_dates(self, values, index, try_parse_dates=True):
-> 1948         if try_parse_dates and self._should_parse_dates(index):
   1949             values = self._date_conv(values)
   1950         return values

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _should_parse_dates(self, i)
   1319             else:
   1320                 name = None
-> 1321             j = self.index_col[i]
   1322
   1323             if is_scalar(self.parse_dates):

TypeError: 'NoneType' object is not subscriptable

Notes

This does not happen with engine="python" (you only get a ParseError there).

@quazgar quazgar added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 21, 2020
@jbrockmendel jbrockmendel added IO CSV read_csv, to_csv and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 5, 2020
@siddartha-RE
Copy link

In Pandas 1.0.3 I am seeing the error with the following simple invoication:

df = pd.read_csv('/tmp/csv_no_dates.csv', nrows=500, low_memory=False, parse_dates=[])

Bottom of stack:

  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 676, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 454, in _read
    data = parser.read(nrows)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1133, in read
    ret = self._engine.read(nrows)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 2080, in read
    values = self._maybe_parse_dates(values, i, try_parse_dates=True)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 2141, in _maybe_parse_dates
    if try_parse_dates and self._should_parse_dates(index):
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1448, in _should_parse_dates
    j = self.index_col[i]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants