Skip to content

BUG: parase_dates column is in dataframe (#31251) #31550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 111 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
cc64192
BUG: parase_dates column is in dataframe (#31251)
sathyz Feb 1, 2020
4808c16
REF: collect MultiIndex indexing, set methods (#31553)
jbrockmendel Feb 1, 2020
07eda6a
REF: do all convert_tolerance casting inside Index.get_loc (#31425)
jbrockmendel Feb 1, 2020
d4d1e32
REF: move _convert_to_indexer to Loc (#31554)
jbrockmendel Feb 1, 2020
5b0fefa
BUG&TST: df.replace fail after converting to new dtype (#31545)
charlesdong1991 Feb 1, 2020
149c772
TST: added test to validate (#31340) (#31483)
prakhar987 Feb 1, 2020
cc1c406
CLN: libperiod (#31527)
jbrockmendel Feb 1, 2020
ac61a8a
WEB: Changed whatsnew href from v0.25.0 to v1.0.0 (#31567)
sohamtiwari3120 Feb 2, 2020
36f7e22
CI: Remove warning raising after new matplotlib release (#31573)
charlesdong1991 Feb 2, 2020
119360d
BUG: objToJson.c - fix return value (#31482)
alimcmaster1 Feb 2, 2020
918cc48
Fix docstring for to_markdown buf parameter (#31565)
jakevdp Feb 2, 2020
84c01bc
REF: de-duplicate Period freq conversion code (#31577)
jbrockmendel Feb 2, 2020
b310ed9
REF: organize Series indexing tests (#31557)
jbrockmendel Feb 2, 2020
8dfb549
BUG accept and deprecate negative integer for max_colwidth (#31569)
glemaitre Feb 2, 2020
120e9d9
CLN: Fix ReadMe Badges (#31579)
alimcmaster1 Feb 2, 2020
f0f058f
REF: organize DataFrame tests for indexing, by-method (#31558)
jbrockmendel Feb 2, 2020
d3c59e4
REF: CategoricalIndex indexing tests (#31559)
jbrockmendel Feb 2, 2020
83e8389
unpinned openpyxl (#31525)
WillAyd Feb 2, 2020
9fb1f91
BUG: Timedelta components no longer rounded with high precision integ…
mroeschke Feb 2, 2020
fb8aaa2
implement test_constructors for Timestamp (#31583)
jbrockmendel Feb 2, 2020
83f84ed
BUG: Fix qcut for nullable integers (#31440)
dsaxton Feb 2, 2020
df5572c
BUG: Preserve key order when using loc on MultiIndex DataFrame (#28933)
nrebena Feb 2, 2020
26329fc
REF: define _convert_to_indexer in Loc (#31585)
jbrockmendel Feb 2, 2020
e2ae57c
BUG: to_json not allowing uploads to S3 (#28375) (#31552)
YagoGG Feb 2, 2020
db16a8f
CLN: inherit PeriodIndex._box_func (#31512)
jbrockmendel Feb 2, 2020
000245a
CLN: remove IndexEngine.set_value (#31510)
jbrockmendel Feb 2, 2020
aab9225
Dead JSON Code Cleanup (#31509)
WillAyd Feb 2, 2020
6d9ba2a
Cleaned up names (#31508)
WillAyd Feb 2, 2020
7880cf0
CLN: No need to use libindex.get_value_at (#31506)
jbrockmendel Feb 2, 2020
f6146a5
CLN: named parameters for GroupBy.(mean|median|var|std) (#31485)
topper-123 Feb 2, 2020
f878ddc
PERF/CLN: float-to-int casting (#31409)
jbrockmendel Feb 2, 2020
b678eca
REF: consolidate DTI/TDI/PI get_value in ExtensionIndex, associated c…
jbrockmendel Feb 2, 2020
b7d3d83
TST/CLN: remove setup_method from test_dtypes (#31343)
simonjayhawkins Feb 2, 2020
cf8c680
BUG: preserve object dtype for Index set ops (#31401)
jbrockmendel Feb 2, 2020
27f365d
TST: troubleshoot npdev build (#31594)
jbrockmendel Feb 3, 2020
64336ff
BUG: fix reindexing with a tz-aware index and method='nearest' (#31511)
kandersolar Feb 3, 2020
44782c0
BUG: no longer raise user warning when plotting tz aware time series …
MarcoGorelli Feb 3, 2020
abab307
DOC fix *_option() docstring (#31571)
glemaitre Feb 3, 2020
c815ffa
DOC: add back google analytics with the new doc theme (#31606)
jorisvandenbossche Feb 3, 2020
a9d2450
DOC: combine regressions in section in v1.0.1 whatsnew (#31607)
jorisvandenbossche Feb 3, 2020
a2721fd
Raise error in read_csv when arguments header and prefix both are not…
rushabh-v Feb 3, 2020
14cd612
DOC: add redirect for moved json_normalize docstring (#31622)
jorisvandenbossche Feb 3, 2020
f75f115
REGR: Fixed setitem with MultiIndex (#31631)
TomAugspurger Feb 3, 2020
9853d0e
replacing .format with f-strings (#31628)
drewseibert Feb 3, 2020
4fa44b7
BUG: Period[us] start_time off by 1 nanosecond (#31475)
jbrockmendel Feb 4, 2020
106ed85
[MRG] f-string updates for issue #29547 (#31556)
abbiepopa Feb 4, 2020
06384b7
CLN: Unreachable branch in Loc._getitem_iterable (#31636)
jbrockmendel Feb 4, 2020
feee467
PERF: avoid is_bool_indexer check where possible (#31399)
jbrockmendel Feb 4, 2020
73ea6ca
DOC: Fix typo in Getting Started docs (#31642)
Feb 4, 2020
01582c4
REGR: Categorical with np.str_ categories (#31528)
jbrockmendel Feb 4, 2020
d28db65
BUG: Fixed IntervalArray[int].shift (#31502)
TomAugspurger Feb 4, 2020
3b2b0d3
PERF: Cache MultiIndex.levels (#31651)
TomAugspurger Feb 4, 2020
79633f9
whatsnew (#31665)
TomAugspurger Feb 4, 2020
c3e32d7
REGR: Fix TypeError in groupby min / max of period column (#31477)
dsaxton Feb 4, 2020
1996b17
BUG: Block.iget not wrapping timedelta64/datetime64 (#31666)
jbrockmendel Feb 5, 2020
d3b7e64
REGR: Fixed handling of Categorical in cython ops (#31668)
TomAugspurger Feb 5, 2020
678bfae
CLN: assorted cleanups in indexes/ (#31674)
jbrockmendel Feb 5, 2020
3a4c310
CLN: Remove deprecations (#31675)
topper-123 Feb 5, 2020
69fe3c0
CLN: misc tslibs (#31673)
jbrockmendel Feb 5, 2020
1d5d5a3
REF: make _convert_scalar_indexer require a scalar (#31676)
jbrockmendel Feb 5, 2020
a1e0752
CLN: MultiIndex.get_value is a hive of scum and villainy (#31662)
jbrockmendel Feb 5, 2020
531a430
Replaced .format with f- strings (#31660)
leandermaben Feb 5, 2020
8e47971
check first and last points' labels are correct (#31659)
MarcoGorelli Feb 5, 2020
be9ee6d
BUG: avoid specifying default coerce_timestamps in to_parquet (#31652)
jorisvandenbossche Feb 5, 2020
bacd48b
CLN Replace format in test_repr_info (#31639)
thomasjpfan Feb 5, 2020
f5409cb
REF: call _maybe_cast_indexer upfront, better exception messages (#31…
jbrockmendel Feb 5, 2020
d84f9eb
BUG: Series.xs boxing datetime64 incorrectly (#31630)
jbrockmendel Feb 5, 2020
a89f7fd
Add test for gh 31605 (#31621)
fjetter Feb 5, 2020
2862b3d
CLN: convert_list_indexer is always kind=loc (#31599)
jbrockmendel Feb 5, 2020
9bfdb0d
REF: implement tests/indexes/objects/ (#31597)
jbrockmendel Feb 5, 2020
42065cd
REF: parametrize indexing tests (#31592)
jbrockmendel Feb 5, 2020
33e86bf
REF: simplify DTI._parse_string_to_bounds (#31519)
jbrockmendel Feb 5, 2020
aa47971
REF: _convert_scalar_indexer up-front (#31655)
jbrockmendel Feb 5, 2020
236f7e6
REF: move convert_scalar out of cython (#31672)
jbrockmendel Feb 5, 2020
881d0b7
REGR: fix non-reduction apply with tz-aware objects (#31614)
jorisvandenbossche Feb 5, 2020
c6c86dd
DOC: fixup v1.0.1 whatsnew entries (#31686)
jorisvandenbossche Feb 5, 2020
2f70e41
TST: add test for regression in groupby with empty MultiIndex level (…
jorisvandenbossche Feb 5, 2020
2f9a446
BUG: read_csv used in file like object RawIOBase is not recognize enc…
paihu Feb 5, 2020
d73ded0
DOC: Update 1.0.1 release notes (#31699)
TomAugspurger Feb 5, 2020
2bf618f
REGR: Fixed AssertionError in groupby (#31616)
TomAugspurger Feb 5, 2020
f0b00f8
TYP: remove type:ignore from pandas/io/common.py (#31700)
simonjayhawkins Feb 5, 2020
e2a6f6b
WEB: Link from the website to the docs (#30891)
datapythonista Feb 5, 2020
0b6debf
DOC: fix contributors listing for v1.0.1 (#31704)
jorisvandenbossche Feb 5, 2020
cc4a3e9
DOC: add plotting backends in visualization.rst (#31066)
rushabh-v Feb 5, 2020
daf5bb4
added type annotation to JSONDtype.na_value (#31718)
SaturnFromTitan Feb 5, 2020
935b6f4
DOC Adds newline to dataframe melt (#31712)
thomasjpfan Feb 5, 2020
120f35f
DOC: Add 1.0.2 whatsnew (#31723)
alimcmaster1 Feb 6, 2020
a2a35a8
TST/CLN: dtype test_construct_from_string (#31727)
simonjayhawkins Feb 6, 2020
9ef92ee
Update travis-37.yaml (#31745)
alimcmaster1 Feb 6, 2020
808004a
CLN: inconsistent kwarg name (#31721)
jbrockmendel Feb 6, 2020
31c1856
F string fixes (#31751)
drewseibert Feb 6, 2020
d0c84ce
BUG: Fix to_excel writers handling of cols (#31729)
alimcmaster1 Feb 6, 2020
ecc72f4
CLN: inline indexing 1-liners (#31625)
jbrockmendel Feb 6, 2020
aa720d4
organize DTI indexing tests (#31753)
jbrockmendel Feb 6, 2020
e5c65bf
REF: Index.get_value call self.get_loc instead of self._engine.get_lo…
jbrockmendel Feb 6, 2020
e1ca66b
API/BUG: make .at raise same exceptions as .loc (#31724)
jbrockmendel Feb 6, 2020
8aa2520
REF: Use _maybe_cast_indexer and dont override Categorical.get_loc (#…
jbrockmendel Feb 6, 2020
beb4859
REF: organize MultiIndex indexing tests (#31715)
jbrockmendel Feb 7, 2020
fdba416
CLN: remove libindex.get_value_at (#31680)
jbrockmendel Feb 7, 2020
0d078c5
CLN: _convert_scalar_indexer only handle "loc" and "getitem" (#31709)
jbrockmendel Feb 7, 2020
cf01369
CI: Add --quiet option to isort command (#31733)
d1618033 Feb 7, 2020
4b142ef
TST: Implement external error raised helper function. (#31130)
ShaharNaveh Feb 7, 2020
6d3cc14
CI: fix feather test (#31791)
jbrockmendel Feb 8, 2020
e2b47ce
REF: Move Loc-only methods to Loc (#31589)
jbrockmendel Feb 8, 2020
0bfb8cb
BUG: Fixed encoding of pd.NA with to_json (#31748)
ArtificialQualia Feb 8, 2020
ca84bd0
ERR: improve error message for invalid indexer (#31769)
jorisvandenbossche Feb 8, 2020
9cd1a30
BUG: add parse_dates check in cparser (#31251)
sathyz Feb 8, 2020
8225c8b
BUG: parase_dates column is in dataframe (#31251)
sathyz Feb 1, 2020
63dc85e
BUG: add parse_dates check in cparser (#31251)
sathyz Feb 8, 2020
29bd964
remove colspec check in _process_date_conversion
sathyz Feb 8, 2020
4d2e1cf
Merge remote-tracking branch 'origin/issue-31251' into issue-31251
sathyz Feb 8, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pandas/io/parsers.py
Original file line number Diff line number Diff line change
Expand Up @@ -3278,6 +3278,8 @@ def _isindex(colspec):
if is_scalar(colspec):
if isinstance(colspec, int) and colspec not in data_dict:
colspec = orig_names[colspec]
elif colspec not in orig_names:
raise ValueError(f"'{colspec}' is not in list")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError(f"'{colspec}' is not in list")
raise ValueError(f"Missing column provided to ‘parse_dates’: '{colspec}'")

Not tied to the wording but I don’t think “is not in list” is super clear to the end user, so could improve a bit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had missed to check CParser, handled it now.

if _isindex(colspec):
continue
data_dict[colspec] = converter(data_dict[colspec])
Expand Down
11 changes: 11 additions & 0 deletions pandas/tests/io/parser/test_parse_dates.py
Original file line number Diff line number Diff line change
Expand Up @@ -1516,3 +1516,14 @@ def test_hypothesis_delimited_date(date_format, dayfirst, delimiter, test_dateti

assert except_out_dateutil == except_in_dateutil
assert result == expected


def test_missing_column(all_parsers):
parser = all_parsers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the github issue number as a comment here?

content = StringIO("time,val\n" "212.23, 32\n")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that the CI will fail because, this string is not concatenated, can ou please concatenate them?

date_cols = ["time"]
msg = "'time' is not in list"
with pytest.raises(ValueError, match=msg):
parser.read_csv(
content, sep=",", usecols=["val"], parse_dates=date_cols,
)