REGR: ValueError raised when both prefix and names are set to None #42690

lithomas1 · 2021-07-23T19:45:48Z

closes BUG: read_csv raises an error when both prefix and names are set to None #42387
tests added / passed
Ensure all linting tests pass, see here for how to run them
whatsnew entry

…efix-names-raises

phofl

Implementation is good, but we are now inconsistent with sep and delimiter, which still raises if None and not lib.no_default is given.

I think we discussed this a year ago or something like that to use lib.no_default?

jreback · 2021-07-23T23:13:13Z

pandas/io/parsers/readers.py

@@ -505,11 +505,11 @@ def read_csv(
    delimiter=None,
    # Column and Index Locations and Names
    header="infer",
-    names=lib.no_default,
+    names=None,


i would rather leave these as no_default and fix the check itself

We should only have one way of representing none/null, so we should either use None or lib.no_default, not have both. I think I prefer None given that lib.no_default is private.

If we want to keep using lib.no_default then, we should probably close OP as wontfix.

no i think you can simply revert these back to lib.no_default and fix the check on L1305. the point is that these are still invalid if actual None is passed.

Not sure iiuc, but pd.read_csv(xxx, names=None, prefix=None) should fail?

I think it should work. It worked at least in previous versions. If None is passed for both, it would just get the column names from header, or use the "0,1,2,..." naming scheme, right?

right i agree this should work, but i also think you can leave the no_default and it will work (just odify the comparison)

OK, thx for clarifying, update incoming soon.

jreback · 2021-07-23T23:13:18Z

pandas/io/parsers/readers.py

@@ -1302,11 +1302,11 @@ def _refine_defaults_read(
    if delimiter and (sep is not lib.no_default):
        raise ValueError("Specified a sep and a delimiter; you can only specify one.")

-    if names is not lib.no_default and prefix is not lib.no_default:
+    if names is not None and prefix is not None:


pep8speaks · 2021-07-29T21:06:42Z

Hello @lithomas1! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-07-29 23:28:33 UTC

phofl · 2021-07-29T21:29:18Z

pandas/io/parsers/readers.py

@@ -1302,7 +1302,8 @@ def _refine_defaults_read(
    if delimiter and (sep is not lib.no_default):
        raise ValueError("Specified a sep and a delimiter; you can only specify one.")

-    if names is not lib.no_default and prefix is not lib.no_default:
+    allowed_names_prefix = {None, lib.no_default}
+    if names not in allowed_names_prefix and prefix not in allowed_names_prefix:


This won't work with lists

jreback · 2021-07-29T23:28:40Z

lgtm. @phofl if you can give a once over.

phofl

lgtm

…x and names are set to None

…s are set to None (#42811) Co-authored-by: Thomas Li <[email protected]>

…andas-dev#42690) * REGR: ValueError raised when both prefix and names are set to None * Update readers.py * whitespace * Update v1.3.1.rst * Update v1.3.2.rst * Update readers.py * Update readers.py Co-authored-by: Jeff Reback <[email protected]>

* TST: Fix doctests for pandas.io.formats.style * Modified: pandas/io/formats/style.py * Added some expected results * Skipped some tests * TST: Add link to redirect to Table Visualization user guide * Modified style.py * Updated the doctest of the apply() * Updated the doctest of the applymap() * Updated the doctest of the set_table_styles() * Updated the doctest of the set_properties() * TST: Add image to pipe function result * Modified style.py * Updated the doctest of the pipe() * TST: Remove unnecessary outputs * Modified pandas/io/formats/style.py * Updated the doctests of the set_tooltips() * Updated the doctests of the to_latex() * Updated the doctests of the set_td_classes() * Updated the doctests of the set_table_attributes() * TST: Add the output to the Styler.format doctest in to_latex() * REG: DataFrame.agg where func returns lists and axis=1 (#42762) * Fix typing issues for CI (#42770) * BUG: groupby.shift returns different columns when fill_value is specified (#41858) * PERF: extract_array earlier in DataFrame construction (#42774) * ENH: `sparse_columns` and `sparse_index` added to `Styler.to_html` (#41946) * TYP: Fix typing for searchsorted (#42788) * DOC GH42756 Update documentation for pandas.DataFrame.drop to clarify tuples. (#42789) * CI: Fix doctests (#42790) * REGR: nanosecond timestamp comparisons to OOB datetimes (#42796) * COMPAT: MPL 3.4.0 (#42803) * Delete duplicates and unused code from reshape tests (#42802) * REGR: ValueError raised when both prefix and names are set to None (#42690) * REGR: ValueError raised when both prefix and names are set to None * Update readers.py * whitespace * Update v1.3.1.rst * Update v1.3.2.rst * Update readers.py * Update readers.py Co-authored-by: Jeff Reback <[email protected]> * TST: Add style.py to the doctest check * TST: fixed eng_formatter doctest for #42671 (#42705) * TST: Revert x and y position in some doctests * Updated the doctest of the hide_columns() Co-authored-by: Richard Shadrach <[email protected]> Co-authored-by: Irv Lustig <[email protected]> Co-authored-by: Thomas Smith <[email protected]> Co-authored-by: jbrockmendel <[email protected]> Co-authored-by: attack68 <[email protected]> Co-authored-by: Mike Phung <[email protected]> Co-authored-by: Matthew Zeitlin <[email protected]> Co-authored-by: Thomas Li <[email protected]> Co-authored-by: Patrick Hoefler <[email protected]> Co-authored-by: Jeff Reback <[email protected]> Co-authored-by: Krishna Chivukula <[email protected]>

…andas-dev#42690) * REGR: ValueError raised when both prefix and names are set to None * Update readers.py * whitespace * Update v1.3.1.rst * Update v1.3.2.rst * Update readers.py * Update readers.py Co-authored-by: Jeff Reback <[email protected]>

* TST: Fix doctests for pandas.io.formats.style * Modified: pandas/io/formats/style.py * Added some expected results * Skipped some tests * TST: Add link to redirect to Table Visualization user guide * Modified style.py * Updated the doctest of the apply() * Updated the doctest of the applymap() * Updated the doctest of the set_table_styles() * Updated the doctest of the set_properties() * TST: Add image to pipe function result * Modified style.py * Updated the doctest of the pipe() * TST: Remove unnecessary outputs * Modified pandas/io/formats/style.py * Updated the doctests of the set_tooltips() * Updated the doctests of the to_latex() * Updated the doctests of the set_td_classes() * Updated the doctests of the set_table_attributes() * TST: Add the output to the Styler.format doctest in to_latex() * REG: DataFrame.agg where func returns lists and axis=1 (pandas-dev#42762) * Fix typing issues for CI (pandas-dev#42770) * BUG: groupby.shift returns different columns when fill_value is specified (pandas-dev#41858) * PERF: extract_array earlier in DataFrame construction (pandas-dev#42774) * ENH: `sparse_columns` and `sparse_index` added to `Styler.to_html` (pandas-dev#41946) * TYP: Fix typing for searchsorted (pandas-dev#42788) * DOC GH42756 Update documentation for pandas.DataFrame.drop to clarify tuples. (pandas-dev#42789) * CI: Fix doctests (pandas-dev#42790) * REGR: nanosecond timestamp comparisons to OOB datetimes (pandas-dev#42796) * COMPAT: MPL 3.4.0 (pandas-dev#42803) * Delete duplicates and unused code from reshape tests (pandas-dev#42802) * REGR: ValueError raised when both prefix and names are set to None (pandas-dev#42690) * REGR: ValueError raised when both prefix and names are set to None * Update readers.py * whitespace * Update v1.3.1.rst * Update v1.3.2.rst * Update readers.py * Update readers.py Co-authored-by: Jeff Reback <[email protected]> * TST: Add style.py to the doctest check * TST: fixed eng_formatter doctest for pandas-dev#42671 (pandas-dev#42705) * TST: Revert x and y position in some doctests * Updated the doctest of the hide_columns() Co-authored-by: Richard Shadrach <[email protected]> Co-authored-by: Irv Lustig <[email protected]> Co-authored-by: Thomas Smith <[email protected]> Co-authored-by: jbrockmendel <[email protected]> Co-authored-by: attack68 <[email protected]> Co-authored-by: Mike Phung <[email protected]> Co-authored-by: Matthew Zeitlin <[email protected]> Co-authored-by: Thomas Li <[email protected]> Co-authored-by: Patrick Hoefler <[email protected]> Co-authored-by: Jeff Reback <[email protected]> Co-authored-by: Krishna Chivukula <[email protected]>

lithomas1 added 2 commits July 23, 2021 12:42

REGR: ValueError raised when both prefix and names are set to None

e0553ab

Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…

f5cc78b

…efix-names-raises

lithomas1 changed the title ~~Prefix names raises~~ REGR: ValueError raised when both prefix and names are set to None Jul 23, 2021

lithomas1 added IO CSV read_csv, to_csv Regression Functionality that used to work in a prior pandas version labels Jul 23, 2021

lithomas1 added this to the 1.3.1 milestone Jul 23, 2021

lithomas1 requested review from phofl, jreback and simonjayhawkins July 23, 2021 19:46

phofl reviewed Jul 23, 2021

View reviewed changes

jreback requested changes Jul 23, 2021

View reviewed changes

simonjayhawkins mentioned this pull request Jul 24, 2021

RLS: 1.3.1 #42343

Closed

lithomas1 removed this from the 1.3.1 milestone Jul 24, 2021

Merge branch 'master' into prefix-names-raises

8dc6abb

simonjayhawkins added this to the 1.3.2 milestone Jul 26, 2021

Update readers.py

f12b311

lithomas1 added 4 commits July 29, 2021 14:07

whitespace

e3c10e1

Update v1.3.1.rst

75d7083

Merge branch 'pandas-dev:master' into prefix-names-raises

56fc7dc

Update v1.3.2.rst

2729d71

phofl reviewed Jul 29, 2021

View reviewed changes

lithomas1 added 2 commits July 29, 2021 14:54

Update readers.py

5006779

Update readers.py

be486b8

jreback approved these changes Jul 29, 2021

View reviewed changes

Merge branch 'master' into prefix-names-raises

9e64088

phofl approved these changes Jul 30, 2021

View reviewed changes

phofl merged commit 1269bc6 into pandas-dev:master Jul 30, 2021

meeseeksmachine mentioned this pull request Jul 30, 2021

Backport PR #42690 on branch 1.3.x (REGR: ValueError raised when both prefix and names are set to None) #42811

Merged

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Jul 30, 2021

Backport PR pandas-dev#42690: REGR: ValueError raised when both prefi…

4f02b43

…x and names are set to None

KrishnaSai2020 mentioned this pull request Jul 30, 2021

BUG: Posix / pytest (actions-39-numpydev.yaml, ...) checks failing on master #42814

Closed

3 tasks

simonjayhawkins pushed a commit that referenced this pull request Jul 30, 2021

Backport PR #42690: REGR: ValueError raised when both prefix and name…

cc98407

…s are set to None (#42811) Co-authored-by: Thomas Li <[email protected]>

lithomas1 deleted the prefix-names-raises branch July 30, 2021 14:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REGR: ValueError raised when both prefix and names are set to None #42690

REGR: ValueError raised when both prefix and names are set to None #42690

lithomas1 commented Jul 23, 2021

phofl left a comment

jreback Jul 23, 2021

lithomas1 Jul 24, 2021

jreback Jul 25, 2021

phofl Jul 25, 2021

lithomas1 Jul 25, 2021 •

edited

Loading

jreback Jul 27, 2021

lithomas1 Jul 27, 2021

jreback Jul 23, 2021

pep8speaks commented Jul 29, 2021 •

edited

Loading

phofl Jul 29, 2021

jreback commented Jul 29, 2021

phofl left a comment

REGR: ValueError raised when both prefix and names are set to None #42690

REGR: ValueError raised when both prefix and names are set to None #42690

Conversation

lithomas1 commented Jul 23, 2021

phofl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lithomas1 Jul 25, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pep8speaks commented Jul 29, 2021 • edited Loading

Comment last updated at 2021-07-29 23:28:33 UTC

Choose a reason for hiding this comment

jreback commented Jul 29, 2021

phofl left a comment

Choose a reason for hiding this comment

lithomas1 Jul 25, 2021 •

edited

Loading

pep8speaks commented Jul 29, 2021 •

edited

Loading