Skip to content

BUG: read_csv used in file like object RawIOBase is not recognize encoding option #31596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 5, 2020

Conversation

paihu
Copy link
Contributor

@paihu paihu commented Feb 3, 2020

@@ -109,6 +109,7 @@ Datetimelike
- :meth:`DatetimeArray.searchsorted`, :meth:`TimedeltaArray.searchsorted`, :meth:`PeriodArray.searchsorted` not recognizing non-pandas scalars and incorrectly raising ``ValueError`` instead of ``TypeError`` (:issue:`30950`)
- Bug in :class:`Timestamp` where constructing :class:`Timestamp` with dateutil timezone less than 128 nanoseconds before daylight saving time switch from winter to summer would result in nonexistent time (:issue:`31043`)
- Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` when reindexing with a tz-aware index (:issue:`26683`)
- Bug in :meth: `read_csv` used in file like object `RawIOBase` is not recognize `encoding` option (:issue:`31575`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this render correctly? (see building the documentation)

I think you might need

:meth:`read_csv`

(without the blank space)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry. And change log position is not correct.

@TomAugspurger
Copy link
Contributor

Is this going in 1.1 or 1.0.1? @jorisvandenbossche added the issue to 1.0.1, and if it's fixing a regression it should go there.

If so, you'll need to move the release note @paihu.

@jreback jreback added Compat pandas objects compatability with Numpy or Python functions IO CSV read_csv, to_csv labels Feb 5, 2020
@jreback jreback modified the milestones: 1.1, 1.0.1 Feb 5, 2020
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. pls move the note; ping on green.

@@ -178,6 +178,7 @@ I/O
^^^
- Bug in :meth:`read_json` where integer overflow was occuring when json contains big number strings. (:issue:`30320`)
- Bug in :meth:`DataFrame.to_json` was raising ``NotFoundError`` when ``path_or_buf`` was an S3 URI (:issue:`28375`)
- Bug in :meth:`read_csv` used in file like object ``RawIOBase`` is not recognize ``encoding`` option (:issue:`31575`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can move to 1.0.1 (same section)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it.

@jreback jreback changed the title BUG: read_csv used in file like object RawIOBase is not recognize encoding option BUG: read_csv used in file like object RawIOBase is not recognize encoding option Feb 5, 2020
@jreback
Copy link
Contributor

jreback commented Feb 5, 2020

@paihu make sure to merge master you have a conflict

@paihu
Copy link
Contributor Author

paihu commented Feb 5, 2020

I think this issue is regression bug. so I insert Fix regressions section.

Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rather check for isinstance(.., io.IOBase) instead of checking both BufferedIOBase and RawIOBase?

IOBase additionally is also the base class of TextIOBase

cc @gfyoung

@paihu
Copy link
Contributor Author

paihu commented Feb 5, 2020

We don't want to wrap text(TextIOBase). We need to wrap only binary(BufferedIOBase and RawIOBase).

@jorisvandenbossche
Copy link
Member

Because in that case (eg something like a StringIO object), it is assumed to be already encoded UTF string, so encoding shouldn't be applied then?

@TomAugspurger TomAugspurger merged commit 2f9a446 into pandas-dev:master Feb 5, 2020
@TomAugspurger
Copy link
Contributor

Thanks @paihu!

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Feb 5, 2020
simonjayhawkins pushed a commit that referenced this pull request Feb 5, 2020
…is not recognize encoding option (#31698)

Co-authored-by: paihu <[email protected]>
@gfyoung
Copy link
Member

gfyoung commented Feb 5, 2020

Thanks @paihu!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pandas 1.0.0 read_csv() is broken use open( buffering=0) option.
6 participants