Skip to content

Check for pyarrow not feather before pyarrow tests #31144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 24, 2020

Conversation

rebecca-palmer
Copy link
Contributor

read_feather/to_feather now use pyarrow.feather, not top-level (feather-format) feather, but some of their tests were still looking for top-level feather.

Two of them that deliberately cause a file not found error were also looking for the wrong form of error message. (Probably nobody noticed because the above was skipping them.)

(read_/to_feather can only use pyarrow.feather, not feather-format feather)
@pep8speaks
Copy link

pep8speaks commented Jan 20, 2020

Hello @rebecca-palmer! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-01-21 22:13:53 UTC

@jreback jreback added CI Continuous Integration Dependencies Required and optional dependencies labels Jan 20, 2020
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conceptually fine. we are running older arrows so may have to handle old / new error messages

@jreback jreback added the IO Parquet parquet, feather label Jan 20, 2020
@rebecca-palmer
Copy link
Contributor Author

The build failures were because my system quotes the filename in the error message and some of yours don't; should be fixed now.

@jreback jreback added this to the 1.1 milestone Jan 20, 2020
@jreback
Copy link
Contributor

jreback commented Jan 20, 2020

lgtm. ping on green.

@rebecca-palmer
Copy link
Contributor Author

rebecca-palmer commented Jan 21, 2020

Versions of this error message from the failed CI runs:
"Failed to open local file 'foo/~/does_not_exist.feather', error: 没有那个文件或目录"
"Failed to open local file '/home/vsts/work/1/s/pandas/tests/io/data/does_not_exist.feather', error: 没有那个文件或目录"
"Failed to open local file 'foo/~/does_not_exist.feather', error: The system cannot find the path specified.\r\n"
"Failed to open local file 'D:/a/1/s/pandas/tests/io/data/does_not_exist.feather', error: The system cannot find the file specified.\r\n"
'Failed to open local file: foo/~/does_not_exist.feather , error: File o directory non esistente'
'Failed to open local file: /home/vsts/work/1/s/pandas/tests/io/data/does_not_exist.feather , error: File o directory non esistente'
'Failed to open local file: /Users/runner/runners/2.164.3/work/1/s/pandas/tests/io/data/does_not_exist.feather , error: No such file or directory'
'Failed to open local file: foo/~/does_not_exist.feather , error: No such file or directory'
(My local error, that I first wrote msg8 to accept, was in the first one's format but in English.)

History: This test was added in #16523 (because read_msgpack was returning garbage instead of an exception), initially checking only the exception type. It began checking the error message in #24676

My latest version changes the pattern to "Failed to open local file.+does_not_exist.{fn_ext}.?, error: .*", i.e. gives up on matching the last part of the message.

@rebecca-palmer
Copy link
Contributor Author

green @jreback

@jreback jreback merged commit 35c9eba into pandas-dev:master Jan 24, 2020
@jreback
Copy link
Contributor

jreback commented Jan 24, 2020

thanks @rebecca-palmer very nice

raspbian-autopush pushed a commit to raspbian-packages/pandas that referenced this pull request Aug 31, 2020
pandas now needs pyarrow.feather (not in Debian),
not python3-feather-format, for read_feather/to_feather

Author: Rebecca N. Palmer <[email protected]>
Forwarded: accepted for 1.1 pandas-dev/pandas#31144


Gbp-Pq: Name pyarrow_feather.patch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Dependencies Required and optional dependencies IO Parquet parquet, feather
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants