Skip to content

replace KeyError with LookupError in the types of possible read_hdf errors #52783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 19, 2023

Conversation

graingert
Copy link
Contributor

@graingert graingert commented Apr 19, 2023

@graingert graingert changed the title add IndexError to the types os possible read_hdf errors add IndexError to the types of possible read_hdf errors Apr 19, 2023
@@ -455,7 +455,7 @@ def read_hdf(
chunksize=chunksize,
auto_close=auto_close,
)
except (ValueError, TypeError, KeyError):
except (ValueError, TypeError, KeyError, IndexError):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably this should just be

Suggested change
except (ValueError, TypeError, KeyError, IndexError):
except BaseException:

as when using with HDFStore(...): it will still close on all exceptions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense (I'm not sure if we have a linting rule that may complain about BaseException but worth a try

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks could you add a test and whatsnew entry for this (v2.1.0.rst)

@mroeschke mroeschke added the IO HDF5 read_hdf, HDFStore label Apr 19, 2023
@graingert
Copy link
Contributor Author

yep I'm just working on a test now

@graingert
Copy link
Contributor Author

@mroeschke what do you think about using BaseException instead?

@graingert graingert changed the title add IndexError to the types of possible read_hdf errors replace KeyError with LookupError in the types of possible read_hdf errors Apr 19, 2023
Comment on lines +54 to +56
# smoke test to test that file is properly closed after
# read with IndexError before another write
df.to_hdf(path, "k1")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# smoke test to test that file is properly closed after
# read with IndexError before another write
df.to_hdf(path, "k1")
# smoke test to test that file is properly closed after
# read with IndexError before another write
with tm.assert_produces_warning(None):
df.to_hdf(path, "k1")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would this produce a warning? If the file is already open an exception is raised regardless of warning filter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change would need to be in test_read_missing_key_opened_store also

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just to help assert something additional with this call (that no warning is raised)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's correct to do that here

@mroeschke mroeschke added this to the 2.1 milestone Apr 19, 2023
@mroeschke mroeschke merged commit e0e1384 into pandas-dev:main Apr 19, 2023
@mroeschke
Copy link
Member

Thanks @graingert

@graingert graingert deleted the patch-2 branch April 19, 2023 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO HDF5 read_hdf, HDFStore
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: pd.read_hdf leaves files open if it raises an IndexError exception
2 participants