-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
BUG: Respect errors="ignore" during extension astype #35979
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jreback
merged 17 commits into
pandas-dev:master
from
dsaxton:ignore-errors-for-extension-astype
Sep 6, 2020
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
d234627
BUG: Respect errors="ignore" during extension astype
dsaxton 5bc2ba5
A whole bunch of stuff
dsaxton c84a9c6
Lint
dsaxton 85bd7f2
Don't type?
dsaxton 73d0ee0
Nit
dsaxton dbc3a76
Revert "Nit"
dsaxton fc6538f
fixup
dsaxton 7efb4ba
Revert
dsaxton a0f2f9b
Merge remote-tracking branch 'upstream/master' into ignore-errors-for…
dsaxton e66bde4
Merge remote-tracking branch 'upstream/master' into ignore-errors-for…
dsaxton c16204b
Update test
dsaxton 3813c96
Merge remote-tracking branch 'upstream/master' into ignore-errors-for…
dsaxton de8091b
Update test
dsaxton 234cfe1
Fix
dsaxton d07b565
Merge remote-tracking branch 'upstream/master' into ignore-errors-for…
dsaxton 88635de
Nit
dsaxton 61b6eb8
Fix
dsaxton File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,6 +8,7 @@ | |
CategoricalDtype, | ||
DataFrame, | ||
DatetimeTZDtype, | ||
Interval, | ||
IntervalDtype, | ||
NaT, | ||
Series, | ||
|
@@ -565,3 +566,24 @@ def test_astype_empty_dtype_dict(self): | |
result = df.astype(dict()) | ||
tm.assert_frame_equal(result, df) | ||
assert result is not df | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do we have tests for these same types for errors='raise' (the default) and errors='coerce'? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think so for all, can add those here |
||
@pytest.mark.parametrize( | ||
"df", | ||
[ | ||
DataFrame(Series(["x", "y", "z"], dtype="string")), | ||
DataFrame(Series(["x", "y", "z"], dtype="category")), | ||
DataFrame(Series(3 * [Timestamp("2020-01-01", tz="UTC")])), | ||
DataFrame(Series(3 * [Interval(0, 1)])), | ||
], | ||
) | ||
@pytest.mark.parametrize("errors", ["raise", "ignore"]) | ||
def test_astype_ignores_errors_for_extension_dtypes(self, df, errors): | ||
# https://github.com/pandas-dev/pandas/issues/35471 | ||
if errors == "ignore": | ||
expected = df | ||
result = df.astype(float, errors=errors) | ||
tm.assert_frame_equal(result, expected) | ||
else: | ||
msg = "(Cannot cast)|(could not convert)" | ||
with pytest.raises((ValueError, TypeError), match=msg): | ||
df.astype(float, errors=errors) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of this
add in this keyword to the EA astype
it's a little more code (as likely you need to refactor into an _astype method
but better i think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can add this, although it becomes a lot more code. I was able to pull a tiny bit into a helper function, but mostly each method seems to have its own idiosyncrasies that makes it hard to abstract away details. This became quite messy so let me know if I should revert and maybe leave as a follow-up enhancement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, Changing the signature on the base extension array is an api change. I'm not so sure if this is a good idea since this only fixes pandas EAs and third party EAs that have overridden astype will need to update their code to support this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems too large of a change for a bug fix, went ahead and reverted