BUG:error in reading a boolean column with blanks from excel file #45903 #46374
Conversation
There was a problem hiding this comment.
Thanks for the work on this @dimitra-karadima.
I added couple of comments, in particular, I'd like to understand better your approach. It may be correct, but it's not immediately clear why this is the right fix.
Also, do you think it could make sense to use a new column in test_types.xls (and the other formats) instead of creating the new files?
| @@ -294,6 +294,7 @@ class BooleanArray(BaseMaskedArray): | |||
| # Fill values used for any/all | |||
| _truthy_value = True | |||
| _falsey_value = False | |||
| _NONE_VALUES = {"nan", "NaN", "None", "NA", "null", "NULL"} | |||
There was a problem hiding this comment.
I fail to understand how this is related to having blanks. In the current version of pandas (without the changes in this PR), what's the value of the blank value in the spreadsheet when constructing the extension array? Feels like it should be None or an empty string, not one of those.
There was a problem hiding this comment.
@datapythonista Hello! Thanks for the review! The point here is that I added this list because the isna() fails. The None values are converted into strings so without this list, the "None" values will raise a ValueError. Of course if you have any suggestion on how to implement this better please tell me.
| "boolean_with_blanks" + read_ext, dtype={"var1": "boolean"} | ||
| ) | ||
| expected = DataFrame( | ||
| [[True], [False], [None], [True]], columns=["var1"], dtype="boolean" |
There was a problem hiding this comment.
| [[True], [False], [None], [True]], columns=["var1"], dtype="boolean" | |
| {'var1': [True, False, None, True}, dtype="boolean" |
This seems simpler and easier to read and maintain.
datapythonista
added
Bug
IO Excel
| @@ -420,8 +420,10 @@ def test_reader_special_dtypes(self, request, read_ext): | |||
| datetime(2013, 12, 14), | |||
| datetime(2015, 3, 14), | |||
| ], | |||
| "BoolColWithBlank": [True, False, None, True, False], | |||
There was a problem hiding this comment.
@datapythonista I upated the test_types files as you suggested but for some reason no False or True values are shown. Only 1 or 0 and I cannot find out why.
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
|
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen. |
doc/source/whatsnew/v1.5.0.rstfile if fixing a bug or adding a new feature.