TST: Add test to ensure DF describe does not throw an error (#32409) #35270
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jreback
requested changes
Jul 14, 2020
| expected = DataFrame( | ||
| {"test": [2, 2, {"a": "1"}, 1]}, index=["count", "unique", "top", "freq"] | ||
| ) | ||
| try: |
There was a problem hiding this comment.
you don't need the try/except if it fails then pytest automatically raises.
There was a problem hiding this comment.
Thank you for the info, I've updated this PR and re-requested a review
|
This is very strange, but I don't know that this test is actually covering the original issue. With pandas 1.0.1 In [6]: >>> df = pd.DataFrame([{"test": {"a": "1"}}, {"test": {"a": "2"}}])
In [7]: r = df.describe()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.map_locations()
TypeError: unhashable type: 'dict'
Exception ignored in: 'pandas._libs.index.IndexEngine._call_map_locations'
Traceback (most recent call last):
File "pandas/_libs/hashtable_class_helper.pxi", line 1652, in pandas._libs.hashtable.PyObjectHashTable.map_locations
TypeError: unhashable type: 'dict'
In [8]: r
Out[8]:
test
count 2
unique 2
top {'a': '1'}
freq 1So the code executed. The @luckyvs1 can you verify that the test you added fails with pandas 1.0.1? You should be able just make a single |
|
Thank you for the feedback @TomAugspurger -- unfortunately I won't be able to look into this till this weekend, but will investigate if the test case written fails in the tagged |
|
@TomAugspurger I just had a chance to verify on the release and the output of the test on my current PR branch which is built off release |
|
something weird going on. On windows with pandas 1.0.1, I get |
|
@simonjayhawkins Using I imported Reproduced example with failed describe in shell with
|
|
I'm not sure how this should be tested (or whether it can be tested). my concern is that the original cause of the issue of not catching exceptions in the cython code is not solved (see #34113 (comment)). I'm not sure how the issue is fixed on master. Maybe I did the git bisect wrong in #32409 (comment). The pr referenced there changed is_scalar and is_iterator, so if that PR did fix the describe issue, it must have changed the behaviour of one of those functions. Either way, I think there is a problem somewhere. |
|
For reference I'm also able to reproduce the issue described in #32409 on my Mac as well using Reproduced example with failed describe in shell with
|
|
If this needs to be fixed in 1.0.1.x then this test or similar can be added to that release and check if the added test passes on Mac, Linux versions supported in addition to Windows. Since the current behaviour seems to be it's working on Windows, but not Mac and Linux (at least the versions I provided). In the versions where the exception is handled while still being outputted like the current Windows 1.0.1, I’m also not sure at the moment on how that can be tested or how to currently test the lack of exception output as seen in latest 1.0.5+ versions. Edit: Maybe a test that writes the output to a file and check if the contents doesn't contain references to |
* TST: Test for issues pandas-dev#26186 and pandas-dev#11465 * BUG: Generate the tick position in BarPlot using convert tools from matlab. Generate the tick position in BarPlot using convert tools from matlab. * TST: Modify tests/plotting/test_frame.test_bar_categorical Ticklocs are now float also for categorical bar data (as they are position on the axis). The test is changed to compare to a array of np.float. * TST: Fix test for windows OS * TST: Add test for plotting MultiIndex bar plot A fix to issue pandas-dev#26186 revealed no tests existed about plotting a bar plot for a MultiIndex, but a section of the user guide visualization did. This section of the user guide is now in the test suite. * BUG: Special case for MultiIndex bar plot * DOC: Add whatsnew entry for PR pandas-dev#28733 * CLN: Clean up in code and doc * CLN: Clean up test_bar_numeric * DOC Move to whatsnew v1.1 * FIX: Make tick dtype int for backwards compatibility * DOC: Improve whatsnew message * ENH: Add UserWarning when plotting bar plot with MultiIndex * CLN: Remove duplicate code line * TST: Capture UserWarning for Bar plot with MultiIndex * TST: Improve test explanation * ENH: Raise UserWarning only if redrawing on existing axis with data * DOC: Move to whatsnew v1.2.9 Co-authored-by: Marco Gorelli <[email protected]>
* BUG: isin incorrectly casting ints to datetimes * GH ref * add asvs
…ev#37984) * BUG: IntervalArray.astype(categorical_dtype) losing ordered * whatsnew
* DOC: add examples to insert and update generally This commit mainly adds examples on usage. This commit also fills in information suggested by `validate_docstrings.py` script. * DOC: remove inferred insert parameter descriptions
Co-authored-by: VirosaLi <2EkF8qUgpNkj>
Co-authored-by: VirosaLi <2EkF8qUgpNkj>
… (pandas-dev#38383) * last dict fixings * last dict fixings * last dict fixings * last dict fixings
Co-authored-by: VirosaLi <2EkF8qUgpNkj>
* Corrected the grammar in Exception Exception on different arrays' length corrected * Updated error message in tests file as well Co-authored-by: Ashwani <[email protected]>
…andas into add-test-for-describe
5 tasks
|
Closing in favor of #38403 |
|
Note for reference: Reason the diff turned out to be the way it is was due to the forked branch not being up to date with the latest, to fix this at this point do the following with the rebase applied can do the following:
From here, we can update the forked master first then rebase off that branch:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diffNotes:
pytest pandas/tests/frame/methods/test_describe.py::TestDataFrameDescribe::test_describe_does_not_raise_errorand tested error format by calling a raise error line in the try blockraise TypeError("Test message")