TST: assert_produces_warning works with filterwarnings #25721

TomAugspurger · 2019-03-14T02:51:43Z

Previously, assert_produces_warning did not play well with pytest's
filterwarnings.

===================================================================================== FAILURES ======================================================================================
____________________________________________________________________ test_assert_produces_warning_honors_filter _____________________________________________________________________

    @pytest.mark.filterwarnings('ignore:f1:FutureWarning')
    def test_assert_produces_warning_honors_filter():
        with tm.assert_produces_warning(RuntimeWarning):
>           f()

pandas/tests/util/test_assert_produces_warning.py:17:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <contextlib._GeneratorContextManager object at 0x11dd61128>, type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('FutureWarning', FutureWarning('f1'), '/Users/taugspurger/sandbox/pandas/pandas/tests/util/test_assert_produces_warni
ng.py', 10)].

/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py:119: AssertionError
============================================================================= 1 failed in 0.20 seconds ==============================================================================

Internally, assert_produces_warning sets a new filter, which
overrides any filters set by pytest. We allow for filter_level=None to
not set a filter.

This will be useful for the SparseDataFrame deprecation, where I'd like to just set a pytest.mark.filterwarning on the class, and not worry about catching individual warnings.

Previously, assert_produces_warning did not play well with pytest's filterwarnings. ``` ===================================================================================== FAILURES ====================================================================================== ____________________________________________________________________ test_assert_produces_warning_honors_filter _____________________________________________________________________ @pytest.mark.filterwarnings('ignore:f1:FutureWarning') def test_assert_produces_warning_honors_filter(): with tm.assert_produces_warning(RuntimeWarning): > f() pandas/tests/util/test_assert_produces_warning.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <contextlib._GeneratorContextManager object at 0x11dd61128>, type = None, value = None, traceback = None def __exit__(self, type, value, traceback): if type is None: try: > next(self.gen) E AssertionError: Caused unexpected warning(s): [('FutureWarning', FutureWarning('f1'), '/Users/taugspurger/sandbox/pandas/pandas/tests/util/test_assert_produces_warni ng.py', 10)]. /usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py:119: AssertionError ============================================================================= 1 failed in 0.20 seconds ==============================================================================``` Internally, `assert_produces_warning` sets a new `filter`, which overrides any filters set by pytest. We allow for `filter_level=None` to not set a filter.

codecov · 2019-03-14T03:28:35Z

Codecov Report

Merging #25721 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #25721      +/-   ##
==========================================
+ Coverage   91.24%   91.24%   +<.01%     
==========================================
  Files         172      172              
  Lines       52967    52968       +1     
==========================================
+ Hits        48331    48332       +1     
  Misses       4636     4636

Flag	Coverage Δ
#multiple	`89.82% <100%> (ø)`	⬆️
#single	`41.75% <100%> (ø)`	⬆️

Impacted Files	Coverage Δ
pandas/util/testing.py	`88.99% <100%> (+0.01%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79205ea...f0ad708. Read the comment docs.

codecov · 2019-03-14T03:28:35Z

Codecov Report

Merging #25721 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #25721      +/-   ##
==========================================
+ Coverage   91.24%   91.25%   +<.01%     
==========================================
  Files         172      172              
  Lines       52967    52973       +6     
==========================================
+ Hits        48331    48339       +8     
+ Misses       4636     4634       -2

Flag	Coverage Δ
#multiple	`89.82% <100%> (ø)`	⬆️
#single	`41.74% <100%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/util/testing.py	`89.18% <100%> (+0.19%)`	⬆️
pandas/core/indexes/range.py	`97.41% <0%> (+0.04%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79205ea...3e01ab4. Read the comment docs.

jreback · 2019-03-14T12:37:03Z

pandas/tests/util/test_assert_produces_warning.py

+
+@pytest.mark.filterwarnings('ignore:f1:FutureWarning')
+def test_assert_produces_warning_message():
+    with tm.assert_produces_warning(FutureWarning, message='f2'):


can you add a test for tm.assert_produces_warning(None)

Asserting that no warning emitted?

jreback · 2019-03-14T12:37:56Z

pandas/util/testing.py

@@ -2608,6 +2610,9 @@ class for all warnings. To check that no warning is returned,
        If True, displays the line that called the function containing
        the warning to show were the function is called. Otherwise, the
        line that implements the function is displayed.
+    message : str, default ''


how would one use message?

It ends up being hard to use effectively, because unhandled warnings are cause us to raise immediately. Just removing for now, and we can re-add later if needed.

jorisvandenbossche · 2019-03-14T19:53:08Z

pandas/tests/util/test_assert_produces_warning.py

+@pytest.mark.filterwarnings('ignore:f1:FutureWarning')
+@pytest.mark.filterwarnings('ignore:f2:RuntimeWarning')
+def test_assert_produces_warning_honors_filter():
+    with tm.assert_produces_warning(RuntimeWarning):


Shouldn't you be testing the new option here?

Yes... I've messed something up, sorry.

TomAugspurger · 2019-03-14T20:20:32Z

Hmm this may require a slightly different approach.

As a test-written, I only care about the RuntimeWarning. But calling f may emit other warnings (from f, or something it calls).

def f():
    warnings.warn('f1', FutureWarning)
    warnings.warn('f2', RuntimeWarning)

So I think we need an option for whether "extra" warnings cause the test to fail (I really don't want to wrap every call to SparseDataFrame in an assert_produces_warning :)

jbrockmendel · 2019-03-15T15:24:54Z

But calling f may emit other warnings

I think this is related to an issue that came up in DatetimeArray._addsub_int in cases where multiple warnings were expected. The conclusion at the time was that tm.assert_produces_warning doesn't handle multiple expected warnings, so the workaround was to just suppress one and only test for the other.

Not a big deal, but any reason why this merits a new test file?

TomAugspurger · 2019-03-15T15:28:19Z

For the sparse array deprecation, I'd like to decorate the test class with `@pytest.mark.filterwarnings(...)` ignore warnings about the deprecation. But we'd like to assert that expected warnings for other things still happen.

…

On Fri, Mar 15, 2019 at 10:25 AM jbrockmendel ***@***.***> wrote: But calling f may emit other warnings I think this is related to an issue that came up in DatetimeArray._addsub_int in cases where multiple warnings were expected. The conclusion at the time was that tm.assert_produces_warning doesn't handle multiple expected warnings, so the workaround was to just suppress one and only test for the other. Not a big deal, but any reason why this merits a new test file? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#25721 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIhbVstqEzakbRSbPldBi3OAc1NM3ks5vW7tMgaJpZM4by-5g> .

gfyoung · 2019-03-16T09:40:16Z

The conclusion at the time was that tm.assert_produces_warning doesn't handle multiple expected warnings

Right - it would be worthwhile though at some point to investigate adding such support.

jreback · 2019-03-19T23:40:12Z

thanks @TomAugspurger

* origin/master: DOC: clean bug fix section in whatsnew (pandas-dev#25792) DOC: Fixed PeriodArray api ref (pandas-dev#25526) Move locale code out of tm, into _config (pandas-dev#25757) Unpin pycodestyle (pandas-dev#25789) Add test for rdivmod on EA array (GH23287) (pandas-dev#24047) ENH: Support datetime.timezone objects (pandas-dev#25065) Cython language level 3 (pandas-dev#24538) API: concat on sparse values (pandas-dev#25719) TST: assert_produces_warning works with filterwarnings (pandas-dev#25721) make core.config self-contained (pandas-dev#25613) CLN: replace %s syntax with .format in pandas.io.parsers (pandas-dev#24721) TST: Check pytables<3.5.1 when skipping (pandas-dev#25773) DOC: Fix typo in docstring of DataFrame.memory_usage (pandas-dev#25770) Replace dicts with OrderedDicts in groupby aggregation functions (pandas-dev#25693) TST: Fixturize tests/frame/test_missing.py (pandas-dev#25640) DOC: Improve the docsting of Series.iteritems (pandas-dev#24879) DOC: Fix function name. (pandas-dev#25751) Implementing iso_week_year support for to_datetime (pandas-dev#25541) DOC: clarify corr behaviour when using a callable (pandas-dev#25732) remove unnecessary check_output (pandas-dev#25755) # Conflicts: # doc/source/whatsnew/v0.25.0.rst

* upstream/master: (55 commits) PERF: Improve performance of StataReader (pandas-dev#25780) Speed up tokenizing of a row in csv and xstrtod parsing (pandas-dev#25784) BUG: Fix _binop for operators for serials which has more than one returns (divmod/rdivmod). (pandas-dev#25588) BUG-24971 copying blocks also considers ndim (pandas-dev#25521) CLN: Panel reference from documentation (pandas-dev#25649) ENH: Quoting column names containing spaces with backticks to use them in query and eval. (pandas-dev#24955) BUG: reading windows utf8 filenames in py3.6 (pandas-dev#25769) DOC: clean bug fix section in whatsnew (pandas-dev#25792) DOC: Fixed PeriodArray api ref (pandas-dev#25526) Move locale code out of tm, into _config (pandas-dev#25757) Unpin pycodestyle (pandas-dev#25789) Add test for rdivmod on EA array (GH23287) (pandas-dev#24047) ENH: Support datetime.timezone objects (pandas-dev#25065) Cython language level 3 (pandas-dev#24538) API: concat on sparse values (pandas-dev#25719) TST: assert_produces_warning works with filterwarnings (pandas-dev#25721) make core.config self-contained (pandas-dev#25613) CLN: replace %s syntax with .format in pandas.io.parsers (pandas-dev#24721) TST: Check pytables<3.5.1 when skipping (pandas-dev#25773) DOC: Fix typo in docstring of DataFrame.memory_usage (pandas-dev#25770) ...

TomAugspurger added this to the 0.25.0 milestone Mar 14, 2019

TomAugspurger added 2 commits March 14, 2019 06:19

take 2

c05306d

handle no warning

a55659c

jreback requested changes Mar 14, 2019

View reviewed changes

jreback added the Testing pandas testing functions or related to the test suite label Mar 14, 2019

remove message

ff66c6b

jreback approved these changes Mar 14, 2019

View reviewed changes

jorisvandenbossche reviewed Mar 14, 2019

View reviewed changes

extra warnings option

3e01ab4

jreback merged commit 2c0575d into pandas-dev:master Mar 19, 2019

TomAugspurger deleted the warning-filter-level branch April 18, 2019 18:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TST: assert_produces_warning works with filterwarnings #25721

TST: assert_produces_warning works with filterwarnings #25721

TomAugspurger commented Mar 14, 2019

codecov bot commented Mar 14, 2019

codecov bot commented Mar 14, 2019 •

edited

Loading

jreback Mar 14, 2019

TomAugspurger Mar 14, 2019

jreback Mar 14, 2019

TomAugspurger Mar 14, 2019

jorisvandenbossche Mar 14, 2019

TomAugspurger Mar 14, 2019

TomAugspurger commented Mar 14, 2019

jbrockmendel commented Mar 15, 2019

TomAugspurger commented Mar 15, 2019 via email

gfyoung commented Mar 16, 2019

jreback commented Mar 19, 2019

TST: assert_produces_warning works with filterwarnings #25721

TST: assert_produces_warning works with filterwarnings #25721

Conversation

TomAugspurger commented Mar 14, 2019

codecov bot commented Mar 14, 2019

Codecov Report

codecov bot commented Mar 14, 2019 • edited Loading

Codecov Report

jreback Mar 14, 2019

Choose a reason for hiding this comment

TomAugspurger Mar 14, 2019

Choose a reason for hiding this comment

jreback Mar 14, 2019

Choose a reason for hiding this comment

TomAugspurger Mar 14, 2019

Choose a reason for hiding this comment

jorisvandenbossche Mar 14, 2019

Choose a reason for hiding this comment

TomAugspurger Mar 14, 2019

Choose a reason for hiding this comment

TomAugspurger commented Mar 14, 2019

jbrockmendel commented Mar 15, 2019

TomAugspurger commented Mar 15, 2019 via email

gfyoung commented Mar 16, 2019

jreback commented Mar 19, 2019

codecov bot commented Mar 14, 2019 •

edited

Loading