Skip to content

Increase timeout in hypothesis for test_apply.py #23849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 27, 2018
Merged

Increase timeout in hypothesis for test_apply.py #23849

merged 1 commit into from
Nov 27, 2018

Conversation

alimcmaster1
Copy link
Member

  • passes git diff upstream/master -u -- "*.py" | flake8 --diff

In this build

We see the following error:

<pandas.tests.frame.test_apply.TestDataFrameAggregate instance at 0x0000000036892A48>, 2018-11-Unreliable test timings! On an initial run, this test took 631.00ms, which exceeded the deadline of 500.00ms, but on a subsequent run it took 3.00 ms, which did not. If you expect this sort of variability in your test timings, consider turning deadlines off for this test by setting deadline=None.

Time difference perhaps due to given parameters for the test case, currently this test case uses the default value which is 500ms. Perhaps I could be more aggressive here and go for 800ms, thoughts?

Issue introduced in this change

@pep8speaks
Copy link

Hello @alimcmaster1! Thanks for submitting the PR.

@codecov
Copy link

codecov bot commented Nov 21, 2018

Codecov Report

Merging #23849 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #23849   +/-   ##
=======================================
  Coverage   92.28%   92.28%           
=======================================
  Files         161      161           
  Lines       51500    51500           
=======================================
  Hits        47528    47528           
  Misses       3972     3972
Flag Coverage Δ
#multiple 90.68% <ø> (ø) ⬆️
#single 42.31% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e405a10...332aa54. Read the comment docs.

@mroeschke
Copy link
Member

I feel like it would be better if we could maybe just simplify this test to test the original bug in question. I'm not convinces some of the hypothesis search parameters are entirely necessary e.g. testing a single dtype dataframe with 2 to 5 columns.

This was a minimal example that would test the same bug: #22150 (comment)

@alimcmaster1
Copy link
Member Author

Thanks @mroeschke for the thoughts - ill update with a few sensible test case that should clearly highlight the original bug.

@gfyoung gfyoung added Testing pandas testing functions or related to the test suite CI Continuous Integration labels Nov 22, 2018
@alimcmaster1
Copy link
Member Author

@mroeschke thoughts on something like this as a replacement test? I varied the number of columns at it was reported in the issue that the bug was dependent on # of cols.

    @pytest.mark.parametrize("num_cols", [2, 3, 5])
    @pytest.mark.parametrize("index",
       [["1950-06-30", "1952-10-24", "1953-05-29"]
    ])
    def test_frequency_is_original(self, index, num_cols):
        index  = pd.DatetimeIndex(index)
        original = index.copy()
        df = DataFrame(1, index=index, columns=range(num_cols))
        df.apply(lambda x: x)
        assert index.freq == original.freq

@mroeschke
Copy link
Member

Sure, that test looks good. If we are only testing one index, there's probably no need to parametrize over that variable.

@jreback jreback added this to the 0.24.0 milestone Nov 27, 2018
@jreback
Copy link
Contributor

jreback commented Nov 27, 2018

I am not sure this will fix some of the recent time out issues, but let's give a try.

@alimcmaster1 also would love a followup based on @mroeschke suggestion here.

@jreback jreback merged commit 11e2cb7 into pandas-dev:master Nov 27, 2018
@alimcmaster1
Copy link
Member Author

alimcmaster1 commented Nov 27, 2018 via email

@mroeschke mroeschke mentioned this pull request Feb 3, 2019
@alimcmaster1 alimcmaster1 deleted the hypo-flakey branch February 17, 2019 18:42
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants