Skip to content

CI/TST: hypothesis tests regularly fail due time-out #23121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jorisvandenbossche opened this issue Oct 13, 2018 · 2 comments
Closed

CI/TST: hypothesis tests regularly fail due time-out #23121

jorisvandenbossche opened this issue Oct 13, 2018 · 2 comments
Labels
CI Continuous Integration Testing pandas testing functions or related to the test suite
Milestone

Comments

@jorisvandenbossche
Copy link
Member

I have seen the following a few times:

______________ TestDataFrameAggregate.test_frequency_is_original _______________
[gw0] linux -- Python 3.7.0 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.frame.test_apply.TestDataFrameAggregate object at 0x7f17a4ae6ef0>
    @given(index=indices(5), num_columns=integers(0, 5))
>   def test_frequency_is_original(self, index, num_columns):
pandas/tests/frame/test_apply.py:1159: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/core.py:695: in evaluate_test_data
    result = self.execute(data, collect=True)
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/core.py:610: in execute
    result = self.test_runner(data, run)
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/executors.py:58: in default_new_style_executor
    return function(data)
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/core.py:606: in run
    return test(*args, **kwargs)
pandas/tests/frame/test_apply.py:1159: in test_frequency_is_original
    def test_frequency_is_original(self, index, num_columns):
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/core.py:566: in test
    runtime, ceil(runtime / 100) * 100,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
message = 'Test took 298.69ms to run. In future the default deadline setting will be 200ms, which will make this an error. You c...ue of e.g. 300 to turn tests slower than this into an error, or you can set it to None to disable this check entirely.'
s = settings(buffer_size=8192, database=DirectoryBasedExampleDatabase('/home/travis/build/jorisvandenbossche/pandas/.hypot...step_count=50, suppress_health_check=[HealthCheck.too_slow], timeout=60, use_coverage=True, verbosity=Verbosity.normal)
    def note_deprecation(message, s=None):
        # type: (str, settings) -> None
        if s is None:
            s = settings.default
        assert s is not None
        verbosity = s.verbosity
        warning = HypothesisDeprecationWarning(message)
        if verbosity > Verbosity.quiet:
>           warnings.warn(warning, stacklevel=3)
E           hypothesis.errors.HypothesisDeprecationWarning: Test took 298.69ms to run. In future the default deadline setting will be 200ms, which will make this an error. You can set deadline to an explicit value of e.g. 300 to turn tests slower than this into an error, or you can set it to None to disable this check entirely.
../../../miniconda3/envs/pandas/lib/python3.7/site-packages/hypothesis/_settings.py:828: HypothesisDeprecationWarning
---------------------------------- Hypothesis ----------------------------------
You can add @seed(169950230187726778200653375172917819369) to this test or run pytest with --hypothesis-seed=169950230187726778200653375172917819369 to reproduce this failure.
@jorisvandenbossche jorisvandenbossche added Testing pandas testing functions or related to the test suite CI Continuous Integration labels Oct 13, 2018
@jorisvandenbossche jorisvandenbossche added this to the 0.24.0 milestone Oct 13, 2018
@TomAugspurger
Copy link
Contributor

cc @Zac-HD

@Zac-HD
Copy link
Contributor

Zac-HD commented Oct 13, 2018

Right, all the timing warnings are tuned for scalar values, so arrays tend to trigger them a bit too easily... I'll open a PR to adjust the deadline shortly 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Testing pandas testing functions or related to the test suite
Projects
None yet
Development

No branches or pull requests

3 participants