Skip to content

CI/TST: Make test_vector_resize more deterministic #46602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 3, 2022

Conversation

mroeschke
Copy link
Member

I had to xfail(strict=True) this test once before which leads me to believe that the initial random data may be causing this test to be flaky.

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=76111&view=logs&jobId=b1891a80-bdb0-5193-5f59-ce68a0874df0&j=b1891a80-bdb0-5193-5f59-ce68a0874df0&t=0a58a9e9-0673-539c-5bde-d78a5a3b655e

=================================== FAILURES ===================================
_ TestHashTableUnsorted.test_vector_resize[False-Int32HashTable-Int32Vector-int32-False-10] _
[gw1] darwin -- Python 3.9.12 /usr/local/miniconda/envs/pandas-dev/bin/python

self = <pandas.tests.libs.test_hashtable.TestHashTableUnsorted object at 0x161a3d7c0>
writable = False
htable = <pandas._libs.hashtable.Int32HashTable object at 0x1764fee30>
uniques = <pandas._libs.hashtable.Int32Vector object at 0x176528e00>
dtype = 'int32', safely_resizes = False, nvals = 10

    @pytest.mark.parametrize("nvals", [0, 10])  # resizing to 0 is special case
    @pytest.mark.parametrize(
        "htable, uniques, dtype, safely_resizes",
        [
            (ht.PyObjectHashTable, ht.ObjectVector, "object", False),
            (ht.StringHashTable, ht.ObjectVector, "object", True),
            (ht.Float64HashTable, ht.Float64Vector, "float64", False),
            (ht.Int64HashTable, ht.Int64Vector, "int64", False),
            (ht.Int32HashTable, ht.Int32Vector, "int32", False),
            pytest.param(
                ht.UInt64HashTable,
                ht.UInt64Vector,
                "uint64",
                False,
                marks=pytest.mark.xfail(
                    reason="Sometimes doesn't raise.",
        else:
            with pytest.raises(ValueError, match="external reference.*"):
>               htable.get_labels(vals, uniques, 0, -1)
E               Failed: DID NOT RAISE <class 'ValueError'>

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite Unreliable Test Unit tests that occasionally fail labels Apr 1, 2022
@mroeschke mroeschke added this to the 1.4.2 milestone Apr 1, 2022
@simonjayhawkins
Copy link
Member

moving to 1.4.3

no problem to merge to main as autobackport is disabled for 1.4.3

@simonjayhawkins simonjayhawkins modified the milestones: 1.4.2, 1.4.3 Apr 1, 2022
@jreback jreback merged commit bea02f3 into pandas-dev:main Apr 3, 2022
@lumberbot-app
Copy link

lumberbot-app bot commented Apr 3, 2022

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 1.4.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 bea02f300bdf54d86aded7d5c4461dc340ddb929
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #46602: CI/TST: Make test_vector_resize more deterministic'
  1. Push to a named branch:
git push YOURFORK 1.4.x:auto-backport-of-pr-46602-on-1.4.x
  1. Create a PR against branch 1.4.x, I would have named this PR:

"Backport PR #46602 on branch 1.4.x (CI/TST: Make test_vector_resize more deterministic)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

@jreback
Copy link
Contributor

jreback commented Apr 3, 2022

@meeseeksdev backport 1.4.x

@lumberbot-app
Copy link

lumberbot-app bot commented Apr 3, 2022

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 1.4.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 bea02f300bdf54d86aded7d5c4461dc340ddb929
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #46602: CI/TST: Make test_vector_resize more deterministic'
  1. Push to a named branch:
git push YOURFORK 1.4.x:auto-backport-of-pr-46602-on-1.4.x
  1. Create a PR against branch 1.4.x, I would have named this PR:

"Backport PR #46602 on branch 1.4.x (CI/TST: Make test_vector_resize more deterministic)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

@mroeschke mroeschke deleted the tst/flaky_vector_resize branch April 3, 2022 15:50
mroeschke added a commit to mroeschke/pandas that referenced this pull request Apr 3, 2022
@simonjayhawkins
Copy link
Member

#46350 was not backported (which this PR fixes), neither was the refactor #46106 that prevented the autobackport from succeeding and made some code changes to pandas/_libs/hashtable_class_helper.pxi.in that we probably don't want to backport (and maybe the reason #46350 was needed?)

change the milestone on this to 1.5 and close the backport PR #46625?

@mroeschke
Copy link
Member Author

Sounds good to me. This doesn't fail often so hopefully it shouldn't affect the 1.4.x branch too often.

@mroeschke mroeschke modified the milestones: 1.4.3, 1.5 Apr 7, 2022
@simonjayhawkins
Copy link
Member

This doesn't fail often so hopefully it shouldn't affect the 1.4.x branch too often.

will keep an eye out but I was thinking that it was maybe #46106 is the cause of the issue which would not affect 1.4.x anyway.

yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this pull request Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing pandas testing functions or related to the test suite Unreliable Test Unit tests that occasionally fail
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants