Debug CI Issue

[WillAyd]

No description provided.

[simonjayhawkins]

lint error ##[error]./pandas/tests/extension/base/dtype.py:1:1:F401:'warnings' imported but unused

[TomAugspurger]

@WillAyd I have some time over the next couple hours to debug this. I'll plan to push to your branch if that's OK.

[TomAugspurger]

Ends up being pretty simple. With older numpys

In [24]: pd.DataFrame({"A": pd.array([], dtype="Int64"), "B": pd.array([], dtype="int64")}).dtypes == 'Int64'
Out[24]:
A    True
B    True
dtype: bool

In [25]: np.dtype("int64") == "Int64"
/Users/taugspurger/.virtualenvs/dask-dev/bin/ipython:1: DeprecationWarning: Numeric-style type codes are deprecated and will result in an error in the future.
  #!/Users/taugspurger/Envs/dask-dev/bin/python
Out[25]: True

With NumPy master

In [1]: import pandas as pd
i
In [2]: import numpy as np

In [3]: pd.DataFrame({"A": pd.array([], dtype="Int64"), "B": pd.array([], dtype="int64")}).dtypes == 'Int64'
Out[3]:
A     True
B    False
dtype: bool

In [4]: np.dtype("int64") == "Int64"
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-49749ad32aa3> in <module>
----> 1 np.dtype("int64") == "Int64"

TypeError: data type 'Int64' not understood

Will push a fix up in a bit.

[TomAugspurger]

I think this is targeted for NumPy 1.20, but verifying in numpy/numpy#16554 (comment).

[TomAugspurger]

I'm not sure that there's anything we can actually do here. Open to suggestions, but the fact is that NumPy controls the behavior of np.dtype.__eq__. So I'm comfortable just leaving this for NumPy<1.20.

[TomAugspurger]

cc @jreback if you have a chance to look. Planning to merge on green to get CI running again though.

[jreback]

lgtm merge when ready

[jreback]

did we merge something to fix CI?

[TomAugspurger]

Sorry, thought this was in. Taking another look.

[TomAugspurger]

Ah, I think this issue is that the numpydev build is NumPy 1.20.dev0.stuff, but it's not new enough to have the changes in the travis-CI job (built from numpy source).

[TomAugspurger]

Right now https://anaconda.org/scipy-wheels-nightly/numpy/files doesn't have a wheel with the new behavior the travis build (building numpy from source) is seeing.

So it's ugly, but we have to make the check dependent on the behavior of np.dtype('int64') == "Int64" until we can get a wheel with the new behavior. Then we can just check the version number.

[WillAyd]

Maybe just skip the test until the wheel is available?

…

Sent from my iPhone

On Jun 12, 2020, at 11:40 AM, Tom Augspurger ***@***.***> wrote:  Right now https://anaconda.org/scipy-wheels-nightly/numpy/files doesn't have a wheel with the new behavior the travis build (building numpy from source) is seeing. So it's ugly, but we have to make the check dependent on the behavior of np.dtype('int64') == "Int64" until we can get a wheel with the new behavior. Then we can just check the version number. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[TomAugspurger]

Slight preference for this, just in case we forget to update or it takes a little while for new wheels to show up.

[TomAugspurger]

No idea why this started failing now

        with tm.assert_produces_warning(UserWarning):
            axes = _check_plot_works(
>               scatter_matrix, filterwarnings="always", frame=df, range_padding=0.1
            )

pandas\tests\plotting\test_misc.py:107: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <contextlib._GeneratorContextManager object at 0x000001A3127D1F48>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('DeprecationWarning', DeprecationWarning("Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working"), 'C:\\Miniconda\\envs\\pandas-dev\\lib\\site-packages\\matplotlib\\cbook\\__init__.py', 2349), ('DeprecationWarning', DeprecationWarning("Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working"), 'C:\\Miniconda\\envs\\pandas-dev\\lib\\site-packages\\matplotlib\\cbook\\__init__.py', 2366)]

[WillAyd]

Thanks @TomAugspurger - pretty tricky stuff here

[simonjayhawkins]

@meeseeksdev backport to 1.0.x

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

$ git checkout 1.0.x
$ git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

$ git cherry-pick -m1 e04868400ae3299a69a1901978a18f55650fca9a

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

$ git commit -am 'Backport PR #34721: Debug CI Issue'

4. Push to a named branch :

git push YOURFORK 1.0.x:auto-backport-of-pr-34721-on-1.0.x

5. Create a PR against branch 1.0.x, I would have named this PR:

"Backport PR #34721 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.