CI debug

[TomAugspurger]

Closes #27541

[WillAyd]

Different error this time right?

[TomAugspurger]

We did have a timeout earlier, but I think it looked a bit different than the last one. I restarted the build.

[jreback]

you need pytest<5.0.0; I thought we had added this (its on all the other 3.6 builds)

[WillAyd]

I don't know if directly relevant here since current build complies but there is a note for pytest >5.0.0 that if used with pytest-xdist that should at least be 1.29.0 or later in the changeling:

https://docs.pytest.org/en/latest/changelog.html#features

Might be related to some of the issues (this one seems suspiciously related to the one we saw last week on Azure)

[WillAyd]

Though it does look like the issue Jeff has in the OP of #27541 has pytest-xdist 1.28.0 and pytest 5.0.1 so issue might just boil down to that

[WillAyd]

Ignore comments above - I should have looked at diff. But yea this seems like the right change

[TomAugspurger]

Huh that sounds extremely relevant. In a meeting now, but will update everything to be pytest >5, pytest-xdist >=1.2.9 (unless someone beats me).

[WillAyd]

Current build here meets those specs but just failed again so I think it's still something else

[TomAugspurger]

Trying to pin that build to pytest<5. Won't have more time for this today unfortunately.

[TomAugspurger]

Well this one failed with out of memory errors when we tried to open a subprocess

=================================== FAILURES ===================================
_____________________________ test_initial_warning _____________________________
[gw0] linux -- Python 3.6.8 /home/travis/miniconda3/envs/pandas-dev/bin/python
    def test_initial_warning():
        code = (
            "import pandas as pd; import matplotlib.pyplot as plt; "
            "s = pd.Series(1, pd.date_range('2000', periods=12)); "
            "fig, ax = plt.subplots(); "
            "ax.plot(s.index, s.values)"
        )
        call = [sys.executable, "-c", code]
>       out = subprocess.check_output(call, stderr=subprocess.STDOUT).decode()

[WillAyd]

Yea was about to comment. I have the feeling this might be the root issue and the original issues we saw were problems with pytest / pytest-xdist compat and actually reporting out the error.

[jbrockmendel]

Is it this same test failing repeatedly? We could wrap the subprocess call in something like:

try:
     subprocess...
except OSError as err:
     if err.errno == errno.ENOMEM:
          pytest.xfail(...)
     raise

[TomAugspurger]

We do seem to have less disk space, which indicates we may be running on a smaller VM. From https://travis-ci.org/pandas-dev/pandas/jobs/561486224 (3 days ago)

�[0K$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G  4.0K  3.7G   1% /dev
tmpfs           748M  292K  748M   1% /run
/dev/sda1        69G   11G   55G  17% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none            3.7G     0  3.7G   0% /run/shm
none            100M     0  100M   0% /run/user
none            768M  179M  590M  24% /var/ramfs

And from today

�[0K$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.8G  4.0K  1.8G   1% /dev
tmpfs           370M  292K  370M   1% /run
/dev/sda1        69G   12G   55G  17% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none            1.9G     0  1.9G   0% /run/shm
none            100M     0  100M   0% /run/user
none            768M  179M  590M  24% /var/ramfs

I'm not sure if we print our RAM in the logs.

[WillAyd]

Shot in the dark but looks like this was deprecated

https://docs.travis-ci.com/user/reference/trusty/#container-based-infrastructure

[TomAugspurger]

FYI, we may want to keep an eye on https://travis-ci.community/t/builds-often-fail-because-of-lack-of-resources-memory-cpu/4331 and travis-ci/worker#604. Seems like the worker images are being given less resources.

For now, we might lighten the load on that build by removing some optional deps. I'd propose removing matplotlib for a start.

[TomAugspurger]

Another OOM crash. I'm removing matplotlib from those two environments.

[TomAugspurger]

OK, that passed. Any objections to merging as is, and cleaning up the CI files in a followup PR? I suspect the pytest & pytest-xdist pins are no longer necessary, but it'd be good to get other PRs going again.

[WillAyd]

Ok by me

[jreback]

hmm I suspect we need to backport this to 0.25.1