CI: downstream tests failing due to environment resolvement (geospatial stack)

[jorisvandenbossche]

The geopandas downstream tests are failing because of an installation issue.

See eg https://travis-ci.org/pandas-dev/pandas/jobs/508237658, giving:

>   from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
E   ImportError: libiconv.so.2: cannot open shared object file: No such file or directory

This environment is defined by this yml file: https://github.com/pandas-dev/pandas/blob/master/ci/deps/travis-36.yaml

It is mixing defaults and conda-forge (with defaults as higher priority one), but given the nomkl it is getting blas / openblas from conda-forge.
Until recently, the full geospatial stack was resolved to come from defaults (see eg https://travis-ci.org/pandas-dev/pandas/jobs/507830158#L1749), but since the last hours, it is getting fiona / gdal from conda-forge as well (but not all of the gdal dependencies, hence giving the error above).

cc @ocefpaf I know this is again an example of mixing conda-forge and defaults (like the recent one we had on the geopandas CI), which is not guaranteed to work. But it is also again something that was working until recently, and a kind of common use case: getting all from defaults, except for a few things that are not available there (and therefore adding conda-forge as the second, lower priority channel)

[ocefpaf]

@isuruf could this be related to the blas migration?

@jorisvandenbossche note that both defaults and conda-forge are updated frequently so there is no promise that a mixed channel, that in theory is already a bad idea, will work forever. Only strict channel can help us debug problem in a single channel, multiple channels are pretty much impossible to debug and to ensure a permanent fix.

[jorisvandenbossche]

@ocefpaf yes, I am well aware. For me a reason to keep those builds is that it can reflect what probably a lot of people are doing in practice, and can catch issues like the above. But of course, that only makes senseif the conda / conda-forge folks are actually interested in hearing about such compatibility issues.
If not, we can certainly convert the CI build here to a clean one, either not using nomkl and everything from defaults, or either everything from conda-forge

(in any case, the pandas CI is probably not the best place to test those mixed builds since we don't want to have the pandas CI failing for such compatibility issue. If we want to keep one, better to do that on the geopandas CI)

[jorisvandenbossche]

@jreback do you know if there is a specific reason we are using nomkl on the travis py3.6 build? Just to reduce the download size, or to actually have a build testing with openblas?

If we want to keep openblas, we can have a full conda-forge based build to avoid such issues in the future. Or otherwise, I can also remove nomkl and have it fully defaults-based.

[ocefpaf]

But of course, that only makes senseif the conda / conda-forge folks are actually interested in hearing about such compatibility issues.

While I appreciate the effort in reporting there is not much we can do. Sorry. I know that is frustrating but we are not able to fix anything unless the report is using strict, there is nothing we can do in the mixed channel case that will last longer than a "packaging cycle." No error reporting a mixed channel problem can be debugged and fixed in the long term.

I would recommend to always have a clean CI with wither strict, only defaults or only conda-forge.

PS: in the past we gave the illusion that mixing channels would be possible but the truth is that conda-forge was, most of the time, ahead of defaults in the packages versions. Since defaults started using conda-forge recipes that lag got shorter and we cannot guarantee stability based on having the latest versions for solver anymore.

[jorisvandenbossche]

The problem with environment.yml files is that it would still be nice to 'strictly' specify which packages can come from another channel, or to have this strict configuration embedded in it (which I know you are well aware of that!)

in the past we gave the illusion that mixing channels would be possible

But mixed with strict priority is still OK right?

[ocefpaf]

The problem with environment.yml files is that it would still be nice to 'strictly' specify which packages can come from another channel, or to have this strict configuration embedded in it (which I know you are well aware of that!)

Yes. We are still make that a default behavior for consistency. As well as the strict option.

But mixed with strict priority is still OK right?

Yes. And, if issues appear in that scenario, then it is a problem we can, and should, act on.

[jreback]

@jorisvandenbossche

@jreback do you know if there is a specific reason we are using nomkl on the travis py3.6 build? Just to reduce the download size, or to actually have a build testing with openblas?

yep exactly. but prob could change that i think.

[jbrockmendel]

@jorisvandenbossche any idea if this has resolved itself?