Windows 3.7 CI Failures

[WillAyd]

Lots of failures recently with CI build. I do see this in the logs:

CondaValueError: Malformed version string '~': invalid character(s).

Which I assume is the root cause, though the dependency file hasn't been updated in some time.

@TomAugspurger if you have any thoughts; maybe a conda-forge issue?

[WillAyd]

May also have to do with conda version. I think last working CI run was using 4.5.12 latest have been using 4.6.14

[gfyoung]

Weird...I can create the environment using 4.6.14 just fine locally on my Windows machine.

[jreback]

let’s leave this open to follow up on the actual gcfs issue

[gfyoung]

xref: #26358 (comment)

[gfyoung]

cc @martindurant : Hello, I'm pinging because you seem to be the prominent contributor to dask/gcsfs. We're running into an issue installing this via conda on Windows 3.7 on Azure (see the issue description). What's weird is that we haven't touched our Azure YAML for awhile, and this suddenly popped up. Might you have any idea what happened?

[martindurant]

Sorry, what exactly is the issue? gcsfs should install on 3.7 as far as I know, but I have no experience with azure - it's a pure-python package, though, so should be simple. There has been no new version of gcsfs, nor google-auth/oauthlib.

[gfyoung]

it's a pure-python package, though, so should be simple

@martindurant: That's what logic would dictate, but CI does not seem to follow basic human logic 😂

Have a look at:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=11345

• Click the "Windows py37_np141" build
• Click the "Before Install" step
• Scroll down to line 962

This is configuration for the conda environment that we set up for this (line 55 of build output):

https://github.com/pandas-dev/pandas/blob/master/ci/deps/azure-windows-37.yaml

Have you seen that (line 962) error ever? I haven't been able to reproduce on a local windows machine at this point, so not really sure what's going on here...

If you have any other questions, I'll be happy to help you out.

[TomAugspurger]

This is a conda issue, not gcsfs, right?

[martindurant]

I haven't ever seen this before, no. I suppose, indeed we should ping someone from conda or forge.

[martindurant]

A text search doesn't reveal the character "~" anywhere in the gcsfs package, outside of strings, docs and comments.

[gfyoung]

Could it be manifesting itself in one of the upstream dependencies for gcsfs? I might have to do some digging on that front to see...then again, why it's only failing 3.7 on Azure and literally nowhere else (my Windows computer included) is still bewildering.

I thought it might be conda as well, but the version number on conda is the same before and after this whole gcsfs thing started failing, so I'm feeling less certain about that being the case.

[martindurant]

Definitely could be, but as I said above, it's not the immediate suspects from google.

[WillAyd]

Shot in the dark but there is a note about issues with running a batch script within a batch script on 4.6+:

https://docs.conda.io/projects/conda/en/latest/user-guide/troubleshooting.html#using-conda-in-windows-batch-script-exits-early

[gfyoung]

@WillAyd : The current Azure builds use 4.5.12, not 4.6+, so that troubleshoot is moot unfortunately for our purposes.

[WillAyd]

Yea as expected then no luck. I did increase verbosity on one of the builds and got the following traceback in case anyone sees use in that:

  File "C:\Miniconda\lib\site-packages\conda\exceptions.py", line 819, in __call__
    return func(*args, **kwargs)
  File "C:\Miniconda\lib\site-packages\conda_env\cli\main.py", line 74, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "C:\Miniconda\lib\site-packages\conda_env\cli\main_create.py", line 108, in execute
    installer.install(prefix, pkg_specs, args, env)
  File "C:\Miniconda\lib\site-packages\conda_env\installers\conda.py", line 35, in install
    unlink_link_transaction = solver.solve_for_transaction(prune=getattr(args, 'prune', False))
  File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 518, in solve_for_transaction
    force_remove, force_reinstall)
  File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 451, in solve_for_diff
    final_precs = self.solve_final_state(deps_modifier, prune, ignore_pinned, force_remove)
  File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 180, in solve_final_state
    index, r = self._prepare(prepared_specs)
  File "C:\Miniconda\lib\site-packages\conda\core\solve.py", line 592, in _prepare
    self.subdirs, prepared_specs)
  File "C:\Miniconda\lib\site-packages\conda\core\index.py", line 217, in get_reduced_index
    push_record(record)
  File "C:\Miniconda\lib\site-packages\conda\core\index.py", line 198, in push_record
    for _spec in record.combined_depends:
  File "C:\Miniconda\lib\site-packages\conda\models\records.py", line 273, in combined_depends
    result = {ms.name: ms for ms in MatchSpec.merge(self.depends)}
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 411, in merge
    match_specs = tuple(cls(s) for s in match_specs)
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 411, in <genexpr>
    match_specs = tuple(cls(s) for s in match_specs)
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 43, in __call__
    return super(MatchSpecType, cls).__call__(**parsed)
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 180, in __init__
    self._match_components = self._build_components(**kwargs)
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 367, in _build_components
    return frozendict(_make(key, value) for key, value in iteritems(kwargs))
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 367, in <genexpr>
    return frozendict(_make(key, value) for key, value in iteritems(kwargs))
  File "C:\Miniconda\lib\site-packages\conda\models\match_spec.py", line 361, in _make
    matcher = _implementors[field_name](value)
  File "C:\Miniconda\lib\site-packages\conda\models\version.py", line 468, in __new__
    self.cmp = VersionOrder(spec)
  File "C:\Miniconda\lib\site-packages\conda\models\version.py", line 163, in __new__
    raise CondaValueError(message + "invalid character(s).")
conda.exceptions.CondaValueError: Malformed version string '~': invalid character(s).

[WillAyd]

I think the Conda issue is conda/conda#7776 so should be fixed with 4.6 at the very least