Skip to content

BUILD: conda unable to solve for a dev env using environment.yml in finite time #43876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
rs2 opened this issue Oct 4, 2021 · 9 comments
Closed
1 task done
Labels
Build Library building on various platforms Windows Windows OS

Comments

@rs2
Copy link
Contributor

rs2 commented Oct 4, 2021

Platform

win-64

Installation Method

conda install

pandas Version

latest master

Python Version

n/a

Installation Logs

$ conda env create -f environment.yml
Collecting package metadata (repodata.json): done
Solving environment: /

  • Full disclosure: this is not a pandas issue per se, this is a conda issue. However, I see two ways to solve this issue for anyone trying to build pandas locally:
  1. Use mamba instead of conda to create dev environment. mamba has a much faster SAT solver compared to conda.
  2. (Probably, preferred) Pin package version numbers for each platform in https://github.com/pandas-dev/pandas/blob/c03cc2245538e3a2c182f6eb15cfb143897a9503/environment.yml.
@rs2 rs2 added Build Library building on various platforms Needs Triage Issue that has not been reviewed by a pandas team member labels Oct 4, 2021
@phofl
Copy link
Member

phofl commented Oct 4, 2021

I build this locally on a linux ubuntu machine yesterday and it worked as usual. Do you have acess to a linux machine to test there?

Pinning is not really an option since we want to develop against the newest versions. Same as the testing suite does test against newest versions

@rs2
Copy link
Contributor Author

rs2 commented Oct 4, 2021

I build this locally on a linux ubuntu machine yesterday and it worked as usual. Do you have acess to a linux machine to test there?

Pinning is not really an option since we want to develop against the newest versions. Same as the testing suite does test against newest versions

I need to test specifically on win-64 as per the issue description. conda was unable to solve for the environment in finite time, whereas mamba took well under 10 seconds to do so.

@attack68
Copy link
Contributor

attack68 commented Oct 4, 2021

note: #43804. My dev env didn't resolve with pip in 3hours becuase I detected at least 2 incompatabilities.

@rs2
Copy link
Contributor Author

rs2 commented Oct 4, 2021

note: #43804. My dev env didn't resolve with pip in 3hours becuase I detected at least 2 incompatabilities.

As a workaround, do use mamba instead of conda. It invariably finds a solution in no time. See https://github.com/mamba-org/mamba#installation for details.

@phofl
Copy link
Member

phofl commented Oct 4, 2021

This was solved for me relatively quickly on win64 too

@rs2
Copy link
Contributor Author

rs2 commented Oct 4, 2021

This was solved for me relatively quickly on win64 too

conda SAT solver can be sensitive to the set of available package versions. It may be able to find a solution in minutes at a particular time, and take a few hours just a short time later.

@mroeschke mroeschke added Windows Windows OS and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Oct 13, 2021
@mroeschke
Copy link
Member

It appears this wasn't reproducible in another windows environment so closing for now. This issue should still remain searchable with the suggestion to use mamba

@asishm
Copy link
Contributor

asishm commented Oct 15, 2021

I've been running into this issue on wsl2 (ubuntu 20.04) as well. At first it was getting killed due to OOM with 4GB RAM allocated via wslconfig, but took 20 minutes to resolve the package dependencies.

Mamba was near instant.

@MarcoGorelli
Copy link
Member

Use mamba instead of conda to create dev environment. mamba has a much faster SAT solver compared to conda.

@rs2 did you want to open a PR for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Library building on various platforms Windows Windows OS
Projects
None yet
Development

No branches or pull requests

6 participants