Skip to content

Builds killed due to memory usage #4460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bdice opened this issue Aug 2, 2018 · 2 comments
Closed

Builds killed due to memory usage #4460

bdice opened this issue Aug 2, 2018 · 2 comments
Assignees
Labels
Support Support question

Comments

@bdice
Copy link

bdice commented Aug 2, 2018

Details

Expected Result

I maintain a library for molecular dynamics trajectory analysis, freud (https://bitbucket.org/glotzer/freud). It's heavily written in Cython and C++. We currently use CMake and Cython to build the package, and frequently run out of memory/time (it fails about 75% of the time). On RTD, we have been building with clang to reduce memory usage (gcc fails every time).

We are trying to make the package easier to maintain, by building more like a "normal" Cython package (using setup.py). Our package has been split from one huge module to several smaller modules, which should also help with memory. We had hoped that this would also make the installation behave better on ReadTheDocs. However, we're still hitting the limits and would like to explore increasing time/memory as has been recently mentioned by other projects (#4432 #4251 #4403). The documentation itself is pretty simple, but we can't render Cython docstrings without building the package. As other packages have mentioned, the installation of conda dependencies takes a lot of time.

Our latest and stable builds use the CMake build system, and the branch pure_cython_rtd is what we're using to test ReadTheDocs support on the new Cython-only build system.

ReadTheDocs is a fantastic service. I really appreciate your time and support, and recognize that asking for more resources is a significant burden. However, it would be nice if we could get the docs to build once, correctly, instead of manually re-triggering builds several times until they work.

Actual Result

Builds fail frequently due to running out of memory/time.

@humitos humitos added the Support Support question label Aug 2, 2018
@humitos humitos added this to the Build stability milestone Aug 2, 2018
@agjohnson agjohnson removed this from the Build stability milestone Aug 2, 2018
@humitos humitos self-assigned this Aug 21, 2018
@bdice
Copy link
Author

bdice commented Aug 27, 2018

Hi, just wanted to give an update -- since the shift to Azure a week ago, it seems like our docs have been building successfully. I'm not sure what changed under the hood of ReadTheDocs, but this issue may be resolved. I'll update this issue again if we continue to have challenges. Thanks!

@humitos
Copy link
Member

humitos commented Aug 28, 2018

Thank you @bdice for reporting this back! I really appreciate.

I will close this issue then, but please, let me know if you experience this again in the future.

We are working on different solutions. You can read more about this at #4573.

I'm not sure what changed under the hood of ReadTheDocs, but this issue may be resolved

Our new servers on Azure are bigger (more CPU and more RAM) which gives us less building time for projects and seems the extra RAM gives us more stability to allow builds that consume more resources in parallel.

@humitos humitos closed this as completed Aug 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Support Support question
Projects
None yet
Development

No branches or pull requests

3 participants