Skip to content

Increase build timeout limit for Koalas project #6254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HyukjinKwon opened this issue Oct 8, 2019 · 22 comments
Closed

Increase build timeout limit for Koalas project #6254

HyukjinKwon opened this issue Oct 8, 2019 · 22 comments
Labels
Support Support question

Comments

@HyukjinKwon
Copy link

Details

Expected Result

Build pass without timeout

Screen Shot 2019-10-08 at 4 50 19 PM

Actual Result

Build timeout

Screen Shot 2019-10-08 at 4 50 09 PM

@HyukjinKwon
Copy link
Author

To speed up, I had to disable PDF build and EPUB build; however, we still intermittently face this timeout (which spams emails .. :( .. ).

Can I ask to increase the timeout?

@humitos
Copy link
Member

humitos commented Oct 8, 2019

Hi @HyukjinKwon! I just took a look at your builds and I found that that all the builds were executed on one the same server (build04). We are having an issue with this server (it's executing too many builds at the same time) that we hopefully fix soon.

@humitos humitos added the Support Support question label Oct 8, 2019
@HyukjinKwon
Copy link
Author

Thanks! I am actively working on that project. I will update if I happen to find out this issue isn't persistent anymore!

@stsewd
Copy link
Member

stsewd commented Oct 8, 2019

The project is failing for another reason now

Collecting package metadata: ...working... done
Solving environment: ...working... failed

ResolvePackageNotFound: 
  - java-jdk=8

The project doesn't have build04 as builder anymore.

@stsewd stsewd added the Needed: more information A reply from issue author is required label Oct 8, 2019
@HyukjinKwon
Copy link
Author

It suddenly fails there. I'll check and update within few days.

@no-response no-response bot removed the Needed: more information A reply from issue author is required label Oct 8, 2019
@stsewd stsewd added the Needed: more information A reply from issue author is required label Oct 8, 2019
@HyukjinKwon
Copy link
Author

HyukjinKwon commented Oct 9, 2019

Hi, the error was temporary. The problematic package started to install properly. https://readthedocs.org/projects/koalas/builds/9773902/

Screen Shot 2019-10-09 at 10 19 31 AM

@no-response no-response bot removed the Needed: more information A reply from issue author is required label Oct 9, 2019
@HyukjinKwon
Copy link
Author

Hi, I still face time out intermittently. Can I ask to increase time out please? I am handling flakiness of java-jdk=8 as well separately but this timeout affects the build more seriously.

@humitos
Copy link
Member

humitos commented Oct 11, 2019

Hi @HyukjinKwon! I increased time and memory for your project. I'm closing this issue. Feel free to re-open if you think it's needed. Also, let me know if it's building properly :)

@humitos humitos closed this as completed Oct 11, 2019
@HyukjinKwon
Copy link
Author

Thank you so much guys!

@HyukjinKwon
Copy link
Author

Screen Shot 2019-10-11 at 8 53 05 PM

Eh, sorry @humitos can you double check if the time is increased?

@HyukjinKwon
Copy link
Author

Also, seems like I can't reopen this issue :(.
Screen Shot 2019-10-11 at 8 54 01 PM

@stsewd
Copy link
Member

stsewd commented Oct 11, 2019

Last builds are passing now

@HyukjinKwon
Copy link
Author

Screen Shot 2019-10-13 at 11 18 01 AM

eh, seems still failing. Can I ask to increase the timeout?

@stsewd
Copy link
Member

stsewd commented Oct 13, 2019

Screenshot_2019-10-12 Builds Read the Docs

@HyukjinKwon
Copy link
Author

Thanks for checking it closely @stsewd

BTW, did the build in Read the docs decrease timeout lately?

Seems the builds started to fail again:

Screen Shot 2019-10-15 at 8 43 59 AM

Screen Shot 2019-10-15 at 7 41 32 AM

@stsewd
Copy link
Member

stsewd commented Oct 15, 2019

BTW, did the build in Read the docs decrease timeout lately?

No, we haven't. But looks like the conda step is taking a lot of the time

Do you need both channels?

https://github.com/databricks/koalas/blob/4022df3d97b3a9c12ed842cc1e7e0f43b100af56/dev/readthedocs-environment.yml#L6-L7

Conda gets slower depending on the number of channels.

Also, if you agree, we can try using the last version of conda for your project (there are some improvements to make it fast)

https://docs.readthedocs.io/en/stable/guides/feature-flags.html#available-flags

@HyukjinKwon
Copy link
Author

Let me give a shot to don't use both channels (opened a PR databricks/koalas#932). I will update you by EOD of tomorrow.

I will try UPDATE_CONDA_STARTUP as well.

However, I was wondering why it suddenly times out at 600 seconds instead of 900 seconds, which was used to be.

@stsewd
Copy link
Member

stsewd commented Oct 15, 2019

However, I was wondering why it suddenly times out at 600 seconds instead of 900 seconds, which was used to be.

Probably a bug in our code, sometimes if the command fails the command doesn't get updated on the db #4288

@HyukjinKwon
Copy link
Author

Thanks, @stsewd, I will wait for the fix and follow what you guided first, and see what happens.

HyukjinKwon added a commit to databricks/koalas that referenced this issue Oct 15, 2019
@stsewd
Copy link
Member

stsewd commented Oct 16, 2019

I have enabled the flag, but it stills times out. I'm increasing the build time to 15 min

@stsewd
Copy link
Member

stsewd commented Oct 16, 2019

Looks like is passing again, let me know if it keeps timing out

@HyukjinKwon
Copy link
Author

Awesome!

HyukjinKwon referenced this issue in databricks/koalas Apr 1, 2020
For a bit of background:
Currently documentation generation is being failed in Read the Docs due to timeout.
The largest portion of time is being consumed for Conda installation, and seems like Read the Docs maintainers also encourage people to just use PIP instead (see also `https://github.com/readthedocs/readthedocs.org/issues/6254#issuecomment-542288316`).
We use Conda installation in order to install JDK 8 because Read the Docs has JDK 11 by default, and Spark 2.x does not support JDK 11.

For what this PR proposes:
This PR switches Conda to PIP to install Java 8 via using `install-jdk` package. Seems like they use AdoptOpenJDK instead but the jobs looks passing see https://readthedocs.org/projects/koalas/builds/10741596/.
Once Spark 3.0 is released, we can remove all related codes as Spark 3.0 supports JDK 11, and we don't need to install JDK 8 manually.

This PR decreases build time 1000 seconds (timeout) to around 900 seconds.
rising-star92 added a commit to rising-star92/databricks-koalas that referenced this issue Jan 27, 2023
rising-star92 referenced this issue in rising-star92/databricks-koalas Jan 27, 2023
For a bit of background:
Currently documentation generation is being failed in Read the Docs due to timeout.
The largest portion of time is being consumed for Conda installation, and seems like Read the Docs maintainers also encourage people to just use PIP instead (see also `https://github.com/readthedocs/readthedocs.org/issues/6254#issuecomment-542288316`).
We use Conda installation in order to install JDK 8 because Read the Docs has JDK 11 by default, and Spark 2.x does not support JDK 11.

For what this PR proposes:
This PR switches Conda to PIP to install Java 8 via using `install-jdk` package. Seems like they use AdoptOpenJDK instead but the jobs looks passing see https://readthedocs.org/projects/koalas/builds/10741596/.
Once Spark 3.0 is released, we can remove all related codes as Spark 3.0 supports JDK 11, and we don't need to install JDK 8 manually.

This PR decreases build time 1000 seconds (timeout) to around 900 seconds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Support Support question
Projects
None yet
Development

No branches or pull requests

3 participants