Use less threads than we have cores on Jenkins, increase max heap

[smarter]

Using less threads should allow us to increase the maximum heap size
without filling up the memory of the Jenkins instance

[DarkDimius]

Sorry, this diff is unavailable.

?

As @odersky correctly pointed out, the threads live in the process and share the same heap. Additionally, scalac runs on the same infrastructure with very same settings: 1 thread per cpu.
By reducing number of threads from 8 to 7(on 8 core instance), we get only slightly more memory(250 MB) per thread.

I have a feeling that we are fighting symptoms, instead of trying to find the reason why memory consumption has spiked recently. Dotty used to be able to compile itself in 800mb heap in early December, without struggling with GCs.

[smarter]

I have a feeling that we are fighting symptoms, instead of trying to find the reason why memory consumption has spiked recently

I agree that we should do that too, but in the meantime being able to have the tests pass would be great.

[DarkDimius]

Sorry, this diff is unavailable.

I still cannot see diffs of commits in this PR.

[smarter]

I still cannot see diffs of commits in this PR.

Yes, something is broken on github's side, I've seen this on other PRs too, I've sent a message to github support.

[smarter]

Here's a trick that seems to work: you can add .patch at the end of a github commit URL to get the commit in patch format, and the patch seems to be available even if the HTML page isn't, for example
dotty-staging@7dbf632 doesn't work but https://github.com/dotty-staging/dotty/commit/7dbf632f5d2c4f6f5c30a7435b9456b711e91bf2.patch does.

[DarkDimius]

• else if (isJenkins)
• sys.props("partest.threads") = ((numCores - 1) max 1).toString

I propose we do not increase number of things that work differently on CI. WDYT?

[smarter]

Additionally, scalac runs on the same infrastructure with very same settings: 1 thread per cpu.

But what happens when several PRs are tested at the same time on the same virtual machine? For example here https://scala-ci.typesafe.com/job/dotty-master-validate-partest/956/ and https://scala-ci.typesafe.com/job/dotty-master-validate-partest/957/ are run on jenkins-worker-behemoth-1 which I guess is one virtual machine, shouldn't the jobs be queued so that we only run one job at a time?

[smarter]

OK, even dividing the number of threads used by 2 doesn't seem to be enough to avoid exhausting the RAM, so closing this.