-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Fix spurious failure in CI #11176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix spurious failure in CI #11176
Conversation
The failure can be seen below (thanks to @griggt): https://github.com/lampepfl/dotty/runs/1732363505?check_suite_focus=true On my local machine, I can reproduce the error both under Java 14 and Java 8. The error stacktrace is in TreeChecker. By reducing the size of the enum, we avoid such instability in test. [test_java8]
I guess my main question here is "is there a test suite that has reduced stack size that we can move the test to?" |
I'm not aware of that. After #11058 , the stacktrace is different: sometimes in Pickler test, sometimes in TreeChecker. I checked recent merges, we didn't change max stack memory. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have some reservations about merging this as-is.
I tested this locally in a variety of configurations today:
- with both JDK 8 and JDK 15
- using the original test (1000 enum cases) and the one proposed by this PR (500 cases)
- at the following merge commits:
- b50372e (before Fix #10174: avoid creating deep nesting union space #11058)
- a5ff84a (merge Fix #10174: avoid creating deep nesting union space #11058)
- c63bd9a (nightly CI of 19 Jan; pass)
- d8e4af7 (nightly CI of 20 Jan; fail)
My primary concern is that the reduced test case proposed by this PR passed in all configurations, including at the commit prior to #11058. It would seem to serve more as a defender of the status quo before #11058 than as a test of the improvements introduced by that PR and a regression test for #10174.
I found no difference in results between the JDK versions.
I also noticed some strange behavior that I am at a loss to explain; running
scala3-compiler-bootstrapped/testCompilation i10174
failed in every configuration with 1000 casesscala3-bootstrapped/test
passed in every configuration with 1000 cases
I don't understand how the results of these should differ with all else being the same. I was finally able to get scala3-bootstrapped/test
(which is what is run by CI) to fail by increasing the number of cases to 1500 (this was actually the first size I tried > 1000).
Also, this test is passing again in the recently completed nightly CI.
Thanks for the detailed feedback, @griggt ! We can view #10174 as a stress test case. I think there are two reasons to reduce its size:
To test the improvements in #11058, I've added a new test in a3156b4 (hopefully other parts can tolerate it). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The failure can be seen below (thanks to @griggt):
https://github.com/lampepfl/dotty/runs/1732363505?check_suite_focus=true
On my local machine, I can reproduce the error both under Java 14 and
Java 8. The error stacktrace is in TreeChecker.
By reducing the size of the enum, we avoid such instability in test.
[test_java8]