Skip to content

Fix spurious failure in CI #11176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 21, 2021
Merged

Fix spurious failure in CI #11176

merged 2 commits into from
Jan 21, 2021

Conversation

liufengyun
Copy link
Contributor

@liufengyun liufengyun commented Jan 20, 2021

The failure can be seen below (thanks to @griggt):

https://github.com/lampepfl/dotty/runs/1732363505?check_suite_focus=true

On my local machine, I can reproduce the error both under Java 14 and
Java 8. The error stacktrace is in TreeChecker.

By reducing the size of the enum, we avoid such instability in test.
[test_java8]

The failure can be seen below (thanks to @griggt):

https://github.com/lampepfl/dotty/runs/1732363505?check_suite_focus=true

On my local machine, I can reproduce the error both under Java 14 and
Java 8. The error stacktrace is in TreeChecker.

By reducing the size of the enum, we avoid such instability in test.

[test_java8]
@bishabosha
Copy link
Member

bishabosha commented Jan 20, 2021

in the issue #10174 i could reduce it to 700 cases but any significant reduction below that would not cause the stack overflow, (with standard settings) do we have any reduced stack size tests?

I guess my main question here is "is there a test suite that has reduced stack size that we can move the test to?"

@liufengyun
Copy link
Contributor Author

I guess my main question here is "is there a test suite that has reduced stack size that we can move the test to?"

I'm not aware of that. After #11058 , the stacktrace is different: sometimes in Pickler test, sometimes in TreeChecker. I checked recent merges, we didn't change max stack memory.

Copy link
Contributor

@griggt griggt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some reservations about merging this as-is.

I tested this locally in a variety of configurations today:

My primary concern is that the reduced test case proposed by this PR passed in all configurations, including at the commit prior to #11058. It would seem to serve more as a defender of the status quo before #11058 than as a test of the improvements introduced by that PR and a regression test for #10174.

I found no difference in results between the JDK versions.

I also noticed some strange behavior that I am at a loss to explain; running

  • scala3-compiler-bootstrapped/testCompilation i10174 failed in every configuration with 1000 cases
  • scala3-bootstrapped/test passed in every configuration with 1000 cases

I don't understand how the results of these should differ with all else being the same. I was finally able to get scala3-bootstrapped/test (which is what is run by CI) to fail by increasing the number of cases to 1500 (this was actually the first size I tried > 1000).

Also, this test is passing again in the recently completed nightly CI.

@griggt griggt assigned liufengyun and unassigned griggt Jan 21, 2021
@liufengyun
Copy link
Contributor Author

liufengyun commented Jan 21, 2021

Thanks for the detailed feedback, @griggt !

We can view #10174 as a stress test case. I think there are two reasons to reduce its size:

  • Many parts of the compiler (including Pickler and TreeChecker) cannot tolerate such a size reliably
  • It causes instability in the CI

To test the improvements in #11058, I've added a new test in a3156b4 (hopefully other parts can tolerate it).

Copy link
Member

@bishabosha bishabosha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bishabosha bishabosha merged commit 6d9e101 into scala:master Jan 21, 2021
@bishabosha bishabosha deleted the fix-ci branch January 21, 2021 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants