Skip to content

PR validation has been failing (for some weeks) since Scalariform switched from maven to sbt #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SethTisue opened this issue Jan 13, 2016 · 22 comments
Labels

Comments

@SethTisue
Copy link
Member

and Scalariform is part of https://github.com/scala-ide/uber-build

example failure: https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/2026/console

@SethTisue SethTisue self-assigned this Jan 13, 2016
@SethTisue SethTisue changed the title PR validation has been failing (for some weeks) since ScalaIDE switched from maven to sbt PR validation has been failing (for some weeks) since Scalariform switched from maven to sbt Jan 13, 2016
@SethTisue
Copy link
Member Author

@sschaef writes:

I just updated the builds for scala-refactoring, scalariform and scala-ide. They now rely on a working sbt installation (instead of maven only). [...] I guess your Jenkins installation has already access to sbt. Can you please tell me how to run 0.13.x sbt on your Jenkins? Is it available through the sbt command or through another one?

@SethTisue
Copy link
Member Author

@sschaef the relevant scripts that may need updating are https://github.com/scala-ide/uber-build/blob/master/uber-build.sh and possibly https://github.com/scala/scala/blob/2.11.x/scripts/jobs/integrate/ide

Can you please tell me how to run 0.13.x sbt on your Jenkins?

scripts/jobs/integrate/ide receives the right sbt command to use in the sbtCmd environment variable, as provided by https://github.com/scala/scala-jenkins-infra/blob/c50554e96637ef9f1e2499dfa1d6caf8e21839ef/attributes/worker.rb#L118

to make local testing easier, in scripts we typically write ${sbtCmd-sbt} so even if sbtCmd isn't set you get a reasonable default.

@kiritsuku
Copy link

Alright, I'm going to try that out.

@SethTisue
Copy link
Member Author

So that Scala PR validation doesn't remain affected, and so there won't be any huge rush on a fix, I'm going to temporarily remove integrate-ide from integrate-main. You should still be able to test your changes by manually triggering jobs at https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/ (at least, I don't know of any reason that wouldn't work, since I don't have firsthand experience with this Jenkins job in particular, but it's certainly something we routinely do with our other job).

SethTisue added a commit to SethTisue/scala-jenkins-infra that referenced this issue Jan 13, 2016
should be reinstated once the ScalaIDE folks have
tackled scala/scala-dev#71
SethTisue added a commit to SethTisue/scala-jenkins-infra that referenced this issue Jan 13, 2016
should be reinstated once the ScalaIDE folks have
tackled scala/scala-dev#71
@SethTisue SethTisue removed their assignment Jan 13, 2016
@dragos
Copy link

dragos commented Jan 14, 2016

@sschaef I guess this involves the uber-build, and there's a validator configuration especially for this purpose. I guess that one should pass locally before you need to test it on Jenkins.

Because of these hidden dependencies I'm always wary of build changes, so if this is too much I'd say we go back to maven. Neither refactoring nor scalariform have complicated builds, so the pain will quickly outweight any benefits.

@kiritsuku
Copy link

Yeah, for the next time I'll remember this. The good side of it is that I understand the build chain better every time better when something breaks.

@SethTisue
Copy link
Member Author

one possible consideration: the Scala community build expects sbt. (uh, well maybe dbuild has some maven support? even if it does, I doubt it's actively maintained)

@dragos
Copy link

dragos commented Jan 19, 2016

@sschaef did you manage to track down the regression? It may be difficult to turn the validation back on, since regressions may creep into Scala

@kiritsuku
Copy link

Yes it is nearly fixed. I'm going to fix it completely probably today or tomorrow.

@kiritsuku
Copy link

@SethTisue I fixed the 2.11.x build locally. I can upload my changes but first I would like to understand how exactly scala-ide gets the Scala dependency. Could you show me how the script of this Jenkins job https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/ runs uber-build?

@SethTisue
Copy link
Member Author

I guess you figured it out and no longer have a question? but I'll answer anyway, there is a one-to-one correspondence between Jenkins job names and files in scripts/jobs, so integrate-ide is just running https://github.com/scala/scala/blob/2.11.x/scripts/jobs/integrate/ide

@kiritsuku
Copy link

Thanks, that is helpful. I already did a test run but it failed at some other point. Still takes me a little bit longer.

@kiritsuku kiritsuku reopened this Jan 20, 2016
@kiritsuku
Copy link

The sbt build works now but it can't find the Scala dependency: https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/2028/console

Maybe the dependency is already gone. Are the Scala dependencies cached or are they removed after a build has been finished?

@SethTisue
Copy link
Member Author

@sschaef In that failure log, I don't see sbt looking for the Scala artifacts where it should be, namely on https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/ (where they are archived indefinitely afaik)

Note that when the integrate/ide script runs uber-build.sh, it sets both prRepoUrl and IDE_M2_REPO to that URL. (Why two variables with the same value, I don't know.) The relevant lines are https://github.com/scala/scala/blob/c06aed997f5a0c98434d7cec0ebe5f0da3de790e/scripts/jobs/integrate/ide#L24-L27 . So I guess you need to change uber-build.sh so that it passes that information on to sbt?

@kiritsuku
Copy link

Oh, you are right. I completely oversaw this part. In that case I only have to add them through a resolver.

@kiritsuku
Copy link

Scala 2.11.x CI is back on green: https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/2034/

Nevertheless, I had to disable two tests because they fail on Scala Jenkins but not on Scala IDE Jenkins. I'm going to look into that next week. If in the next days I can't find out why the tests are failing, we can let them on ignored status and reactivate scala-ide integration in order to avoid potential breaking changes.

SethTisue added a commit to scala/scala-jenkins-infra that referenced this issue Jan 22, 2016
…main"

This reverts commit 46b6aa1.

at scala/scala-dev#71 , @sschaef says
the coast should be clear now
@SethTisue
Copy link
Member Author

I re-added integrate-ide to integrate-main and deployed the change. optimistically closing this ticket. thanks @sschaef!

@kiritsuku
Copy link

In order to be more safe in future, would you please add me to the email notifications about failed builds on the scala-2.11.x-integrate-ide job? I don't have permissions to add myself. The email is [email protected]

@SethTisue
Copy link
Member Author

would you please add me to the email notifications about failed builds on the scala-2.11.x-integrate-ide job

I don't have manager rights on https://groups.google.com/a/typesafe.com/forum/#!forum/pr-scala-integrate-ide, but I believe @dragos does

@kiritsuku
Copy link

Haha, that is cool. I didn't know the Scala Jenkins works so differently to the Scala IDE Jenkins. Even more interesting is that so many things can be plugged into Jenkins. Some kind of powerful software.

@dragos
Copy link

dragos commented Jan 22, 2016

On 22 ian. 2016, at 17:07, Simon Schäfer [email protected] wrote:

Scala 2.11.x CI is back on green: https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/2034/

Nevertheless, I had to disable two tests because they fail on Scala Jenkins but not on Scala IDE Jenkins. I'm going to look into that next week. If in the next days I can't find out why the tests are failing, we can let them on ignored status and reactivate scala-ide integration in order to avoid potential breaking changes.

How can we make sure that they are not regressions in the 2.12 code base?


Reply to this email directly or view it on GitHub.

@kiritsuku
Copy link

On 01/22/2016 07:22 PM, Iulian Dragos wrote:

How can we make sure that they are not regressions in the 2.12 code base?
Some of the tests that fail on Scala Jenkins also fail on my computer.
That is why I don't believe they are regressions. Also, I don't think
scalac changes can break hot code replacement in the IDE but of course
that is no guarantee that there are no regressions. I'm going to find
out more about it next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants