Upgrade to Groovy 3.0.0

[dreis2211]

Hi,

as Groovy 3.0.0 is finally released, I was wondering if Spring-Boot should support it. I initially hoped that the upgrade would be easy enough, but there are some things that seem to be not working when I upgrade it. I have a WIP branch where you can see the changes I made so far: master...dreis2211:groovy-3.0.0

Specifically, the Spock framework worries me a bit, because we pull in org.spockframework:spock-core:1.3-groovy-2.5. Also the other tests that seem to fail seem to require further investigation and might needs work on Spring-Framework side (see the bean test that I disabled).

Given that Groovy 2.5.x doesn't really support JDK 14, I thought it might be a good point in time to discuss this as 2.3.0 is not out yet, but on the other hand it's already quite late in its development.

Anyhow, I wanted to start a discussion on this. ;-)

Cheers,
Christoph

[snicoll]

Thanks Christoph, you're beating me to it again :-)

I was discussing this with @jhoeller yesterday and I assumed the upgrade on the framework side was completed (at least locally). Will investigate a bit more.

[dreis2211]

It might on Boot side though. I just couldn't find anything on Boot side, yet. And as the beans support is in the Framework I was thinking that it might needs investigation there. Spring-Framework is only using 3.x if a custom java home is set, so in the end it's not really included, just on the CI as far as I got it. See spring-projects/spring-framework@d521d37

[dreis2211]

spock-2.0-M2 has Groovy 3.x support, but I doubt there will be a final release in time. Nonetheless, updated my branch with it and that fixes some more stuff.

[jhoeller]

As far as we are testing it, all core Spring Framework tests pass with Groovy 3.0 on JDK 14. Granted, there may be subtle issues with our standard binaries (compiled against Groovy 2.5.9 on JDK 8) running against Groovy 3.0; maybe that disabled bean test in Boot in one such case. If you could track it down from Boot's side, I'd be happy to consider specific adaptations at core framework level.

We intend to fully upgrade to Groovy 3.0 for Spring Framework 5.3 later this year. For the time being, it'd be great to remain compatible with both Groovy 2.5 and 3.0 at runtime.

[dreis2211]

I think I nailed it down to the changes made in https://issues.apache.org/jira/browse/GROOVY-8399 for the ImportCustomizer. It seems that they cause the import processing to be skipped in cases where it shouldn't be skipped. Why exactly this is only happening when the beans DSL is involved is not yet 100% clear to me, though. The naive approach of copying the old mechanism to Boot's SmartImportCustomizer fixes the failing bean test.

[dreis2211]

The groovy-3.0.0 branch should now be green (at least it is locally).

[dreis2211]

I wonder if there's value in communicating the ImportCustomizer issues to the Groovy team. I tried isolating an example, but wasn't successful. What do you think?

[wilkinsona]

Thanks for investigating the ImportCustomizer problem, @dreis2211. I think it's worth bringing it to the Groovy team's attention.

@paulk-asert We've noticed a behaviour change in Groovy 3.0, although we've been unable to isolate the cause thus far. @dreis2211 has kindly dug a bit and believes it may be related to the changes made for GROOVY-8399. Can we please collaborate on identifying and hopefully fixing the problem?

[paulk-asert]

Sure, we'd be happy to help. Getting an isolated reproducer would be ideal but I can see that hasn't happened yet. The next best thing would be to describe in some detail what is going on in your scenario that is causing the problem with GROOVY-8399. I'd suggest raising a new issue in the GROOVY issue tracker with those details. If we can understand what cases are being skipped, we should be able to work out a fix.

[dreis2211]

I will investigate a bit further, @wilkinsona if you don't mind and report back if I could find something.

[dreis2211]

I've finally managed to reproduce it in a minimal example. See https://github.com/dreis2211/groovy3-import-bug

I would open a new issue at Groovy if that's okay for you, @wilkinsona & @paulk-asert ?

[paulk-asert]

Sounds good to me.

[dreis2211]

Here we go: https://issues.apache.org/jira/browse/GROOVY-9407

[dreis2211]

I've applied the suggestion in the Groovy ticket above, which seems to work around the behaviour change. At least it's better than copying the old ImportCustomizer logic to SmartImportCustomizer.

[lpandzic]

Any idea when this might get resolved? JDK 14 is GA and it doesn't work with current version of groovy specified in spring-boot-dependencies.

[dreis2211]

@lpandzic Which version are you referring to? 2.5.10?
Apart from that the only thing missing here is a Spock release that aligns with Groovy 3.x. I have everything else ready.

[lpandzic]

In Spring Boot 2.2.5.RELEASE groovy.version is 2.5.9

[dreis2211]

The current snapshots of 2.3.0 build already against 2.5.10, which enables (basic) JDK 14 support.

[snicoll]

@lpandzic same conversation as the one we had recently on Gitter. 2.5.10 was released after Spring Boot 2.2.5. We'll pick it up in due course. JDK 14 official support is scheduled with Spring Boot 2.3. In general, you shouldn't expect a maintenance release of Spring Boot to be (officially) compatible with a newer JDK.

[jonathanlermitage]

Switching to Groovy 3 would fix issues with some libraries that rely on Groovy 3, like RestAssured (e.g. rest-assured/rest-assured#1283 (comment))

[dreis2211]

I wonder if there's a way forward without having a Spock release with Groovy 3 support.

[snicoll]

@jonathanlermitage I don't think Spring Boot is holding you up there. You can override the groovy.version property to use Groovy 3.

[drewtwitchell]

Has Spock been upgraded to leverage Groovy 3? Would love to see this fixed.

[dreis2211]

No, unfortunately there is no release version of Spock with Groovy 3 support. There is only a milestone available. Like @snicoll mentioned before, for most use cases it's perfectly reasonable (and possible) to override the groovy.version property if you need it, @drewtwitchell .

[dreis2211]

Maybe @szpak or any of the other members of the Spock team can shed some light on the release plans for a 2.0 GA release of Spock.

[szpak]

Spock 2.0-M3 will be released "soon" (with a bunch of really nice features ;-) ). Unfortunately. there is no schedule for GA.

Using Spock 1.3 with Groovy 3 is unsupported, but what's more important there is no simply way to run the non-SNAPSHOT version of Spock 1.3 with Groovy 3 due to the sanity check in Spock (many people using Spock 1.3-SNAPSHOT with Groovy 3 reports it generally works fine). I implemented a way to override that restriction in the future (e.g. for Groovy 4), but it will be available only in Spock 2.0-M3+.

[wilkinsona]

Once we've upgraded, we should revert 452fbf3 and upgrade REST Assured as well.

[dreis2211]

Just to mention it once. Is it maybe an option to drop support for Spock? Or depend on the milestone releases? It's now blocking Groovy & REST Assured.

[wilkinsona]

I was just typing the below when your comment popped up, @dreis2211.

I'm getting increasingly tempted to proceed with this as it feels a bit like the tail is wagging the dog at the moment. We don't have any public dependency management for Spock and we only have a testImplementation dependency on it (in spring-boot-test). Spock's milestones are published to Central so there's nothing technical stopping us from upgrading.

If we did upgrade to Groovy 3.0, users of Spock would have two choices:

1. Upgrade to the latest Spock 2.0 milestone
2. Stick with Spock 1.3 and downgrade Boot's Groovy version to 2.5.x.

Looking at the changes in the upgrade branch, only those in the CLI's GroovyCompiler would appear to require Groovy 3.0. I think we could probably live with that as the CLI controls its Groovy version anyway.

Let's see what the rest of the team thinks.

[philwebb]

We've decided not to wait for Spock. @dreis2211 Are you interested in submitting a PR for this based on the work you've already done?

[dreis2211]

Surely. But to get this straight: Should I upgrade Boot to Spock's Milestone or should I get rid of the few internal Spock usages?

[wilkinsona]

Thank you. Upgrade Boot to use Spock's latest 2.0 milestone please, Christoph. IIRC, the dependency is there to test that we detect Spock test annotations correctly when figuring out the context cache key. That's functionality that it would be good to keep if we can.

[dreis2211]

Hence my question...I can't promise it this week still, but feel free to assign it to me. I'll provide something next week the latest. In case you need it for 2.5.0-M1 I could try to pull it off tomorrow.

[philwebb]

There's no rush to get it into M1 from our side, we can merge it whenever.

[philwebb]

Closing in favor of PR #24946