Reactive client is initialised with an invalid (null) database selection.

[mrksph]

Hi,

I'm experiencing issues when upgrading SDN from 6.0.3 (included with spring-data-starter-neo4j 2.4.2) to 6.2.1 (either via dependencyManagement or direct dependency)

When I run the app I get the following error message:

The provided database selection provider differs from the Neo4jClient's one.

If I try to upgrade to 6.1.x instead I get

The provided database selection provider differs from the ReactiveNeo4jClient's one.

[mrksph]

I think the first error might be related to the fact that SDN 6.2.x requires Spring 5.3.16 https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/#dependencies.spring-framework

But I can't find which version of Spring Framework is needed for SDN 6.1.X

Are there any plans on implementing template.saveAs() for SDN 6.0.X?

Thank you

[mrksph]

After trying to update Spring Boot 2.4.2 to 2.5.x and 2.6.x finally I made it work upgrading to Spring Boot 2.4.13 which brings Spring's spring-core 5.3.13 which seems enough to make SDN work.

Another issue I'm still experiencing is that my @DataNeo4jTest won't run as it fails to load Application Context with the same error as my first comment.

@DataNeo4jTest
@ActiveProfiles("test")
public class ThingRepositoryTest {

    @Autowired
    private ThingRepository repository;

    @Test
    public void test() {
        Optional<Thing> e = repository.findByCode("CODE");
        Assertions.assertNotNull(e.get());
    }
}

And this is the error message

The provided database selection provider differs from the ReactiveNeo4jClient's one.

Note how it tries to load the ReactiveNeo4jClient even though I'm using the imperative repositories.

Another thing to note is that it looks like it's ignoring my @activeprofiles("test) annotation because it is loading some configurations that only exist in my src/resources/application.yml file

[meistermeier]

Starting with 6.1.x we pushed the ability to use the database selection down to the Neo4jClient. It was only available in the Neo4jTemplate before and for the Neo4jClient you had to define the database in its fluent API.

I assume that you are defining them in several places. Most likely: You are defining the Neo4jTemplate yourself and providing the databaseSelection.

A reproducer with your / a similar configuration would be helpful.

Side note: I do not know what exactly happens in your configuration, but the ReactiveNeo4jClient message will only appear in the reactive Neo4jClient.

[mrksph]

Thanks for the reply, @meistermeier .

Yes, I'm autowiring the Neo4jTemplate in several places in our application. The thing is I only get that error when running our tests, even more so, when I only run that test because we also have some unit tests that run without any problems.
Also, when starting the application we don't get any errors.

Ok, I'll try to make a reproducer for you guys. I'll be updating this comment as soon as I have it ready.

Thank you.

[meistermeier]

Is there any progress on a reproducer or did you find the problem in your test code?

[mrksph]

Hi @meistermeier, thanks for reaching out.
We've been busy these days so we couldnt put much time into this issue.
We did try to reproduce the error making a project from scratch but unsurprisingly, it worked so we have to check which configuration might be messing things up.

[meistermeier]

I assume this really could have been just a configuration problem. If it's ok for you, I will close this issue for now. If you are facing it again and :) can give us something to reproduce it, feel free to re-open it.

[mrksph]

Hi @meistermeier,

Yesterday we made it work. We did so by explicitly excluding ReactiveNeo4jDataAutoconfiguration when running our Integration Tests using

@EnableAutoConfiguration(exclude= Neo4jReactiveDataAutoConfiguration.class)

I still don't know why the annotations we are using forces Spring to load ReactiveNeo4jDataAutoconfiguration along with other ReactiveNeo4j Beans, guess we'll have to take a deeper look but I think it's mainly because we're not using exclusively the required ones for IT testing.

Is there any sample project with IT tests + embedded database? So we can take a look and see how to configure ours properly

Thanks

[meistermeier]

I cannot provide you any guidelines. Not that we don't have any projects or examples in various repositories but it seems specific for you project.
The only thing, I can think of is that you are somehow creating a configuration bean or similar that uses parts of SDN's reactive API
OR
you have reactor on your classpath. The configuration gets triggered by our very own classes (incl. Driver) and the Flux being on the classpath:
https://github.com/spring-projects/spring-boot/blob/6d7b14a5a2f4555f2f083103c6e6d09a5cfb272b/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/data/neo4j/Neo4jReactiveDataAutoConfiguration.java#L45

[mrksph]

Hi,

Nope, we don't have any of those as dependencies.

I just asked a question in StackOverflow in the hope that someone can enlighten us as to why those beans get instantiated.
(https://stackoverflow.com/questions/71736083/springboottest-loads-unrequired-bean-when-making-it)

I've seen some of Michael Simons' and others' posts using only @SpringBootTest annotation but when I try to reproduce it, I still get the same error.

[mrksph]

After further inspecting, I see that we do have io.projectreactor:reactor-core as a indirect dependency.
I'll try to exclude it and test again.

[mrksph]

Ok, we got it working.

Apparently, neo4j-harness is the one to blame here, as it brings io.projectreactor as an indirect dependency through neo4j-fabric which is the root of our problems.

To fix it, we just had to exclude it from our pom just like this:

<!--  Enables testing Neo4j repositories in memory -->
<dependency>
    <groupId>org.neo4j.test</groupId>
    <artifactId>neo4j-harness</artifactId>
    <version>${neo4j-harness.version}</version>
    <scope>test</scope>
	<exclusions>
		<exclusion>
		    <groupId>org.slf4j</groupId>
		    <artifactId>slf4j-nop</artifactId>
		</exclusion>
		<exclusion>
		    <groupId>io.projectreactor</groupId>
		    <artifactId>*</artifactId>
		</exclusion>
	</exclusions>
</dependency>

Thank you @meistermeier for the clarification on how ReactiveNeo4jDataAutoConfiguration conditional instantiation works.

Thanks to that we could solve this!

[mrksph]

@michael-simons I'm following your blogpost https://medium.com/neo4j/testing-your-neo4j-based-java-application-34bef487cc3c where you suggest use neo4j-harness

The thing is, neo4j-harness depends on neo4j and neo4j depends on neo4j-fabric, which is bringing Reactor (Flux.class)
If SDN's Reactive AutoConfiguration always instantiates reactive beans if Flux is on the classpath, how can I avoid this behavior?

What I just tried in the previous comments didn't seem to work after re-adding the embedded database to our IT tests.

I thought that only excluding Reactor would work because it would be implemented just the way SDN is, separating imperative from reactive implementations but nope, here is a call to Hooks which belongs to Reactor project.

Any suggestions appreciated

[michael-simons]

Puh. You best bet is not using Harness, tbh.

Testcontainers is the better choice.
I can see if I can come up with something, but that's what happens when you pull in a whole DBMS into a test (not your fault, of course, the amount of dependencies in such a DBMS).

[mrksph]

Woah, thanks for the quick answer.

I'll take a look at it and see if it's possible to make the change. Last thing I remember we chose Harness over Testcontainers because the way our pipeline (Jenkins pipeline) is designed, it wouldn't allow us to spawn Docker containers inside it.

[michael-simons]

Yeah, I hear you. That is sometimes an issue.

I see the point to use an embedded instance… But I know the dependencies Neo4j (the database) comes with, and those are a lot.

Could you try if setting this property (either globally in your app or in test, it's a Spring Boot property) helps in your case?

spring.data.neo4j.repositories.type=imperative

haven't gone through this whole issue here from top to bottom…

[mrksph]

Nope, it just fails while trying to instantiate ReactiveNeo4jTemplate :(

[michael-simons]

Can you please share your setup? Config classes, and pom… Domain irrelevant.
In such a form that I can debug the failure. I'm happy to look at it, but I'm not gonna recreate a project myself.

[mrksph]

I've created a Gist containing our pom.xml, a Base class for our IT tests that instantiates the embedded db, and the concrete IT class that has a simple test that fails before being executed

It's basically a Spring Boot (2.4.13) + SDN (6.1.9) project, we don't make much apart from typical CRUD operations against Neo4j.

https://gist.github.com/mrksph/84d095dc39628a59faab2e3d9eb0c68f

[michael-simons]

So, indeed, Spring Boot does not let you disable the reactive clients config if Project reactor is on the classpath. This is their decision, but a decision I agree with.

2nd: It is indeed a bug on our side, but not a compatibility issue.
The bug has been introduced with the changes mentioned by @meistermeier #2488 (comment)

I will fix this later this week.

Here are two workaround in the meantime:

a)
On your Main-Application class

@SpringBootApplication(exclude = Neo4jReactiveDataAutoConfiguration.class)

OR

b)

package com.gs.scib.jquest.it;

import org.neo4j.driver.Driver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.neo4j.core.ReactiveDatabaseSelectionProvider;
import org.springframework.data.neo4j.core.ReactiveNeo4jClient;
import org.springframework.data.neo4j.repository.config.ReactiveNeo4jRepositoryConfigurationExtension;

@Configuration
public class FixInvalidConfigurationOfReactiveClient {

	@Bean(ReactiveNeo4jRepositoryConfigurationExtension.DEFAULT_NEO4J_CLIENT_BEAN_NAME)
	public ReactiveNeo4jClient reactiveNeo4jClient(Driver driver,
		ReactiveDatabaseSelectionProvider reactiveDatabaseSelectionProvider) {
		return ReactiveNeo4jClient.create(driver, reactiveDatabaseSelectionProvider);
	}
}

[michael-simons]

This will btw only happen on the combination of SDN 6.1+ with Spring Boot 2.4.

We fix this as a courtesy.

[mrksph]

Was just thinking about that as we are tied for now to Spring Boot 2.4 because compatibility with other Spring Cloud services we depend on.

Thank you guys. Appreciate your work.

[michael-simons]

You are welcome.

Btw, as I can't find a release date for 6.1.x, 6.2.3 will also work with Boot 2.4 with a superficial check (your reproducer) and the workaround applied to imperative, too:

@Configuration
public class FixInvalidConfigurationOfReactiveClient {

	@Bean
	public ReactiveNeo4jClient reactiveNeo4jClient(Driver driver,
		ReactiveDatabaseSelectionProvider reactiveDatabaseSelectionProvider) {
		return ReactiveNeo4jClient.create(driver, reactiveDatabaseSelectionProvider);
	}

	@Bean
	public Neo4jClient neo4jClient(Driver driver,
		DatabaseSelectionProvider databaseSelectionProvider) {
		return Neo4jClient.create(driver, databaseSelectionProvider);
	}
}

at least in 6.2x we broke it consistently.

Sorry about that, the area was in too much flux (no pun intended) since we developed that part.

[mrksph]

Awesome, thanks Michael!
Yeah sometimes too much flux isn't good either I suppose :)