Propagate Bean `ClassLoader` to `MongoTypeMapper` #3905

chadshowalter · 2021-12-07T13:19:39Z

Description

When I read a document using a repository, if the document has an abstract field with a concrete implementation and the first read of the document happens in a ForkJoinPool thread, the read fails with the following stacktrace:

java.util.concurrent.ExecutionException: org.springframework.data.mapping.model.MappingInstantiationException: Failed to instantiate com.example.demo.AbstractBase using constructor public com.example.demo.AbstractBase() with arguments
	at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006) ~[na:na]
	at com.example.demo.DocService.readDocs(DocService.java:42) ~[classes!/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) ~[spring-beans-5.3.13.jar!/:5.3.13]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.13.jar!/:5.3.13]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.13.jar!/:5.3.13]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:765) ~[spring-boot-2.5.7.jar!/:2.5.7]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:445) ~[spring-boot-2.5.7.jar!/:2.5.7]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) ~[spring-boot-2.5.7.jar!/:2.5.7]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-2.5.7.jar!/:2.5.7]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-2.5.7.jar!/:2.5.7]
	at com.example.demo.DemoApplication.main(DemoApplication.java:10) ~[classes!/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[demo-0.0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[demo-0.0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[demo-0.0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[demo-0.0.1-SNAPSHOT.jar:na]
Caused by: org.springframework.data.mapping.model.MappingInstantiationException: Failed to instantiate com.example.demo.AbstractBase using constructor public com.example.demo.AbstractBase() with arguments
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator$MappingInstantiationExceptionEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:321) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:89) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:367) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readDocument(MappingMongoConverter.java:340) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$ConversionContext.convert(MappingMongoConverter.java:1932) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1640) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$AssociationAwareMongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1689) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$AssociationAwareMongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1652) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:74) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mapping.model.SpELExpressionParameterValueProvider.getParameterValue(SpELExpressionParameterValueProvider.java:53) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.extractInvocationArguments(ClassGeneratingEntityInstantiator.java:276) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter.createInstance(ClassGeneratingEntityInstantiator.java:248) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:89) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:367) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readDocument(MappingMongoConverter.java:340) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:276) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:272) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:101) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:3178) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2813) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2543) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2525) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:847) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findAll(SimpleMongoRepository.java:428) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findAll(SimpleMongoRepository.java:150) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findAll(SimpleMongoRepository.java:57) ~[spring-data-mongodb-3.2.7.jar!/:3.2.7]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:289) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:137) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:121) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:529) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:599) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.13.jar!/:5.3.13]
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:163) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:138) ~[spring-data-commons-2.5.7.jar!/:2.5.7]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.13.jar!/:5.3.13]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-5.3.13.jar!/:5.3.13]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.13.jar!/:5.3.13]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.13.jar!/:5.3.13]
	at com.sun.proxy.$Proxy45.findAll(Unknown Source) ~[na:na]
	at com.example.demo.DocService.lambda$readDocs$1(DocService.java:37) ~[classes!/:na]
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1409) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[na:na]
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.example.demo.AbstractBase]: Class is abstract
	... 51 common frames omitted

Steps to Reproduce

I have created a project on github that illustrates the issue.

Check out the repository (https://github.com/chadshowalter/forkjoin-mongodb) and cd to the repo root
Build the jar: gradlew bootJar
Ensure mongodb is running and available at localhost:27017 (or alternately, provide cl args when running for an available mongodb)
Run the built app: java -jar build/libs/demo-0.0.1-SNAPSHOT.jar
Observe the problem
Run the built app using a non ForkJoinPool executor: java -jar build/libs/demo-0.0.1-SNAPSHOT.jar --demo.pool.forkjoin=false
Read happens without exception in this case

Environment I'm Using

OSX 11.6
mongodb 4.2.10
Spring Boot 2.5.7
java:

openjdk 15.0.2 2021-01-19
OpenJDK Runtime Environment Corretto-15.0.2.7.1 (build 15.0.2+7)
OpenJDK 64-Bit Server VM Corretto-15.0.2.7.1 (build 15.0.2+7, mixed mode, sharing)

Observations

I believe the issue is due to the classloader available when using ForkJoinPool. I do not consistently see the problem when running in my IDE (IntelliJ 2021.2.2), but I do consistently see it when running the built app. I can work around the issue using a custom ForkJoinWorkerThreadFactory, e.g.

public class MyForkJoinWorkerThreadFactory implements ForkJoinPool.ForkJoinWorkerThreadFactory {

     @Override
    public final ForkJoinWorkerThread newThread(ForkJoinPool pool) {
        return new MyForkJoinWorkerThread(pool);
    }

    private static class MyForkJoinWorkerThread extends ForkJoinWorkerThread {

        private MyForkJoinWorkerThread(final ForkJoinPool pool) {
            super(pool);
            // set the classloader here
            setContextClassLoader(Thread.currentThread().getContextClassLoader());
        }
    }
}

The text was updated successfully, but these errors were encountered:

mp911de · 2021-12-08T13:45:38Z

This issue is indeed related to the class loader that is being used. SimpleTypeInformationMapper tries to load the class that is used as a type hint.

Running the packaged jar uses Spring Boot code to create a class loader for the packaged jar files. Running the project from the IDE comes with the JDK's app-classloader that has already all jars on the class path.

Specifically, obtaining the default classloader from the Runnable returns jdk.internal.loader.ClassLoaders$AppClassLoader for ForkJoinPool. Executors.newFixedThreadPool retains the Spring Boot classloader org.springframework.boot.loader.LaunchedURLClassLoader.

When you look into ThreadFactory of the individual executor services, you'll see that ThreadPoolExecutor doesn't set any class loaders while DefaultForkJoinWorkerThreadFactory uses ClassLoader.getSystemClassLoader() to set the class loaders for newly created threads.

Likely the difference is motivated by the assumption of a single ForkJoinPool that gets typically configured through system properties.

You can supply a custom ForkJoinWorkerThreadFactory if you want to use a custom fork-join thread pool.

Alternatively, annotate your concrete classes with a type alias annotation @TypeAlias("com.example.demo.Concrete1") and that will pre-register the classes upon startup with the MappingContext.

Paging @wilkinsona for awareness.

wilkinsona · 2021-12-08T14:05:35Z

Thanks, @mp911de. This is due to a somewhat unfortunate change in Java 9 that we've seen a few times before. See spring-projects/spring-boot#19427, for example.

chadshowalter · 2021-12-08T17:06:15Z

Thanks. Too bad this inconsistency is there. It's a bit of a booby trap.

I had not picked up on the ability to work around this issue with the @TypeAlias annotation. Thanks for pointing that out.

I also observe that it requires some extra work in the build to set ForkJoinPool.commonPool()'s threadFactory, e.g.,

java -Djava.util.concurrent.ForkJoinPool.common.threadFactory=com.example.demo.MyForkJoinWorkerThreadFactory -jar build/libs/demo-0.0.1-SNAPSHOT.jar

does not work unless the custom factory class is at the root of the jar. But it looks like the @TypeAlias workaround works with either a custom or the common ForkJoinPool.

mp911de · 2021-12-08T19:05:32Z

I wonder whether we could default to the Spring ClassLoader in SimpleTypeInformationMapper. I'm going to create a ticket for it as falling back to the contextual class loader creates quite some inconvenience.

mp911de · 2021-12-08T19:08:04Z

See spring-projects/spring-data-commons#2508

chadshowalter · 2021-12-08T20:02:09Z

Thanks! 👍

mp911de · 2021-12-09T10:24:42Z

Reopening the ticket as we can use the container's ClassLoader instead of falling back to the contextual class loader to avoid issues if the entire stack is container-managed.

We now set the ClassLoader from the ApplicationContext to the type mapper to ensure the type mapper has access to entities. Previously, `SimpleTypeInformationMapper` used the contextual class loader and that failed in Fork/Join-Pool threads such as parallel streams as ForkJoinPool uses the system classloader. Running e.g. a packaged Boot application sets up an application ClassLoader that has access to packaged code while the system ClassLoader does not. Also, consistently access the MongoTypeMapper through its getter. Closes #3905

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Dec 7, 2021

mp911de closed this as completed Dec 8, 2021

mp911de added status: invalid An issue that we don't feel is valid and removed status: waiting-for-triage An issue we've not yet triaged labels Dec 8, 2021

mp911de mentioned this issue Dec 8, 2021

Loading data fails with "Specified class is an interface" when invoking repository from worker thread #3908

Closed

mp911de changed the title ~~Reading documents with abstract types may fail when using ForkJoinPool~~ Propagate Bean ClassLoader to DefaultTypeInformationMapper Dec 9, 2021

mp911de added type: enhancement A general enhancement and removed status: invalid An issue that we don't feel is valid labels Dec 9, 2021

mp911de added this to the 3.3.1 (2021.1.1) milestone Dec 9, 2021

mp911de reopened this Dec 9, 2021

mp911de changed the title ~~Propagate Bean ClassLoader to DefaultTypeInformationMapper~~ Propagate Bean ClassLoader to MongoTypeMapper Dec 9, 2021

mp911de self-assigned this Dec 9, 2021

mp911de closed this as completed in 75999d9 Dec 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Propagate Bean `ClassLoader` to `MongoTypeMapper` #3905

Propagate Bean `ClassLoader` to `MongoTypeMapper` #3905

chadshowalter commented Dec 7, 2021

mp911de commented Dec 8, 2021 •

edited

Loading

wilkinsona commented Dec 8, 2021 •

edited

Loading

chadshowalter commented Dec 8, 2021

mp911de commented Dec 8, 2021

mp911de commented Dec 8, 2021

chadshowalter commented Dec 8, 2021

mp911de commented Dec 9, 2021

Propagate Bean ClassLoader to MongoTypeMapper #3905

Propagate Bean ClassLoader to MongoTypeMapper #3905

Comments

chadshowalter commented Dec 7, 2021

Description

Steps to Reproduce

Environment I'm Using

Observations

mp911de commented Dec 8, 2021 • edited Loading

wilkinsona commented Dec 8, 2021 • edited Loading

chadshowalter commented Dec 8, 2021

mp911de commented Dec 8, 2021

mp911de commented Dec 8, 2021

chadshowalter commented Dec 8, 2021

mp911de commented Dec 9, 2021

Propagate Bean `ClassLoader` to `MongoTypeMapper` #3905

Propagate Bean `ClassLoader` to `MongoTypeMapper` #3905

mp911de commented Dec 8, 2021 •

edited

Loading

wilkinsona commented Dec 8, 2021 •

edited

Loading