Optimize SystemEnvironmentPropertyMapper.isLegacyAncestorOf

[avnersin]

demo.zip

are checking the performance impact of adding Consumer beans with spring-cloud-function.
We added 100 Consumer beans and configured them in spring.cloud.function.definition and under spring.cloud.stream.function.bindings.

We see that the initialization time of the BindingServiceProperties bean increased from 59 milliseconds for a single topic to 5581 milliseconds for 100 topics.

We profiled the application using JProfiler.
We see the top hotspot in:

org.springframework.boot.context.properties.source.ConfigurationPropertyName$ElementsParser.parse

Most calls are coming from:

org.springframework.boot.context.properties.source.SystemEnvironmentPropertyMapper.isLegacyAncestorOf

Debugging it, we see that isLegacyAncestorOf is called many times for the same name (each time with a different candidate).
But buildLegacyCompatibleName depends only on the name, not on the candidate it checks against.
Since buildLegacyCompatibleName is relatively time-consuming, we think it makes sense to cache it per input name for optimization.
Or possibly, some caching can help at a higher level in the call stack of Binder.bindObject (see the stacktrace below).

Attaching a demo application that shows this behavior on startup.

Thanks, Avner.

Stacktrace

main@1" prio=5 tid=0x1 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at org.springframework.boot.context.properties.source.SystemEnvironmentPropertyMapper.buildLegacyCompatibleName(SystemEnvironmentPropertyMapper.java:124)
	  at org.springframework.boot.context.properties.source.SystemEnvironmentPropertyMapper.isLegacyAncestorOf(SystemEnvironmentPropertyMapper.java:119)
	  at org.springframework.boot.context.properties.source.SystemEnvironmentPropertyMapper.isAncestorOf(SystemEnvironmentPropertyMapper.java:112)
	  at org.springframework.boot.context.properties.source.SystemEnvironmentPropertyMapper$$Lambda$162/0x0000000125119ee8.test(Unknown Source:-1)
	  at java.util.function.BiPredicate.lambda$or$2(BiPredicate.java:105)
	  at java.util.function.BiPredicate$$Lambda$163/0x00000001250fdfe0.test(Unknown Source:-1)
	  at org.springframework.boot.context.properties.source.SpringIterableConfigurationPropertySource.containsDescendantOf(SpringIterableConfigurationPropertySource.java:136)
	  at org.springframework.boot.context.properties.bind.Binder.containsNoDescendantOf(Binder.java:509)
	  at org.springframework.boot.context.properties.bind.Binder.bindObject(Binder.java:398)
	  at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:350)
	  at org.springframework.boot.context.properties.bind.Binder.lambda$bindDataObject$5(Binder.java:477)
	  at org.springframework.boot.context.properties.bind.Binder$$Lambda$171/0x00000001251242e8.bindProperty(Unknown Source:-1)
	  at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:100)
	  at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:88)
	  at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:64)
	  at org.springframework.boot.context.properties.bind.Binder.lambda$bindDataObject$6(Binder.java:480)
	  at org.springframework.boot.context.properties.bind.Binder$$Lambda$175/0x0000000125124738.apply(Unknown Source:-1)
	  at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	  at java.util.AbstractList$RandomAccessSpliterator.tryAdvance(AbstractList.java:706)
	  at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
	  at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
	  at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
	  at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	  at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	  at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
	  at org.springframework.boot.context.properties.bind.Binder.fromDataObjectBinders(Binder.java:488)
	  at org.springframework.boot.context.properties.bind.Binder.lambda$bindDataObject$7(Binder.java:479)
	  at org.springframework.boot.context.properties.bind.Binder$$Lambda$173/0x0000000125124510.get(Unknown Source:-1)
	  at org.springframework.boot.context.properties.bind.Binder$Context.withIncreasedDepth(Binder.java:597)
	  at org.springframework.boot.context.properties.bind.Binder$Context.withDataObject(Binder.java:583)
	  at org.springframework.boot.context.properties.bind.Binder.bindDataObject(Binder.java:479)
	  at org.springframework.boot.context.properties.bind.Binder.bindObject(Binder.java:418)
	  at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:350)
	  at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:339)
	  at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:269)
	  at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:256)
	  at org.springframework.boot.context.properties.ConfigurationPropertiesBinder.bind(ConfigurationPropertiesBinder.java:94)
	  at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.bind(ConfigurationPropertiesBindingPostProcessor.java:96)
	  at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.postProcessBeforeInitialization(ConfigurationPropertiesBindingPostProcessor.java:79)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:423)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:601)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:339)
	  at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$430/0x0000000125267010.getObject(Unknown Source:-1)
	  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:346)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:337)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	  at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1664)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1552)
	  at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:913)
	  at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)
	  at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:546)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1361)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1191)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:563)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:339)
	  at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$430/0x0000000125267010.getObject(Unknown Source:-1)
	  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:346)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:337)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.instantiateSingleton(DefaultListableBeanFactory.java:1155)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingleton(DefaultListableBeanFactory.java:1121)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:1056)
	  at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:987)
	  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:627)
	  at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:752)
	  at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439)
	  at org.springframework.boot.SpringApplication.run(SpringApplication.java:318)
	  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1361)
	  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1350)
	  at com.example.demo.DemoApplication.main(DemoApplication.java:10)

[philwebb]

Thanks for the sample @avnersin, it was very useful for profiling.

Issue #42361 is similar to this one and is also looking to improve performance when binding lots of YAML data. However, your use-case is slightly more complex due to the use of Spring Cloud Stream.

I have made some optimizations in Spring Boot 3.5.x with the following issues:

• ApplicationInfoPropertySource should be marked as immutable #44856
• Improve ConfigurationPropertyName equals/hashCode performance #44857
• Improve SystemEnvironmentPropertyMapper performance by caching results #44858
• SystemEnvironmentPropertySource is no longer considered immutable #44859
• Allow ConfigurationPropertyCaching overriding for the duration of a bind operation #44860
• Introduce a cache for expensive JavaBeanBinder and ValueObjectBinder #44861
• Bypass SystemEnvironmentPropertySource.resolvePropertyName calls for configuration properties #44862
• Add containsDescendantOfCache for system property source #44863

I don't think we have much more that we can now do on our side, however, there may still be performance optimizations that Spring Cloud Stream can apply. I've opened spring-cloud/spring-cloud-stream#3101 as one such example.

I'm going to close this one for now, but if you find issues using 3.5.x after spring-cloud/spring-cloud-stream#3101 has been fixed, please comment back here and we can re-open the issue.

[avnersin]

Thanks a lot for your help. We are looking forward to test it with spring boot 3.5 once it's released. We will then update here.