Profile-specific property files are incorrectly ordered when both '.' and './config' folders are used in the same application

[focdanisch]

Hello,

when upgrading our application from 2.3.0.RELEASE to spring boot 2.4.5, I realised that spring boot now sorts the application*.properties files in a different order. I was reading the upgrade documentation for 2.4.x, but as far as I understand it, we should not have to make any adjustments in our configuration. Furthermore, the sort order I see does not match your documented behaviour. Quote from here: https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-external-config-files-profile-specific

Profile-specific properties are loaded from the same locations as standard application.properties, with profile-specific files always overriding the non-specific ones. If several profiles are specified, a last-wins strategy applies. For example, if profiles prod,live are specified by the spring.profiles.active property, values in application-prod.properties can be overridden by those in application-live.properties.

I created a small spring boot project from your initial sample project (git clone https://github.com/spring-guides/gs-spring-boot.git) from the gs-spring-boot/initial folder. You can find it here: https://github.com/focdanisch/spring-boot-properties-order

As we are using maven as our build system, I only built the test project with maven. I changed the following things: I added two modules and switched the parent pom.xml to pom packaging. module1 only contains a collection of application*.properties in src/main/resources:

config/application-config.properties
config/application-local.properties
application.properties
application-root.properties

module2 has another collection of application*.properties:

in src/main/resources:

application.properties
application-local.properties

in config folder under root:
application-local.properties

application.properties defines the active profiles as local,config,root. According to your documentation, application-root.properties must win.

In module2 under src/test/resources is a junit test class (TestProfileSpecificOrder) which passes with 2.3.10.RELEASE (I used the latest 2.3.x version for the test) and fails with 2.4.5 (you can easily switch the versions in the root pom, I left both version tags in the file). The test class contains two tests:

For the first test,

@Value("${test.prop}")
String testProp;

is injected, and it is simply checked if the value of this property is "root" (which must be that way, as application-root.properties must win).

For the second test, the environment gets injected:

@Autowired
Environment environment;

With 2.3.10.RELEASE, I see this order of propertySources (when I import the project into Eclipse and put a breakpoint at the beginning of the method):

ConfigurationPropertySourcesPropertySource {name='configurationProperties'}
MapPropertySource {name='Inlined Test Properties'}
PropertiesPropertySource {name='systemProperties'}
OriginAwareSystemEnvironmentPropertySource {name='systemEnvironment'}
RandomValuePropertySource {name='random'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/application-root.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/config/application-config.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [file:./config/application-local.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/config/application-local.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/application-local.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/application.properties]'}

which seems correct and matches your documentstion. The properties are primarily sorted by the active profiles, and secondarily by classpath/file. Spring searches properties from specific to general, so the result is correct.

With 2.4.5, I see this order:

ConfigurationPropertySourcesPropertySource {name='configurationProperties'}
MapPropertySource {name='test'}MapPropertySource {name='Inlined Test Properties'}
PropertiesPropertySource {name='systemProperties'}
OriginAwareSystemEnvironmentPropertySource {name='systemEnvironment'}
RandomValuePropertySource {name='random'}
OriginTrackedMapPropertySource {name='Config resource 'file [config\application-local.properties]' via location 'optional:file:./config/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [config/application-config.properties]' via location 'optional:classpath:/config/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [config/application-local.properties]' via location 'optional:classpath:/config/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [application-root.properties]' via location 'optional:classpath:/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [application-local.properties]' via location 'optional:classpath:/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [application.properties]' via location 'optional:classpath:/''}

So it sorts by classpath/file, then by root folder/config folder, and then by active profiles. That seems wrong to me, as this way, the file [config\application-local.properties] overwrites everything else, even though "local" is the first in the active profiles list and therefore should be overriden by [config/application-config.properties] as well as [application-root.properties]. Also, the remaining properties are not ordered correctly, as the files should be listed in the reverse order of the active profiles, so all application-root.properties files should be on top, followed by all all application-config.properties files, followed by all application-local.properties files.

Our application is dependent on a working last win strategy. We could theoretically combine some of the separate property files into one of the new multi-property-files, but not all of them. Our application has not only two, but a lot of maven modules, and some of them must be able to override or even add additional profile specific property files, and the active profiles setting allows us to specify the correct order.

Am I missing something here? Is there something I have misunderstood in your upgrade guide? Or is this now intended behaviour with 2.4.5 (in that case, the documentation would be incorrect)?

Best regards,
Manuel

[wilkinsona]

#25058 is somewhat similar.

[focdanisch]

So, you are basically saying, that the documentation is wrong in that case. External files always win over files found in the classpath. This would be OK for me, but how can I sort the different inner properties into the correct order? Everything found in the classpath does not respect local,config,root either. Perhaps I spend some more time playing with my test project so I get a better feeling of what is going on with properties-sorting now.

[wilkinsona]

No, it wasn't my intention to say that. My comment was only intended to link two possibly related issues together. We haven't had a chance to look at this one in detail so we don't yet know what the outcome will be.

[focdanisch]

Ah OK, sorry for the misunderstanding. Nevertheless, I think I will expand the example I've created a little bit, as soon as I find the time to do it. :D

[mbhave]

@focdanisch The documentation specifies the order in which profile-specific files are loaded based on whether they are packaged in the jar or present outside of the jar. This is from the documentation:

Config data files are considered in the following order:

    Application properties packaged inside your jar (application.properties and YAML variants).

    Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).

    Application properties outside of your packaged jar (application.properties and YAML variants).

    Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).

See #3845 for additional context.

I am going to close this issue because this is working as documented.

@focdanisch If you have a scenario where the new arrangement does not work for you, please create a new issue and we can address that separately.

[focdanisch]

@mbhave I disagree, the documentation I linked in my original post clearly states that a last-wins strategy applies. This is NOT the case any more with 2.4.x.

So I am fine with external files overwriting files in the jar. The problem is, that the files inside the jar are not sorted by active profile either, depending on their location (config-folder or root folder). The classpath files sort order with 2.3.x was:

OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/application-root.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/config/application-config.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/config/application-local.properties]'}
OriginTrackedMapPropertySource {name='applicationConfig: [classpath:/application-local.properties]'}

according to the active profiles (local,config,root). With 2.4.x I get:

OriginTrackedMapPropertySource {name='Config resource 'class path resource [config/application-config.properties]' via location 'optional:classpath:/config/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [config/application-local.properties]' via location 'optional:classpath:/config/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [application-root.properties]' via location 'optional:classpath:/''}
OriginTrackedMapPropertySource {name='Config resource 'class path resource [application-local.properties]' via location 'optional:classpath:/''}

Which is NOT correctly sorted by active profiles. The example I created shows all the problems we are seeing with 2.4.x. So there is no last-wins strategy here any more. External files overwrite internal files (which is OK), but internal files have no last-wins-stategay as well (when they are spread over config and root folders), because they are not sorted according to the active profiles.

[philwebb]

I think this issue might be similar to #25766.

What's happening is that the order is correct only within the context of the parent node. We're effectively getting a tree that looks like this:

file:./config/
    optional:file:./config/ file [config/application-config.properties]
    optional:file:./config/ file [config/application-local.properties]
classpath:/config
    optional:classpath:/config/ class path resource [config/application-config.properties]
    optional:classpath:/config/ class path resource [config/application-local.properties]
classpath:/
    optional:classpath:/ class path resource [application.properties]
    optional:classpath:/ class path resource [application-root.properties]
    optional:classpath:/ class path resource [application-local.properties]

Each of the child nodes is ordered correctly, but when we iterate over the tree the overall order isn't what's expected.

I'm really not sure how to fix this yet.