Introduce support for escaping the escape character for property placeholders

[bitb4ker]

Up until spring-boot 3.3.8, an escaped backslash followed by a placeholder like:

prop1=value1  
prop2=value2\\${prop1}  

Resulted in prop2 resolving to:

value2\value1

Starting with spring-boot 3.4.0, the result is:

value2value1

Doubling the backslashes like this:

prop1=value1  
prop2=value2\\\\${prop1}  

Results in:

value2${prop1}

This is due to the escaping logic introduced in org.springframework.util.PlaceholderParser which does two passes of PlaceholderParser.SimplePlaceholderPart.resolveRecursively(PartResolutionContext, String), each calling PlaceholderParser.parse(String, boolean) in which the escaping logic is implemented.

This last method does not handle escaping the escape character, causing the issue.

[snicoll]

The parser in 6.2 has been rewritten to fix, amongst other things, #9628. Unfortunately, this means that if the placeholder is escaped, it's rendered as is (i.e. not evaluated).

I wrote a test:

@Test
void gh34315() {
    Properties properties = new Properties();
    properties.setProperty("prop1", "value1");
    properties.setProperty("prop2", "value2\\${prop1}");
    PlaceholderParser parser = new PlaceholderParser("${", "}", ":", '\\', true);
    assertThat(parser.replacePlaceholders("${prop2}", properties::getProperty)).isEqualTo("value2${prop1}");
}

and it passes. I've also added those two properties in an empty Spring 6.2 app and it resolved the same way. Can you share a sample that actually fails the way you've described?

[bitb4ker]

props-spring-3.3.zip

props-spring-3.4.zip

To run:

gradlew bootRun

The 3.3 one shows the expected behavior, the 3.4 shows what I described as the broken behavior in both '\' and '\\' in the original post.

[snicoll]

I got it now. The problem is PropertySourcesPlaceholderConfigurer getting in the way of the placeholder resolution. It does its own round, removes the backslash to render the placeholder as is. As a result, the parser continues its work and sees a new placeholder to resolve.

[snicoll]

The issue as reported can't be fixed without re-introducing the bug that we fixed. The side effect that was found as part of the report is going to be handled by #34326.

For that case above, you'll need to restructure your configuration to move the backslash elsewhere to avoid the escaping, something like:

prop1=\\value1
prop2=value2${prop1}

[bitb4ker]

Couldn't we introduce a way to escape the escape character?
This is essentially the root of the issue I raised.
Why does it break previous fixes?

[snicoll]

Everything is possible, I guess but we’re not keen to make the parser more complex at this time.

[bitb4ker]

My issue is that this broke something that was working for years.
We have dozen of config files now broken because we build windows path and windows logins with placeholders.

[neoludo]

@bitb4ker is right, that's a breaking change. You should at least mention it in the release notes.

[bclozel]

@neoludo Can you suggest what's missing from the release notes?

[neoludo]

The sentence If you used the escaped character right before the placeholder, you will need to modify your configuration structure to move \\ in the value itself. seems to be correct for properties files only.

But in a YAML file, and given this YAML snippet :

username: johndoe
login: DOMAIN\${username}

login will be evaluated as : DOMAINjohndoe

and given this YAML snippet :

username: johndoe  
login: DOMAIN\\${username}

login will be evaluated correctly as : DOMAIN\johndoe

So there is no need to move backslashes to value itself, and the upgrade notes should dissociate YAML parsing from properties parsing.

[bclozel]

I guess this is a Yaml specific concern; but I guess the advice to move the \ directly into one of the values still applies.

It could be that this currently works because of some subtle YAML+parser interaction right now, but is still not advised.

[bitb4ker]

@bclozel I see a paragraph was added on that in the release notes however I still think that this is an issue that needs to be addressed as being forced to add the \ in the value being interpolated is just not right.
In my case I deal with downlevel user accounts (DOMAIN\USERNAME) and paths and in neither case is adding the \ in the value being interpolated appropriate.

Could we add implement an escape mechanism for the escape character somewhere in the roadmap?

[sbrannen]

Hi everyone,

Thanks for all of the constructive feedback! 👍

We have decided to introduce support for "escaping the escape character" in Spring Framework 7.0.

In light of that, I have changed the title of this issue and reopened this issue to address that.

In addition, we will be addressing #34861 and related issues in Spring Framework 6.2.7.

Cheers,

Sam

[bitb4ker]

That's a great news, thanks!
I can't wait to test it!

[sbrannen]

In the interim, once #34861 is complete you should be able to explicitly set the escapeCharacter to null via custom configuration.

Whereas, once #34865 is complete you should be able to globally disable the escape character by default, via a JVM system property, directly via SpringProperties, or within your project in a spring.properties file.

Thus, 6.2.7 will provide mechanisms that can be used as an alternative to "escaping the escape character" within configuration strings.