Malformed api-docs JSON when custom HttpMessageConverter is used

[peter-szepe-geneea]

When custom HttpMessageConverter is used malformed api-docs endpoint produces malformed JSON

Steps to reproduce the behavior:

• spring-boot version 2.2.6.RELEASE
• springdoc-openapi 1.3.5-SNAPSHOT
• the actual result is a JSON String created from the String produced by org.springdoc.webmvc.api.OpenApiResource#openapiJson. The original response is wrapped with a leading and trailing double quote and all the double internal double quotes are escaped:
  "{\"openapi\":\"3.0.1\",\"info\":{\"title\":\"OpenAPI definition\",\"version\":\"v0\"},\"servers\":[{\"url\":\"http://localhost:8080\",\"description\":\"Generated server url\"}],\"paths\":{\"/get\":{\"get\":{\"tags\":[\"controller\"],\"operationId\":\"getSomeMap\",\"responses\":{\"200\":{\"description\":\"default response\",\"content\":{\"*/*\":{\"schema\":{\"$ref\":\"#/components/schemas/ImmutableMultimapStringString\"}}}}}}}},\"components\":{\"schemas\":{\"ImmutableMultimapStringString\":{\"type\":\"object\",\"properties\":{\"empty\":{\"type\":\"boolean\"}}}}}}"
• a sample application can be found in https://github.com/Geneea/springdoc-test

Expected behavior

• I would expect that org.springdoc.webmvc.api.OpenApiResource#openapiJson should not do the conversion from OpenAPI to String. Just return OpenAPI or ResponseEntity<OpenAPI> and leave the serialization for the registered HttpMessageConverter.

Additional context
An ugly fix for this problem is https://github.com/Geneea/springdoc-test/blob/master/src/main/java/com/geneea/springdoc/FixApiDocsWithFilter.java
registering a Filter what corrects the malformed response.

Same problem is described in https://stackoverflow.com/questions/59393191/endpoint-api-docs-doesnt-work-with-custom-gsonhttpmessageconverter

[bnasslahsen]

@peter-szepe-geneea,

Only one @Configuration class may have the @EnableWebMvc annotation to import the Spring Web MVC configuration.

If you can remove @EnableWebMvc from your sample, it will work directly.
If you can not, because you changing the default list of Http Converters with our own, you should override OpenApiResource as follow:

@Component
public class OpenApiResource extends org.springdoc.webmvc.api.OpenApiResource {

	public OpenApiResource(OpenAPIBuilder openAPIBuilder, AbstractRequestBuilder requestBuilder, GenericResponseBuilder responseBuilder, OperationBuilder operationParser, RequestMappingInfoHandlerMapping requestMappingHandlerMapping, Optional<ActuatorProvider> servletContextProvider, Optional<List<OpenApiCustomiser>> openApiCustomisers, SpringDocConfigProperties springDocConfigProperties, Optional<SecurityOAuth2Provider> springSecurityOAuth2Provider) {
		super(openAPIBuilder, requestBuilder, responseBuilder, operationParser, requestMappingHandlerMapping, servletContextProvider, openApiCustomisers, springDocConfigProperties, springSecurityOAuth2Provider);
	}

	@Operation(hidden = true)
	@GetMapping(value = API_DOCS_URL, produces = MediaType.TEXT_PLAIN_VALUE)
	public String openapiJson(HttpServletRequest request, @Value(API_DOCS_URL) String apiDocsUrl)
			throws JsonProcessingException {
		calculateServerUrl(request, apiDocsUrl);
		OpenAPI openAPI = this.getOpenApi();
		return Json.mapper().writeValueAsString(openAPI);
	}

	private void calculateServerUrl(HttpServletRequest request, String apiDocsUrl) {
		String requestUrl = decode(request.getRequestURL().toString());
		String calculatedUrl = requestUrl.substring(0, requestUrl.length() - apiDocsUrl.length());
		openAPIBuilder.setServerBaseUrl(calculatedUrl);
	}
}

OpenApiResource, will return TEXT_PLAIN_VALUE starting from the next release v1.3.5, so you won't need to override it for this use case.

[Aloren]

@bnasslahsen The fix provided in ee08333 is breaking all our automated tests for Swagger endpoints in all services. Is it possible to leave content-type to be json as it was previously, but instead of returning raw string-- return some dto, so that spring does all json conversion?

[bnasslahsen]

Hi @Aloren,

Changing the return type to DTO, is not easy. It require a custom serializer, which is already provided by swagger-core.
Also if your tests are based on the payload, Changing to DTO may change the result if we don't use the same serializer.
Can't you just adapt your failing tests by adapring the content-type ?

[Aloren]

We can of course add additional workarounds for our tests, but the main issue is that such response format violates HTTP specs. With such change content negotiation is completely broken and whoever knows what functionality it might break in the future. This is a type of fix that gives short-term benefits, but may lead to interesting new issues :)

[bnasslahsen]

I would say that the impact should be limited because the return type was already of type String.
But i agree its better to return it as object.
Hopefully, If i have some time, i will have a look at it.

[bnasslahsen]

@Aloren,

This change is reverted back, as it seems more going to cause problems than resolving them.

@peter-szepe-geneea,

You can override the OpenApiResource as proposed earlier. This seems the less harmful solution

[bnasslahsen]

its now available on v1.3.7.

[MikeMitterer]

@bnasslahsen I'm using 1.3.9 but I have still the same problem???
openapi_version=1.3.9+ gson_version=2.8.6

    implementation("org.springdoc:springdoc-openapi-webmvc-ui:${openapi_version}")
    implementation("org.springdoc:springdoc-openapi-webmvc-core:${openapi_version}")
    implementation("org.springdoc:springdoc-openapi-security:${openapi_version}")
    implementation("org.springdoc:springdoc-openapi-kotlin:${openapi_version}")

    implementation("com.google.code.gson:gson:$gson_version")

With SpringFox I can register an Adapter: springfox/springfox#2758...

[bnasslahsen]

@MikeMitterer,

springdoc-openapi is built on top of swagger-core, which is based on jackson;
There is already existing request to add support for GSON, but not sure it will be added in short term:

• Add GSON support swagger-api/swagger-core#1826

As described before, you can override the OpenApiResource as follow:

@Component
public class OpenApiResource extends org.springdoc.webmvc.api.OpenApiResource {

	public OpenApiResource(OpenAPIBuilder openAPIBuilder, AbstractRequestBuilder requestBuilder, GenericResponseBuilder responseBuilder, OperationBuilder operationParser, RequestMappingInfoHandlerMapping requestMappingHandlerMapping, Optional<ActuatorProvider> servletContextProvider, Optional<List<OperationCustomizer>> operationCustomizers, Optional<List<OpenApiCustomiser>> openApiCustomisers, SpringDocConfigProperties springDocConfigProperties, Optional<SecurityOAuth2Provider> springSecurityOAuth2Provider, Optional<RouterFunctionProvider> routerFunctionProvider) {
		super(openAPIBuilder, requestBuilder, responseBuilder, operationParser, requestMappingHandlerMapping, servletContextProvider, operationCustomizers, openApiCustomisers, springDocConfigProperties, springSecurityOAuth2Provider, routerFunctionProvider);
	}

	@Operation(hidden = true)
	@GetMapping(value = Constants.API_DOCS_URL, produces = MediaType.TEXT_PLAIN_VALUE)
	public String openapiJson(HttpServletRequest request, @Value(Constants.API_DOCS_URL) String apiDocsUrl)
			throws JsonProcessingException {
		calculateServerUrl(request, apiDocsUrl);
		OpenAPI openAPI = this.getOpenApi();
		return Json.mapper().writeValueAsString(openAPI);
	}
}

[sergio11]

Hello, I have incorporated into my project version 1.3.9 OpenAPI to migrate from SpringFox and I have suffered the same problem that is discussed in this entry

implementation "org.springdoc:springdoc-openapi-ui:1.3.9"

We are removing the Jackson AutoConfiguration class from startup and manually configuring the HttpMessageConverter we need

@SpringBootApplication(exclude = {JacksonAutoConfiguration.class})
@Profile("dev")
public class AgrocietyDevelopmentApplication {

    public static void main(final String[] args) {
        SpringApplication.run(AgrocietyDevelopmentApplication.class, args);
    }

}

@Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
        converters.add(new JsonMergePatchHttpMessageConverter());
        converters.add(new JsonPatchHttpMessageConverter());
        converters.add(new MappingJackson2HttpMessageConverter(objectMapper));
        converters.add(new ByteArrayHttpMessageConverter());
    }

Should I inject the Jackson ObjectMapper into a class that inherits from 'OpenApiResource' and do the JSON conversion in the openApi method ? Or how should I deal with this problem?
Thank you very much for your attention, regards

[bnasslahsen]

@sergio11,

If you are using jackson, there is no need by default to override OpenApiResource, unless your are adding a custom HttpMessageConverter, which changes the default jackson behaviour.

You can find attached a sample code using: @SpringBootApplication(exclude = {JacksonAutoConfiguration.class})

If you are still having issues. Please make sure you provide a minimal reproducible example as the one i have attached.

initial.zip

[sergio11]

@bnasslahsen

Thank you very much for the response and the example provided. I have reduced the project to the minimum to show you the problem, I would ask you please if you can verify what is wrong in this example project.

I have left a simple controller and the WebConfig class with the ObjectMapper that we use the project.

Thank you very much for your attention, regards

spring_open_api_test.zip

[bnasslahsen]

@sergio11,

You need to add StringHttpMessageConverter to your method configureMessageConverters,
(StringHttpMessageConverter has to be the first one):

    @Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
    	converters.add(new StringHttpMessageConverter());
        converters.add(new MappingJackson2HttpMessageConverter(provideObjectMapper()));
        converters.add(new ByteArrayHttpMessageConverter());
    }

You can find attached, the working example from the source that you have provided.

spring_open_api_test-corrected.zip

[micw]

Hello,
I wonder why this is closed. IMO, it actually is a bug:

The endpoint returns a ResponseEntity of type "String". The client requests a application/json. So the string must correctly be a JSON string, meaning that it's quoted accordingly.

Adding StringHttpMessageConverter (which is by default added with spring boot) changeds this behaviour in an inconsistent way: If the response entity is a string, it is returned as unquoted string (which is, in most cases, invalid json). Returning anything else will be encoded correctly as JSON. So StringHttpMessageConverter introduces buggy behaviour.

So for the following endpoint, the behaviour differs with pressence/absence of StringHttpMessageConverter:

@GetMapping(value = "/test1")
public ResponseEntity<String> test1() {
  return ResponseEntity.ok("Test1");
}

This will always produce text/json. With StringHttpMessageConverter enabled, the response will be an invalid JSON of Test1. Without StringHttpMessageConverter, it will produce a valid JSON if "Test1.

If an endpoint creates a String that is already JSON, It must not rely on that inconsistent behaviour to hopefully deliver this unquoted to the client. A valid implementation of auch an endpoint could look that like:

@GetMapping(value = "/test2", produces = "application/json")
public ResponseEntity<byte[]> test2() {
    return ResponseEntity.ok("Test2".getBytes(StandardCharset.UTF_8));
}

This endpoint will always return Test2, regardless of the pressence/absence of StringHttpMessageConverter.

Edit: it would even better to directly write the JSON directly to HttpServletResponse do de-couple the behaviour from specific message converters.

[in3des]

Hi,
Can anyone suggests how this formatting was done?
that was a direct link just under the info TITLE at main swagger-ui html page.

trying to figure out how to get this result... Thx

So far I have only default formatting which looks like a mess ((
cannot find any setting for customization

[jiangxiaoqiang]

the new version tweak the response type to MediaType.APPLICATION_JSON_VALUE, but the result string still be eacaped the second time. The final result was the escaped version.

the new version tweak the response type to MediaType.APPLICATION_JSON_VALUE, but the result string still be eacaped the second time. The final result was the escaped version.

[jiangxiaoqiang]

Hi, Can anyone suggests how this formatting was done? that was a direct link just under the info TITLE at main swagger-ui html page.

trying to figure out how to get this result... Thx So far I have only default formatting which looks like a mess (( cannot find any setting for customization

add StringHttpMessageConverter to your custom converter, add to the first one.

[in3des]

this is not a good way to solve the problem. It seems like a hack way.

@jiangxiaoqiang
I found some quick and very simple solution ))
just need to add this string to your properties 👍
springdoc.writer-with-default-pretty-printer=true

[in3des]

did not work for me.

Very strange...
Just opened clean project and added this string, and it works

try same thing, I used this example project
https://github.com/springdoc/springdoc-openapi-demos/tree/master/springdoc-openapi-spring-boot-2-webmvc
In my project this solution also works
springdoc.writer-with-default-pretty-printer=true