Indexes dont work as expected on spring-data-mongodb 3.2.5

[adi-gido]

Since the auto indexes were set to false by default, creating the indexes programmatically on a spring application causes indexes to be created wrong.

It seems the cause for this is a fix for spring-data-mongodb in versions 3 and up [https://github.com/spring-projects/spring-data-mongodb/commit/ab5b1f01409b301483a0692a9417522731ce40a5]

mapper.getMappedSort call leads to collection.createIndex parameter to be wrong for some of our indexes
and leads to errors like

{ v: 2, key: { interactionElements.conditions.name: 1 }, name: "interactionElements.conditions.name", has the same name as the requested index: { v: 2, key: { interactionElements.conditions.conditions.name: 1 }

Note the redundant '.conditions' in the request.

This occurs when one of the parameters is a map.

The problem is severe - we cannot trust Spring Data with programmatic indexes creation from Java.

`
private void ensureAnnotatedIndexes(MongoConnection connection, String tenantId, String environmentName) {

	try {
		log.debug("Going to ensure annotated indexes for {}, {}", tenantId, environmentName);
		MongoTemplate tenantEnvMongoTemplate = connection.mongoTemplate;
		MappingContext<? extends MongoPersistentEntity<?>, MongoPersistentProperty> mappingContext = tenantEnvMongoTemplate.getConverter().getMappingContext();
		IndexResolver resolver = new MongoPersistentEntityIndexResolver(mappingContext);

		ClassPathScanningCandidateComponentProvider scanner = new ClassPathScanningCandidateComponentProvider(false);
		scanner.addIncludeFilter(new AnnotationTypeFilter(Document.class));

		for (BeanDefinition bd : scanner.findCandidateComponents(BASE_PACKAGE_FOR_ANNOTATED_INDEXED_ENTITIES)) {

			String documentClassName = bd.getBeanClassName();

			try {
				Class<?> documentClass = Class.forName(documentClassName);
				log.trace("Going to ensure annotated indexes for {}, {}, {}", documentClassName, tenantId, environmentName);

				Iterable<? extends IndexDefinition> indexes = resolver.resolveIndexFor(documentClass);
				if (!indexes.iterator().hasNext()) {
					log.trace("No annotated indexes found for {}, {}, {}", documentClassName, tenantId, environmentName);
					continue;
				}

				IndexOperations indexOps = tenantEnvMongoTemplate.indexOps(documentClass);
				indexes.forEach(indexOps::ensureIndex);
				log.debug("Done ensuring indexes for {}, {}, {}", documentClassName, tenantId, environmentName);

			} catch (Exception e) {
				log.error("Error while trying to ensure indexes for {}, {}, {}", documentClassName, tenantId, environmentName, e);
			}
		}
		log.info("Done with ensurement of annotated indexes for {}, {}", tenantId, environmentName);

	} catch (Exception e) {
		log.error("Failed to ensure indexes for {}, {}", tenantId, environmentName, e);
	}
}

}`

[boris-rybakov]

I am going to give you a bit more information on the issue. The duplicate condition caused by Map of InteractionElements. Here is the code that caused it:
private HashMap<String, AbstractInteractionElement> interactionElements = new LinkedHashMap<>();

When spring-data-mongo 2.2.9 was processing it to get mappedKey, it uses the method
protected String mapPropertyName(MongoPersistentProperty property) in spring-data-mongodb/spring-data-mongodb/src/main/java/org/springframework/data/mongodb/core/convert/QueryMapper.java
So the code from 2.2.9 was identifing the property as isPositional false and didn't proceed to check property.isMap() .
In spring-data-mongo 3.2.5, the condition has changed and the code continues to property.isMap() , which causes the additional condition to be added to interactionElements.

[christophstrobl]

Thanks for reporting. @brybakov1973 could you please add the structure of the involved domain types to reproduce the issue.

[boris-rybakov]

{
"_id" : ObjectId("5c4840bf8b87bc0001705dce"),
"_class" : "com.jacada.is.as.domain.InteractionBody",
"interactionElements" : {
"bbabac0c6df1-167c56723e09c3e5-edef" : {
"_id" : "bbabac0c6df1-167c56723e09c3e5-edef",
"text" : "",
"footerText" : "",
"formLayout" : "MULTIPLE_QUESTIONS",
"splitQuestionId" : "",
"clickToContinueQuestionId" : "",
"provideInUserSummary" : false,
"helpText" : "",
"isHelp" : false,
"displayMenuBar" : false,
"enableCallNow" : false,
"enableChat" : false,
"enableHelp" : false,
"enableCallback" : false,
"enableEmail" : false,
"breadCrumbEnabled" : false,
"breadCrumbTitle" : "",
"pageName" : "p2",
"finishType" : "SYSTEM_DECIDE",
"showAnchorButton" : false,
"anchorPage" : "",
"anchorButtonLabel" : "",
"anchorButtonLocation" : "HEADER",
"showAnchorButtonOnAllPages" : false,
"anchorButtonImage" : "",
"setAsFinished" : false,
"rulesInElementSupported" : false,
"xLocation" : NumberInt(559),
"yLocation" : NumberInt(364),
"name" : "p2",
"inboundConnectors" : [
{
"_id" : "7e495fff98ba-de5aed27577d3d00-e761",
"targetContainer" : "8b5019ea4d82-bbf54862775bf796-2817",
"sourceContainerType" : NumberInt(6),
"targetContainerType" : NumberInt(6),
"result" : false,
"useResult" : false,
"text" : "",
"containerId" : ""
}
],
"outboundConnectors" : [

        ], 
        "conditions" : [
            {
                "name" : "test"
            }
        ], 
        "decisionType" : "AND", 
        "conditionOutcome" : true, 
        "show" : true, 
        "_class" : "com.jacada.is.as.domain.elements.FormElement"
    }
}, 

}

[christophstrobl]

thanks - I was more thinking of the Java type that is involved in the mapping.

[boris-rybakov]

public class InteractionBody {

private HashMap<String, InteractionElement> interactionElements = new LinkedHashMap<>();
private HashMap<String, InteractionElement> variables = new HashMap<>();
private String name;


public HashMap<String, InteractionElement> getInteractionElements() {
	return interactionElements;
}

public void setInteractionElements(HashMap<String, InteractionElement> interactionElements) {
	this.interactionElements = interactionElements;
}

public HashMap<String, InteractionElement> getVariables() {
	return variables;
}

public void setVariables(HashMap<String, InteractionElement> variables) {
	this.variables = variables;
}

public String getName() {
	return name;
}

public void setName(String name) {
	this.name = name;
}

}

[christophstrobl]

Thanks for the update. I do see the duplication of the key now when mapping the generated index against the domain type. This should not happen. We'll take care of it.

I’d be interested if the index ever worked because by looking at the snippets provided the index interactionElements.conditions.name does not cover the Map key segment defined by Map<...> interactionElements. The Map is treated like a nested document and the keys would need to be part of the index unless it’s a wildcard one.

[boris-rybakov]

You are correct, Mongo never used this kind of indexes in the search. I opened MongoDb support ticket and we concluded (together with Mongo support) that until they implement wildcard one (*) this doesn't do a thing. Mongo has this feature request already in their road map.