Partially populated PersistentEntity instances not properly removed from cache [DATACMNS-1574]

[spring-projects-issues]

Thomas Spendier opened DATACMNS-1574 and commented

I am facing the following problem, which is very rarely reproducible:

Documents get sometimes stored to MongoDB although not triggered by our business logic.

Find attached the class structure for a class where a document got wrongly inserted.

An example of a document representing this class:

{
 "_id" : {
   "userDefinedId" : "ps-chat-service",
   "tenantId" : "T594cbe61-7cfc-4054-baa4-2f2cdf942e91"
 },
 "_class" : "chat.CategoryAssignmentConfig",
 "containerId" : "ps-chat-service",
 "categoryAssignmentRules" : [],
 "tenantId" : "T594cbe61-7cfc-4054-baa4-2f2cdf942e91"
} 

Still we end up that sometimes documents show up in MongoDB with the following serialized format:

{
 "_id" : ObjectId("5c73e667f7631921f8f4eb54"),
 "_class" : "chat.CategoryAssignmentConfig"
} 

The corresponding MongoDB log where this document got inserted:

command DB.chat.CategoryAssignmentConfig command: insert { insert: "chat.CategoryAssignmentConfig", ordered: true, documents: [ { _id: ObjectId('5c73e667f7631921f8f4eb54'), _class: "chat.CategoryAssignmentConfig" } ] } ninserted:1 keysInserted:1 numYields:0 reslen:44 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_query 13ms

It looks like as if our business logic simply inserts an object of the referred class where no properties are set and an auto-generated ID is created by spring-data-mongodb (which doesn't match our complex typed ID).

Still we log the content of every object before performing a MongoOperations::save() call and there is no log line which shows such an "empty" object. All log lines basically match the following output (with the only difference being the concrete values for the containerId and tenantId property:

Persisting document PersistedCategoryAssignmentRulesContainer[spaceInstanceId=SpaceInstanceId[userDefinedId=ps-chat-service,userIdClassName=java.lang.String,tenantId=T4a5c5bd9-bb3c-4e9d-a935-c34b78e55ec0],tenantId=T4a5c5bd9-bb3c-4e9d-a935-c34b78e55ec0,routingKey=0,masterPartitionInstanceId=3,containerId=ps-chat-service,categoryAssignmentRules=[]]; 

I couldn't find any similar issues so I created this ticket. I am not certain whether the problem is within spring-data-mongodb, but I have no reason to belief that our business logic is triggering these wrong inserts (otherwise we would have seen it in the logs). Do you know of any similar occurrences in any spring-data-mongodb version? I am in particular interested why the _id field gets auto-generated although it actually is set

---

Attachments:

• Java classes.zip (3.05 kB)

Backported to: 2.1.11 (Lovelace SR11)

[spring-projects-issues]

Thomas Spendier commented

I finally identified the problem and could reproduce it. The initial trigger for persisting documents with an unexpected auto-generated ID is a class loading problem in our used middle-ware which exposes a bug in spring-data-commons.

While we are in progress of getting the middle-ware fixed, I suggest to fix the following bug in spring-data-commons. The problem is in org.springframework.data.mapping.context.AbstractMappingContext (in v1.13.6 it is line 221) where only a MappingException is catched in order to remove the eagerly added type information from the persistentEntities map. In case another RuntimeException is thrown (in our case java.lang.TypeNotPresentException) the exception is thrown but the type information remains in the persistentEntities map, as there exists no catch clause.

Persisting a document with an auto-generated ID then happens on the next save operation for the affected document where the class loading problem got resolved, but since the (let's call it "corrupted") type information didn't get removed from the persistentEntities map before, spring-data-mongodb uses it which in the end results in persisting the document with an auto-generated ID.

The simple fix is to replace catching MappingException with catching all RuntimeExceptions.

Oliver Drotbohm: I have created an integration test demonstrating the problem where in the end the database contains the document with an auto-generated ID, but it uses many dependencies from our code base. I could of course isolate the issue with solely using spring-data-mongodb as dependency, but is it worth the effort? Is it enough to create a pull request for spring-data-commons only having a unit test for it?

[spring-projects-issues]

Oliver Drotbohm commented

While I agree that we should properly clean up the cache, I still suspect the client code to wrongfully continue to request the type as it seems to ignore the runtime exception that it sees in the first place. What I am trying to say is that the second call saving that document shouldn't actually never happen as the initial lookup should've been in handled in a way that it doesn't reattempt to save an instance of that type, right?

I.e. even if we cleaned up the cache properly, wouldn't you code just try to over and over lookup the PersistentEntity for that broken type and thus cause exceptions to be thrown and neglected forever?

[spring-projects-issues]

Oliver Drotbohm commented

I've pushed the cleanups to master and the bugfix branch for Lovelace. Feel free to give the snapshots a try

[spring-projects-issues]

Thomas Spendier commented

Thank you for the quick fix!

Regarding taking care of this exception in the client code: Yes your are right. However we have a mitigation for this problem by restarting the component where the class loading problem occurs. As long as the restart didn't happen, we pile up all the not successfully persistence operations to get executed once the problem is fixed (or we simply start rejecting once the queue has filled up). Our middle-ware is doing this for us automatically (when regularly failing to persist), so there is no need to remember each problematic class definition on our own