Skip to content

TTL index creation/verification fails when @Indexed(expireAfter) expression result in 0 seconds #4844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Alcarmor opened this issue Nov 25, 2024 · 4 comments
Assignees
Labels
type: bug A general bug

Comments

@Alcarmor
Copy link

Alcarmor commented Nov 25, 2024

with spring data mongodb 4.4.0 the annotation Indexed has deprecated the expireAfterSeconds in favor of expireAfter
https://docs.spring.io/spring-data/data-mongodb/docs/current/api/org/springframework/data/mongodb/core/index/Indexed.html#expireAfter()

so, according to documentation an entity mapped like this

@Indexed(expireAfterSeconds = 0) private LocalDateTime retentionDate;

should be changed in this

@Indexed(expireAfter = "0s") private LocalDateTime retentionDate;

here comes the problem: the new attribute seems ignore any value that result in 0 seconds

generating a non valid ttl index and in case of an existing index a relative validation failure

Caused by: com.mongodb.MongoCommandException: Command failed with error 85 (IndexOptionsConflict): 'An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { retentionDate: 1 }, name: "retentionDate" }, existing index: { v: 2, key: { retentionDate: 1 }, name: "retentionDate", expireAfterSeconds: 0 }' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { retentionDate: 1 }, name: \"retentionDate\" }, existing index: { v: 2, key: { retentionDate: 1 }, name: \"retentionDate\", expireAfterSeconds: 0 }", "code": 85, "codeName": "IndexOptionsConflict", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1732554648, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "AAAAAAAAAAAAAAAAAAAAAAAAAAA=", "subType": "00"}}, "keyId": 0}}, "operationTime": {"$timestamp": {"t": 1732554648, "i": 1}}}
	at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:210)
	at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:520)
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceiveInternal(InternalStreamConnection.java:448)
	at com.mongodb.internal.connection.InternalStreamConnection.lambda$sendAndReceive$0(InternalStreamConnection.java:375)
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:378)
	at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:111)
	at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:748)
	at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:68)
	at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:208)
	at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:113)
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:82)
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:73)
	at com.mongodb.internal.connection.DefaultServer$OperationCountTrackingConnection.command(DefaultServer.java:298)
	at com.mongodb.internal.operation.SyncOperationHelper.lambda$executeCommand$5(SyncOperationHelper.java:209)
	at com.mongodb.internal.operation.SyncOperationHelper.lambda$withSourceAndConnection$0(SyncOperationHelper.java:131)
	at com.mongodb.internal.operation.SyncOperationHelper.withSuppliedResource(SyncOperationHelper.java:156)
	at com.mongodb.internal.operation.SyncOperationHelper.lambda$withSourceAndConnection$1(SyncOperationHelper.java:130)
	at com.mongodb.internal.operation.SyncOperationHelper.withSuppliedResource(SyncOperationHelper.java:156)
	at com.mongodb.internal.operation.SyncOperationHelper.withSourceAndConnection(SyncOperationHelper.java:129)
	at com.mongodb.internal.operation.SyncOperationHelper.executeCommand(SyncOperationHelper.java:207)
	at com.mongodb.internal.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:105)
	at com.mongodb.internal.operation.CreateIndexesOperation.execute(CreateIndexesOperation.java:60)
	at com.mongodb.client.internal.MongoClusterImpl$OperationExecutorImpl.execute(MongoClusterImpl.java:379)
	at com.mongodb.client.internal.MongoCollectionImpl.executeCreateIndexes(MongoCollectionImpl.java:941)
	at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:923)
	at com.mongodb.client.internal.MongoCollectionImpl.createIndexes(MongoCollectionImpl.java:918)
	at com.mongodb.client.internal.MongoCollectionImpl.createIndex(MongoCollectionImpl.java:903)
	at org.springframework.data.mongodb.core.DefaultIndexOperations.lambda$ensureIndex$0(DefaultIndexOperations.java:131)
	at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:603)
	at org.springframework.data.mongodb.core.DefaultIndexOperations.execute(DefaultIndexOperations.java:217)
	at org.springframework.data.mongodb.core.DefaultIndexOperations.ensureIndex(DefaultIndexOperations.java:121)
	at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexCreator.java:152)

ttl indexes are generted succesfully only if the expression result in more than 0 secods, but this is a big problem for existing indexes

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Nov 25, 2024
@christophstrobl
Copy link
Member

thanks for bringing this to our attention. It would be great if you could provide a complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem.

@Alcarmor
Copy link
Author

Alcarmor commented Nov 28, 2024

thanks for bringing this to our attention. It would be great if you could provide a complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem.

demo.zip
in this spring boot sample project i have prepared 2 entities
MyEntityWithDeprecatedTTL and MyEntityWithNewTTL
with the same attributes as indexes, one with the new ttl annotation attribute and the other with the deprecated one.

Executing "mvn test" the DemoApplicationTests class will compare the generated indexes.
(the project uses test containers, so requires docker)
You will see that the test wich compare the 1 second expression ttl indexes will pass, but the test with the 0 seconds expression will fail.

Or it can be deployed to see the differences between the two entities directly on mongodb

@fmeiser
Copy link

fmeiser commented Dec 5, 2024

We have the same problem. The source of the problem is MongoPersistentEntityIndexResolver:567 where 0 is ignored.

if (!timeout.isZero() && !timeout.isNegative()) {
   indexDefinition.expire(timeout);
}

This behavior does not exist with expireAfterSeconds because it compares to greater than or equal to 0.

@christophstrobl christophstrobl self-assigned this Dec 6, 2024
@christophstrobl
Copy link
Member

thank you @Alcarmor for the reproducer and @fmeiser for pointing to MongoPersistentEntityIndexResolver.

@mp911de mp911de changed the title TTL index creation/verification fails on spring data mongodb 4.4.0 when expression result in 0 seconds TTL index creation/verification fails when @Indexed(expireAfter) expression result in 0 seconds Dec 10, 2024
@mp911de mp911de added type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Dec 10, 2024
@mp911de mp911de added this to the 4.3.7 (2024.0.7) milestone Dec 10, 2024
mp911de pushed a commit that referenced this issue Dec 10, 2024
This commit fixes an issue where expireAfter=0s behaves differently from expireAfterSeconds=0 where the former would not be applied.

Closes #4844
Original pull request: #4848
mp911de pushed a commit that referenced this issue Dec 10, 2024
This commit fixes an issue where expireAfter=0s behaves differently from expireAfterSeconds=0 where the former would not be applied.

Closes #4844
Original pull request: #4848
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants