count() $maxTimeMS not work

[zhaohuiio]

mongodb version 3.6.23
jdk 11
system: MacOS 12.4

Criteria criteria = Criteria.where("token").is("true");
Query query = new Query(criteria).maxTimeMsec(1000);;
long count = mongoTemplate.count(query, DBObject.class, "public_leads");

The desired effect is that after 1 second of execution, if it doesn't compute, then immediately break it. But this will not happen, and count() will complete the entire evaluation without being interrupted by a query timeout

db.collection.count({query}, {maxTimeMS: 1000})
Native mongodb js queries work, but not in the framework, I tried the two versions respectively, spring-data-mongodb version: 3.4.11 , 4.0.5 will be repeated

Criteria criteria = Criteria.where("token").is("true");
Query query = new Query(criteria).maxTimeMsec(1000);
List list = mongoTemplate.find(query, DBObject.class, "public_leads");

There's no problem with using it that way

[zhaohuiio]

db.public_leads.count({token:"true"}, {maxTimeMS: 1000})
Generate logs in mongodb using native queries Include $maxTimeMS in the log discovery query statement

springboot TestCode( Base on spring-data-mongodb version: 4.0.5 )

public long test(){
        Criteria criteria = Criteria.where("token").is("true");
        Query query = new Query(criteria).maxTimeMsec(5);;
        Object count = mongoTemplate.count(query, DBObject.class, "public_leads");
        return 0;
}

Execute the above code to generate logs in mongodb，count() uses aggregation at the bottom of the framework

[zhaohuiio]

Add maxTimeMS to AggregationOptions [DATAMONGO-2390] #3249

Based on this PR, it seems that aggregation is needed to solve this problem

[christophstrobl]

@zhaohuiio thank you for the additional explanation. I see, maxTimeMsec is from the query is not carried forward to CountOptions#maxTime when performing an exact count.

[zhaohuiio]

I found a solution( Base on spring-data-mongodb version: 4.0.5 )

public long test(){
        Duration duration = Duration.ofMillis(2000);
        AggregationOptions options = Aggregation.newAggregationOptions().maxTime(duration).build();
        Criteria criteria = Criteria.where("token").is("true");
        Aggregation aggregationCount = Aggregation.newAggregation(
                Aggregation.match(criteria),
                Aggregation.group().count().as("count")
        );
        Aggregation aggregation = aggregationCount.withOptions(options);
        List record = mongoTemplate.aggregate(aggregation, "public_leads", DBObject.class).getMappedResults();
        BasicDBObject obj = (BasicDBObject)record.get(0);
        String count = obj.get("count") + "";
        return Long.valueOf(count);
    }

But In earlier versions, this method did not work because Aggregation.newAggregationOptions().maxTime() did not be provided

[zhaohuiio]

One last question, What I don't understand here is why the count method of the high version of spring-data-mongodb should be implemented by aggregation at the bottom ！

public long test(){
        Criteria criteria = Criteria.where("token").is("true");
        Query query = new Query(criteria).maxTimeMsec(1000);;
        long count = mongoTemplate.count(query, DBObject.class, "public_leads");
        return count;
 }

Execute code, generated log

2023-04-27T16:31:17.211+0800 I COMMAND [conn3320915] command xxx.public_leads appName: "MongoDB Shell" command: aggregate { aggregate: "public_leads", pipeline: [ { $match: { token: "true" } }, { $group: { _id: 1.0, count: { $sum: 1.0 } } } ], cursor: {}, lsid: { id: UUID("e1c3816c-0fff-49a1-9cb7-e6cf1096c371") }, $clusterTime: { clusterTime: Timestamp(1682584274, 1), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $readPreference: { mode: "secondaryPreferred" }, $db: "xxx" } planSummary: COUNT_SCAN { token: 1, user_company_id: 1, discard_time: 1 } keysExamined:3852981 docsExamined:0 cursorExhausted:1 numYields:30104 nreturned:1 reslen:257 locks:{ Global: { acquireCount: { r: 60212 } }, Database: { acquireCount: { r: 30106 } }, Collection: { acquireCount: { r: 30106 } } } protocol:op_msg 2515ms

After many times of verification, the efficiency of the old version of mongodb is higher than that of the new version using aggregation to calculate values.

db.public_leads.count({token:"true"}, {maxTimeMS: 5000})

2.04s

db.getCollection('public_leads').aggregate([          
{$match:{token:"true"}},
{$group:{_id:1, count: {$sum:1}}}
])          

2.63s

Looking forward to your reply !

[christophstrobl]

Count can operate upon collection statistics which is fast but may return an inaccurate result. Details can be found in the MongoDB documentation.