Skip to content

MongoTemplate#executeFindMultiInternal allocations could be optimized #3941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
constanine opened this issue Jan 19, 2022 · 1 comment
Closed
Assignees
Labels
type: enhancement A general enhancement

Comments

@constanine
Copy link

constanine commented Jan 19, 2022

In my case, when pagequery the collections in mongodb, which size more than 1000, such as mongoTemplate.find(new Query().limit(1000),xxx.class)
the cost time of querying is too much, and then I find some reason in the code as:

  private <T> List<T> executeFindMultiInternal(CollectionCallback<FindIterable<Document>> collectionCallback,
      CursorPreparer preparer, DocumentCallback<T> documentCallback, String collectionName) {
    try {
      try (MongoCursor<Document> cursor = preparer
          .initiateFind(getAndPrepareCollection(doGetDatabase(), collectionName), collectionCallback::doInCollection)
          .iterator()) {
        List<T> result = new ArrayList<>();
        while (cursor.hasNext()) {
          Document object = cursor.next();
          result.add(documentCallback.doWith(object));
        }
        return result;
      }
    } catch (RuntimeException e) {
      throw potentiallyConvertRuntimeException(e, exceptionTranslator);
    }
  }

The point is new ArrayList() ?! , which length of the list is 1000,this means the list will be copyed 18 times, I think the funtion getAndPrepareCollection() which one returns the bean of MongoCollection.java,and this bean has a funtion called countDocuments(), so how about code like this:

  private <T> List<T> executeFindMultiInternal(CollectionCallback<FindIterable<Document>> collectionCallback,
      CursorPreparer preparer, DocumentCallback<T> documentCallback, String collectionName) {
    try {
	  MongoCollection collections = getAndPrepareCollection(doGetDatabase(), collectionName)
      try (MongoCursor<Document> cursor = preparer
          .initiateFind(collections, collectionCallback::doInCollection)
          .iterator()) {
        List<T> result = new ArrayList<>(collections.countDocuments());
        while (cursor.hasNext()) {
          Document object = cursor.next();
          result.add(documentCallback.doWith(object));
        }
        return result;
      }
    } catch (RuntimeException e) {
      throw potentiallyConvertRuntimeException(e, exceptionTranslator);
    }
  }

The code would reduce the funtcion:java.util.ArrayList#grow be called and save time

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 19, 2022
@mp911de
Copy link
Member

mp911de commented Jan 24, 2022

Introducing a call to collections.countDocuments() introduces another I/O operation that outweighs its benefit if the result is empty or contains a small number of documents. Why don't you use the stream() method that bypasses ArrayList creation entirely?

@christophstrobl christophstrobl linked a pull request Mar 3, 2022 that will close this issue
@mp911de mp911de changed the title org.springframework.data.mongodb.core.MongoTemplate#executeFindMultiInternal should be optimized MongoTemplate#executeFindMultiInternal allocations could be optimized Sep 16, 2022
@mp911de mp911de added type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 16, 2022
@mp911de mp911de added this to the 3.3.7 (2021.1.7) milestone Sep 16, 2022
mp911de pushed a commit that referenced this issue Sep 16, 2022
mp911de pushed a commit that referenced this issue Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants