Skip to content

Create and document Java sync improved bulk write API #1458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
48b9614
Create and document Java sync improved bulk write API
stIncMale Jul 23, 2024
af854ed
Remove the type parameter from `ClientWriteModel`
stIncMale Jul 23, 2024
1eb7466
Remove `ClientBulkWriteException.create` as we can get by with the co…
stIncMale Jul 25, 2024
5567324
Merge branch 'master' into JAVA-5527
stIncMale Jul 27, 2024
644d561
Merge branch 'master' into JAVA-5527
stIncMale Aug 13, 2024
f566303
Do minor improvements
stIncMale Aug 15, 2024
3d13ffd
Make changes needed for the implementation
stIncMale Aug 15, 2024
cf46b46
Fix formatting in ClientUpdateManyModel
stIncMale Aug 15, 2024
e049234
Make a few minor changes
stIncMale Aug 15, 2024
0e16427
Merge branch 'master' into JAVA-5527
stIncMale Aug 15, 2024
0d518d8
Add more info to the API docs, add `ClientWriteModel` subtypes
stIncMale Aug 22, 2024
6b77f78
Add `ClientWriteModelWithNamespace`
stIncMale Aug 22, 2024
fdb90d2
Remove `throws` declarations from the API
stIncMale Aug 22, 2024
39386fe
Make wording on `ClientWriteModel` methods consistent with that on `C…
stIncMale Aug 22, 2024
f67af3f
Move constructor methods to `ClientNamespacedWriteModel`
stIncMale Aug 22, 2024
395af7a
Use `Optional` to express verbose/summary results
stIncMale Aug 22, 2024
a11e5f6
Make an API doc improvement
stIncMale Aug 27, 2024
f41ed59
Use `Integer` for indexes in `ClientBulkWriteResult.Verbose`. Make `C…
stIncMale Aug 28, 2024
4846e0b
Use `Integer` for indexes in `ClientBulkWriteException`.
stIncMale Aug 28, 2024
dbf9a26
Take `ClientBulkWriteException` into account in `OperationExecutor.ex…
stIncMale Aug 28, 2024
21bb22e
Make `ClientInsertOneResult.getInsertedId` return `Optional`
stIncMale Aug 28, 2024
f026ee3
Update the documentation of `ClientBulkWriteException` and remove a TODO
stIncMale Aug 28, 2024
3584de5
Move results to `com.mongodb.client.model.bulk`
stIncMale Aug 30, 2024
5a17ea0
Merge branch 'master' into JAVA-5527
stIncMale Aug 30, 2024
8e1d770
Move internal results to `com.mongodb.internal.client.model.bulk`
stIncMale Aug 30, 2024
e973615
Document that `bulkWrite` is not supported by serverless instances
stIncMale Aug 30, 2024
e33f592
Add subtypes of `ClientNamespacedWriteModel` and hide `ClientWriteModel`
stIncMale Sep 3, 2024
301a2ba
Remove a garbage comment
stIncMale Sep 3, 2024
6f01e61
Replace `ConcreteClientNamespacedWriteModel` with multiple more speci…
stIncMale Sep 9, 2024
c3224e9
Rearrange `ClientWriteModel` inheritance: "one" should extend "many",…
stIncMale Sep 9, 2024
c057390
Make internal `AbstractClientNamespacedWriteModel` public
stIncMale Sep 9, 2024
dbb6ec8
Rearrange `ClientWriteModel` inheritance: neither "one" nor "many" sh…
stIncMale Sep 9, 2024
f781bca
Make internal `AbstractClientUpdateModel`, `AbstractClientDeleteModel…
stIncMale Sep 9, 2024
3fc86bb
Stop linking to `Filters` from the documentation of `ClientNamespaced…
stIncMale Sep 17, 2024
86e5234
Update driver-core/src/main/com/mongodb/client/model/bulk/ClientBulkW…
stIncMale Sep 18, 2024
fb134f8
Update driver-core/src/main/com/mongodb/client/model/bulk/ClientDelet…
stIncMale Sep 18, 2024
a4bf4d0
Fix typos in API docs
stIncMale Sep 18, 2024
2837235
Change code to always refer to `ClientBulkWriteResult.Verbose` with t…
stIncMale Sep 18, 2024
b6702a6
Rename `ClientBulkWriteResult.Verbose` to `ClientBulkWriteResult.Verb…
stIncMale Sep 23, 2024
24ce6db
Improve partial result API documentation wording
stIncMale Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions driver-core/src/main/com/mongodb/ClientBulkWriteException.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb;

import com.mongodb.bulk.WriteConcernError;
import com.mongodb.client.model.bulk.ClientWriteModel;
import com.mongodb.client.result.bulk.ClientBulkWriteResult;
import com.mongodb.lang.Nullable;

import java.util.List;
import java.util.Map;
import java.util.Optional;

import static com.mongodb.assertions.Assertions.isTrueArgument;
import static java.util.Collections.emptyList;
import static java.util.Collections.emptyMap;
import static java.util.Collections.unmodifiableList;
import static java.util.Collections.unmodifiableMap;
import static java.util.Optional.ofNullable;

/**
* The result of an unsuccessful or partially unsuccessful client-level bulk write operation.
*
* @see ClientBulkWriteResult
* @since 5.3
* @serial exclude
*/
public final class ClientBulkWriteException extends MongoServerException {
private static final long serialVersionUID = 1;

@Nullable
private final MongoException error;
private final List<WriteConcernError> writeConcernErrors;
private final Map<Long, WriteError> writeErrors;
@Nullable
private final ClientBulkWriteResult partialResult;

private ClientBulkWriteException(
final String message,
@Nullable final MongoException error,
@Nullable final List<WriteConcernError> writeConcernErrors,
@Nullable final Map<Long, WriteError> writeErrors,
@Nullable final ClientBulkWriteResult partialResult,
final ServerAddress serverAddress) {
super(message, serverAddress);
// BULK-TODO Should ClientBulkWriteException.getCode be the same as error.getCode,
// and getErrorLabels/hasErrorLabel contain the same labels as error.getErrorLabels?
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe reviewers have thoughts on this comment. My understanding is that the existing bulk write API does not wrap the top-level errors in BulkWriteException, but the new API wraps the top-level errors in ClientBulkWriteException. Given that the driver uses error code and labels to implement some of its functionality, the question seems reasonable. I don't currently have an answer.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will have to think about this more. I'm not sure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all the error labels the driver knows about and uses: TransientTransactionError, UnknownTransactionCommitResult, RetryableWriteError, NoWritesPerformed.

This question splits into two:

  1. What error codes/labels should be copied to ClientBulkWriteException from ClientBulkWriteException.error (the top-level error) for consumption by applications?
    • Simply copying some labels seems to be just incorrect, for example, RetryableWriteError - even if a specific bulkWrite command within a bulk operation can be retried, it does not mean that all previous commands can also be repeated when an application attempts to retry the whole operation. The same reasoning seems to apply to NoWritesPerformed. However, if no bulkWrite commands succeeded, then copying both of these labels seems safe.
    • I don't think we should copy any error codes for consumption by applications.
  2. What error codes/labels should be copied to ClientBulkWriteException from ClientBulkWriteException.error (the top-level error) for consumption by the driver?
    • Given that ClientBulkWriteException will be created by ClientBulkWriteOperation.execute, only the following driver code will have a chance to handle labels and the code from this exception:
      • MongoClusterImpl.OperationExecutorImpl.execute - reads TransientTransactionError, UnknownTransactionCommitResult, and code 91.
      • ClientSessionImpl.withTransaction - reads TransientTransactionError, UnknownTransactionCommitResult.
    • It seems safe to copy the code if its 91.
    • It seems safe to copy the label UnknownTransactionCommitResult.
    • I am not sure about copying TransientTransactionError. @jyemin does this label always result in driver aborting the transaction?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems safest not to copy any error codes / error labels to the top level Exception.

If a user needs to inspect them can't they use the writeErrors map?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems safest not to copy any error codes / error labels to the top level Exception.

I agree that we should not copy error codes/labels for consumption by applications. However, we may still decide to do that for consumption by the driver (and if we do so, the applications will be able to observe that).

I am OK with not copying anything, and instead updating the logic of MongoClusterImpl.OperationExecutorImpl.execute, ClientSessionImpl.withTransaction, and their reactive counterparts such that they look at ClientBulkWriteException.getError.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user needs to inspect them can't they use the writeErrors map?

I don't think that ClientBulkWriteException.getWriteErrors is relevant here. Only the code/labels from the top-level error seem to make sense to consider in the context of copying them to ClientBulkWriteException.

Copy link
Member Author

@stIncMale stIncMale Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dbf9a26, f026ee3.

this.error = error;
this.writeConcernErrors = writeConcernErrors == null ? emptyList() : unmodifiableList(writeConcernErrors);
this.writeErrors = writeErrors == null ? emptyMap() : unmodifiableMap(writeErrors);
this.partialResult = partialResult;
}

/**
* Creates a {@link ClientBulkWriteException}.
* At least one of {@code error}, {@code writeConcernErrors}, {@code writeErrors}, {@code partialResult}
* must be non-{@code null} or non-empty.
*
* @param error The {@linkplain #getError() top-level error}.
* @param writeConcernErrors The {@linkplain #getWriteConcernErrors() write concern errors}.
* @param writeErrors The {@linkplain #getWriteErrors() write errors}.
* @param partialResult The {@linkplain #getPartialResult() partial result}.
* @param serverAddress The {@linkplain MongoServerException#getServerAddress() server address}.
* @return The requested {@link ClientBulkWriteException}.
*/
public static ClientBulkWriteException create(
@Nullable final MongoException error,
@Nullable final List<WriteConcernError> writeConcernErrors,
@Nullable final Map<Long, WriteError> writeErrors,
@Nullable final ClientBulkWriteResult partialResult,
final ServerAddress serverAddress) {
isTrueArgument("At least one of `writeConcernErrors`, `writeErrors`, `partialResult` must be non-null or non-empty",
!(writeConcernErrors == null || writeConcernErrors.isEmpty())
|| !(writeErrors == null || writeErrors.isEmpty())
|| partialResult != null);
String message = "Client-level bulk write operation error on server " + serverAddress + "."
+ (error == null ? "" : " Top-level error: " + error + ".")
+ (writeErrors == null || writeErrors.isEmpty() ? "" : " Write errors: " + writeErrors + ".")
+ (writeConcernErrors == null || writeConcernErrors.isEmpty() ? "" : " Write concern errors: " + writeConcernErrors + ".")
+ (partialResult == null ? "" : " Partial result: " + partialResult + ".");
return new ClientBulkWriteException(message, error, writeConcernErrors, writeErrors, partialResult, serverAddress);
}

/**
* The top-level error. That is an error that is neither a {@linkplain #getWriteConcernErrors() write concern error},
* nor is an {@linkplain #getWriteErrors() error of an individual write operation}.
*
* @return The top-level error. {@linkplain Optional#isPresent() Present} only if a top-level error occurred.
*/
public Optional<MongoException> getError() {
return ofNullable(error);
}

/**
* The {@link WriteConcernError}s that occurred while executing the client-level bulk write operation.
* <p>
* There are no guarantees on mutability of the {@link List} returned.</p>
*
* @return The {@link WriteConcernError}s.
*/
public List<WriteConcernError> getWriteConcernErrors() {
return writeConcernErrors;
}

/**
* The indexed {@link WriteError}s.
* The {@linkplain Map#keySet() keys} are the indexes of the corresponding {@link ClientWriteModel}s
* in the corresponding client-level bulk write operation.
* <p>
* There are no guarantees on mutability or iteration order of the {@link Map} returned.</p>
*
* @return The indexed {@link WriteError}s.
* @see ClientBulkWriteResult#getInsertResults()
* @see ClientBulkWriteResult#getUpdateResults()
* @see ClientBulkWriteResult#getDeleteResults()
*/
public Map<Long, WriteError> getWriteErrors() {
return writeErrors;
}

/**
* The result of the successful part of a client-level bulk write operation.
*
* @return The successful partial result. {@linkplain Optional#isPresent() Present} only if at least one
* {@linkplain ClientWriteModel individual write operation} succeed.
*/
public Optional<ClientBulkWriteResult> getPartialResult() {
return ofNullable(partialResult);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb.client.model.bulk;

import com.mongodb.annotations.Sealed;
import com.mongodb.client.model.Filters;
import com.mongodb.client.result.bulk.ClientBulkWriteResult;
import com.mongodb.internal.client.model.bulk.ConcreteClientBulkWriteOptions;
import com.mongodb.lang.Nullable;
import org.bson.BsonValue;
import org.bson.conversions.Bson;

/**
* The options to apply when executing a client-level bulk write operation.
*
* @since 5.3
*/
@Sealed
public interface ClientBulkWriteOptions {
/**
* Creates the default options.
*
* @return The default options.
*/
static ClientBulkWriteOptions clientBulkWriteOptions() {
return new ConcreteClientBulkWriteOptions();
}

/**
* Enables or disables ordered execution of {@linkplain ClientWriteModel individual write operations}.
* In an ordered execution a failure of an individual operation prevents the rest of them
* from being executed.
* In an unordered execution failures of individual operations do not prevent the rest of them
* from being executed.
*
* @param ordered The ordered flag. If {@code null}, the client defaults to {@code true}.
* @return {@code this}.
*/
ClientBulkWriteOptions ordered(@Nullable Boolean ordered);

/**
* Disables or enables checking against document validation rules, a.k.a., schema validation.
*
* @param bypassDocumentValidation The flag specifying whether to bypass the document validation rules.
* {@code null} represents the server default.
* @return {@code this}.
*/
ClientBulkWriteOptions bypassDocumentValidation(@Nullable Boolean bypassDocumentValidation);

/**
* Sets variables that can be referenced from {@linkplain ClientWriteModel individual write operations}
* with the {@code "$$"} syntax, which in turn requires using {@link Filters#expr(Object)} when specifying filters.
* Values must be constants or expressions that do not reference fields.
*
* @param let The variables. {@code null} represents the server default.
* @return {@code this}.
* @mongodb.driver.manual reference/aggregation-variables/ Variables in Aggregation Expressions
*/
ClientBulkWriteOptions let(@Nullable Bson let);

/**
* Sets the comment to attach to the {@code bulkWrite} administration command.
*
* @param comment The comment. {@code null} represents the server default.
* @return {@code this}.
*/
ClientBulkWriteOptions comment(@Nullable BsonValue comment);

/**
* Enables or disables requesting {@linkplain ClientBulkWriteResult#hasVerboseResults() verbose results}.
*
* @param verboseResults The flag specifying whether to request verbose results.
* If {@code null}, the client defaults to {@code false}.
* This value corresponds inversely to the {@code errorsOnly} field of the {@code bulkWrite} administration command.
* @return {@code this}.
*/
ClientBulkWriteOptions verboseResults(@Nullable Boolean verboseResults);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb.client.model.bulk;

import com.mongodb.annotations.Sealed;
import com.mongodb.client.model.Collation;
import com.mongodb.internal.client.model.bulk.ConcreteClientDeleteOptions;
import com.mongodb.lang.Nullable;
import org.bson.conversions.Bson;

/**
* The options to apply when deleting documents.
*
* @since 5.3
*/
@Sealed
public interface ClientDeleteOptions {
/**
* Creates the default options.
*
* @return The default options.
*/
static ClientDeleteOptions clientDeleteOptions() {
return new ConcreteClientDeleteOptions();
}

/**
* Sets the collation.
*
* @param collation The collation. {@code null} represents the server default.
* @return {@code this}.
*/
ClientDeleteOptions collation(@Nullable Collation collation);

/**
* Sets the index specification,
* {@code null}-ifies {@linkplain #hintString(String) hint string}.
*
* @param hint The index specification. {@code null} represents the server default.
* @return {@code this}.
*/
ClientDeleteOptions hint(@Nullable Bson hint);

/**
* Sets the index name,
* {@code null}-ifies {@linkplain #hint(Bson) hint}.
*
* @param hintString The index name. {@code null} represents the server default.
* @return {@code this}.
*/
ClientDeleteOptions hintString(@Nullable String hintString);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb.client.model.bulk;

import com.mongodb.annotations.Sealed;
import com.mongodb.client.model.Collation;
import com.mongodb.internal.client.model.bulk.ConcreteClientReplaceOptions;
import com.mongodb.lang.Nullable;
import org.bson.conversions.Bson;

/**
* The options to apply when replacing documents.
*
* @since 5.3
*/
@Sealed
public interface ClientReplaceOptions {
/**
* Creates the default options.
*
* @return The default options.
*/
static ClientReplaceOptions clientReplaceOptions() {
return new ConcreteClientReplaceOptions();
}

/**
* Sets the collation.
*
* @param collation The collation. {@code null} represents the server default.
* @return {@code this}.
*/
ClientReplaceOptions collation(@Nullable Collation collation);

/**
* Sets the index specification,
* {@code null}-ifies {@linkplain #hintString(String) hint string}.
*
* @param hint The index specification. {@code null} represents the server default.
* @return {@code this}.
*/
ClientReplaceOptions hint(@Nullable Bson hint);

/**
* Sets the index name,
* {@code null}-ifies {@linkplain #hint(Bson) hint}.
*
* @param hintString The index name. {@code null} represents the server default.
* @return {@code this}.
*/
ClientReplaceOptions hintString(@Nullable String hintString);

/**
* Enables or disables creation of a document if no documents match the filter.
*
* @param upsert The upsert flag. {@code null} represents the server default.
* @return {@code this}.
*/
ClientReplaceOptions upsert(@Nullable Boolean upsert);
}
Loading