Skip to content

Treat ABORTED writes as retryable #1456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 8, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions packages/firestore/src/remote/remote_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ import {
PersistentWriteStream
} from './persistent_stream';
import { RemoteSyncer } from './remote_syncer';
import { isPermanentError } from './rpc_error';
import { isPermanentError, isPermanentWriteError } from './rpc_error';
import {
DocumentWatchChange,
ExistenceFilterChange,
Expand Down Expand Up @@ -644,9 +644,10 @@ export class RemoteStore implements TargetMetadataProvider {
}

private async handleHandshakeError(error: FirestoreError): Promise<void> {
// Reset the token if it's a permanent error or the error code is
// ABORTED, signaling the write stream is no longer valid.
if (isPermanentError(error.code) || error.code === Code.ABORTED) {
// Reset the token if it's a permanent error, signaling the write stream is
// no longer valid. Note that the handshake does not count as a write: see
// comments on isPermanentWriteError for details.
if (isPermanentError(error.code)) {
log.debug(
LOG_TAG,
'RemoteStore error before completed handshake; resetting stream token: ',
Expand All @@ -664,7 +665,9 @@ export class RemoteStore implements TargetMetadataProvider {
}

private async handleWriteError(error: FirestoreError): Promise<void> {
if (isPermanentError(error.code)) {
// Only handle permanent errors here. If it's transient, just let the retry
// logic kick in.
if (isPermanentWriteError(error.code)) {
// This was a permanent error, the request itself was the problem
// so it's not going to succeed if we resend it.
const batch = this.writePipeline.shift()!;
Expand Down
22 changes: 22 additions & 0 deletions packages/firestore/src/remote/rpc_error.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ enum RpcCode {
DATA_LOSS = 15
}

/**
* Determines whether an error code represents a permanent error when received
* in response to a non-write operation.
*
* See isPermanentWriteError for classifying write errors.
*/
export function isPermanentError(code: Code): boolean {
switch (code) {
case Code.OK:
Expand Down Expand Up @@ -80,6 +86,22 @@ export function isPermanentError(code: Code): boolean {
}
}

/**
* Determines whether an error code represents a permanent error when received
* in response to a write operation.
*
* Write operations must be handled specially because as of b/119437764, ABORTED
* errors on the write stream should be retried too (even though ABORTED errors
* are not generally retryable).
*
* Note that during the initial handshake on the write stream an ABORTED error
* signals that we should discard our stream token (i.e. it is permanent). This
* means a handshake error should be classified with isPermanentError, above.
*/
export function isPermanentWriteError(code: Code): boolean {
return isPermanentError(code) && code !== Code.ABORTED;
}

/**
* Maps an error Code from a GRPC status identifier like 'NOT_FOUND'.
*
Expand Down
5 changes: 3 additions & 2 deletions packages/firestore/test/unit/specs/spec_builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import {
import { DocumentKey } from '../../../src/model/document_key';
import { JsonObject } from '../../../src/model/field_value';
import {
isPermanentError,
isPermanentWriteError,
mapCodeFromRpcCode,
mapRpcCodeFromCode
} from '../../../src/remote/rpc_error';
Expand Down Expand Up @@ -504,7 +504,8 @@ export class SpecBuilder {

// If this is a permanent error, the write is not expected to be sent
// again.
const isPermanentFailure = isPermanentError(mapCodeFromRpcCode(error.code));
const code = mapCodeFromRpcCode(error.code);
const isPermanentFailure = isPermanentWriteError(code);
const keepInQueue =
options.keepInQueue !== undefined
? options.keepInQueue
Expand Down
2 changes: 1 addition & 1 deletion packages/firestore/test/unit/specs/write_spec.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -633,7 +633,6 @@ describeSpec('Writes:', [], () => {
Code.ALREADY_EXISTS,
Code.PERMISSION_DENIED,
Code.FAILED_PRECONDITION,
Code.ABORTED,
Code.OUT_OF_RANGE,
Code.UNIMPLEMENTED,
Code.DATA_LOSS
Expand Down Expand Up @@ -697,6 +696,7 @@ describeSpec('Writes:', [], () => {
);

for (const code of [
Code.ABORTED,
Code.CANCELLED,
Code.UNKNOWN,
Code.DEADLINE_EXCEEDED,
Expand Down