Skip to content

Handle IndexedDB errors in unsubscribe callback #3162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Jul 9, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9686261
Handle IndexedDb errors in unsubscribe callback
schmidt-sebastian Jun 4, 2020
131c6cd
Merge branch 'master' into mrschmidt/unsubscribe
schmidt-sebastian Jun 26, 2020
024c2e7
Review
schmidt-sebastian Jun 26, 2020
0e00088
Create smart-cars-doubt.md
schmidt-sebastian Jun 26, 2020
1f56feb
Update smart-cars-doubt.md
schmidt-sebastian Jun 26, 2020
182d100
WIP
schmidt-sebastian Jun 29, 2020
b45bb4f
Merge branch 'mrschmidt/unsubscribe' of github.com:firebase/firebase-…
schmidt-sebastian Jun 29, 2020
91e915c
update codeowner to allow approving changeset changes (#3308)
Feiyang1 Jun 26, 2020
9a5358f
Add getDocFromCache() & getDocFromServer() (#3285)
schmidt-sebastian Jun 26, 2020
f124462
Remove makeConstructorPrivate (#3309)
schmidt-sebastian Jun 26, 2020
657def9
Reduce size impact of Target/TargetImpl pattern (#3272)
schmidt-sebastian Jun 26, 2020
f334261
Update dependency chalk to v4 (#3298)
renovate[bot] Jun 26, 2020
282cc41
Fix updateToken (#3312)
Feiyang1 Jun 27, 2020
09672eb
Add getQuery(), getQueryFromCache() and getQueryFromServer() (#3294)
schmidt-sebastian Jun 27, 2020
837cc1b
Add setDoc, updateDoc, deleteDoc and addDoc (#3306)
schmidt-sebastian Jun 27, 2020
3947788
Clean up unused FCM secret (#3311)
zwu52 Jun 27, 2020
4b8b439
Add set() overrides to lite sdk (#3291)
Jun 27, 2020
726697f
Add writeBatch(), WriteBatch (#3307)
schmidt-sebastian Jun 29, 2020
7cdfa36
Add terminate() and snapshotEqual() to firestore-exp (#3313)
schmidt-sebastian Jun 29, 2020
d3ca86c
Make exp release script work again (#3301)
Feiyang1 Jun 29, 2020
5ae98a4
Make onSnapshot work with rxjs observers (#3318)
Feiyang1 Jun 29, 2020
7bcb1ff
Update dependency firebase-tools to v8 (#3300)
renovate[bot] Jun 29, 2020
d88058e
Point functions-exp to the right typing file (#3322)
Feiyang1 Jun 29, 2020
6960241
Publish firebase@exp 0.800.2
Feiyang1 Jun 29, 2020
5b26aed
Update dependency eslint to v7 (#3299)
renovate[bot] Jun 30, 2020
0d2b01d
Add Snapshot Listeners to firestore-exp (#3317)
schmidt-sebastian Jun 30, 2020
7042a71
Ignore failures
schmidt-sebastian Jun 30, 2020
01abb6b
Merge branch 'master' into mrschmidt/unsubscribe
schmidt-sebastian Jun 30, 2020
0bdd73e
Merge branch 'master' into mrschmidt/unsubscribe
schmidt-sebastian Jun 30, 2020
af7e24d
Shorter comment
schmidt-sebastian Jun 30, 2020
db788d1
Fix test
schmidt-sebastian Jun 30, 2020
1ba0fe1
Review
schmidt-sebastian Jul 9, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions packages/firestore/src/core/event_manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,21 +102,19 @@ export class EventManager implements SyncEngineListener {

async unlisten(listener: QueryListener): Promise<void> {
const query = listener.query;
let lastListen = false;

const queryInfo = this.queries.get(query);
if (queryInfo) {
const i = queryInfo.listeners.indexOf(listener);
if (i >= 0) {
queryInfo.listeners.splice(i, 1);
lastListen = queryInfo.listeners.length === 0;
if (queryInfo.listeners.length > 1) {
queryInfo.listeners.splice(i, 1);
} else {
await this.syncEngine.unlisten(query);
this.queries.delete(query);
}
}
}

if (lastListen) {
this.queries.delete(query);
return this.syncEngine.unlisten(query);
}
}

onWatchChange(viewSnaps: ViewSnapshot[]): void {
Expand Down
2 changes: 1 addition & 1 deletion packages/firestore/src/core/firestore_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ export class FirestoreClient {
if (this.clientTerminated) {
return;
}
this.asyncQueue.enqueueAndForget(() => {
this.asyncQueue.enqueueRetryable(() => {
Copy link
Contributor

@wilhuff wilhuff Jun 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary? I thought that if we failed we tore down the streams? At that point haven't we effectively unlistened anyway? All that's missing is updating our in-memory bookkeeping to indicate that we don't want to restart the target when IndexedDB becomes available again.

One other concern is that if IndexedDB is failing we've likely been put in the background and may not come back for a while--possibly more than the 30 minutes for which a resume token means free query resumption. If we were asleep that long the user would be charged for the query anew--for a query they were trying to stop listening to. If all we're losing is the resume token and the sequence numbers for exact GC ordering, it seems better to let the unlisten succeed despite failing to write.

So in the primary tab, if we fail trying to save metadata data about unlistening that's not really a big deal. Can a secondary tab fail trying to tell the primary that it wants to unlisten? In that case we should probably retry.

If this squares with your understanding, it seems like this line may actually be correct: we do want to retry unlistens but only when it's the secondary tab asking the primary tab to do it. The part that's missing is that primary tabs should handle IndexedDB errors and succeed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path calls into LocalStore to invoke "Release Target" and then calls RemoteStore. We could change the plumbing here to drive this from RemoteStore, in which case RemoteStore could go offline and do its own retry. That sounds a like a fair bit of code churn that may lead to some unexpected side effects - the mapping between the lastListen state in EventManager and the active list of targets in SyncEngine/LocalStore may go out of sync.

One easy thing we could do here is just swallow IndexedDB failures if "Release Target" fails, which would even remove some of the newly added error handling in RemoteStore. Do you think that's worth exploring?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just swallow IndexedDB failures if "Release Target" fails

This is what I was suggesting: releasing a target in the LocalStore is just about making sure the final updates are made for GC metadata (and the in-memory tracking of the target). It may be that even LocalStore itself may need to change so that even if the transaction fails it will remove the target from its own set of live things.

I'm not sure this undoes the error handling in RemoteStore, since it's really only unsubscribe can work this way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the PR to ignore errors in "Release target". Unfortunately, since a failed Target can generate a "fake" Remote Event (for Limbo resolutions), the code path that calls into this from RemoteStore still has to use the IndexedDB fallback.

return this.eventMgr.unlisten(listener);
});
}
Expand Down
54 changes: 54 additions & 0 deletions packages/firestore/test/unit/specs/recovery_spec.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -746,4 +746,58 @@ describeSpec('Persistence Recovery', ['no-ios', 'no-android'], () => {
});
}
);

specTest('Does not raise events when unlisten fails', [], () => {
const query = Query.atPath(path('collection'));
const doc1 = doc('collection/key1', 1, { foo: 'a' });
const doc2 = doc('collection/key2', 2, { foo: 'b' });
return (
spec()
.userListens(query)
.watchAcksFull(query, 1000, doc1)
.expectEvents(query, {
added: [doc1]
})
.failDatabaseTransactions('Release target')
.userUnlistens(query)
// Target remains active in RemoteStore, but events are suppressed
.expectActiveTargets({ query })
.watchSends({ affects: [query] }, doc2)
.recoverDatabase()
.runTimer(TimerId.AsyncQueueRetry)
.expectActiveTargets()
);
});

specTest('Can re-listen to query when unlisten fails', [], () => {
const query = Query.atPath(path('collection'));
const doc1 = doc('collection/key1', 1, { foo: 'a' });
const doc2 = doc('collection/key2', 2, { foo: 'b' });
return (
spec()
.userListens(query)
.watchAcksFull(query, 1000, doc1)
.expectEvents(query, {
added: [doc1]
})
.failDatabaseTransactions('Release target')
.userUnlistens(query)
// Target remains active since 'Release target' failed
.expectActiveTargets({ query })
.userListens(query)
.expectEvents(query, {
added: [doc1]
})
.recoverDatabase()
.runTimer(TimerId.AsyncQueueRetry)
// Target remains active as it has now been listened to twice and
// unsubscribed from only once
.watchSends({ affects: [query] }, doc2)
.watchSnapshots(2000)
.expectEvents(query, {
added: [doc2]
})
.userUnlistens(query)
);
});
});
6 changes: 5 additions & 1 deletion packages/firestore/test/unit/specs/spec_builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,11 @@ export class SpecBuilder {
const targetId = this.queryMapping.get(target)!;
this.removeQueryFromActiveTargets(query, targetId);

if (this.config.useGarbageCollection && !this.activeTargets[targetId]) {
if (
this.config.useGarbageCollection &&
!this.injectFailures &&
!this.activeTargets[targetId]
) {
this.queryMapping.delete(target);
this.queryIdGenerator.purge(target);
}
Expand Down
25 changes: 23 additions & 2 deletions packages/firestore/test/unit/specs/spec_test_runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -399,8 +399,29 @@ abstract class TestRunner {
const query = parseQuery(querySpec);
const eventEmitter = this.queryListeners.get(query);
debugAssert(!!eventEmitter, 'There must be a query to unlisten too!');
this.queryListeners.delete(query);
await this.queue.enqueue(() => this.eventManager.unlisten(eventEmitter!));

const deferred = new Deferred<void>();
await this.queue.enqueueRetryable(async () => {
try {
await this.eventManager.unlisten(eventEmitter!);
const currentEventEmitter = this.queryListeners.get(query);
// Before removing the listener, verify that the query hasn't been
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must be missing some context for this comment. If a user attempts to unlisten, that fails, gets in the retry queue, and then they listen to the same query while we're retrying, doesn't the event manager have two listeners for that query? The first query will stil succeed in unlistening.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

queryListeners is a map of active listeners that the Spec tests care about. This code here was my first attempt at making sure that we don't remove a query from the Spec tests view if another operation has re-registered it. I was able to simplify this though by removing the query directly and outside of the queue, which should have no effect on the spec tests since this step would have previously executed in a very similar order.

// relistened to in between retry events. We only remove the listener
// if the current event emitter still matches the event emitter at the
// time of the unlisten.
if (currentEventEmitter === eventEmitter) {
this.queryListeners.delete(query);
}
} catch (e) {
expect(this.persistence.injectFailures).contains('Release target');
throw e;
} finally {
// Resolve the deferred Promise even if the operation failed. This allows
// the spec tests to manually retry the failed user change.
deferred.resolve();
}
});
return deferred.promise;
}

private doSet(setSpec: SpecUserSet): Promise<void> {
Expand Down