Skip to content

Make applyRemoteEvent idempotent #2263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Oct 15, 2019

Conversation

schmidt-sebastian
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian commented Oct 11, 2019

This one was the most "fun" so far.

Major changes:

  • Make queryDataByTarget an immutable map, so I can discard changes if the transaction inside of applyRemoteEvent fails.
  • Move the checks whether a CollectionParent index has been written outside of the RDC and the MC and instead move it to the callsites (this can lead to slightly more calls since the data structure is no longer used throughout the entire lifetime of the client).

});

// Reset newQueryDataByTargetMap in case this transaction gets re-run.
newQueryDataByTargetMap = this.queryDataByTarget;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: The only change in this block is line 486 to 491 and the fact that everything below uses newQueryDataByTargetMap.

export interface ActiveTargets {
[id: number]: unknown;
}
export type ActiveTargets = SortedMap<TargetId, unknown>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this type so I could continue to parse the target map from LocalStore.

@schmidt-sebastian schmidt-sebastian changed the title Mrschmidt/applyremoteevent Make applyRemoteEvent idempotent Oct 11, 2019
@@ -31,30 +30,17 @@ import { SimpleDbStore } from './simple_db';
* A persisted implementation of IndexManager.
*/
export class IndexedDbIndexManager implements IndexManager {
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find where this cache is kept now. Unconditionally writing it would be a pretty major regression.

Or is the equivalent supposed to be the collectionParents addition in indexeddb_mutation_queue.ts? This cache ensures that we only ever write a collection parent index entry once for the entire duration of the app. The indexeddb_mutation_queue.ts changes ensure uniqueness within a mutation batch, but that's a far lesser benefit.

On the whole I think unwinding all of this in the name of idempotency is a bad deal. If we added transaction.onCommitted this wouldn't be impacted at all. This is the second case we were looking for--I think it makes sense to pursue that instead of these changes.

@wilhuff wilhuff assigned schmidt-sebastian and unassigned wilhuff Oct 11, 2019
@schmidt-sebastian schmidt-sebastian force-pushed the mrschmidt/applyremoteevent branch from 414beec to 2843d42 Compare October 11, 2019 18:40
@schmidt-sebastian
Copy link
Contributor Author

I am not sure why this doesn't compile according to Travis (I will check after 1 pm), but I modified this PR to pull in the latest changes. Unfortunately, I still had to keep some of the new logic that determines whether we have already seen a collection parent before we write it, since the accounting now only happens after a transaction commit. We might be able to clean this up more if we add a onAborted callback, but that adds even more complexity.

Copy link
Contributor

@wilhuff wilhuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wilhuff wilhuff removed their assignment Oct 14, 2019
@schmidt-sebastian schmidt-sebastian merged commit 5cf8e53 into mrschmidt/idempotent Oct 15, 2019
hsubox76 pushed a commit that referenced this pull request Oct 15, 2019
* Add transaction retries (#2250)

* Marking SimpleDb calls as idempotent (#2251)

* Mark mostly readonly calls as idempotent (#2252)

* Fix test failure (#2256)

* Make handleUserChange idempotent (#2257)

* Temporarily disable CountingQueryEngine tests (#2258)

*  Improve test hack (#2259)

* Improve test hack

* Comment in test hack

* Make getNewDocumentChanges() idempotent (#2255)

* Add onCommitted listeners for transactions (#2265)

* Fix build

* Fix Lint

* Make applyRemoteEvent idempotent (#2263)

* Make notifyLocalViewChanges idempotent (#2268)

* Make releaseQuery idempotent (#2266)

* Mark acknowledgeBatch and rejectBatch idempotent (#2269)

* idempotent `allocateQuery` and `notifyLocalViewChanges` (#2264)

* Mark collectGarbage idempotent (#2267)

* Idempotency: Address TODOs, add Changelog (#2270)
@schmidt-sebastian schmidt-sebastian deleted the mrschmidt/applyremoteevent branch October 15, 2019 22:24
@firebase firebase locked and limited conversation to collaborators Nov 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants