Skip to content

Dequeue limbo resolutions when their respective queries are stopped #4395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 5, 2021

Conversation

dconeybe
Copy link
Contributor

@dconeybe dconeybe commented Feb 2, 2021

Fix a bug where enqueued limbo resolutions are left in the queue even after all targets that care about their resolutions are stopped. This erroneous behavior was found in and reported against the Android SDK (firebase/firebase-android-sdk#2311) but also applies to the Web and iOS SDKs. This fix will be ported to those other SDKs. This bug was introduced when limbo resolution throttling was implemented almost a year ago (#2790).

@changeset-bot
Copy link

changeset-bot bot commented Feb 2, 2021

🦋 Changeset detected

Latest commit: 03a9c22

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 8 packages
Name Type
@firebase/firestore Patch
firebase Patch
@firebase/rules-unit-testing Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

'@firebase/firestore': patch
---

Fix a bug where local cache inconsistencies were unnecessarily being resolved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes or Fixed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

*/
enqueuedLimboResolutions: DocumentKey[] = [];
enqueuedLimboResolutions = new Set<string>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should maybe mention that we can use this datastructure in JS because it preserves insertion order.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const expectedLimboDocs = Array.from(this.expectedEnqueuedLimboDocs, key =>
key.path.canonicalString()
).sort();
expect(actualLimboDocs).to.deep.equal(expectedLimboDocs);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep some of the descriptiveness of the original error, you should add a message argument (to the last function call) that mentions that this codepath validates limbo resolutions. Otherwise, an error will show up, but it will not be all that helpful.

Furthermore, you should use to.have.members and remove sort().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const doc2 = doc('collection2/doc', 1000, { key: 2 });
const query2 = query('collection2');
const filteredQuery2 = query('collection2', filter('key', '==', 2));
const filteredQuery3 = query('collection2', filter('key', '>=', 2));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe:

s/filteredQuery2/filteredQuery2a
s/filteredQuery3/filteredQuery2b

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 945 to 952
.userListens(query1)
.watchAcksFull(query1, 1000, doc1)
.expectEvents(query1, { added: [doc1] })
.userUnlistens(query1)
.userListens(filteredQuery1)
.expectEvents(filteredQuery1, { added: [doc1], fromCache: true })
.watchAcksFull(filteredQuery1, 1001)
.expectLimboDocs(doc1.key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you flip the queries around, the test becomes easier to follow. Here I am now asking: "Doesn't it make sense that the filtered query filters out doc1?" If you flip it and run the filteredQuery before the fullQuery (which might be a better name) then the test becomes a bit more obvious.

Applies everywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean. I had actually originally written it that way. But then I added a third filtered query to some of the tests and it seemed to be a detriment to readability to issue a filtered query followed by a full query followed by another filtered query. For consistency between the test cases, I modified all of the tests to issue the full queries first.

What are your thoughts on the following options to address this:

  1. Leave it as-is, with the full queries issued first.
  2. Always issue the filtered queries first, unless there are two filtered queries, in which case issue the full queries first.
  3. Find another way to put the same document into limbo with two different queries, maybe with a limit query?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going with "one of the above seems ok". I think the argument for consistency makes sense, and the test can generally be followed, as long as the person looking at it also looks at the actual contents of the docs/queries.

// verify that doc2 *is* removed from the limbo resolution queue.
.userUnlistens(filteredQuery2)
.expectLimboDocs(doc1.key)
.expectEnqueuedLimboDocs()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Both of the tests below seem to be subsets of this test. I would therefore move this test under the other tests to first verify the basic implementation before going into the more completed cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@schmidt-sebastian
Copy link
Contributor

Let me know if you change the spec tests so much that you want another review.

@dconeybe
Copy link
Contributor Author

dconeybe commented Feb 4, 2021

Let me know if you change the spec tests so much that you want another review.

I had a few questions based on the comments you left.

@google-oss-bot
Copy link
Contributor

Binary Size Report

Affected SDKs

  • @firebase/analytics

    Type Base (6766da2) Head (c0088ad) Diff
    esm2017 ? 18.6 kB ? (?)
    main ? 24.4 kB ? (?)
    module ? 23.3 kB ? (?)
  • @firebase/analytics-exp

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 22.2 kB ? (?)
    esm2017 ? 17.6 kB ? (?)
    main ? 23.1 kB ? (?)
    module ? 22.2 kB ? (?)
  • @firebase/app

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 11.2 kB ? (?)
    esm2017 ? 9.60 kB ? (?)
    lite ? 9.27 kB ? (?)
    lite-esm2017 ? 7.89 kB ? (?)
    main ? 11.0 kB ? (?)
    module ? 11.2 kB ? (?)
    react-native ? 10.7 kB ? (?)
  • @firebase/auth

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 181 kB ? (?)
    main ? 181 kB ? (?)
    module ? 181 kB ? (?)
  • @firebase/component

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 5.18 kB ? (?)
    esm2017 ? 3.99 kB ? (?)
    main ? 5.90 kB ? (?)
    module ? 5.18 kB ? (?)
  • @firebase/database

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 276 kB ? (?)
    esm2017 ? 243 kB ? (?)
    main ? 279 kB ? (?)
    module ? 276 kB ? (?)
  • @firebase/database-exp

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 277 kB ? (?)
    esm2017 ? 241 kB ? (?)
    main ? 279 kB ? (?)
    module ? 277 kB ? (?)
  • @firebase/firestore

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 259 kB ? (?)
    esm2017 ? 201 kB ? (?)
    main ? 531 kB ? (?)
    module ? 259 kB ? (?)
    react-native ? 202 kB ? (?)
  • @firebase/firestore-exp

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 199 kB ? (?)
    main ? 505 kB ? (?)
    module ? 199 kB ? (?)
    react-native ? 200 kB ? (?)
  • @firebase/firestore-lite

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 64.7 kB ? (?)
    main ? 143 kB ? (?)
    module ? 64.7 kB ? (?)
    react-native ? 65.0 kB ? (?)
  • @firebase/firestore/bundle

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 266 kB ? (?)
    esm2017 ? 155 kB ? (?)
    main ? 527 kB ? (?)
    module ? 266 kB ? (?)
    react-native ? 155 kB ? (?)
  • @firebase/firestore/memory

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 196 kB ? (?)
    esm2017 ? 152 kB ? (?)
    main ? 324 kB ? (?)
    module ? 196 kB ? (?)
    react-native ? 152 kB ? (?)
  • @firebase/firestore/memory-bundle

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 205 kB ? (?)
    esm2017 ? 155 kB ? (?)
    main ? 321 kB ? (?)
    module ? 205 kB ? (?)
    react-native ? 155 kB ? (?)
  • @firebase/functions

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 9.87 kB ? (?)
    esm2017 ? 7.69 kB ? (?)
    main ? 10.3 kB ? (?)
    module ? 9.87 kB ? (?)
  • @firebase/installations

    Type Base (6766da2) Head (c0088ad) Diff
    esm2017 ? 16.6 kB ? (?)
    main ? 22.7 kB ? (?)
    module ? 21.6 kB ? (?)
  • @firebase/logger

    Type Base (6766da2) Head (c0088ad) Diff
    esm2017 ? 3.25 kB ? (?)
    main ? 5.75 kB ? (?)
    module ? 4.83 kB ? (?)
  • @firebase/messaging

    Type Base (6766da2) Head (c0088ad) Diff
    esm2017 ? 26.2 kB ? (?)
    main ? 34.9 kB ? (?)
    module ? 34.4 kB ? (?)
  • @firebase/performance

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 27.6 kB ? (?)
    esm2017 ? 25.9 kB ? (?)
    main ? 28.5 kB ? (?)
    module ? 27.6 kB ? (?)
  • @firebase/polyfill

    Type Base (6766da2) Head (c0088ad) Diff
    main ? 747 B ? (?)
    module ? 705 B ? (?)
  • @firebase/remote-config

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 22.4 kB ? (?)
    esm2017 ? 17.4 kB ? (?)
    main ? 23.4 kB ? (?)
    module ? 22.4 kB ? (?)
  • @firebase/rules-unit-testing

    Type Base (6766da2) Head (c0088ad) Diff
    main ? 12.2 kB ? (?)
  • @firebase/storage

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 60.2 kB ? (?)
    esm2017 ? 51.9 kB ? (?)
    main ? 61.2 kB ? (?)
    module ? 60.2 kB ? (?)
  • @firebase/storage/exp

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 49.2 kB ? (?)
    main ? 49.9 kB ? (?)
    module ? 49.2 kB ? (?)
  • @firebase/util

    Type Base (6766da2) Head (c0088ad) Diff
    browser ? 20.2 kB ? (?)
    esm2017 ? 19.0 kB ? (?)
    main ? 22.1 kB ? (?)
    module ? 20.2 kB ? (?)
  • @firebase/webchannel-wrapper

    Type Base (6766da2) Head (c0088ad) Diff
    esm2017 ? 39.5 kB ? (?)
    main ? 47.2 kB ? (?)
    module ? 40.8 kB ? (?)
  • firebase

    Click to show 15 binary size changes.
    Type Base (6766da2) Head (c0088ad) Diff
    firebase-analytics.js ? 35.7 kB ? (?)
    firebase-app.js ? 20.1 kB ? (?)
    firebase-auth.js ? 177 kB ? (?)
    firebase-database.js ? 195 kB ? (?)
    firebase-firestore.js ? 304 kB ? (?)
    firebase-firestore.memory.js ? 243 kB ? (?)
    firebase-functions.js ? 9.99 kB ? (?)
    firebase-installations.js ? 19.0 kB ? (?)
    firebase-messaging.js ? 40.8 kB ? (?)
    firebase-performance-standalone.es2017.js ? 71.8 kB ? (?)
    firebase-performance-standalone.js ? 48.3 kB ? (?)
    firebase-performance.js ? 38.5 kB ? (?)
    firebase-remote-config.js ? 36.8 kB ? (?)
    firebase-storage.js ? 39.8 kB ? (?)
    firebase.js ? 857 kB ? (?)

Test Logs

@google-oss-bot
Copy link
Contributor

Size Analysis Report

Affected Products

Diffs between base commit (6766da2) and head commit (c0088ad) are too large (470,930 characters) to display.

Please check below links to see details from the original test log.

@dconeybe dconeybe merged commit a718518 into master Feb 5, 2021
@dconeybe dconeybe deleted the dconeybe/LimboRemoveFromQueueFix branch February 5, 2021 16:15
@google-oss-bot google-oss-bot mentioned this pull request Feb 9, 2021
@firebase firebase locked and limited conversation to collaborators Mar 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants