Skip to content

Bundle's named query resume. #3395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 51 commits into from
Aug 6, 2020
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
9602712
Renaming interfaces without leading I
wu-hui May 15, 2020
c5e783e
Initial commit of bundle reading - for web only.
wu-hui May 21, 2020
5e7fb89
Tests only run when it is not Node.
wu-hui May 21, 2020
1ee1615
Fix redundant imports
wu-hui May 21, 2020
18f0be1
Fix missing textencoder
wu-hui May 21, 2020
aa455bf
Remove generator.
wu-hui May 29, 2020
78248cd
Support bundle reader for Node
wu-hui May 23, 2020
83160a1
Fix rebase errors.
wu-hui May 29, 2020
24e10cb
Remote 'only'
wu-hui May 29, 2020
9d6edc5
Merge branch 'wuandy/Bundles' into wuandy/BundleReaderNode
wu-hui Jun 5, 2020
4cbe608
Merge branch 'wuandy/Bundles' into wuandy/BundleReaderNode
wu-hui Jun 5, 2020
4313e51
Added more comments, and more tests for Node.
wu-hui Jun 5, 2020
296cfc4
Implement BundleCache.
wu-hui Jun 5, 2020
fff3d36
Add applyBundleDocuments to local store.
wu-hui Jun 1, 2020
fb762de
Add rest of bundle service to localstore
wu-hui Jun 1, 2020
1ec4182
Simplify change buffer get read time logic.
wu-hui Jun 2, 2020
cd3ab7a
Fix lint errors
wu-hui Jun 2, 2020
d991c75
Add comments.
wu-hui Jun 2, 2020
af097c5
Change localstore to check for newer bundle directly.
wu-hui Jun 2, 2020
e735e23
temp checkin
wu-hui Jun 3, 2020
17ba434
Implement without async/await
wu-hui Jun 4, 2020
f808d8d
Major code complete.
wu-hui Jun 4, 2020
db1d864
Integration tests added.
wu-hui Jun 6, 2020
979ffd9
Added spec tests.
wu-hui Jun 9, 2020
f7ff495
Add comments and move types to d.ts
wu-hui Jun 9, 2020
556a007
Support loading string for real.
wu-hui Jun 9, 2020
adf1504
Makes sure SDK still works after loading bad bundles.
wu-hui Jun 11, 2020
b364ab0
Better spect test.
wu-hui Jun 12, 2020
2588f20
Load bundles from secondary clients should raise snapshots
wu-hui Jun 13, 2020
93bc964
Support secondary to secondary.
wu-hui Jun 13, 2020
cb56334
Implementation.
wu-hui Jun 14, 2020
cd88414
Implements bundled query loading and resuming.
wu-hui Jun 15, 2020
99610d7
Merge branch 'wuandy/Bundles' into wuandy/BundlesLoadFromSecondary
wu-hui Jul 11, 2020
a6963dc
Merge branch 'wuandy/BundlesLoadFromSecondary' into wuandy/BundleQuer…
wu-hui Jul 12, 2020
d738d9d
Undo rename
wu-hui Jul 12, 2020
9b00a85
Merge branch 'wuandy/Bundles' into wuandy/BundleQueryResume
wu-hui Jul 24, 2020
23ea4d8
Make loadBundle work with exp build
wu-hui Jul 27, 2020
27e0fd8
Add TODO
wu-hui Jul 27, 2020
8147aea
Merge branch 'wuandy/ExpBundlesApi' into wuandy/BundleQueryResume
wu-hui Jul 27, 2020
a3d0bb0
Undo some merges
wu-hui Jul 27, 2020
48658b4
Merge branch 'wuandy/BundleQueryResume' of github.com:firebase/fireba…
wu-hui Jul 27, 2020
8baf8fc
Use legacy.LoadBundleTask
wu-hui Jul 28, 2020
7abcf19
Merge branch 'wuandy/ExpBundlesApi' into wuandy/BundleQueryResume
wu-hui Jul 28, 2020
f228dfb
Address first round feedbacks
wu-hui Jul 28, 2020
5a6e8ea
Merge branch 'wuandy/Bundles' into wuandy/BundleQueryResume
wu-hui Jul 28, 2020
70d350a
Save readtime to target cache and remove readtime from Query
wu-hui Jul 31, 2020
8ea9e7b
Save named queries to target cache as well.
wu-hui Jul 31, 2020
487982c
Fix more errors.
wu-hui Jul 31, 2020
9f1e135
Address feedback
wu-hui Aug 5, 2020
5eb05d9
Remove readTime from allocateTarget
wu-hui Aug 5, 2020
e057fc1
drop resume token when updating read time.
wu-hui Aug 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/firestore-types/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ export class FirebaseFirestore {
bundleData: ArrayBuffer | ReadableStream<ArrayBuffer> | string
): LoadBundleTask;

namedQuery(name: string): Promise<Query | null>;

INTERNAL: { delete: () => Promise<void> };
}

Expand Down
1 change: 1 addition & 0 deletions packages/firestore/exp/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ export {
waitForPendingWrites,
disableNetwork,
enableNetwork,
namedQuery,
loadBundle,
terminate
} from './src/api/database';
Expand Down
15 changes: 15 additions & 0 deletions packages/firestore/exp/src/api/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ import {
indexedDbClearPersistence
} from '../../../src/local/indexeddb_persistence';
import { LoadBundleTask } from '../../../src/api/bundle';
import { Query } from '../../../lite';

/**
* The root reference to the Firestore database and the entry point for the
Expand Down Expand Up @@ -310,3 +311,17 @@ export function loadBundle(

return resultTask;
}

export async function namedQuery(
firestore: firestore.FirebaseFirestore,
name: string
): Promise<firestore.Query | null> {
const firestoreImpl = cast(firestore, Firestore);
const client = await firestoreImpl._getFirestoreClient();
const namedQuery = await client.getNamedQuery(name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs merging again. The call to getNamedQuery should call a free-standing version that takes LocalStore as an argument. All methods in FirestoreClient have been updated.

Let me know if you need help with this. You can also defer to a follow up PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'd like to defer to my a follow up PR.

if (!namedQuery) {
return null;
}

return new Query(firestoreImpl, null, namedQuery.query, namedQuery.readTime);
}
7 changes: 7 additions & 0 deletions packages/firestore/exp/test/shim.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ import {
increment,
initializeFirestore,
loadBundle,
namedQuery,
onSnapshot,
onSnapshotsInSync,
parent,
Expand Down Expand Up @@ -173,6 +174,12 @@ export class FirebaseFirestore implements legacy.FirebaseFirestore {
return loadBundle(this._delegate, bundleData)!;
}

async namedQuery(name: string): Promise<legacy.Query | null> {
return namedQuery(this._delegate, name).then(query => {
return query ? new Query(this, query) : null;
});
}

INTERNAL = {
delete: () => terminate(this._delegate)
};
Expand Down
10 changes: 10 additions & 0 deletions packages/firestore/src/api/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -504,6 +504,16 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {
return resultTask;
}

async namedQuery(name: string): Promise<firestore.Query | null> {
this.ensureClientConfigured();
const namedQuery = await this._firestoreClient!.getNamedQuery(name);
if (!namedQuery) {
return null;
}

return new Query(namedQuery.query, this, null);
}

ensureClientConfigured(): FirestoreClient {
if (!this._firestoreClient) {
// Kick off starting the client but don't actually wait for it.
Expand Down
8 changes: 7 additions & 1 deletion packages/firestore/src/core/firestore_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

import { CredentialsProvider } from '../api/credentials';
import { User } from '../auth/user';
import { LocalStore } from '../local/local_store';
import { getNamedQuery, LocalStore } from '../local/local_store';
import { GarbageCollectionScheduler, Persistence } from '../local/persistence';
import { Document, NoDocument } from '../model/document';
import { DocumentKey } from '../model/document_key';
Expand Down Expand Up @@ -50,6 +50,7 @@ import { BundleReader } from '../util/bundle_reader';
import { LoadBundleTask } from '../api/bundle';
import { newTextEncoder } from '../platform/serializer';
import { toByteStreamReader } from '../platform/byte_stream_reader';
import { NamedQuery } from './bundle';

const LOG_TAG = 'FirestoreClient';
const MAX_CONCURRENT_LIMBO_RESOLUTIONS = 100;
Expand Down Expand Up @@ -533,4 +534,9 @@ export class FirestoreClient {
});
});
}

getNamedQuery(queryName: string): Promise<NamedQuery | undefined> {
this.verifyNotTerminated();
return getNamedQuery(this.localStore, queryName);
}
}
12 changes: 4 additions & 8 deletions packages/firestore/src/local/indexeddb_bundle_cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export class IndexedDbBundleCache implements BundleCache {
.get(bundleId)
.next(bundle => {
if (bundle) {
return fromDbBundle(this.serializer, bundle);
return fromDbBundle(bundle);
}
return undefined;
});
Expand All @@ -57,9 +57,7 @@ export class IndexedDbBundleCache implements BundleCache {
transaction: PersistenceTransaction,
bundleMetadata: bundleProto.BundleMetadata
): PersistencePromise<void> {
return bundlesStore(transaction).put(
toDbBundle(this.serializer, bundleMetadata)
);
return bundlesStore(transaction).put(toDbBundle(bundleMetadata));
}

getNamedQuery(
Expand All @@ -70,7 +68,7 @@ export class IndexedDbBundleCache implements BundleCache {
.get(queryName)
.next(query => {
if (query) {
return fromDbNamedQuery(this.serializer, query);
return fromDbNamedQuery(query);
}
return undefined;
});
Expand All @@ -80,9 +78,7 @@ export class IndexedDbBundleCache implements BundleCache {
transaction: PersistenceTransaction,
query: bundleProto.NamedQuery
): PersistencePromise<void> {
return namedQueriesStore(transaction).put(
toDbNamedQuery(this.serializer, query)
);
return namedQueriesStore(transaction).put(toDbNamedQuery(query));
}
}

Expand Down
27 changes: 6 additions & 21 deletions packages/firestore/src/local/local_serializer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,7 @@ function isDocumentQuery(dbQuery: DbQuery): dbQuery is api.DocumentsTarget {
}

/** Encodes a DbBundle to a Bundle. */
export function fromDbBundle(
serializer: LocalSerializer,
dbBundle: DbBundle
): Bundle {
export function fromDbBundle(dbBundle: DbBundle): Bundle {
return {
id: dbBundle.bundleId,
createTime: fromDbTimestamp(dbBundle.createTime),
Expand All @@ -292,10 +289,7 @@ export function fromDbBundle(
}

/** Encodes a BundleMetadata to a DbBundle. */
export function toDbBundle(
serializer: LocalSerializer,
metadata: bundleProto.BundleMetadata
): DbBundle {
export function toDbBundle(metadata: bundleProto.BundleMetadata): DbBundle {
return {
bundleId: metadata.id!,
createTime: toDbTimestamp(fromVersion(metadata.createTime!)),
Expand All @@ -304,22 +298,16 @@ export function toDbBundle(
}

/** Encodes a DbNamedQuery to a NamedQuery. */
export function fromDbNamedQuery(
serializer: LocalSerializer,
dbNamedQuery: DbNamedQuery
): NamedQuery {
export function fromDbNamedQuery(dbNamedQuery: DbNamedQuery): NamedQuery {
return {
name: dbNamedQuery.name,
query: fromBundledQuery(serializer, dbNamedQuery.bundledQuery),
query: fromBundledQuery(dbNamedQuery.bundledQuery),
readTime: fromDbTimestamp(dbNamedQuery.readTime)
};
}

/** Encodes a NamedQuery from a bundle proto to a DbNamedQuery. */
export function toDbNamedQuery(
serializer: LocalSerializer,
query: bundleProto.NamedQuery
): DbNamedQuery {
export function toDbNamedQuery(query: bundleProto.NamedQuery): DbNamedQuery {
return {
name: query.name!,
readTime: toDbTimestamp(fromVersion(query.readTime!)),
Expand All @@ -334,7 +322,6 @@ export function toDbNamedQuery(
* including features exists only in SDKs (for example: limit-to-last).
*/
export function fromBundledQuery(
serializer: LocalSerializer,
bundledQuery: bundleProto.BundledQuery
): Query {
const query = convertQueryTargetToQuery({
Expand All @@ -353,19 +340,17 @@ export function fromBundledQuery(

/** Encodes a NamedQuery proto object to a NamedQuery model object. */
export function fromProtoNamedQuery(
serializer: LocalSerializer,
namedQuery: bundleProto.NamedQuery
): NamedQuery {
return {
name: namedQuery.name!,
query: fromBundledQuery(serializer, namedQuery.bundledQuery!),
query: fromBundledQuery(namedQuery.bundledQuery!),
readTime: fromVersion(namedQuery.readTime!)
};
}

/** Encodes a BundleMetadata proto object to a Bundle model object. */
export function fromBundleMetadata(
serializer: LocalSerializer,
metadata: bundleProto.BundleMetadata
): Bundle {
return {
Expand Down
Loading