Skip to content

Implement bundle features in local store. #3200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jun 25, 2020
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9602712
Renaming interfaces without leading I
wu-hui May 15, 2020
c5e783e
Initial commit of bundle reading - for web only.
wu-hui May 21, 2020
5e7fb89
Tests only run when it is not Node.
wu-hui May 21, 2020
1ee1615
Fix redundant imports
wu-hui May 21, 2020
18f0be1
Fix missing textencoder
wu-hui May 21, 2020
aa455bf
Remove generator.
wu-hui May 29, 2020
78248cd
Support bundle reader for Node
wu-hui May 23, 2020
83160a1
Fix rebase errors.
wu-hui May 29, 2020
24e10cb
Remote 'only'
wu-hui May 29, 2020
9d6edc5
Merge branch 'wuandy/Bundles' into wuandy/BundleReaderNode
wu-hui Jun 5, 2020
4cbe608
Merge branch 'wuandy/Bundles' into wuandy/BundleReaderNode
wu-hui Jun 5, 2020
4313e51
Added more comments, and more tests for Node.
wu-hui Jun 5, 2020
296cfc4
Implement BundleCache.
wu-hui Jun 5, 2020
fff3d36
Add applyBundleDocuments to local store.
wu-hui Jun 1, 2020
fb762de
Add rest of bundle service to localstore
wu-hui Jun 1, 2020
1ec4182
Simplify change buffer get read time logic.
wu-hui Jun 2, 2020
cd3ab7a
Fix lint errors
wu-hui Jun 2, 2020
d991c75
Add comments.
wu-hui Jun 2, 2020
af097c5
Change localstore to check for newer bundle directly.
wu-hui Jun 2, 2020
8f0d5da
Merge branch 'wuandy/Bundles' into wuandy/BundleLocalStore
wu-hui Jun 14, 2020
7ea8335
A little more comments.
wu-hui Jun 14, 2020
5a38f0a
Address comments.
wu-hui Jun 17, 2020
9340d6a
Address comments 2
wu-hui Jun 19, 2020
6fc7f4a
Make it tree-shakeable.
wu-hui Jun 19, 2020
bdbd70a
More comments addressing.
wu-hui Jun 23, 2020
8a94ed5
Merge branch 'wuandy/Bundles' into wuandy/BundleLocalStore
wu-hui Jun 24, 2020
6fc92ee
Another around of comments
wu-hui Jun 25, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions packages/firestore/src/core/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@

import { Query } from './query';
import { SnapshotVersion } from './snapshot_version';
import { JsonProtoSerializer } from '../remote/serializer';
import * as bundleProto from '../protos/firestore_bundle_proto';
import * as api from '../protos/firestore_proto_api';
import { DocumentKey } from '../model/document_key';
import { MaybeDocument, NoDocument } from '../model/document';
import { debugAssert } from '../util/assert';

/**
* Represents a Firestore bundle saved by the SDK in its local storage.
Expand All @@ -40,3 +46,43 @@ export interface NamedQuery {
/** The time at which the results for this query were read. */
readonly readTime: SnapshotVersion;
}

/**
* Represents the [metadata, document] pairs from the bundles in an array.
*/
export type BundledDocuments = Array<
[bundleProto.BundledDocumentMetadata, api.Document | undefined]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if you a nested object here: Array<{metadata: ..., document?: }>. Index-based typing is very hard to parse for readers of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

>;

/**
* Helper to convert objects from bundles to model objects in the SDK.
*/
export class BundleConverter {
constructor(private serializer: JsonProtoSerializer) {}

toDocumentKey(name: string): DocumentKey {
return this.serializer.fromName(name);
}

/**
* Converts a [metadata, document] pair to a MaybeDocument.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Converts a [metadata, document] pair to a MaybeDocument.
* Converts a BundledDocument to a MaybeDocument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

*/
toMaybeDocument(
metadata: bundleProto.BundledDocumentMetadata,
doc: api.Document | undefined
): MaybeDocument {
if (metadata.exists) {
debugAssert(!!doc, 'Document is undefined when metadata.exist is true.');
return this.serializer.fromDocument(doc!, false);
} else {
return new NoDocument(
this.toDocumentKey(metadata.name!),
this.toSnapshotVersion(metadata.readTime!)
);
}
}

toSnapshotVersion(time: api.Timestamp): SnapshotVersion {
return this.serializer.fromVersion(time);
}
}
9 changes: 7 additions & 2 deletions packages/firestore/src/core/component_provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ import {
MemoryEagerDelegate,
MemoryPersistence
} from '../local/memory_persistence';
import { BundleConverter } from './bundle';

const MEMORY_ONLY_PERSISTENCE_ERROR_MESSAGE =
'You are using the memory-only build of Firestore. Persistence support is ' +
Expand Down Expand Up @@ -125,10 +126,12 @@ export class MemoryComponentProvider implements ComponentProvider {
}

createLocalStore(cfg: ComponentConfiguration): LocalStore {
const serializer = cfg.platform.newSerializer(cfg.databaseInfo.databaseId);
return new LocalStore(
this.persistence,
new IndexFreeQueryEngine(),
cfg.initialUser
cfg.initialUser,
new BundleConverter(serializer)
);
}

Expand Down Expand Up @@ -209,10 +212,12 @@ export class IndexedDbComponentProvider extends MemoryComponentProvider {
}

createLocalStore(cfg: ComponentConfiguration): LocalStore {
const serializer = cfg.platform.newSerializer(cfg.databaseInfo.databaseId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider sharing this serializer instance with this.persistence if it is not too much hassle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

return new MultiTabLocalStore(
this.persistence,
new IndexFreeQueryEngine(),
cfg.initialUser
cfg.initialUser,
new BundleConverter(serializer)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also just pass the Serializer or Database ID and handle initializing BundleConverter inside LocalStore, which would greatly increase encapsulation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -445,20 +445,20 @@ export class IndexedDbRemoteDocumentCache implements RemoteDocumentCache {
primitiveComparator(l.canonicalString(), r.canonicalString())
);

this.changes.forEach((key, maybeDocument) => {
this.changes.forEach((key, documentChange) => {
const previousSize = this.documentSizes.get(key);
debugAssert(
previousSize !== undefined,
`Cannot modify a document that wasn't read (for ${key})`
);
if (maybeDocument) {
if (documentChange.maybeDoc) {
debugAssert(
!this.readTime.isEqual(SnapshotVersion.min()),
!this.getReadTime(key).isEqual(SnapshotVersion.min()),
'Cannot add a document with a read time of zero'
);
const doc = this.documentCache.serializer.toDbRemoteDocument(
maybeDocument,
this.readTime
documentChange.maybeDoc!,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop bang here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

this.getReadTime(key)
);
collectionParents = collectionParents.add(key.path.popLast());

Expand All @@ -474,7 +474,7 @@ export class IndexedDbRemoteDocumentCache implements RemoteDocumentCache {
// preserved in `getNewDocumentChanges()`.
const deletedDoc = this.documentCache.serializer.toDbRemoteDocument(
new NoDocument(key, SnapshotVersion.min()),
this.readTime
this.getReadTime(key)
);
promises.push(
this.documentCache.addEntry(transaction, key, deletedDoc)
Expand Down
Loading