Skip to content

Add a type parameter to Persistence #1047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jul 31, 2018
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
2c93300
Cherry pick sequence number starting point
Jul 23, 2018
96d732c
Working on typed transactions
Jul 23, 2018
59121ba
Start plumbing in sequence number
Jul 24, 2018
4b75267
Back out sequence number changes
Jul 25, 2018
29505fc
[AUTOMATED]: Prettier Code Styling
Jul 25, 2018
d03f608
Fix tests
Jul 25, 2018
e7efc7b
[AUTOMATED]: Prettier Code Styling
Jul 25, 2018
fd1359a
Fix lint
Jul 25, 2018
9a76422
[AUTOMATED]: Prettier Code Styling
Jul 25, 2018
a7b46dd
Merge branch 'master' into typed_transactions
Jul 25, 2018
3abdbbe
Uncomment line
Jul 25, 2018
5b66917
Fix the merge conflicts
Jul 25, 2018
a2848ff
MemoryPersistenceTransaction -> MemoryTransaction
Jul 26, 2018
70e8e51
[AUTOMATED]: Prettier Code Styling
Jul 26, 2018
ca4bc5d
Review updates
Jul 27, 2018
7909aef
Style
Jul 27, 2018
1bd431e
Merge branch 'master' into typed_transactions
Jul 27, 2018
9fc2b89
Lint and style
Jul 27, 2018
d4f1dcc
Review feedback
Jul 30, 2018
a94c026
[AUTOMATED]: Prettier Code Styling
Jul 30, 2018
fa05379
Merge branch 'master' into typed_transactions
Jul 30, 2018
068a4af
Revert some unintentional import churn
Jul 30, 2018
8e46ab9
Line 44 should definitely be empty
Jul 30, 2018
091038c
Checkpoint before adding helper function for stores
Jul 30, 2018
f0ed2dd
Use a helper for casting PersistenceTransaction to IndexedDbTransaction
Jul 30, 2018
ca53e31
[AUTOMATED]: Prettier Code Styling
Jul 30, 2018
4193323
Remove errant generic type
Jul 30, 2018
1a39850
Lint
Jul 30, 2018
80a2459
Merge branch 'master' into typed_transactions
Jul 30, 2018
7d14f14
Merge branch 'master' into typed_transactions
Jul 31, 2018
9e570c8
Fix typo
Jul 31, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/firestore/src/core/firestore_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ export class FirestoreClient {
// undefined checks.
private eventMgr: EventManager;
private garbageCollector: GarbageCollector;
// Note that localStore will end up with the same type parameter as
// persistence because the LocalStore constructor enforces this.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obsolete

private persistence: Persistence;
private localStore: LocalStore;
private remoteStore: RemoteStore;
Expand Down
1 change: 0 additions & 1 deletion packages/firestore/src/core/sync_engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ import { ObjectMap } from '../util/obj_map';
import { Deferred } from '../util/promise';
import { SortedMap } from '../util/sorted_map';
import { isNullOrUndefined } from '../util/types';

import { Query } from './query';
import { SnapshotVersion } from './snapshot_version';
import { TargetIdGenerator } from './target_id_generator';
Expand Down
9 changes: 5 additions & 4 deletions packages/firestore/src/local/indexeddb_mutation_queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,10 @@ import {
} from './indexeddb_schema';
import { LocalSerializer } from './local_serializer';
import { MutationQueue } from './mutation_queue';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { SimpleDb, SimpleDbStore } from './simple_db';
import { IndexedDbTransaction } from './indexeddb_persistence';
import { PersistenceTransaction } from './persistence';

/** A mutation queue for a specific user, backed by IndexedDB. */
export class IndexedDbMutationQueue implements MutationQueue {
Expand Down Expand Up @@ -564,7 +565,7 @@ function mutationsStore(
txn: PersistenceTransaction
): SimpleDbStore<DbMutationBatchKey, DbMutationBatch> {
return SimpleDb.getStore<DbMutationBatchKey, DbMutationBatch>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about an IndexedDbPersistence.getStore<>() helper to centralize this cast (and add a runtime assert please so if we do somehow mix/match things we get a helpful error immediately).

(FYI- And then the IndexedDbTransaction import will probably no longe be needed).

DbMutationBatch.store
);
}
Expand All @@ -576,7 +577,7 @@ function documentMutationsStore(
txn: PersistenceTransaction
): SimpleDbStore<DbDocumentMutationKey, DbDocumentMutation> {
return SimpleDb.getStore<DbDocumentMutationKey, DbDocumentMutation>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
DbDocumentMutation.store
);
}
Expand All @@ -588,7 +589,7 @@ function mutationQueuesStore(
txn: PersistenceTransaction
): SimpleDbStore<DbMutationQueueKey, DbMutationQueue> {
return SimpleDb.getStore<DbMutationQueueKey, DbMutationQueue>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
DbMutationQueue.store
);
}
41 changes: 29 additions & 12 deletions packages/firestore/src/local/indexeddb_persistence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import {
} from './indexeddb_schema';
import { LocalSerializer } from './local_serializer';
import { MutationQueue } from './mutation_queue';
import { Persistence } from './persistence';
import { Persistence, PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { QueryCache } from './query_cache';
import { RemoteDocumentCache } from './remote_document_cache';
Expand All @@ -59,6 +59,12 @@ const UNSUPPORTED_PLATFORM_ERROR_MSG =
' IndexedDB or is known to have an incomplete implementation. Offline' +
' persistence has been disabled.';

export class IndexedDbTransaction extends PersistenceTransaction {
constructor(readonly simpleDbTransaction: SimpleDbTransaction) {
super();
}
}

/**
* An IndexedDB-backed instance of Persistence. Data is stored persistently
* across sessions.
Expand Down Expand Up @@ -176,7 +182,7 @@ export class IndexedDbPersistence implements Persistence {

runTransaction<T>(
action: string,
operation: (transaction: SimpleDbTransaction) => PersistencePromise<T>
operation: (transaction: IndexedDbTransaction) => PersistencePromise<T>
): Promise<T> {
if (this.persistenceError) {
return Promise.reject(this.persistenceError);
Expand All @@ -186,10 +192,17 @@ export class IndexedDbPersistence implements Persistence {

// Do all transactions as readwrite against all object stores, since we
// are the only reader/writer.
return this.simpleDb.runTransaction('readwrite', ALL_STORES, txn => {
// Verify that we still have the owner lease as part of every transaction.
return this.ensureOwnerLease(txn).next(() => operation(txn));
});
return this.simpleDb.runTransaction(
'readwrite',
ALL_STORES,
simpleDbTxn => {
// Verify that we still have the owner lease as part of every transaction.
const txn = new IndexedDbTransaction(simpleDbTxn);
return this.ensureOwnerLease(txn.simpleDbTransaction).next(() =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simpleDbTxn?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's all the same, I'd rather keep the longer name. I think the abbreviation is a little harder to read.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was suggesting you use the existing variable that's in scope.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. That makes more sense. Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general rule if I ever ask somebody to abbreviate a name in code review feedback you can assume my account has been hacked. 😁

operation(txn)
);
}
);
}

static isAvailable(): boolean {
Expand Down Expand Up @@ -326,12 +339,16 @@ export class IndexedDbPersistence implements Persistence {
// would increase the chances of us not refreshing on time if the queue is
// backed up for some reason.
this.ownerLeaseRefreshHandle = setInterval(() => {
const txResult = this.runTransaction('Refresh owner timestamp', txn => {
// NOTE: We don't need to validate the current owner contents, since
// runTransaction does that automatically.
const store = txn.store<DbOwnerKey, DbOwner>(DbOwner.store);
return store.put('owner', new DbOwner(this.ownerId, Date.now()));
});
const txResult = this.simpleDb.runTransaction(
'readwrite',
ALL_STORES,
txn => {
// NOTE: We don't need to validate the current owner contents, since
// runTransaction does that automatically.
const store = txn.store<DbOwnerKey, DbOwner>(DbOwner.store);
return store.put('owner', new DbOwner(this.ownerId, Date.now()));
}
);

txResult.catch(reason => {
// Probably means we lost the lease. Report the error and stop trying to
Expand Down
16 changes: 11 additions & 5 deletions packages/firestore/src/local/indexeddb_query_cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,12 @@ import {
DbTargetKey
} from './indexeddb_schema';
import { LocalSerializer } from './local_serializer';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { QueryCache } from './query_cache';
import { QueryData } from './query_data';
import { SimpleDb, SimpleDbStore } from './simple_db';
import { IndexedDbTransaction } from './indexeddb_persistence';
import { PersistenceTransaction } from './persistence';

export class IndexedDbQueryCache implements QueryCache {
constructor(private serializer: LocalSerializer) {}
Expand All @@ -52,7 +53,7 @@ export class IndexedDbQueryCache implements QueryCache {
/**
* A cached copy of the metadata for the query cache.
*/
private metadata = null;
private metadata: DbTargetGlobal = null;

/** The garbage collector to notify about potential garbage keys. */
private garbageCollector: GarbageCollector | null = null;
Expand Down Expand Up @@ -297,6 +298,8 @@ export class IndexedDbQueryCache implements QueryCache {
this.garbageCollector = gc;
}

// TODO(gsoltis): we can let the compiler assert that txn !== null if we
// drop null from the type bounds on txn.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something blocking us from doing this right now?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, kinda. It compiles, but only because we don't enforce strict function types. This is replicating the type signature from GarbageSource, which allows null. So actually removing it from this function gets us no safety and a more obscure runtime error than what you would currently get.

The solution will eventually arrive when we get rid of GarbageSource.

containsKey(
txn: PersistenceTransaction | null,
key: DocumentKey
Expand Down Expand Up @@ -335,7 +338,10 @@ export class IndexedDbQueryCache implements QueryCache {
function targetsStore(
txn: PersistenceTransaction
): SimpleDbStore<DbTargetKey, DbTarget> {
return SimpleDb.getStore<DbTargetKey, DbTarget>(txn, DbTarget.store);
return SimpleDb.getStore<DbTargetKey, DbTarget>(
(txn as IndexedDbTransaction).simpleDbTransaction,
DbTarget.store
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same feedback here (and elsewhere) about an IndexedDbPersistence.getStore<>() helper.

}

/**
Expand All @@ -345,7 +351,7 @@ function globalTargetStore(
txn: PersistenceTransaction
): SimpleDbStore<DbTargetGlobalKey, DbTargetGlobal> {
return SimpleDb.getStore<DbTargetGlobalKey, DbTargetGlobal>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
DbTargetGlobal.store
);
}
Expand All @@ -357,7 +363,7 @@ function documentTargetStore(
txn: PersistenceTransaction
): SimpleDbStore<DbTargetDocumentKey, DbTargetDocument> {
return SimpleDb.getStore<DbTargetDocumentKey, DbTargetDocument>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
DbTargetDocument.store
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@ import { Document, MaybeDocument } from '../model/document';
import { DocumentKey } from '../model/document_key';

import { DbRemoteDocument, DbRemoteDocumentKey } from './indexeddb_schema';
import { IndexedDbTransaction } from './indexeddb_persistence';
import { LocalSerializer } from './local_serializer';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { RemoteDocumentCache } from './remote_document_cache';
import { SimpleDb, SimpleDbStore } from './simple_db';
import { PersistenceTransaction } from './persistence';

export class IndexedDbRemoteDocumentCache implements RemoteDocumentCache {
constructor(private serializer: LocalSerializer) {}
Expand Down Expand Up @@ -89,7 +90,7 @@ function remoteDocumentsStore(
txn: PersistenceTransaction
): SimpleDbStore<DbRemoteDocumentKey, DbRemoteDocument> {
return SimpleDb.getStore<DbRemoteDocumentKey, DbRemoteDocument>(
txn,
(txn as IndexedDbTransaction).simpleDbTransaction,
DbRemoteDocument.store
);
}
Expand Down
2 changes: 1 addition & 1 deletion packages/firestore/src/local/memory_mutation_queue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ import { SortedSet } from '../util/sorted_set';

import { GarbageCollector } from './garbage_collector';
import { MutationQueue } from './mutation_queue';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { DocReference } from './reference_set';
import { PersistenceTransaction } from './persistence';

export class MemoryMutationQueue implements MutationQueue {
/**
Expand Down
13 changes: 9 additions & 4 deletions packages/firestore/src/local/memory_persistence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ export class MemoryPersistence implements Persistence {
* will make the in-memory persistence layer behave as if it were actually
* persisting values.
*/
private mutationQueues: { [user: string]: MutationQueue } = {};
private mutationQueues: {
[user: string]: MutationQueue;
} = {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea why there are spurious re-formattings in this PR? Maybe run yarn from the root to make sure you have the right version of prettier installed.

private remoteDocumentCache = new MemoryRemoteDocumentCache();
private queryCache = new MemoryQueryCache();

Expand Down Expand Up @@ -85,9 +87,12 @@ export class MemoryPersistence implements Persistence {
operation: (transaction: PersistenceTransaction) => PersistencePromise<T>
): Promise<T> {
debug(LOG_TAG, 'Starting transaction:', action);
return operation(new MemoryPersistenceTransaction()).toPromise();
return operation(new MemoryTransaction()).toPromise();
}
}

/** Dummy class since memory persistence doesn't actually use transactions. */
class MemoryPersistenceTransaction implements PersistenceTransaction {}
/**
* Memory persistence is not actually transactional, but future implementations
* may have transaction-scoped state.
*/
export class MemoryTransaction implements PersistenceTransaction {}
2 changes: 1 addition & 1 deletion packages/firestore/src/local/memory_query_cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ import { DocumentKey } from '../model/document_key';
import { ObjectMap } from '../util/obj_map';

import { GarbageCollector } from './garbage_collector';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { QueryCache } from './query_cache';
import { QueryData } from './query_data';
import { ReferenceSet } from './reference_set';
import { assert } from '../util/assert';
import { PersistenceTransaction } from './persistence';

export class MemoryQueryCache implements QueryCache {
/**
Expand Down
3 changes: 1 addition & 2 deletions packages/firestore/src/local/memory_remote_document_cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@ import {
} from '../model/collections';
import { Document, MaybeDocument } from '../model/document';
import { DocumentKey } from '../model/document_key';

import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { RemoteDocumentCache } from './remote_document_cache';
import { PersistenceTransaction } from './persistence';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm. Does prettier not order these? (I'm not sure why this moved here)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I asked vscode to organize imports. And I guess prettier didn't object either way? I'll put it back and see what happens.


export class MemoryRemoteDocumentCache implements RemoteDocumentCache {
private docs = maybeDocumentMap();
Expand Down
3 changes: 1 addition & 2 deletions packages/firestore/src/local/persistence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
import { User } from '../auth/user';

import { MutationQueue } from './mutation_queue';
import { PersistenceTransaction } from './persistence';
import { PersistencePromise } from './persistence_promise';
import { QueryCache } from './query_cache';
import { RemoteDocumentCache } from './remote_document_cache';
Expand All @@ -29,7 +28,7 @@ import { RemoteDocumentCache } from './remote_document_cache';
* pass it to your callback. You then pass it to any method that operates
* on persistence.
*/
export interface PersistenceTransaction {}
export abstract class PersistenceTransaction {}

/**
* Persistence is the lowest-level shared interface to persistent storage in
Expand Down
26 changes: 7 additions & 19 deletions packages/firestore/src/local/simple_db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,12 @@
* limitations under the License.
*/

import { assert, fail } from '../util/assert';
import { assert } from '../util/assert';
import { debug } from '../util/log';
import { AnyDuringMigration } from '../util/misc';

import { PersistencePromise } from './persistence_promise';
import { SCHEMA_VERSION } from './indexeddb_schema';
import { Deferred } from '../util/promise';
import { PersistenceTransaction } from './persistence';
import { Code, FirestoreError } from '../util/error';

const LOG_TAG = 'SimpleDb';
Expand Down Expand Up @@ -150,14 +148,10 @@ export class SimpleDb {

/** Helper to get a typed SimpleDbStore from a transaction. */
static getStore<KeyType extends IDBValidKey, ValueType>(
txn: PersistenceTransaction,
txn: SimpleDbTransaction,
store: string
): SimpleDbStore<KeyType, ValueType> {
if (txn instanceof SimpleDbTransaction) {
return txn.store<KeyType, ValueType>(store);
} else {
return fail('Invalid transaction object provided!');
}
return txn.store<KeyType, ValueType>(store);
}

constructor(private db: IDBDatabase) {}
Expand Down Expand Up @@ -534,25 +528,19 @@ export class SimpleDbStore<KeyType extends IDBValidKey, ValueType> {
}

private cursor(options: IterateOptions): IDBRequest {
let direction = 'next';
let direction: IDBCursorDirection = 'next';
if (options.reverse) {
direction = 'prev';
}
if (options.index) {
const index = this.store.index(options.index);
if (options.keysOnly) {
return index.openKeyCursor(
options.range,
direction as AnyDuringMigration
);
return index.openKeyCursor(options.range, direction);
} else {
return index.openCursor(options.range, direction as AnyDuringMigration);
return index.openCursor(options.range, direction);
}
} else {
return this.store.openCursor(
options.range,
direction as AnyDuringMigration
);
return this.store.openCursor(options.range, direction);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ describe('EagerGarbageCollector', () => {
expect(referenceSet.isEmpty()).to.equal(false);

referenceSet.removeReferencesForId(2);
return gc.collectGarbage(true).toPromise();
return gc.collectGarbage(null).toPromise();
})
.then(garbage => {
expectSetToEqual(garbage, [key3]);
Expand Down
7 changes: 5 additions & 2 deletions packages/firestore/test/unit/local/local_store.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,10 @@ import { IndexedDbPersistence } from '../../../src/local/indexeddb_persistence';
import { LocalStore, LocalWriteResult } from '../../../src/local/local_store';
import { LocalViewChanges } from '../../../src/local/local_view_changes';
import { NoOpGarbageCollector } from '../../../src/local/no_op_garbage_collector';
import { Persistence } from '../../../src/local/persistence';
import {
Persistence,
PersistenceTransaction
} from '../../../src/local/persistence';
import {
documentKeySet,
DocumentMap,
Expand Down Expand Up @@ -270,7 +273,7 @@ describe('LocalStore w/ IndexedDB Persistence', () => {
genericLocalStoreTests(persistenceHelpers.testIndexedDbPersistence);
});

function genericLocalStoreTests(
function genericLocalStoreTests<TransactionType extends PersistenceTransaction>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obsolete?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

getPersistence: () => Promise<Persistence>
): void {
let persistence: Persistence;
Expand Down
Loading