Skip to content

Memory-only Firestore build #2608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Mar 18, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion integration/firestore/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "1.0.1",
"private": true,
"scripts": {
"build:deps": "cd ../../ ; yarn build",
"build:deps": "cd ../../packages/firestore ; yarn build",
"build:persistence": "INCLUDE_FIRESTORE_PERSISTENCE=true gulp compile-tests",
"build:memory": "INCLUDE_FIRESTORE_PERSISTENCE=false gulp compile-tests",
"test": "yarn build:memory; karma start --single-run; yarn build:persistence; karma start --single-run;",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much like yarn test does everything, I think it's reasonable to still have a yarn build that builds everything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now, gulp uses the same output folder for both modes. This means that the builds overwrite each other. We can probably use different output folders, but this would require some extra plumbing. Let me know if this is something that you want me to do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. I see. It's certainly not something to tackle in this PR. If gulp were better about doing incremental builds this could be worthwhile, but I suspect it's not going to actually buy us much or possibly even anything.

Expand Down
5 changes: 5 additions & 0 deletions packages/firebase/firestore/memory/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,9 @@
* limitations under the License.
*/

/**
* This file serves as the public entrypoint for users that import
* `firebase/firestore/memory`.
*/

import '../../../firestore/dist/index.memory.esm';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still the case that looking at this file I can't tell why it exists, what bits of the build it interacts with, and whether or not I should use for anything when e.g. writing Firestore tests. I think this file is establishing the public interface to Firestore as seen when a user imports @firebase/firestore/memory, right? Comments that address these concerns would be helpful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a short comment that explains this a little bit.

3 changes: 2 additions & 1 deletion packages/firebase/firestore/memory/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"name": "firebase/firestore/memory",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also have a description and a license and all those standard fields?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is modeled after the package.json files in firebase@next, but I believe you are right. @Feiyang1 should we add license/description fields?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no license needed for json files.
Those fields are not needed in this file because they are already in the root package.json. You can provide a description for sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Descriptions added to this package.json and top-level package.json.

"description": "A memory-only build of the Cloud Firestore JS SDK.",
"main": "dist/index.cjs.js",
"module": "dist/index.esm.js",
"typings": "../../empty-import.d.ts"
"typings": "../../empty-import.d.ts",
}
1 change: 1 addition & 0 deletions packages/firebase/firestore/package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"name": "firebase/firestore",
"description": "The Cloud Firestore component of the Firebase JS SDK.",
"main": "dist/index.cjs.js",
"module": "dist/index.esm.js",
"typings": "../empty-import.d.ts"
Expand Down
8 changes: 6 additions & 2 deletions packages/firestore/index.memory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
import firebase from '@firebase/app';
import { FirebaseNamespace } from '@firebase/app-types';

import {MemoryPersistenceProvider} from "./src/local/memory_persistence";
import { Firestore } from './src/api/database';
import { MemoryPersistenceProvider } from './src/local/memory_persistence';
import { configureForFirebase } from './src/platform/config';

import './register-module';
Expand All @@ -30,7 +31,10 @@ import { name, version } from './package.json';
* Registers the memory-only Firestore build with the components framework.
*/
export function registerFirestore(instance: FirebaseNamespace): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done (here and elsewhere)

configureForFirebase(instance, () => new MemoryPersistenceProvider());
configureForFirebase(
instance,
(app, auth) => new Firestore(app, auth, new MemoryPersistenceProvider())
);
instance.registerVersion(name, version);
}

Expand Down
10 changes: 7 additions & 3 deletions packages/firestore/index.node.memory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,23 @@
import firebase from '@firebase/app';
import { FirebaseNamespace } from '@firebase/app-types';

import {MemoryPersistenceProvider} from "./src/local/memory_persistence";
import { Firestore } from './src/api/database';
import { MemoryPersistenceProvider } from './src/local/memory_persistence';
import { configureForFirebase } from './src/platform/config';
import './register-module';
import './src/platform_node/node_init';

import { name, version } from './package.json';

/**
* Registers the memory-only Firestore build for Node with the components
* Registers the memory-only Firestore build for Node with the components
* framework.
*/
export function registerFirestore(instance: FirebaseNamespace): void {
configureForFirebase(instance, () => new MemoryPersistenceProvider());
configureForFirebase(
instance,
(app, auth) => new Firestore(app, auth, new MemoryPersistenceProvider())
);
instance.registerVersion(name, version);
}

Expand Down
5 changes: 3 additions & 2 deletions packages/firestore/index.node.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
import firebase from '@firebase/app';
import { FirebaseNamespace } from '@firebase/app-types';

import {IndexedDbPersistenceProvider} from "./src/local/indexeddb_persistence";
import { Firestore } from './src/api/database';
import { IndexedDbPersistenceProvider } from './src/local/indexeddb_persistence';
import { configureForFirebase } from './src/platform/config';

import './register-module';
Expand All @@ -32,7 +33,7 @@ import { name, version } from './package.json';
export function registerFirestore(instance: FirebaseNamespace): void {
configureForFirebase(
instance,
() => new IndexedDbPersistenceProvider()
(app, auth) => new Firestore(app, auth, new IndexedDbPersistenceProvider())
);
instance.registerVersion(name, version, 'node');
}
Expand Down
6 changes: 4 additions & 2 deletions packages/firestore/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
import firebase from '@firebase/app';
import { FirebaseNamespace } from '@firebase/app-types';

import {IndexedDbPersistenceProvider} from "./src/local/indexeddb_persistence";
import { Firestore } from './src/api/database';
import { IndexedDbPersistenceProvider } from './src/local/indexeddb_persistence';
import { configureForFirebase } from './src/platform/config';
import { name, version } from './package.json';

Expand All @@ -31,7 +32,8 @@ import './src/platform_browser/browser_init';
*/
export function registerFirestore(instance: FirebaseNamespace): void {
configureForFirebase(
instance, () => new IndexedDbPersistenceProvider()
instance,
(app, auth) => new Firestore(app, auth, new IndexedDbPersistenceProvider())
);
instance.registerVersion(name, version);
}
Expand Down
1 change: 1 addition & 0 deletions packages/firestore/memory/package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"name": "@firebase/firestore/memory",
"description": "A memory-only build of the Cloud Firestore JS SDK.",
"main": "../dist/index.memory.node.cjs.js",
"browser": "../dist/index.memory.cjs.js",
"module": "../dist/index.memory.esm.js",
Expand Down
6 changes: 3 additions & 3 deletions packages/firestore/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@firebase/firestore",
"version": "1.12.1",
"description": "",
"version": "1.11.2",
"description": "The Cloud Firestore component of the Firebase JS SDK.",
"author": "Firebase <[email protected]> (https://firebase.google.com/)",
"scripts": {
"prebuild": "tsc -d --downlevelIteration --declarationDir dist/lib --emitDeclarationOnly && tsc -m es2015 --moduleResolution node scripts/*.ts ",
Expand All @@ -10,7 +10,7 @@
"dev": "rollup -c -w",
"lint": "eslint -c .eslintrc.js '**/*.ts' --ignore-path '../../.gitignore'",
"lint:fix": "eslint --fix -c .eslintrc.js '**/*.ts' --ignore-path '../../.gitignore'",
"prettier": "prettier --write 'src/**/*.js' 'test/**/*.js' 'src/**/*.ts' 'test/**/*.ts'",
"prettier": "prettier --write '*.ts' '*.js' 'src/**/*.js' 'test/**/*.js' 'src/**/*.ts' 'test/**/*.ts'",
"test": "run-s lint test:all",
"test:all": "run-p test:browser test:travis test:minified",
"test:browser": "karma start --single-run",
Expand Down
17 changes: 11 additions & 6 deletions packages/firestore/rollup.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import memoryPkg from './memory/package.json';

import {
appendPrivatePrefixTransformers,
manglePrivatePropertiesOptions,
manglePrivatePropertiesOptions
} from './terser.config';

// This Firestore Rollup configuration provides a number of different builds:
Expand All @@ -45,8 +45,11 @@ import {
// The in-memory builds are roughly 130 KB smaller, but throw an exception
// for calls to `enablePersistence()` or `clearPersistence()`.
//
// All browser builds rely on Terser's property name mangling to reduce code
// All browser builds rely on Terser's property name mangling to reduce code
// size.
//
// See https://g3doc.corp.google.com/firebase/jscore/g3doc/contributing/builds.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, you can shorten this to http://g3doc/firebase/jscore/g3doc/contributing/builds.md if you wanted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// for a description of the various JavaScript formats used in our SDKs.

// MARK: Browser builds

Expand All @@ -67,8 +70,8 @@ function resolveNodeExterns(id) {
}

/**
* Resolves the external dependencies for the Memory-based Firestore
* implementation. Verifies that no persistence sources are used by Firestore's
* Resolves the external dependencies for the Memory-based Firestore
* implementation. Verifies that no persistence sources are used by Firestore's
* memory-only implementation.
*/
export function resolveMemoryExterns(deps, externsId, referencedBy) {
Expand Down Expand Up @@ -138,7 +141,8 @@ const browserBuilds = [
sourcemap: true
},
plugins: es5BuildPlugins,
external: (id, referencedBy) => resolveMemoryExterns(browserDeps, id, referencedBy)
external: (id, referencedBy) =>
resolveMemoryExterns(browserDeps, id, referencedBy)
},
// ES2017 ESM build (with persistence)
{
Expand All @@ -160,7 +164,8 @@ const browserBuilds = [
sourcemap: true
},
plugins: es2017BuildPlugins,
external: (id, referencedBy) => resolveMemoryExterns(browserDeps, id, referencedBy)
external: (id, referencedBy) =>
resolveMemoryExterns(browserDeps, id, referencedBy)
},
// ES5 CJS Build (with persistence)
//
Expand Down
61 changes: 22 additions & 39 deletions packages/firestore/src/api/database.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,7 @@ import { FirebaseApp } from '@firebase/app-types';
import { FirebaseService, _FirebaseApp } from '@firebase/app-types/private';
import { DatabaseId, DatabaseInfo } from '../core/database_info';
import { ListenOptions } from '../core/event_manager';
import {
FirestoreClient,
IndexedDbPersistenceSettings,
InternalPersistenceSettings,
MemoryPersistenceSettings
} from '../core/firestore_client';
import { FirestoreClient, PersistenceSettings } from '../core/firestore_client';
import {
Bound,
Direction,
Expand Down Expand Up @@ -82,7 +77,7 @@ import * as log from '../util/log';
import { LogLevel } from '../util/log';
import { AutoId } from '../util/misc';
import * as objUtils from '../util/obj';
import { Rejecter, Resolver } from '../util/promise';
import { Deferred, Rejecter, Resolver } from '../util/promise';
import { FieldPath as ExternalFieldPath } from './field_path';

import {
Expand Down Expand Up @@ -121,11 +116,6 @@ const DEFAULT_FORCE_LONG_POLLING = false;
*/
export const CACHE_SIZE_UNLIMITED = LruParams.COLLECTION_DISABLED;

const PERSISTENCE_MISSING_ERROR_MSG =
'You are using the memory-only build of Firestore. Persistence support is ' +
'only available via the @firebase/firestore bundle or the ' +
'firebase-firestore.js build.';

// enablePersistence() defaults:
const DEFAULT_SYNCHRONIZE_TABS = false;

Expand Down Expand Up @@ -393,13 +383,6 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {
}

enablePersistence(settings?: firestore.PersistenceSettings): Promise<void> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be be worth just removing these methods altogether? Or does that require separate typings?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but we would at the very least have to publish two different typings (and go through API review). Without duplicated typing, having these method here to throw an error seems like a better experience.

if (!this._persistenceProvider.isDurable) {
throw new FirestoreError(
Code.FAILED_PRECONDITION,
PERSISTENCE_MISSING_ERROR_MSG
);
}

if (this._firestoreClient) {
throw new FirestoreError(
Code.FAILED_PRECONDITION,
Expand Down Expand Up @@ -428,23 +411,14 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {
);
}

return this.configureClient(
this._persistenceProvider,
new IndexedDbPersistenceSettings(
this._settings.cacheSizeBytes,
synchronizeTabs
)
);
return this.configureClient(this._persistenceProvider, {
durable: true,
cacheSizeBytes: this._settings.cacheSizeBytes,
synchronizeTabs
});
}

async clearPersistence(): Promise<void> {
if (!this._persistenceProvider.isDurable) {
throw new FirestoreError(
Code.FAILED_PRECONDITION,
PERSISTENCE_MISSING_ERROR_MSG
);
}

if (
this._firestoreClient !== undefined &&
!this._firestoreClient.clientTerminated
Expand All @@ -455,7 +429,17 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {
);
}

return this._persistenceProvider.clearPersistence();
const deferred = new Deferred<void>();
this._queue.enqueueAndForgetEvenAfterShutdown(async () => {
try {
const databaseInfo = this.makeDatabaseInfo();
await this._persistenceProvider.clearPersistence(databaseInfo);
deferred.resolve();
} catch (e) {
deferred.reject(e);
}
});
return deferred.promise;
}

terminate(): Promise<void> {
Expand Down Expand Up @@ -514,10 +498,9 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {
if (!this._firestoreClient) {
// Kick off starting the client but don't actually wait for it.
// eslint-disable-next-line @typescript-eslint/no-floating-promises
this.configureClient(
new MemoryPersistenceProvider(),
new MemoryPersistenceSettings()
);
this.configureClient(new MemoryPersistenceProvider(), {
durable: false
});
}
return this._firestoreClient as FirestoreClient;
}
Expand All @@ -534,7 +517,7 @@ export class Firestore implements firestore.FirebaseFirestore, FirebaseService {

private configureClient(
persistenceProvider: PersistenceProvider,
persistenceSettings: InternalPersistenceSettings
persistenceSettings: PersistenceSettings
): Promise<void> {
assert(!!this._settings.host, 'FirestoreSettings.host is not set');

Expand Down
Loading