Skip to content

Mila/count #6597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Sep 16, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
b92e492
Mila/count update proto (#6482)
milaGGL Jul 28, 2022
598e948
compile protos and upload the generated json file (#6491)
milaGGL Jul 29, 2022
7f42490
Mila/count add api interface (#6499)
milaGGL Aug 4, 2022
981e6f1
create sample count test cases (#6506)
milaGGL Aug 4, 2022
bc61df7
Mila/count implement count query (#6528)
milaGGL Aug 24, 2022
c21304b
add test case for terminated firestore
milaGGL Aug 26, 2022
a3242ce
Mila/count add tests (#6566)
milaGGL Aug 30, 2022
6da4f4f
Mila/count export aggregate to api (#6575)
milaGGL Sep 1, 2022
fe871ce
change int32 to int64 (#6577)
milaGGL Sep 2, 2022
507e984
check client offline state and test (#6579)
milaGGL Sep 7, 2022
98189d5
Mila/count non lite api tests (#6581)
milaGGL Sep 8, 2022
0ebab69
Mila/count update api surface (#6589)
milaGGL Sep 13, 2022
4b273fe
remove DocumentFieldValue
milaGGL Sep 13, 2022
a9de825
Merge branch 'master' into mila/count
milaGGL Sep 13, 2022
576d202
fix lint error: Symbol not found for identifier
milaGGL Sep 13, 2022
dd693c6
format to pass lint check
milaGGL Sep 13, 2022
83fbc8d
remove the exports of aggregate query functions/types
milaGGL Sep 14, 2022
26a0bd2
Update aggregation.test.ts
milaGGL Sep 14, 2022
9cced28
skip count query test cases if not using emulator
milaGGL Sep 14, 2022
f490bfe
update the type of skipTestUnlessUsingEmulator
milaGGL Sep 14, 2022
078d581
roll back to any type for skipTestUnlessUsingEmulator
milaGGL Sep 15, 2022
a9ee57a
move aggregation test file into api_internal folder
milaGGL Sep 15, 2022
dbbc2f8
resolve comments
milaGGL Sep 15, 2022
c12d5a0
reformat to pass lint check
milaGGL Sep 15, 2022
74116a4
add changeset as requested by github
milaGGL Sep 15, 2022
de9dd9a
remove the changeset
milaGGL Sep 15, 2022
9438594
Merge branch 'master' into mila/count
milaGGL Sep 15, 2022
3a6cc27
add the changeset back
milaGGL Sep 15, 2022
0267c69
resolve comments
milaGGL Sep 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/hot-insects-wink.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@firebase/firestore': minor
---

Release count query for internal use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "Release" -> "Implement"

10 changes: 0 additions & 10 deletions packages/firestore/src/api/aggregate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,6 @@ import { cast } from '../util/input_validation';

import { ensureFirestoreConfigured, Firestore } from './database';

export {
AggregateField,
AggregateFieldType,
AggregateQuerySnapshot,
aggregateQuerySnapshotEqual,
AggregateSpec,
AggregateSpecData,
count
} from '../lite-api/aggregate';

/**
* Executes the query and returns the results as a `AggregateQuerySnapshot` from the
* server. Returns an error if the network is not available.
Expand Down
34 changes: 18 additions & 16 deletions packages/firestore/src/core/firestore_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,17 @@

import { GetOptions } from '@firebase/firestore-types';

import { AggregateField, AggregateQuerySnapshot } from '../api/aggregate';
import { LoadBundleTask } from '../api/bundle';
import {
CredentialChangeListener,
CredentialsProvider
} from '../api/credentials';
import { User } from '../auth/user';
import { getCount } from '../lite-api/aggregate';
import {
AggregateField,
AggregateQuerySnapshot,
getCount
} from '../lite-api/aggregate';
import { Query as LiteQuery } from '../lite-api/reference';
import { LocalStore } from '../local/local_store';
import {
Expand Down Expand Up @@ -513,21 +516,20 @@ export function firestoreClientRunCountQuery(
AggregateQuerySnapshot<{ count: AggregateField<number> }>
>();
client.asyncQueue.enqueueAndForget(async () => {
const remoteStore = await getRemoteStore(client);
if (!canUseNetwork(remoteStore)) {
deferred.reject(
new FirestoreError(
Code.UNAVAILABLE,
'Failed to get count result because the client is offline.'
)
);
} else {
try {
const result = await getCount(query);
deferred.resolve(result);
} catch (e) {
deferred.reject(e as Error);
try {
const remoteStore = await getRemoteStore(client);
if (!canUseNetwork(remoteStore)) {
deferred.reject(
new FirestoreError(
Code.UNAVAILABLE,
'Failed to get count result because the client is offline.'
)
);
}
const result = await getCount(query);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 529 and 530 still need to be in an "else" block.

deferred.resolve(result);
} catch (e) {
deferred.reject(e as Error);
}
});
return deferred.promise;
Expand Down
4 changes: 2 additions & 2 deletions packages/firestore/src/lite-api/aggregate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,12 @@ export function getCount(
/**
* Compares two `AggregateQuerySnapshot` instances for equality.
* Two `AggregateQuerySnapshot` instances are considered "equal" if they have
* the same underlying query, the same metadata, and the same data.
* the same underlying query, and the same data.
*
* @param left - The `AggregateQuerySnapshot` to compare.
* @param right - The `AggregateQuerySnapshot` to compare.
*
* @returns true if the AggregateQuerySnapshos are equal.
* @returns true if the AggregateQuerySnapshots are equal.
*/
export function aggregateQuerySnapshotEqual<T extends AggregateSpec>(
left: AggregateQuerySnapshot<T>,
Expand Down
2 changes: 2 additions & 0 deletions packages/firestore/src/platform/node/grpc_connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,8 @@ export class GrpcConnection implements Connection {
private cachedStub: GeneratedGrpcStub | null = null;

get shouldResourcePathBeIncludedInRequest(): boolean {
// Both `invokeRPC()` and `invokeStreamingRPC()` ignore their `path` arguments, and expect
// the "path" to be part of the given `request`.
return true;
}

Expand Down
2 changes: 2 additions & 0 deletions packages/firestore/src/remote/rest_connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ export abstract class RestConnection implements Connection {
private readonly databaseRoot: string;

get shouldResourcePathBeIncludedInRequest(): boolean {
// Both `invokeRPC()` and `invokeStreamingRPC()` use their `path` arguments to determine
// where to run the query, and expect the `request` to NOT specify the "path".
return false;
}

Expand Down
106 changes: 54 additions & 52 deletions packages/firestore/test/integration/api_internal/aggregation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,51 +27,57 @@ import {
query,
terminate,
where,
writeBatch
writeBatch,
DocumentData,
QueryDocumentSnapshot
} from '../util/firebase_export';
import {
apiDescribe,
postConverter,
withEmptyTestCollection,
withTestCollection,
withTestDb,
skipTestUnlessUsingEmulator
withTestDb
} from '../util/helpers';
import { USE_EMULATOR } from '../util/settings';

apiDescribe('Count query', (persistence: boolean) => {
it('can run count query getCountFromServer', () => {
const testDocs = {
a: { author: 'authorA', title: 'titleA' },
b: { author: 'authorB', title: 'titleB' }
};
return skipTestUnlessUsingEmulator(() =>
withTestCollection(persistence, testDocs, async coll => {
(USE_EMULATOR ? apiDescribe : apiDescribe.skip)(
'Count quries',
(persistence: boolean) => {
it('can run count query getCountFromServer', () => {
const testDocs = {
a: { author: 'authorA', title: 'titleA' },
b: { author: 'authorB', title: 'titleB' }
};
return withTestCollection(persistence, testDocs, async coll => {
const snapshot = await getCountFromServer(coll);
expect(snapshot.data().count).to.equal(2);
})
);
});
});
});

it('count query supports withConverter', () => {
const testDocs = {
a: { author: 'authorA', title: 'titleA' },
b: { author: 'authorB', title: 'titleB' }
};
return skipTestUnlessUsingEmulator(() =>
withTestCollection(persistence, testDocs, async coll => {
it("count query doesn't use converter", () => {
const testDocs = {
a: { author: 'authorA', title: 'titleA' },
b: { author: 'authorB', title: 'titleB' }
};
const throwingConverter = {
toFirestore(obj: never): DocumentData {
throw new Error('should never be called');
},
fromFirestore(snapshot: QueryDocumentSnapshot): never {
throw new Error('should never be called');
}
};
return withTestCollection(persistence, testDocs, async coll => {
const query_ = query(
coll,
where('author', '==', 'authorA')
).withConverter(postConverter);
).withConverter(throwingConverter);
const snapshot = await getCountFromServer(query_);
expect(snapshot.data().count).to.equal(1);
})
);
});
});
});

it('count query supports collection groups', () => {
return skipTestUnlessUsingEmulator(() =>
withTestDb(persistence, async db => {
it('count query supports collection groups', () => {
return withTestDb(persistence, async db => {
const collectionGroupId = doc(collection(db, 'aggregateQueryTest')).id;
const docPaths = [
`${collectionGroupId}/cg-doc1`,
Expand All @@ -89,36 +95,32 @@ apiDescribe('Count query', (persistence: boolean) => {
collectionGroup(db, collectionGroupId)
);
expect(snapshot.data().count).to.equal(2);
})
);
});
});
});

it('getCountFromServer fails if firestore is terminated', () => {
return withEmptyTestCollection(persistence, async (coll, firestore) => {
await terminate(firestore);
expect(() => getCountFromServer(coll)).to.throw(
'The client has already been terminated.'
);
it('getCountFromServer fails if firestore is terminated', () => {
return withEmptyTestCollection(persistence, async (coll, firestore) => {
await terminate(firestore);
expect(() => getCountFromServer(coll)).to.throw(
'The client has already been terminated.'
);
});
});
});

it("terminate doesn't crash when there is count query in flight", () => {
return skipTestUnlessUsingEmulator(() =>
withEmptyTestCollection(persistence, async (coll, firestore) => {
it("terminate doesn't crash when there is count query in flight", () => {
return withEmptyTestCollection(persistence, async (coll, firestore) => {
void getCountFromServer(coll);
await terminate(firestore);
})
);
});
});
});

it('getCountFromServer fails if user is offline', () => {
return skipTestUnlessUsingEmulator(() =>
withEmptyTestCollection(persistence, async (coll, firestore) => {
it('getCountFromServer fails if user is offline', () => {
return withEmptyTestCollection(persistence, async (coll, firestore) => {
await disableNetwork(firestore);
await expect(getCountFromServer(coll)).to.be.eventually.rejectedWith(
'Failed to get count result because the client is offline'
);
})
);
});
});
});
});
}
);
41 changes: 1 addition & 40 deletions packages/firestore/test/integration/util/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ import {
PrivateSettings,
SnapshotListenOptions,
newTestFirestore,
QueryDocumentSnapshot,
newTestApp,
FirestoreError
newTestApp
} from './firebase_export';
import {
ALT_PROJECT_ID,
Expand Down Expand Up @@ -238,22 +236,6 @@ export async function withNamedTestDbsOrSkipUnlessUsingEmulator(
}
}
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export function skipTestUnlessUsingEmulator<
T extends (...params: any[]) => Promise<void>
>(fn: T, ...params: Parameters<T>): Promise<void> {
/**
* This is a wrapper to execute test functions that can only run on emulator till
* production test environment is ready.
*/
if (!USE_EMULATOR) {
return Promise.resolve();
}

return fn(...params).catch((error: FirestoreError) => {
throw error;
});
}

export function withTestDoc(
persistence: boolean,
Expand Down Expand Up @@ -341,24 +323,3 @@ export function withTestCollectionSettings(
}
);
}

export class Post {
constructor(
readonly title: string,
readonly author: string,
readonly ref: DocumentReference | null = null
) {}
byline(): string {
return this.title + ', by ' + this.author;
}
}

export const postConverter = {
toFirestore(post: Post): DocumentData {
return { title: post.title, author: post.author };
},
fromFirestore(snapshot: QueryDocumentSnapshot): Post {
const data = snapshot.data();
return new Post(data.title, data.author, snapshot.ref);
}
};
18 changes: 1 addition & 17 deletions packages/firestore/test/lite/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@ import { QueryDocumentSnapshot } from '../../src/lite-api/snapshot';
import { AutoId } from '../../src/util/misc';
import {
DEFAULT_PROJECT_ID,
DEFAULT_SETTINGS,
USE_EMULATOR
DEFAULT_SETTINGS
} from '../integration/util/settings';

let appCount = 0;
Expand Down Expand Up @@ -102,21 +101,6 @@ export function withTestCollection(
});
}

// eslint-disable-next-line @typescript-eslint/no-explicit-any
export function skipTestUnlessUsingEmulator<
T extends (...params: any[]) => Promise<void>
>(fn: T, ...params: Parameters<T>): Promise<void> {
/**
* This is a wrapper to execute test functions that can only run on emulator till
* production test environment is ready.
*/
if (!USE_EMULATOR) {
return Promise.resolve();
}

return fn(...params);
}

// Used for testing the FirestoreDataConverter.
export class Post {
constructor(
Expand Down
Loading