Skip to content

Online Count with tentative package private api #3847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 51 commits into from
Aug 29, 2022
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
12fc017
Update protos to googleapis HEAD at https://github.com/googleapis/goo…
dconeybe Jun 2, 2022
3f3a823
write.proto: add back string verify = 5
dconeybe Jun 2, 2022
d8dab8e
Update protos to googleapis HEAD of the preview branch at https://git…
dconeybe Jun 2, 2022
3ffc859
write.proto: add back string verify = 5
dconeybe Jun 2, 2022
f5b5ec4
Add API skeleton to aggregate queries and COUNT
dconeybe Jun 3, 2022
3112d79
Datastore.java: runCountQuery() added
dconeybe Jun 3, 2022
728f4cb
wire up count queries
dconeybe Jun 3, 2022
ec2be89
delete group-by stuff, since it's not needed for this experiment
dconeybe Jun 3, 2022
3f2a539
CountTest.java added
dconeybe Jun 4, 2022
dccef12
Revert changes to Firestore.kt, since this prototype only uses java
dconeybe Jun 4, 2022
20b6f61
AggregateField.java: remove everything except COUNT
dconeybe Jun 4, 2022
4eee46c
AggregateSnapshot.java deleted, since it's not used in this prototype
dconeybe Jun 4, 2022
41ce0a8
revert a bunch of other unnecssary changes
dconeybe Jun 4, 2022
931c8a6
Collapse CountQuery.java into AggregateQuery.java for simplicity
dconeybe Jun 4, 2022
57aa89b
/gradlew :firebase-firestore:googleJavaFormat
dconeybe Jun 4, 2022
e95066c
AggregateField.java: remove CountAggregateField.upTo, since it's neve…
dconeybe Jun 4, 2022
20a7e10
Update protos to googleapis HEAD at https://github.com/googleapis/goo…
dconeybe Jun 2, 2022
92c5218
write.proto: add back string verify = 5
dconeybe Jun 2, 2022
5130d16
Update protos to googleapis HEAD of the preview branch at https://git…
dconeybe Jun 2, 2022
9c3a336
write.proto: add back string verify = 5
dconeybe Jun 2, 2022
74e6d14
Add API skeleton to aggregate queries and COUNT
dconeybe Jun 3, 2022
4155ac6
Datastore.java: runCountQuery() added
dconeybe Jun 3, 2022
6b34e44
wire up count queries
dconeybe Jun 3, 2022
8dded2c
delete group-by stuff, since it's not needed for this experiment
dconeybe Jun 3, 2022
12b5cd1
CountTest.java added
dconeybe Jun 4, 2022
77beb18
Revert changes to Firestore.kt, since this prototype only uses java
dconeybe Jun 4, 2022
bcf0bcf
AggregateField.java: remove everything except COUNT
dconeybe Jun 4, 2022
9aef43a
AggregateSnapshot.java deleted, since it's not used in this prototype
dconeybe Jun 4, 2022
01420e4
revert a bunch of other unnecssary changes
dconeybe Jun 4, 2022
965144b
Collapse CountQuery.java into AggregateQuery.java for simplicity
dconeybe Jun 4, 2022
4aa3ceb
/gradlew :firebase-firestore:googleJavaFormat
dconeybe Jun 4, 2022
df05424
AggregateField.java: remove CountAggregateField.upTo, since it's neve…
dconeybe Jun 4, 2022
09685a4
Merge remote-tracking branch 'origin/dconeybe/CountFromDFE' into dcon…
wu-hui Jun 20, 2022
8d640fe
Temp checkin
wu-hui Jun 28, 2022
af28b4c
It runs now
wu-hui Jul 7, 2022
e748b6a
Format and lint
wu-hui Jul 7, 2022
3efce02
Undo change
wu-hui Jul 7, 2022
d4ced5e
Merge branch 'master' into wuandy/CountFromDFE
wu-hui Jul 8, 2022
8d98662
Public doc and changelog
wu-hui Jul 8, 2022
4d7da5a
api.txt
wu-hui Jul 8, 2022
c33a960
Support network change and address feedback
wu-hui Jul 13, 2022
e33fb81
Address more feedback
wu-hui Jul 18, 2022
ca9387d
Merge branch 'master' into wuandy/CountFromDFE
wu-hui Jul 18, 2022
89dc144
More
wu-hui Jul 20, 2022
2dff823
More feedback
wu-hui Jul 27, 2022
d6e7098
Remove retry
wu-hui Aug 15, 2022
419f3fe
Fix CI
wu-hui Aug 15, 2022
ae618b4
Hide this feature for now
wu-hui Aug 22, 2022
3f3cb57
Update API
wu-hui Aug 22, 2022
2dc20b5
Only run tests with emulator
wu-hui Aug 22, 2022
7f090e2
More feedback
wu-hui Aug 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions firebase-firestore/api.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,19 @@ package com.google.firebase {

package com.google.firebase.firestore {

public class AggregateQuery {
method @NonNull public com.google.android.gms.tasks.Task<com.google.firebase.firestore.AggregateQuerySnapshot> get(@NonNull com.google.firebase.firestore.AggregateSource);
method @NonNull public com.google.firebase.firestore.Query getQuery();
}

public class AggregateQuerySnapshot {
method @Nullable public Long getCount();
}

public enum AggregateSource {
enum_constant public static final com.google.firebase.firestore.AggregateSource SERVER_DIRECT;
}

public class Blob implements java.lang.Comparable<com.google.firebase.firestore.Blob> {
method public int compareTo(@NonNull com.google.firebase.firestore.Blob);
method @NonNull public static com.google.firebase.firestore.Blob fromBytes(@NonNull byte[]);
Expand Down Expand Up @@ -290,6 +303,7 @@ package com.google.firebase.firestore {
method @NonNull public com.google.firebase.firestore.ListenerRegistration addSnapshotListener(@NonNull com.google.firebase.firestore.MetadataChanges, @NonNull com.google.firebase.firestore.EventListener<com.google.firebase.firestore.QuerySnapshot>);
method @NonNull public com.google.firebase.firestore.ListenerRegistration addSnapshotListener(@NonNull java.util.concurrent.Executor, @NonNull com.google.firebase.firestore.MetadataChanges, @NonNull com.google.firebase.firestore.EventListener<com.google.firebase.firestore.QuerySnapshot>);
method @NonNull public com.google.firebase.firestore.ListenerRegistration addSnapshotListener(@NonNull android.app.Activity, @NonNull com.google.firebase.firestore.MetadataChanges, @NonNull com.google.firebase.firestore.EventListener<com.google.firebase.firestore.QuerySnapshot>);
method @NonNull public com.google.firebase.firestore.AggregateQuery count();
method @NonNull public com.google.firebase.firestore.Query endAt(@NonNull com.google.firebase.firestore.DocumentSnapshot);
method @NonNull public com.google.firebase.firestore.Query endAt(java.lang.Object...);
method @NonNull public com.google.firebase.firestore.Query endBefore(@NonNull com.google.firebase.firestore.DocumentSnapshot);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package com.google.firebase.firestore;

import static com.google.firebase.firestore.testutil.IntegrationTestUtil.testCollection;
import static com.google.firebase.firestore.testutil.IntegrationTestUtil.testCollectionWithDocs;
import static com.google.firebase.firestore.testutil.IntegrationTestUtil.testFirestore;
import static com.google.firebase.firestore.testutil.IntegrationTestUtil.waitFor;
import static com.google.firebase.firestore.testutil.IntegrationTestUtil.waitForException;
import static com.google.firebase.firestore.testutil.TestUtil.map;
import static org.hamcrest.CoreMatchers.instanceOf;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotEquals;

import androidx.test.ext.junit.runners.AndroidJUnit4;
import com.google.firebase.firestore.testutil.IntegrationTestUtil;
import org.junit.After;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(AndroidJUnit4.class)
public class CountTest {

@After
public void tearDown() {
IntegrationTestUtil.tearDown();
}

@Test
public void testCountQueryEquals() {
CollectionReference coll1 = testCollection("foo");
CollectionReference coll1_same = coll1.firestore.collection(coll1.getPath());
AggregateQuery query1 = coll1.count();
AggregateQuery query1_same = coll1_same.count();
AggregateQuery query2 =
coll1.document("bar").collection("baz").whereEqualTo("a", 1).limit(100).count();
AggregateQuery query2_same =
coll1.document("bar").collection("baz").whereEqualTo("a", 1).limit(100).count();
AggregateQuery query3 =
coll1.document("bar").collection("baz").whereEqualTo("b", 1).orderBy("c").count();
AggregateQuery query3_same =
coll1.document("bar").collection("baz").whereEqualTo("b", 1).orderBy("c").count();

assertEquals(query1, query1_same);
assertEquals(query2, query2_same);
assertEquals(query3, query3_same);

assertEquals(query1.hashCode(), query1_same.hashCode());
assertEquals(query2.hashCode(), query2_same.hashCode());
assertEquals(query3.hashCode(), query3_same.hashCode());

assertNotEquals(null, query1);
assertNotEquals(query1, query2);
assertNotEquals(query2, query3);
assertNotEquals(query1.hashCode(), query2.hashCode());
assertNotEquals(query2.hashCode(), query3.hashCode());
}

@Test
public void testCanRunCount() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "b"),
"c", map("k", "c")));

AggregateQuerySnapshot snapshot =
waitFor(collection.count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(3), snapshot.getCount());
}

@Test
public void testCanRunCountWithFilters() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "b"),
"c", map("k", "c")));

AggregateQuerySnapshot snapshot =
waitFor(collection.whereEqualTo("k", "b").count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(1), snapshot.getCount());
}

@Test
public void testSnapshotEquals() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "b"),
"c", map("k", "c")));

AggregateQuerySnapshot snapshot1 =
waitFor(collection.whereEqualTo("k", "b").count().get(AggregateSource.SERVER_DIRECT));
AggregateQuerySnapshot snapshot1_same =
waitFor(collection.whereEqualTo("k", "b").count().get(AggregateSource.SERVER_DIRECT));

AggregateQuerySnapshot snapshot2 =
waitFor(collection.whereEqualTo("k", "a").count().get(AggregateSource.SERVER_DIRECT));
waitFor(collection.document("d").set(map("k", "a")));
AggregateQuerySnapshot snapshot2_different =
waitFor(collection.whereEqualTo("k", "a").count().get(AggregateSource.SERVER_DIRECT));

assertEquals(snapshot1, snapshot1_same);
assertEquals(snapshot1.hashCode(), snapshot1_same.hashCode());
assertEquals(snapshot1.getQuery(), collection.whereEqualTo("k", "b").count());

assertNotEquals(snapshot1, snapshot2);
assertNotEquals(snapshot1.hashCode(), snapshot2.hashCode());
assertNotEquals(snapshot2, snapshot2_different);
assertNotEquals(snapshot2.hashCode(), snapshot2_different.hashCode());
}

@Test
public void testCanRunCollectionGroupQuery() {
FirebaseFirestore db = testFirestore();
// Use .document() to get a random collection group name to use but ensure it starts with 'b'
// for predictable ordering.
String collectionGroup = "b" + db.collection("foo").document().getId();

String[] docPaths =
new String[] {
"abc/123/${collectionGroup}/cg-doc1",
"abc/123/${collectionGroup}/cg-doc2",
"${collectionGroup}/cg-doc3",
"${collectionGroup}/cg-doc4",
"def/456/${collectionGroup}/cg-doc5",
"${collectionGroup}/virtual-doc/nested-coll/not-cg-doc",
"x${collectionGroup}/not-cg-doc",
"${collectionGroup}x/not-cg-doc",
"abc/123/${collectionGroup}x/not-cg-doc",
"abc/123/x${collectionGroup}/not-cg-doc",
"abc/${collectionGroup}"
};
WriteBatch batch = db.batch();
for (String path : docPaths) {
batch.set(db.document(path.replace("${collectionGroup}", collectionGroup)), map("x", 1));
}
waitFor(batch.commit());

AggregateQuerySnapshot snapshot =
waitFor(db.collectionGroup(collectionGroup).count().get(AggregateSource.SERVER_DIRECT));
assertEquals(
Long.valueOf(5), // "cg-doc1", "cg-doc2", "cg-doc3", "cg-doc4", "cg-doc5",
snapshot.getCount());
}

@Test
public void testCanRunCountWithFiltersAndLimits() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "a"),
"c", map("k", "a"),
"d", map("k", "d")));

AggregateQuerySnapshot snapshot =
waitFor(
collection.whereEqualTo("k", "a").limit(2).count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(2), snapshot.getCount());

snapshot =
waitFor(
collection
.whereEqualTo("k", "a")
.limitToLast(2)
.count()
.get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(2), snapshot.getCount());

snapshot =
waitFor(
collection
.whereEqualTo("k", "d")
.limitToLast(1000)
.count()
.get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(1), snapshot.getCount());
}

@Test
public void testCanRunCountOnNonExistentCollection() {
CollectionReference collection = testFirestore().collection("random-coll");

AggregateQuerySnapshot snapshot =
waitFor(collection.count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(0), snapshot.getCount());

snapshot =
waitFor(collection.whereEqualTo("k", 100).count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(0), snapshot.getCount());
}

@Test
public void testFailWithoutNetwork() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "b"),
"c", map("k", "c")));
waitFor(collection.getFirestore().disableNetwork());

Exception e = waitForException(collection.count().get(AggregateSource.SERVER_DIRECT));
assertThat(e, instanceOf(FirebaseFirestoreException.class));
assertEquals(
FirebaseFirestoreException.Code.UNAVAILABLE, ((FirebaseFirestoreException) e).getCode());

waitFor(collection.getFirestore().enableNetwork());
AggregateQuerySnapshot snapshot =
waitFor(collection.count().get(AggregateSource.SERVER_DIRECT));
assertEquals(Long.valueOf(3), snapshot.getCount());
}

@Test
public void testExponentialBackoffWorks() {
CollectionReference collection =
testCollectionWithDocs(
map(
"a", map("k", "a"),
"b", map("k", "b"),
"c", map("k", "c")));
waitFor(collection.getFirestore().disableNetwork());

Exception e =
waitForException(
collection.count().get(AggregateSource.SERVER_DIRECT, /* maxAttempts */ 2));
assertThat(e, instanceOf(FirebaseFirestoreException.class));
assertEquals(
FirebaseFirestoreException.Code.UNAVAILABLE, ((FirebaseFirestoreException) e).getCode());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package com.google.firebase.firestore;

import androidx.annotation.NonNull;

/**
* Represents which field to aggregate on for a {@link AggregateQuery}, and what type of
* aggregations to perform.
*/
abstract class AggregateField {

private AggregateField() {}

/**
* Returns a {@link CountAggregateField} which counts the number of documents matching the {@link
* AggregateQuery}.
*/
@NonNull
public static CountAggregateField count() {
return new CountAggregateField();
}

static final class CountAggregateField extends AggregateField {
CountAggregateField() {}
}
}
Loading