Firestore queries performance problem

[algrid]

Versions:

• Android Studio version: 3.4.1
• Firebase Component: Firestore
• Component version: firebase-firestore:20.0.0

The problem:

Firestore queries can work slow, especially when they return a lot of documents. And after such a query another query (that returns even a single document) start to work slower too.

Some numbers:

• I have a sample data set of ~5k documents
• I'm making a request that returns ~500 documents and it takes 10-18 sec
• After that a 1-document (limit(1)) request becomes slower and takes 1-2 sec (vs 400-500 ms before)
• First requests are always slower. For example the 1-document request takes 1.5-3 sec for the first time.
• The problem is more prominent on older/low end devices

If I turn off persistence the queries become ~2x faster.

When doing these slow queries I often see in logcat lines like this:

2019-06-24 23:59:42.364 30808-30836/com.sample W/CursorWindow: Window is full: requested allocation 7095 bytes, free space 2925 bytes, window size 2097152 bytes
2019-06-24 23:59:42.681 30808-30836/com.sample W/CursorWindow: Window is full: requested allocation 7095 bytes, free space 2889 bytes, window size 2097152 bytes
2019-06-24 23:59:42.991 30808-30819/com.sample I/com.sample: Background concurrent copying GC freed 402351(12MB) AllocSpace objects, 0(0B) LOS objects, 50% free, 14MB/29MB, paused 46us total 115.483ms

To reproduce the problem I created a sample project: https://github.com/algrid/Firestore-slow-sample (don't forget to add your own google-services.json)

• Every "Populate" click adds a 500-docs batch (it's slow so be patient, wait for a toast message). Click 10 times to create a 5k data set.
• "Query" button - the big query (wait for the toast message)
• "Query lim 1" button - the small query

Probably this is related to the problems reported here: flutter/flutter#18570

[schmidt-sebastian]

@algrid Thanks for reporting this! Unfortunately, there is no quick fix, but we are currently prioritizing Query performance on Android. We hope to have some fixes for you soon!

[HalgandP]

I'm facing the same problem of slow performance.
The more i addSnapshotListener the more the performance are slowing.

If i disabled persistence it's also faster, but not enough. I've tried to set the cache to unlimited, didn't help. I see the same messages as algrid in the logcat.

Testing on Huawei P8 lite.
'com.google.firebase:firebase-firestore:19.0.2'
'com.google.firebase:firebase-core:16.0.9'

[algrid]

@schmidt-sebastian Thank you! Glad to know that someone is working on this.
Firestore is a great tool and is nicely integrated with other Firebase stuff, but these performance issues sometimes make its usage problematic.

[schmidt-sebastian]

While this is still ongoing work, I have merged two PRs that will improve Query execution performance with the next release:

• Performance: Don't deserialize full document Proto for Query execution #561
• Performance: Decode documents in background thread #559

These PRs improve our deserialization form Persistence by a) skipping the step when it is not needed and b) moving it to a background thread if it is.

[schmidt-sebastian]

Closing this issue, due to inactivity and the hope that it has been addressed by the code in #727. Please file a new issue should this problem persist.