Firestore SDK onSnapshot doesn't get all responses from fetches. #7641
Assignees
Comments
x1i4b1x2
added
new
jbalidiong
added
needs-attention
and removed
new
|
Hi @x1i4b1x2. Could you enable Firestore debug logging, reproduce, and provide the logs? If you're not comfortable publishing the logs publicly then we can find a way to get them to me privately. To enable debug logging, do the following: import { setLogLevel } from 'firebase/firestore';
setLogLevel('debug'); |
|
This is the entire log of the faulty attempt, copied and pasted, the "doublevalue" I imagine are the data of the documents (number with timestamps), they are 7 documents: |
|
This is a successful loading with the same documents: |
|
Rolled back to 9.23.0, so far it looks good. |
|
Hi @x1i4b1x2. Thanks for letting us know that downgrading to 9.23.0 fixed the problem. Are you able to try any later versions, such as 10.0.0 (the next version after 9.23.0)? If you are able to find the exact version in which this problem was introduced that would be very helpful. Also, thank you for the logs. They are helpful too, although I'm not quite sure what's going on yet. Here are some observations (note that "target"="query sent to the server"):
I may end up needing a minimal reproduction app from you in order to investigate, like a github repo that I can clone and use. The app would first populate the database with documents and then run the queries that cause the problem. I know that may be difficult to create, but it may end up being needed to fully investigate and find the root cause. Just something to think about. |
|
I can confirm the problem starts from version 10.0.0. Tested side by side, at the same time, with the same documents, 9.23 is fine, 10.0.0 infinite loading. I can't replicate a repository though. |
|
@x1i4b1x2 Thank you for confirming that the issue started in 10.0.0. Another question: Are you by any chance using multitab persistence? That is, calling |
|
No, nothing sophisticated. |
|
@x1i4b1x2 Does this problem happen the first time you run these queries, or do you need to run your app for a while before this problem starts occurring? Do you have any sense of how many concurrent snapshot listeners you have registered? I ask because I'm going to attempt to reproduce and am trying to narrow down the use case. |
|
It happens occasionally, randomly. If it is infinite loading, usually closing the snapshots and retrying is enough. It happens with 5/7+ concurrent snapshot listeners. Snapshot listeners are initiated in a for-of-loop sequentially. Each document is not very light, is about 200kb each. |
|
Does your app unregister the snapshot listeners once the documents are no longer of interest? Does your app enable indexeddb persistence? Or does it just use the default (no persistence, a.k.a. "memory persistence)? |
|
Yes. Default. Standard, with no specific settings. |
|
I suspect that the problem that you are seeing is somehow related to the sheer size of the documents that you are downloading. I'd like to replicate those documents in my testing Firestore database to see if I can reproduce. In the logs you provided the lines that show the contents of the documents are truncated at 5145 characters. Would you be able to provide me with the full contents of those documents. Specifically, these documents:
Maybe a JSON dump or some way where the logs are not truncated? |
|
This is the data exported from one document, but it has to be changed in the following format:
example:
So in my example, they are 7 documents like this. |
|
FYI I'm re-assigning this issue to @tom-andersen since he has some time to investigate further. After doing some pair debugging, what we've found is that the multiple |
|
It is possibly the library from 10.0.0 because we haven't encountered the issue so far with 9.23. |
|
Can confirm that we have also run into this issue in production – we've down-graded to 9.23 until it's resolved. |
|
I have a similar issue where our Firestore query often hangs 3 out of 5 times with the error client is offline. |
|
We are looking for a consistent reproduction of this issue. If anyone can reproduce this problem consistently, that would be of great help. Also if there are any surrounding circumstances such as poor network connectivity, authentication or going offline that would be helpful clues as well. Specifically, the multiple @tonprince If there is network connectivity problems, the client will go offline, and then resume when network issue is resolved. So at first glance, this appears to be expected behaviour. I am however curious whether the query ever resumes or hangs permanently even after client goes back online? For this to be the same problem, there needs to be log output that shows duplicate |
|
@x1i4b1x2 Are you able to provide a longer log with what happened before error appeared? |
|
The logs are the complete logs of the operation with setLogLevel('debug') |
|
@x1i4b1x2, we released a fix for hanging queries so you can now use v10.5.2. This will address the issue shown in your Firestore logs from v10.4.0 in #7641 (comment). Since you also mentioned an issue in v10.0.0, there could be a separate issue. Please let us know if you have any other issues after upgrading. |
|
It's working so far, thank you. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Operating System
Web
Browser Version
latest chrome
Firebase SDK Version
10.4.0
Firebase SDK Product:
Firestore
Describe your project's tooling
create-react-app, typescript, firebase sdk
Describe the problem
Web Analytics SaaS needs to make fetches in a sequential for-of-loop. For each loop (which represents a document on Firestore) it sets up an onSnapshot to that document. The number of loops usually is between 1 to 10.
Sometimes, when doing a loop of 5-8 fetches it doesn't get a response from all the documents fetched. It sets up the onSnapshot but it doesn't get a response. Then if you retry, of course, it works.
In the case of the image, it received a response from 4 fetches successfully, for the other 2, it waited for a lot and then got the response. But usually, it just waits for a response forever.
Thank you!
Steps and code to reproduce issue
This is how onSnapshot is initiated sequentially in a for-of-loop. The code correctly sets up all the onSnapshot, it is just that sometimes, not all of them receive a response, or sometimes it takes a lot of time for some of them to get a response.
I tried to rate-limit the for each loop, but it still has this error sometimes.
The text was updated successfully, but these errors were encountered: