FirebaseError: Document already exists #5549
Comments
|
@mafergus Thanks for reporting this!
You can enable debug logging by invoking |
|
Yes we're happy to assist in any way possible, you guys are doing great work. Would you mind pointing me to the code that generates this identifier so I can better understand the algorithm? Perhaps we can change our code to mitigate any collisions. We've noticed this specifically during periods of peak activity. We'll enable the debug logging, I would expect we'll hit this again over the weekend. |
|
The code is here: |
|
Hey @mafergus. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically. If you have more information that will help us get to the bottom of this, just add a comment! |
|
Hey @schmidt-sebastian, we captured a bunch of examples of this over the weekend in Sentry. As far as I'm aware, Sentry doesn't give a nice way to export breadcrumb logs. The two options I see are either pass you JSON of the breadcrumbs, or temporarily invite you to our Sentry project. Let me know how you'd like to proceed. |
|
It depends how much logging we are talking about here. Please contact me at mrschmidt(at)google.com. |
|
I'm having this issue too |
|
Are you able to catch and ignore these errors for now? We might need to adjust how these documents are processed internally to address this, which is major change for us. Essentially, what is happening here is that the SDK sends a write and then the connection gets dropped. The backend however executed the write. When we then try to send the write again, the backend tells us that the document already exists as our previous attempt did in fact succeed. |
|
For me this is a critical part of my application - i use the firestore document as a state record for a user that is created when they open my application, and a cloud function reads that document to determine where to route the user internally. I've temporarily made a workaround (in the below code, This works most of the time. Is there something I can do besides this to prevent this issue? It only started appearing a couple weeks ago. I'm using firebase-js version 8 |
|
I suppose a workaround is wrapping this |
|
It is very likely that you can just ignore "Document already exists" error since it means that our previous attempt to write this document actually succeeded. We cannot do this from the SDK side (yet) as some other client may have interleaved a write and changed the document after the initial write attempt. |
|
I need the id of the document, so I don't think try catching and swallowing it would work? For now I will wrap it in a cloud fn. |
|
Can you use |
|
Also seeing the error very frequently in our application. We don't have that much traffic, so I also don't think, an ID collision is very likely. |
|
Hope, |
|
I am using Code: Error: |
|
I am also running into this same issue with |
|
Thanks for continuing to bring this to our attention, your feedback helps us understand how this affects you. We still believe the root cause to be as described in this Feb 14th comment. Additionally, please continue to refer to the workaround in this Feb 17th comment. |
|
I have figured out why this is happening for me. I am testing some ML libraries that use deterministic seeding (such as seed_everything). In some cases, this can cause the Firestore library to generate the exact same document ID as before. To fix this, I basically just re-seed the random libraries before calling .add(). This fixed the issue for me, hope it helps someone else. |
|
We're getting this error in a transaction that looks like this: await runTransaction(db, async (tx) => {
const snapshot = await ref.get()
if (snapshot.exists()) {
throw new Error("document exists")
}
await ref.set(...)
})I can't reproduce it, but we have a lot of Sentry logs with it. The
Are you telling me that the change is being saved, and I should just ignore the error? What if you decide to change things in SDK and now the error becomes something I shouldn't ignore? Is there any chance you could at least rename that error to something like "Document maybe wasn't saved" or something more accurate?' |
|
@vojto did you recently update your firbase version in your code base. So between get and set, if the document state changes the transaction gets failed. |
Yep - the error started showing up only recently. And I've been able to reproduce it by running the code twice at the same time! I thought in case of concurrent edits, the transaction would be automatically rerun:
|
|
Even I am also getting same issue with firebase version : 9.9.4 |
|
@vojto The "Document Already Exists" error you experienced is specific to transactions and was a bug that was accidentally introduced in v9.9.4. It will be fixed in the next release. See #6659 for details. The "Document Already Exists" issue reported in the OP is not related to transactions, and still exists. |
|
I am also seeing this with "firebase": "8.2.3" |
|
@appsgenie That error is probably safe to ignore. What is likely happening is that the request is sent to the server to create the new document, then the client loses connection before the response is received from the server. When the client reconnects then it re-sends the "create document" request, which fails because while the document already exists because while the client was briefly offline the server completed the request to create the new document. |
|
Just to add we're having this as well. We're going to implement the workaround but a fix about this would of course be appreciated. |
|
Hi, I faced this issue in version 9.6.1 as well. I cannot ignore this issue because after The workarround solution now is to use Thank you. |
|
Hi @vutoan, yes it sounds like the workaround |
|
I'm having this same issue. There is certainly something currently amiss with My workaround to ensure that I always receive the document reference of the newly created document from const collectionReference = collection(firestore, 'collection');
const documentReference = await addDoc(collectionReference, { data: 'data'}).catch((error) => {
return doc(collectionReference, error.message.split('/').at(-1));
});This gets the newly added document's identifier from the error message itself: |
Possible Problem Source: Auth Reconnections?I think I have an insight here that hasn't been mentioned yet. Someone earlier in this thread said that this is likely because there's some sort of server disconnection for a moment and then the attempt to add the document reconnect to the server and I'm also seeing a lot of people that are reporting that this is happening in areas where people are logging in. So, I think may be something that could be happening here is that auth is triggering some type of reconnection because the user has a new token or something like that and so if you're trying to basically add a document during this process, there's something going on where there's a sort of "blip" in the connection because of auth, which is causing this error to occur. |
|
@ydax (or anyone else) Please enable Firestore debug logging (by calling |
|
Thanks @dconeybe for your tip. We're seeing the same error when we try to create new documents while the app is loading (takes long time to load all the documents as there are multiple Firebase round trips). I did It seems the write ( |
|
@GaurangTandon Thank you for the logs. That is indeed quite helpful. I noticed that you redacted some information from the logs, notably this log line which occurs 2 times: |
|
@dconeybe It's the same user ID in both cases. |
|
@GaurangTandon Could you tell me what auth method you are using (anonymous, email/password, google auth)? Also, are you using App Check? I'm asking because it's odd to me that the credentials are being refreshed in 2 seconds and I'm wondering if it's related to a specific auth mechanism or App Check. |
|
This is using google auth. I don't think we use app check. Note that when our app starts, we load several Firebase documents one after the other (in waterfall style), and we call Now, we don't plan to set forceRefresh=false in these cases, because it fixes specific bugs we observed in production historically. Regardless of that, it would be helpful if Firebase SDK could handle that correctly. |
|
@GaurangTandon Ahh that makes perfect sense that calling getIdToken() with forceRefresh=true would result in Firestore getting the updated credentials and rebuilding the "listen" and "write" streams, causing this issue. I agree that Firestore should handle this case correctly too. I'll look into it more and reply back once I have something concrete. In the meantime, the only workaround I can think of is to simply catch and ignore the "document already exists" error and treat it like a success. Alternately, you could try regenerating the document ID and creating it again if you get the "document already exists" error. |
|
@GaurangTandon I am able to reproduce this issue following your usage pattern with the code below: import { collection, Firestore, addDoc } from 'firebase/firestore';
import { getAuth, signInWithEmailAndPassword, getIdToken } from 'firebase/auth';
export async function issue5549(db: Firestore): Promise<void> {
const auth = getAuth(db.app);
const user = await signInWithEmailAndPassword(auth, "[email protected]", "password");
const collectionRef = collection(db, 'issue5549');
const promises: Array<Promise<unknown>> = [];
for (let i=0; i<30; i++) {
const documentName = `0000${i+1}`.slice(-3);
promises.push(addDoc(collectionRef, {name: documentName}));
}
for (const promise of promises) {
getIdToken(user.user, /* forceRefresh= */true)
await promise; // fails occasionally due to "document already exists" error.
}
}Here are the (redacted) logs: https://gist.github.com/dconeybe/d5ea0081828523302a34fbad117a4ba2 |
|
I talked to the team and we don't have a solution for the problem yet. I don't have an ETA either. I will keep you posted. Another workaround we came up with would be to use a transaction (i.e. |
|
@dconeybe can you provide some example code on how to use a transaction to replace The best workaround seems to be @joshkautz's suggestion on Feb 19, 2023. However, the error message format has changed since then (I'm using firestore API version 11.1.0). A more robust solution could be to change the error thrown by the firestore SDK to include the document reference as a field such as |
This suggestion sounds like it would be a reasonable, temporary fix. I get the impression that the changes required to address the root cause might be pretty substantial, and this may never be much of a priority for the Firebase team.🤷♂️ |
|
Agree with you Josh that this is at best a temporary fix, since
Note that I experience this problem almost 100% of the time while using the firebase emulator suite to write tests. |
|
@joshkautz after re-reading this thread, I realized that I missed @schmidt-sebastian's comment on Feb 17, 2022 where he explains that |
and others
[REQUIRED] Describe your environment
We are seeing this error when trying to add a document to a collection. I've scoured Google and Stack-Overflow, gone through the source code and also gprc source code. I can't understand the reason for this error:
firebase.firestore().collection('some-collection').add({ data: 1 })As far as I can tell, this should never throw a "document exists" error. Is this valid behavior or a Firebase bug?
The text was updated successfully, but these errors were encountered: