SSR / Firestore / Not able to load proto file

[vroussel35]

[REQUIRED] Describe your environment

• Operating System version: Node 10.18.1
• Browser version: N/A (node)
• Firebase SDK version: 7.15.1
• Firebase Product: firestore (auth, database, storage, etc)

[REQUIRED] Describe the problem

In an Angular Universal (SSR) context, calling firestore method on the server does not work.

Error enabling offline persistence. Falling back to persistence disabled: Error: ENOENT: no such file or directory, open '[project-directory-path]/dist/server/src/protos/google/firestore/v1/firestore.proto'

Problem identified in the function loadProtos() : node_modules/@firebase/firestore/dist/index.node.cjs.js (line 22969)

function loadProtos() {
    var root = path.resolve(__dirname, "src/protos");
    ....   
}

The value of __dirname is the project directory path and therefore, during execution, node_modules/@firebase/firestore/dist/src/protos path can't be found

Steps to reproduce:

Download this repository https://github.com/vroussel35/error-proto then
npm install
npm run dev:ssr
Browse http://localhost:4200/

// TODO(you): code here to reproduce the problem

[schmidt-sebastian]

@vroussel35 Thanks for filing this.

Can you try the steps outlined here: https://stackoverflow.com/questions/61182267/nestjs-angular-universal-not-working-error-enabling-offline-persistence ?

Otherwise, it is possible to specify a custom proto directory via the environment variable FIRESTORE_PROTO_ROOT.

[vroussel35]

Hi @schmidt-sebastian - thanks for your reply.

SO link
The steps outlined in the SO link you have shared are not exactly matching our context : we are not using angularfire - we just want to use the basic firebase/firestore package and make a basic firestore query.

FIRESTORE_PROTO_ROOT
This environment variable is not used in the file node_modules/@firebase/firestore/dist/index.node.cjs.js (line 22969) so using it won't change the wrong path.

[Feiyang1]

The problem is that firestore is bundled while it should not be in the Node environment. Firestore has a special Nodejs build and assumes it runs in node_modules/. Generally you don't want to bundle libraries in node apps including when doing ssr, because some libs may rely on native code that can't be bundled.

As a workaround, you should exclude any firebase package including firestore from being bundled when doing SSR. Can you try the solution I provided for a similar issue? Does it solve your problem?

[vroussel35]

The problem if we add firestore packages to the externalDependencies server build block in our angular.json like this:

"server": {
 "builder": "@angular-devkit/build-angular:server",
 "options": {
  "outputPath": "dist/server",
  "main": "server.ts",
  "tsConfig": "tsconfig.server.json",
  "externalDependencies": [
   "@firebase/firestore"
  ]
 },
 ....

We have now a new error :

ERROR Error: Uncaught (in promise): TypeError: firebase.firestore is not a function
TypeError: firebase.firestore is not a function

[vroussel35]

Progressing on our investigations...

We found a way to fix our last issue TypeError: firebase.firestore is not a function looking at how the @angular/fire package is managing this.

In the module/class/service calling the firebase.initializeApp(...) function, we use the registerFirestore function from the @firebase/firestore package like this :

import { ...,  Inject, PLATFORM_ID } from "@angular/core";
import { isPlatformServer } from "@angular/common";
import { registerFirestore } from '@firebase/firestore';
...
constructor(
    @Inject(PLATFORM_ID) private platformId: Object,
) { ...}

if (isPlatformServer(this.platformId)) {
 if (registerFirestore) {
  registerFirestore(firebase);
 }
}
...
firebase.initializeApp({...})

Now, we are facing another issue : our query to Firestore is hanging and never return to the client. I think it's something related to the Angular Zone topic.

If someone has any information related to this, let me know.

Regarding this specific issue: I suspect we are not the only ones facing this issue so what can be done to fix it without the need of these workarounds? I guess the firebase package could identify if it's running on Node and then do the mentioned workarounds above?

[schmidt-sebastian]

Calling registerFirestore is definitely something you shouldn't need to do. I am however not all that familiar with the Angular ecosystem (maybe @jamesdaniels can chime in). For now, I would like to see if you tried using https://www.npmjs.com/package/@angular/fire?

[vroussel35]

Calling registerFirestore is definitely something you shouldn't need to do

@schmidt-sebastian What is the purpose of this function? Can you explain why we shouldn't need to do that?

From what we have seen, @angular/fire (and @jamesdaniels) seems to call it when in a node context.

For now, I would like to see if you tried using https://www.npmjs.com/package/@angular/fire

We tried it and it's kind of working on SSR - our problem is this package is based on Observables and our need is to perform several Promises - so that's why we would like to use the default firebase package.

If we are using the @angular/fire package, we could have used the toPromise() function on the Observable but this doesn't work either.

For example:

This works => query is done and SSR is rendered.

this.angularFirestore.collection(
 'collectionId', ref => ref.where('field', '==', 'value'))
 .snapshotChanges().subscribe(values => {
 ...
});

This doesn't work: calling toPromise() on the Observable => the SSR execution is hanging and only "resolve" after 60 seconds:

this.angularFirestore.collection(
 'collectionId', ref => ref.where('field', '==', 'value'))
 .snapshotChanges().pipe(take(1)).toPromise().then(values => {
 ...
});

[jamesdaniels]

Hey @vroussel35, seems like you are finding all the sharp edges with Firebase + Angular that we've been working on over on AngularFire.

The root of your rendering issue is that Firebase isn't Zone.js patched. It's side-effects (Timers, sockets, etc.) destabilize Zone.js and keep SSR from rendering and your service workers from initializing in the browser. As you've noted we've solved this in AngularFire but are Observable based.

TBH toPromise has too many sharp corners for me to suggest it to folk, can you try get() on the AF collection instead? That will return a promise that should be appropriately Zone wrapped, though I don't recall if I've tested it on SSR.

this.angularFirestore.collection(
 'collectionId', ref => ref.where('field', '==', 'value'))
).get()

You could always make a promise out of the subscription yourself:

new Promise((resolve, reject) => 
  this.angularFirestore.collection(
   'collectionId', ref => ref.where('field', '==', 'value'))
  .snapshotChanges().pipe((take(1)).subscribe(values => {
    resolve(values)
  })
);

[schmidt-sebastian]

Calling registerFirestore is definitely something you shouldn't need to do

@schmidt-sebastian What is the purpose of this function? Can you explain why we shouldn't need to do that?

registerFirestore() is an internal function that we use to inject Firestore into firebase's namespace. We offer no guarantee about its functionality or its name, and it is even mangled in some of our builds. Only APIs exposed via our Typescript definition files (and hence mentioned in our docs) are guaranteed to be stable.

[jamesdaniels]

@schmidt-sebastian I can definitely confirm that you need to call registerFirestore to get Firestore added to the Firebase namespace in an Angular app. Recently I’ve had to start doing similar for other modules too.

[schmidt-sebastian]

@jamesdaniels @Feiyang1 We probably need to find a solution for this as registerFirestore could be renamed by our build tooling.

[jamesdaniels]

FWIW I imagine the root cause is over aggressive tree-shaking somewhere in the ngcc build tooling, many of the problems started with developers who were going over to Angular Ivy. I imagine it's scrambling the CJS somehow, which is why Angular is depreciating CJS support. I've blown a lot of time digging into a root cause with no avail and had that work-around... I'm just hoping the @firebase/* packages give us a better path forward with Angular & frameworks with similar tooling that expect side-effect free packages in the mid-term.

[Feiyang1]

@vroussel35 Can you add all firebase packages to externalDependencies, e.g. @firebase/app?(you shouldn't need to call registerFirestore manually)

If you only exclude @firebase/firestore from being bundled, it causes multiple firebase(@firebase/app) instances issue - one in node_modules/ and one in your bundle.

The unbundled @firebase/firestore will register with firebase in the node_modules/, while your app will use the bundled firebase. It's why you are seeing TypeError: firebase.firestore is not a function. You should exclude all @firebase packages from being bundled, so they all reference the same firebase (the one in the node_modules/).

[vroussel35]

TBH toPromise has too many sharp corners for me to suggest it to folk, can you try get() on the AF collection instead? That will return a promise that should be appropriately Zone wrapped, though I don't recall if I've tested it on SSR.

@jamesdaniels the get() function returns an Observable ;). I tested it and it seems the get function is not appropriately Zone wrapped: the page with a simple get call renders always exactly after 60 seconds.

Can you add all firebase packages to externalDependencies, e.g. @firebase/app?(you shouldn't need to call registerFirestore manually)

@Feiyang1 I can confirm adding all firebase packages to externalDependencies works. The workaround (calling registerFirestore manually) is not needed anymore.

Here is my angular.json "server" config:

"server": {
 "builder": "@angular-devkit/build-angular:server",
 "options": {
  "outputPath": "dist/web/server",
  "main": "projects/web/server.ts",
  "tsConfig": "projects/web/tsconfig.server.json",
  "externalDependencies": [
   "@firebase/analytics",
   "@firebase/analytics-types",
   "@firebase/app",
   "@firebase/app-types",
   "@firebase/auth",
   "@firebase/auth-interop-types",
   "@firebase/auth-types",
   "@firebase/component",
   "@firebase/database",
   "@firebase/database-types",
   "@firebase/firestore",
   "@firebase/firestore-types",
   "@firebase/functions",
   "@firebase/functions-types",
   "@firebase/installations",
   "@firebase/installations-types",
   "@firebase/logger",
   "@firebase/messaging",
   "@firebase/messaging-types",
   "@firebase/performance",
   "@firebase/performance-types",
   "@firebase/polyfill",
   "@firebase/remote-config",
   "@firebase/remote-config-types",
   "@firebase/storage",
   "@firebase/storage-types",
   "@firebase/util",
   "@firebase/webchannel-wrapper",
  ]
 },
 "configurations": {
  "production": { ... }
 },
 ...
}

@jamesdaniels @Feiyang1 @schmidt-sebastian I will close this issue as this specific issue is resolved by adding all the @firebase packages to the externalDependancies.

But I will create another issue (with repo and steps to reproduce) to tackle the other issue with - for example - a very basic firestore get() call like shown below which is never SSR'ed / rendered:

firebase.firestore().collection(collectionId).doc(docId).get().then(result => {
 console.log(result.id);
});