Skip to content

Can't import Firebase SDK #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rick-stevens opened this issue Feb 2, 2019 · 31 comments
Closed

Can't import Firebase SDK #112

rick-stevens opened this issue Feb 2, 2019 · 31 comments
Labels

Comments

@rick-stevens
Copy link

rick-stevens commented Feb 2, 2019

Related

https://stackoverflow.com/questions/51522280/use-firebase-sdk-with-netlify-lambda-functions

Dependencies

{
  "firebase": "^5.8.2",
  "firebase-admin": "^7.0.0",
  "netlify-lambda": "^1.3.1",
}

The issue

I'd love to use Firebase (Firestore/Storage) in my Netlify function, though it won't even let me import any related SDK using netlify-lambda serve. I've tried both firebase and firebase-admin which respectively result in these delayed (5 - 10 seconds) errors:

Using firebase:

import firebase from 'firebase/app'
import 'firebase/firestore'

exports.handler = (event, context, callback) => callback(null, {
  statusCode: 200,
  body: 'OK'
})
Output:
netlify-lambda: Starting server
Lambda server is listening on 9000
Hash: e3817619e6c76f23fc78
Version: webpack 4.29.0
Time: 3210ms
Built at: 02/02/2019 11:55:21 PM
      Asset      Size  Chunks             Chunk Names
firebase.js  2.72 MiB       0  [emitted]  firebase
Entrypoint firebase = firebase.js
 [10] external "fs" 42 bytes {0} [built]
 [12] ../node_modules/firebase-admin/lib/utils/error.js 34.8 KiB {0} [built]
 [14] ../node_modules/firebase-admin/lib/utils/validator.js 7.25 KiB {0} [built]
 [56] ../node_modules/firebase-admin/lib/utils/deep-copy.js 2.6 KiB {0} [built]
[115] ../node_modules/firebase-admin/lib/auth/credential.js 15 KiB {0} [built]
[235] ../node_modules/firebase-admin/lib/index.js 1.36 KiB {0} [built]
[261] ../node_modules/firebase-admin/lib/auth/auth.js 20.5 KiB {0} [built]
[262] ../node_modules/@firebase/database/dist/index.esm.js 592 KiB {0} [built]
[264] ../node_modules/firebase-admin/lib/messaging/messaging.js 48.8 KiB {0} [built]
[265] ../node_modules/firebase-admin/lib/storage/storage.js 4.98 KiB {0} [built]
[354] ../node_modules/@google-cloud/firestore/build/src/index.js 52.9 KiB {0} [built]
[422] ./components/serviceAccountKey.json 2.32 KiB {0} [built]
[423] ./firebase.js 1.17 KiB {0} [built]
[424] ../node_modules/firebase-admin/lib/default-namespace.js 1.16 KiB {0} [built]
[425] ../node_modules/firebase-admin/lib/firebase-namespace.js 18.4 KiB {0} [built]
    + 780 hidden modules

WARNING in ../node_modules/grpc/src/grpc_extension.js 32:12-33
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js 20:22-48
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/util/versioning.js 17:20-67
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/hash-stream-validation/index.js
Module not found: Error: Can't resolve 'fast-crc32c' in '/repos/test/node_modules/hash-stream-validation'
 @ ../node_modules/hash-stream-validation/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/bytebuffer/dist/bytebuffer-node.js
Module not found: Error: Can't resolve 'memcpy' in '/repos/test/node_modules/bytebuffer/dist'
 @ ../node_modules/bytebuffer/dist/bytebuffer-node.js
 @ ../node_modules/protobufjs/dist/protobuf.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/write-file-atomic/index.js
Module not found: Error: Can't resolve 'worker_threads' in '/repos/test/node_modules/write-file-atomic'
 @ ../node_modules/write-file-atomic/index.js
 @ ../node_modules/gcs-resumable-upload/node_modules/configstore/index.js
 @ ../node_modules/gcs-resumable-upload/build/src/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

Or using firebase-admin:

import * as admin from 'firebase-admin'

exports.handler = (event, context, callback) => callback(null, {
  statusCode: 200,
  body: 'OK'
})
Output:
netlify-lambda: Starting server
Lambda server is listening on 9000
Hash: e3817619e6c76f23fc78
Version: webpack 4.29.0
Time: 3102ms
Built at: 02/02/2019 11:53:21 PM
      Asset      Size  Chunks             Chunk Names
firebase.js  2.72 MiB       0  [emitted]  firebase
Entrypoint firebase = firebase.js
 [10] external "fs" 42 bytes {0} [built]
 [12] ../node_modules/firebase-admin/lib/utils/error.js 34.8 KiB {0} [built]
 [14] ../node_modules/firebase-admin/lib/utils/validator.js 7.25 KiB {0} [built]
 [56] ../node_modules/firebase-admin/lib/utils/deep-copy.js 2.6 KiB {0} [built]
[115] ../node_modules/firebase-admin/lib/auth/credential.js 15 KiB {0} [built]
[235] ../node_modules/firebase-admin/lib/index.js 1.36 KiB {0} [built]
[261] ../node_modules/firebase-admin/lib/auth/auth.js 20.5 KiB {0} [built]
[262] ../node_modules/@firebase/database/dist/index.esm.js 592 KiB {0} [built]
[264] ../node_modules/firebase-admin/lib/messaging/messaging.js 48.8 KiB {0} [built]
[265] ../node_modules/firebase-admin/lib/storage/storage.js 4.98 KiB {0} [built]
[354] ../node_modules/@google-cloud/firestore/build/src/index.js 52.9 KiB {0} [built]
[422] ./components/serviceAccountKey.json 2.32 KiB {0} [built]
[423] ./firebase.js 1.17 KiB {0} [built]
[424] ../node_modules/firebase-admin/lib/default-namespace.js 1.16 KiB {0} [built]
[425] ../node_modules/firebase-admin/lib/firebase-namespace.js 18.4 KiB {0} [built]
    + 780 hidden modules

WARNING in ../node_modules/grpc/src/grpc_extension.js 32:12-33
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js 20:22-48
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/grpc/node_modules/node-pre-gyp/lib/util/versioning.js 17:20-67
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/grpc/node_modules/node-pre-gyp/lib/pre-binding.js
 @ ../node_modules/grpc/src/grpc_extension.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/hash-stream-validation/index.js
Module not found: Error: Can't resolve 'fast-crc32c' in '/repos/test/node_modules/hash-stream-validation'
 @ ../node_modules/hash-stream-validation/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/bytebuffer/dist/bytebuffer-node.js
Module not found: Error: Can't resolve 'memcpy' in '/repos/test/node_modules/bytebuffer/dist'
 @ ../node_modules/bytebuffer/dist/bytebuffer-node.js
 @ ../node_modules/protobufjs/dist/protobuf.js
 @ ../node_modules/grpc/index.js
 @ ../node_modules/google-gax/build/src/grpc.js
 @ ../node_modules/google-gax/build/src/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
 @ ../node_modules/@google-cloud/firestore/build/src/v1beta1/index.js
 @ ../node_modules/@google-cloud/firestore/build/src/index.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js

WARNING in ../node_modules/write-file-atomic/index.js
Module not found: Error: Can't resolve 'worker_threads' in '/repos/test/node_modules/write-file-atomic'
 @ ../node_modules/write-file-atomic/index.js
 @ ../node_modules/gcs-resumable-upload/node_modules/configstore/index.js
 @ ../node_modules/gcs-resumable-upload/build/src/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ../node_modules/firebase-admin/lib/storage/storage.js
 @ ../node_modules/firebase-admin/lib/firebase-namespace.js
 @ ../node_modules/firebase-admin/lib/default-namespace.js
 @ ../node_modules/firebase-admin/lib/index.js
 @ ./firebase.js
@danew
Copy link

danew commented Feb 3, 2019

I believe this is because that package does't like being bundled with netlify-lambda's default webpack config.
I'm currently also having the same issue, I'm hoping to have some time to investigate it further tomorrow.

@danew
Copy link

danew commented Feb 3, 2019

It turns out that this issue is because the packages are being bundled with Webpack. Since the developer never intended this, since it's a "backend" module and wasn't intended to be bundled.

A fix for this would be to package your functions as a zip file like @DavidWells has done here https://github.com/DavidWells/function-zips/

@swyxio
Copy link
Contributor

swyxio commented Feb 3, 2019

i dont have experience with firebase sdk but @danew makes sense. zip it up. i'd also welcome a PR for instructions on zipping.

we will be releasing more native support for this kind of thing in a couple months - please stay tuned.

@swyxio
Copy link
Contributor

swyxio commented Feb 3, 2019

@rick-stevens
Copy link
Author

rick-stevens commented Feb 3, 2019

Thanks @danew / @sw-yx! This workaround does, however, cut out netlify-lambda, which was the reason for this issue. There's no need to zip the Firebase SDK if I'm not using the transpiler in the first place (as also shown in the blog post you linked from the README).

It makes developing harder as I'll have to write my own file watcher and proxy server to integrate with my app's framework. Looking forward to the native support!

@swyxio
Copy link
Contributor

swyxio commented Feb 3, 2019

ok, i see. i love using firebase too and am keen to provide better DX for this. let me know if you figure out a way.

@danew
Copy link

danew commented Feb 3, 2019

@rick-stevens as an alternative you can still use netlify-lambda serve for local development by using a custom webpack config which includes webpack-node-externals. This will stop webpack from bundling your bundles these packages together and require them from your node_modules instead.

Then you just need to create a simple build script similar to the one I linked above for build time.

@chinchang
Copy link

Even "zip n ship" is not a solution. node_modules installed on user's machine won't necessarily work on Netlify function server. grpc has the same issues. I am guessing it gets compiled according to the host machine architecture.

@swyxio
Copy link
Contributor

swyxio commented Feb 15, 2019

hmm. if you can make a small repro i can take it to our platform devs. i would v much like to support this usecase in production (even without netlify-lambda)

@swyxio
Copy link
Contributor

swyxio commented Feb 15, 2019

@rick-stevens just noting a similar stackoverflow answer https://stackoverflow.com/questions/51522280/use-firebase-sdk-with-netlify-lambda-functions/51656271 it seems that instead of using the SDK you can just use their REST endpoint?

@rick-stevens
Copy link
Author

Hi @sw-yx, I think this is the same Stack Overflow issue I noted originally. I already tried looking into their REST API, but while this is okay for some of their services, it doesn't provide one for Firebase Storage for instance (one I need in my case).

I've already switched to a full Firebase stack for my project (including Hosting and Functions), so I don't have a quick example anymore. But including firebase-admin (or other libraries like grpc as @chinchang stated) into an empty netlify-lambda project function, should be sufficient to reproduce!

@swyxio
Copy link
Contributor

swyxio commented Feb 15, 2019

my bad, i skipped right past that SO answer. i see, its firebase storage thats needed. good enough for us to work with, thank you!

@chinchang
Copy link

I too switched to Firebase functions. And wanted to highlight that their functions deployment experience is much better than netlify as there is no need of packaging nor uploading node_modules. It just works. Maybe we can get a similar experience in Netlify also :)

@ghost
Copy link

ghost commented Feb 15, 2019

hey @sw-yx , yea also the REST solution does not work for Cloud Firestore, only for the Realtime Database. An integrated netlify-lambda solution would be much appreciated.

@renatodex
Copy link

renatodex commented Mar 29, 2019

+1000000 here.

This issue is massive for us.
We had to basically to implement the Whole Firebase SDK in Rest to allow this integration.
Of course we love Netlify Lambda functions much more than Firebase functions, since we are hosting the project on Netlify, and we don't want to lose the awesome Deploy features provided by Netlify.

The firebase authentication using Service Accounts is still a huge problem for us, since there is not enough documentation to clarify this process without using Google SDKs.

I opened a thread on firebase-admin also to grab more attention to this.
firebase/firebase-admin-node#491

@mklan
Copy link

mklan commented May 2, 2019

Hey do you still have the issue? I found a reasonable solution:

add this webpack configuration to your project

//./config/webpack.functions.js
const nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()],
};

and update your scripts inside your package.json to take a custom webpack config:

"scripts": {
    "build:lambda": "netlify-lambda build --config ./config/webpack.functions.js src/lambda",
    "start:lambda": "netlify-lambda serve --config ./config/webpack.functions.js src/lambda"
  },

@rick-stevens
Copy link
Author

Hi @mklan, I've personally moved away from this approach all together, but by looking at the thread activity I think other people will probably have a go at this. Thanks!

@swyxio
Copy link
Contributor

swyxio commented May 2, 2019

thats awesome @mklan! do you think it makes sense to build it into netlify-lambda?

@mklan
Copy link

mklan commented May 3, 2019

@sw-yx I am not sure what side effects this config has.

@swyxio
Copy link
Contributor

swyxio commented May 3, 2019

ok. cc @renatodex

@juanojeda
Copy link

juanojeda commented May 16, 2019

//./config/webpack.functions.js
const nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()],
};

and update your scripts inside your package.json to take a custom webpack config:

"scripts": {
    "build:lambda": "netlify-lambda build --config ./config/webpack.functions.js src/lambda",
    "start:lambda": "netlify-lambda serve --config ./config/webpack.functions.js src/lambda"
  },

I've been banging my head against this issue today, and finally came across this solution. It works for me, and it'd be awesome if it was documented in netlify-lambda somewhere.

👍 Thanks gang!

@swyxio
Copy link
Contributor

swyxio commented May 16, 2019

@apotox
Copy link

apotox commented Jun 27, 2019

in my case when I tried to bundle firebase SDK with my lambda function I faced a lot of errors and missed requirements.., so decided to move the big external dependencies with new package.json file inside “lambda” folder with the help of generate-json-webpack-plugin webpack plugin. and I edited the package.json build script , so after the “yarn build: server” I change the directory by the command “cd dist/server/” and then “yarn install”, this will create a new node_modules into the lambda folder and netlify will zip it using zip-it-and-ship-it to AWS lambda function.

Check This Medium Post

@stale
Copy link

stale bot commented Aug 26, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@shindeshubhamm
Copy link

Hey do you still have the issue? I found a reasonable solution:

add this webpack configuration to your project

//./config/webpack.functions.js
const nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()],
};

and update your scripts inside your package.json to take a custom webpack config:

"scripts": {
    "build:lambda": "netlify-lambda build --config ./config/webpack.functions.js src/lambda",
    "start:lambda": "netlify-lambda serve --config ./config/webpack.functions.js src/lambda"
  },

Only working solution for this problem. Thanks @juanojeda

@MKrupauskas
Copy link

Getting this same annoying issue on multiple nextjs websites. It makes the API routes unusable. Is there a solutions for websites using nextjs API routes and the firebase SDK? Modifying the webpack config is not possible as far as I know. Could you point me in the direction of solving this same issue just with nextjs? Thanks 🚀

@erezrokah
Copy link
Contributor

erezrokah commented Jul 12, 2021

Modifying the webpack config is not possible as far as I know.

Do you mean modifying the Next.js webpack config or the Next.js one?
We have a dedicated Firebase section under https://github.com/netlify/netlify-lambda#webpack-configuration.

Also you might want to use the Essential Next.js plugin https://github.com/netlify/netlify-plugin-nextjs, or if you're already using it open an issue on that repo (if they don't already have one).

@MKrupauskas
Copy link

The nextjs plugin is used by default as far as I know. I have tried modifying the next.js webpack config through the next.config.js file but applying the changes that were recommended additionally broke css imports. Is this something the netlify-plugin-nextjs team should handle?

@erezrokah
Copy link
Contributor

Are you using netlify-lambda to transform your functions?
If so, can you please share a repo with your setup? I'm still not sure if this is a netlify-lambda issue, a Next.js issue or a plugin issue.

@MKrupauskas
Copy link

Not using netlify-lambda just created an API route pages/api/form.js. I think the plugin then handles transforming that API route into a netlify lambda function and that's where it errors out.

@erezrokah
Copy link
Contributor

Thanks @MKrupauskas that makes sense. I suggest opening an issue in https://github.com/netlify/netlify-plugin-nextjs.

This issue thread is is for a tool called Netlify Lambda that lets you add a build step to your functions. See:
https://github.com/netlify/netlify-lambda#netlify-lambda

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests