Skip to content

Functions not being built and packaged? #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jimniels opened this issue Apr 16, 2019 · 25 comments
Closed

Functions not being built and packaged? #142

jimniels opened this issue Apr 16, 2019 · 25 comments
Labels

Comments

@jimniels
Copy link

I've been trying to get a simple set of functions setup and running and it keeps failing and I can't figure out why.

My netlify.toml looks like this:

[build]
  command = "npm run build"
  functions = "dist"

And my npm run build command is just netlify-lambda build src

If I run npm run build locally, everything works just fine.

> npm run build

> [email protected] build /Users/jimnielsen/Sites/jim-nielsen.com/api
> netlify-lambda build src

netlify-lambda: Building functions
Hash: 8faa90c77ae883f3a8c3
Version: webpack 4.30.0
Time: 1117ms
Built at: 04/16/2019 4:04:00 PM
              Asset      Size  Chunks             Chunk Names
           hello.js  1.03 KiB       0  [emitted]  hello
icg-slack-deploy.js  18.5 KiB       1  [emitted]  icg-slack-deploy
Entrypoint hello = hello.js
Entrypoint icg-slack-deploy = icg-slack-deploy.js
[0] external "stream" 42 bytes {1} [built]
[1] external "zlib" 42 bytes {1} [built]
[2] external "url" 42 bytes {1} [built]
[3] external "http" 42 bytes {1} [built]
[4] external "https" 42 bytes {1} [built]
[5] ./hello.js 123 bytes {0} [built]
[6] ./icg-slack-deploy.js 2.19 KiB {1} [built]
[7] ../node_modules/node-fetch/lib/index.mjs 39.6 KiB {1} [built]

But as soon as I push, the build fails:

3:55:55 PM: Executing user command: npm run build
3:55:56 PM: > [email protected] build /opt/build/repo
3:55:56 PM: > netlify-lambda build src
3:55:56 PM: netlify-lambda: Building functions
3:55:58 PM: Hash: a6669134aaaa082f8a2e
3:55:58 PM: Version: webpack 4.30.0
3:55:58 PM: Time: 1634ms
3:55:58 PM: Built at: 04/16/2019 9:55:58 PM
3:55:58 PM:               Asset      Size  Chunks             Chunk Names
3:55:58 PM:            hello.js  1.03 KiB       0  [emitted]  hello
3:55:58 PM: icg-slack-deploy.js  19.5 KiB       1  [emitted]  icg-slack-deploy
3:55:58 PM: Entrypoint hello = hello.js
3:55:58 PM: Entrypoint icg-slack-deploy = icg-slack-deploy.js
3:55:58 PM:  [0] external "stream" 42 bytes {1} [built]
3:55:58 PM:  [1] external "zlib" 42 bytes {1} [built]
3:55:58 PM:  [2] external "url" 42 bytes {1} [built]
3:55:58 PM:  [3] external "http" 42 bytes {1} [built]
3:55:58 PM:  [4] external "https" 42 bytes {1} [built]
3:55:58 PM:  [5] ./hello.js 123 bytes {0} [built]
3:55:58 PM:  [6] ./icg-slack-deploy.js 2.19 KiB {1} [built]
3:55:58 PM:  [7] ../node_modules/dotenv/lib/main.js 2.87 KiB {1} [built]
3:55:58 PM:  [8] external "fs" 42 bytes {1} [built]
3:55:58 PM:  [9] external "path" 42 bytes {1} [built]
3:55:58 PM: [10] ../node_modules/node-fetch/lib/index.mjs 39.6 KiB {1} [built]
3:55:58 PM: Build script success
3:55:58 PM: Starting to prepare functions from '/'
3:55:58 PM: Zipping functions from /opt/build/repo/dist to /tmp/zisi-940663858
3:56:04 PM: Error: Could not find "encoding" module in file: /repo/dist/icg-slack-deploy.js. 
          
Please ensure "encoding" is installed in the project.
3:56:04 PM: Failing build: Failed to prepare functions for deployment
3:56:04 PM: failed during stage 'preparing functions for deployment': exit status 1
3:56:05 PM: Finished processing build request in 23.159125539s
3:56:05 PM: Shutting down logging, 0 messages pending

This is the interesting line to me: "Could not find "encoding" module in file: /repo/dist/icg-slack-deploy.js."

What's interesting is that I am only requireing one dependency in that file:

const fetch = require('node-fetch');

That line of code is making everything fail. If I comment that file out, Netlify builds everything as expected (but then the function doesn't work, because I can't use fetch).

Am I missing something here?

@bcomnes
Copy link
Contributor

bcomnes commented Apr 16, 2019

We recently added support for unbundled function deployment to the buildbot and CLI (via https://github.com/netlify/zip-it-and-ship-it if you are curious). That means, you can deploy js functions with dependencies without bundling them first with a tool like netlify-lambda. Your deployed functions folder can contain function files that require modules in a node_modules in our outside the deployed functions folder. On deployment, we walk the dependency tree and grab all the dependencies the functions need. In general, it should work for bundled and unbundled functions. For some reason, its finding node-fetch requires an encoding module that appears to be consumed as a peer dependency (e.g. node-fetch requires it, but doesn't ship a reference in its package.json).

The solution now is to install encoding in the nearest package.json. We need to look into if this is an issue with node-fetch that needs correcting, or if our dependency algorithm is just missing stuff, but there is an action you can take now.

@bcomnes
Copy link
Contributor

bcomnes commented Apr 16, 2019

I'll open a tracking issue for encoding + node-fetch and reference this.

@swyxio
Copy link
Contributor

swyxio commented Apr 17, 2019

hell is underspecified dependencies.

@jimniels
Copy link
Author

We recently added support for unbundled function deployment to the buildbot and CLI...That means, you can deploy js functions with dependencies without bundling them first with a tool like netlify-lambda

That's actually pretty neat. I haven't messed with functions and netlify-lambda since a 0.x release of some sort, so it's cool to see the new stuff. Plus this model fits what I'm trying to do. My functions are simple and I just need their node_modules deps brought in.

So, in this case, I decided that rather than troubleshoot netlify-lamdba + encoding, I would just switch over to this "new" way of doing things.

Unfortunately, I didn't get very far.

First, I installed the CLI then ran netlify dev functions:create hello

> netlify functions:create hello
? Pick a template js-node-fetch
◈ Creating function hello
◈ Created src/hello/node-fetch.js
◈ Created src/hello/package.json
✔ installed dependencies for hello

Which resulted in this:

Screen Shot 2019-04-16 at 11 36 02 PM

I removed the build command from my netlify.toml file and had it only pointing at my folder of functions:

[build]
  functions = "./src"

As stated in the docs (and as you alluded to):

all the functions created by netlify functions:create require no build step

So It would seem, at this point, that I have the most basic setup you could have for deploying functions: I created a new function with netlify dev functions:create, I point at that folder in my netlify.toml, and it's still not working. It works locally when I run netlify dev, I can go to http://localhost:8888/.netlify/functions/hello and I get the expected response. But when I push to Netlify, the build fails:

11:39:34 PM: Found netlify.toml. Overriding site configuration
11:39:34 PM: Different functions path detected, going to use the one specified in the toml file: './src' versus '' in the site
11:39:34 PM: No build command found, continuing to publishing
11:39:34 PM: Starting to prepare functions from '/'
11:39:34 PM: Zipping functions from /opt/build/repo/src to /tmp/zisi-624832131
11:39:35 PM: Error: Could not find "node-fetch" module in file: /hello/hello.js. 
          
Please ensure "node-fetch" is installed in the project.
11:39:35 PM: Failing build: Failed to prepare functions for deployment
11:39:35 PM: failed during stage 'preparing functions for deployment': exit status 1
11:39:35 PM: Finished processing build request in 5.468181506s
11:39:35 PM: Shutting down logging, 0 messages pending

Am I missing something here?

@swyxio
Copy link
Contributor

swyxio commented Apr 17, 2019

yea you caught us at the early stage where we still havent implemented package installing per function folder i think (@bcomnes knows, i dont work on that part). so basically right now the only way to do it is build locally and then netlify deploy —prod. but ofc we will be making sure those node modules install eventually.

@jimniels
Copy link
Author

Haha ok well, so I stopped using this "new" method and instead went back to the "old" way of doing things, which funnily enough is where I started when I first posted this issue.

So to resolve the first issue I had, I added encoding to my package.json (as recommended) and that finally made the build succeed in netlify when pushed to GitHub.

However, now my simple hello.js function doesn't actually work. After some troubleshooting, I found once again it seems to stem from use of node-fetch. When I don't use node-fetch at all, my functions deploy and they actually work. Example of hello.js:

exports.handler = (event, context, callback) => {
  callback(null, {
    statusCode: 200,
    body: "Hello, World!"
  });
}

But as soon as I try to bring in a fetch call, it fails.

const fetch = require("node-fetch");
exports.handler = (event, context, callback) => {
  callback(null, {
    statusCode: 200,
    body: "Hello, World!"
  });

  fetch(
    "https://icanhazdadjoke.com/",
    { method: "GET", headers: { Accept: "application/json" } }
  )
    .then(res => res.json())
    .then(json => { console.log(json) })
}

That code results in Netlify functions logging this:

8:39:33 AM: hello invoked
8:39:33 AM: TypeError: o is not a function
    at t.handler (/var/task/hello.js:1:18480)
RequestId: 249caac6-9e6b-4c76-86a8-9b45d70d44a5 Process exited before completing request

When I run netlify-lamdba serve locally to test this, I get something similar when I hit http://localhost:9000/.netlify/functions/hello

Lambda server is listening on 9000
Request from ::1: GET /.netlify/functions/hello
Response with status 200 in 10 ms.
TypeError: o is not a function
    at Object.t.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/build/hello.js:1:18480)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/lib/serve.js:133:27
    at Layer.handle [as handle_request] (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:137:13)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at Route.dispatch (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/layer.js:95:5)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:281:22
    at param (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:354:14)
    at param (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:365:14)
    at Function.process_params (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:410:3)

In fact, if I change that my hello.js file while the server is running locally, netlify-lambda seems to choke on something and just dies:

Hash: 80255c95f9b2a80ce027
Version: webpack 4.30.0
Time: 93ms
Built at: 04/17/2019 8:44:35 AM
              Asset      Size  Chunks             Chunk Names
           hello.js  18.1 KiB       0  [emitted]  hello
icg-slack-deploy.js  19.5 KiB       1  [emitted]  icg-slack-deploy
Entrypoint hello = hello.js
Entrypoint icg-slack-deploy = icg-slack-deploy.js
 [0] external "stream" 42 bytes {0} {1} [built]
 [1] external "zlib" 42 bytes {0} {1} [built]
 [2] external "url" 42 bytes {0} {1} [built]
 [3] external "http" 42 bytes {0} {1} [built]
 [4] external "https" 42 bytes {0} {1} [built]
 [5] ../node_modules/node-fetch/lib/index.mjs 39.6 KiB {0} {1} [built]
 [6] ./hello.js 349 bytes {0} [built]
 [7] ./icg-slack-deploy.js 2.19 KiB {1} [built]
 [8] ../node_modules/dotenv/lib/main.js 2.87 KiB {1} [built]
 [9] external "fs" 42 bytes {1} [built]
[10] external "path" 42 bytes {1} [built]
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE: address already in use :::9000
    at Server.setupListenHandle [as _listen2] (net.js:1270:14)
    at listenInCluster (net.js:1318:12)
    at Server.listen (net.js:1405:7)
    at Function.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/application.js:618:24)
    at Object.exports.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/lib/serve.js:179:7)
    at Watching.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/bin/cmd.js:38:26)
    at compiler.hooks.done.callAsync (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:98:9)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Watching._done (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:97:28)
    at compiler.emitRecords.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:72:19)
    at Compiler.emitRecords (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:449:39)
    at compiler.emitAssets.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:53:20)
    at hooks.afterEmit.callAsync.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:435:14)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at asyncLib.forEachLimit.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:432:27)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/neo-async/async.js:2813:7
Emitted 'error' event at:
    at emitErrorNT (net.js:1297:8)
    at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `netlify-lambda serve src`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jimnielsen/.npm/_logs/2019-04-17T14_44_35_788Z-debug.log

netlify dev seems to have a similar issue. If I run netlify dev and then hit http://localhost:8888/.netlify/functions/hello, it gives me a similar o is not a function (but netlify dev doesn't seem to choke when the server is running locally and I change the file, the server stays up and running with my updated file changes, but it still chokes on the fetch call)

Connected!

   ┌─────────────────────────────────────────────────┐
   │                                                 │
   │   ◈ Server now ready on http://localhost:8888   │
   │                                                 │
   └─────────────────────────────────────────────────┘

Request from ::ffff:127.0.0.1: GET /.netlify/functions/hello
Response with status 200 in 8 ms.
TypeError: o is not a function
    at Object.t.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/build/hello.js:1:18480)
    at /usr/local/lib/node_modules/netlify-cli/node_modules/netlify-dev-plugin/src/utils/serveFunctions.js:158:29
    at Layer.handle [as handle_request] (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/layer.js:95:5)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:137:13)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)

@sergio-toro
Copy link

sergio-toro commented Apr 20, 2019

In fact, if I change that my hello.js file while the server is running locally, netlify-lambda seems to choke on something and just dies:

Hash: 80255c95f9b2a80ce027
Version: webpack 4.30.0
Time: 93ms
Built at: 04/17/2019 8:44:35 AM
              Asset      Size  Chunks             Chunk Names
           hello.js  18.1 KiB       0  [emitted]  hello
icg-slack-deploy.js  19.5 KiB       1  [emitted]  icg-slack-deploy
Entrypoint hello = hello.js
Entrypoint icg-slack-deploy = icg-slack-deploy.js
 [0] external "stream" 42 bytes {0} {1} [built]
 [1] external "zlib" 42 bytes {0} {1} [built]
 [2] external "url" 42 bytes {0} {1} [built]
 [3] external "http" 42 bytes {0} {1} [built]
 [4] external "https" 42 bytes {0} {1} [built]
 [5] ../node_modules/node-fetch/lib/index.mjs 39.6 KiB {0} {1} [built]
 [6] ./hello.js 349 bytes {0} [built]
 [7] ./icg-slack-deploy.js 2.19 KiB {1} [built]
 [8] ../node_modules/dotenv/lib/main.js 2.87 KiB {1} [built]
 [9] external "fs" 42 bytes {1} [built]
[10] external "path" 42 bytes {1} [built]
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE: address already in use :::9000
    at Server.setupListenHandle [as _listen2] (net.js:1270:14)
    at listenInCluster (net.js:1318:12)
    at Server.listen (net.js:1405:7)
    at Function.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/application.js:618:24)
    at Object.exports.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/lib/serve.js:179:7)
    at Watching.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/bin/cmd.js:38:26)
    at compiler.hooks.done.callAsync (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:98:9)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Watching._done (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:97:28)
    at compiler.emitRecords.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:72:19)
    at Compiler.emitRecords (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:449:39)
    at compiler.emitAssets.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:53:20)
    at hooks.afterEmit.callAsync.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:435:14)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at asyncLib.forEachLimit.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:432:27)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/neo-async/async.js:2813:7
Emitted 'error' event at:
    at emitErrorNT (net.js:1297:8)
    at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `netlify-lambda serve src`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jimnielsen/.npm/_logs/2019-04-17T14_44_35_788Z-debug.log

I'm experiencing the same EADDRINUSE error. Where you able to fix it?

@jimniels
Copy link
Author

No I haven’t been able to get this working. I basically gave up. Figured I’d try again later, or when somebody has some insight into the issue.

@suits-at
Copy link

We recently added support for unbundled function deployment to the buildbot and CLI (via https://github.com/netlify/zip-it-and-ship-it if you are curious). That means, you can deploy js functions with dependencies without bundling them first with a tool like netlify-lambda. Your deployed functions folder can contain function files that require modules in a node_modules in our outside the deployed functions folder. On deployment, we walk the dependency tree and grab all the dependencies the functions need. In general, it should work for bundled and unbundled functions. For some reason, its finding node-fetch requires an encoding module that appears to be consumed as a peer dependency (e.g. node-fetch requires it, but doesn't ship a reference in its package.json).

The solution now is to install encoding in the nearest package.json. We need to look into if this is an issue with node-fetch that needs correcting, or if our dependency algorithm is just missing stuff, but there is an action you can take now.

I just had the same problem after using node-fetch inside a function, but for me installing encoding solved the problem!

@swyxio
Copy link
Contributor

swyxio commented Apr 23, 2019

thanks everyone for the feedback and sorry about these early troubles. i’m going to recommend people use axios for the time being. but also would love to fix the underlying problem

@bcomnes
Copy link
Contributor

bcomnes commented Apr 23, 2019

(This API requires an optional dependency on npm package encoding, which you need to install manually. webpack users may see a warning message due to this optional dependency.) -- https://github.com/bitinn/node-fetch/blob/master/package.json

Just to get the info out there, node-fetch uses an undeclared peer dependency in some cases. Our bundling tree shaker sometimes fails the build when it can't find this dependency. Failing the build due to missing dependencies is generally the desired behavior (vs a successful deploy that will fail during runtime due to missing deps), however in some cases, when modules do weird things, it's sometimes not what you want. We are currently investigating available options to handle this edge case in a cleaner manner. In the meantime, just install the encoding peer dependency. Track here: netlify/zip-it-and-ship-it#30

@tomas-padrieza
Copy link

This problem gets really nasty. I'm using googleapis, that has a nested dependency on node-fetch (from google-auth-library>gtoken>gaxios). Adding encoding to my own package.json doesn't affect the build as netlify-lambda build is not including all of the dependancies i guess. This was not a problem before.

@jdhenckel
Copy link

jdhenckel commented May 3, 2019

Argh! I wasted a couple hours reverting and rewriting various parts of my code... only then did I stumble upon this thread and discover that it was not my fault. Please try to be a little more careful when you make breaking changes! How am I supposed to know that I all I need to do is add "encoding" to my package.json? I've never even heard of that package before.

@babycourageous
Copy link

babycourageous commented May 6, 2019

Hi there - I'm running into this with all other libraries (other than node-fetch) and have searched for a solution/answer but coming up blank...
When using the approach of having function dependencies in the folder
i.e.
image

This structure works fine locally with local dev (netlify dev) as well as live with a command line deploy (netlify deploy --prod) but errors out on live with a "cannot find module" as described in above comments when using git push to trigger the deploy. In my case it's a "cannot find axios..." fail.

My question - are we required to upload the corresponding node_modules folder within that function folder for github push deploys to work?

Thanks!

@bcomnes
Copy link
Contributor

bcomnes commented May 7, 2019

I opened a PR to add a single exception for node-fetch oddness: netlify/zip-it-and-ship-it#35 netlify/zip-it-and-ship-it#34

@babycourageous
Copy link

@bcomnes - Nice - although I'm experiencing it with any of my "folder-style" functions (none of mine have node-fetch). I'm having to commit the function folder's node_modules folder with the git push in order for the function not to fail.

@bcomnes
Copy link
Contributor

bcomnes commented May 8, 2019

The buildbot only automatically installs top level node_modules currently. If you have nested package.json files inside of your functions folder, then you need to add something to your build command that cds into your functions folder and populates your node modules with npm i or yarn or something. Or you can commit node_modules to git (less convenient) but this will bloat your repo size.

Additionally, you can also have a single top level package.json and node_modules shared between your site build and functions, and unbundled functions without their own package.json will use whatever is in there. This is probably the smoothest workflow for unbundled functions at the moment.

In the PR mentioned above, we are experimenting with automatic package.json detection and npm i/yarning in functions that have their own nested package.json, but it isn't available to use yet.

@babycourageous
Copy link

Thank you @bcomnes! I figured this was the case (having to add a build step) but wasn't 100%. I wound up going with the classic netlify-lambda build step that I used to use. Only key difference I am noticing is that when serving functions with netlify-dev, the received POST request is base64 encoded and using netlify-lambda to serve does not. Not a big deal, just something I noticed.

Thanks again for the detailed update!

@bcomnes
Copy link
Contributor

bcomnes commented May 9, 2019

Sorry I posted the wrong PR: netlify/zip-it-and-ship-it#34 is the correct one that I intended to refer to.

@cinnamonKale
Copy link

I was able to fix this temporarily by changing

const fetch = require('node-fetch');

to

const fetch = require('node-fetch').default;

@stale
Copy link

stale bot commented Jul 20, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 20, 2019
@jimniels
Copy link
Author

I never did see this fixed.

My use case was pretty simple, so I ended up just removing my dependence on node-fetch and opted to wrangle the https lib that's part of node instead. So, in the end, it was a win for me because now I have 0 deps in this particular case.

But it's unfortunate to see so many people had the same issue. Hopefully some of the workarounds listed in here help.

@johnedvard
Copy link

looked like node-fetch was the issue for me as well, so I switched to https. Now it builds on Netlify

@kapilgorve
Copy link

kapilgorve commented Mar 16, 2020

I was using node-fetch in netlify-lambda . I had to do
import fetch from 'node-fetch' . If I used require, I would get undefined error. I also had to install encoding package.

johnnyoshika added a commit to johnnyoshika/react-apollo-lambda-netlify that referenced this issue Apr 28, 2020
@samxtu
Copy link

samxtu commented Dec 2, 2022

Haha ok well, so I stopped using this "new" method and instead went back to the "old" way of doing things, which funnily enough is where I started when I first posted this issue.

So to resolve the first issue I had, I added encoding to my package.json (as recommended) and that finally made the build succeed in netlify when pushed to GitHub.

However, now my simple hello.js function doesn't actually work. After some troubleshooting, I found once again it seems to stem from use of node-fetch. When I don't use node-fetch at all, my functions deploy and they actually work. Example of hello.js:

exports.handler = (event, context, callback) => {
  callback(null, {
    statusCode: 200,
    body: "Hello, World!"
  });
}

But as soon as I try to bring in a fetch call, it fails.

const fetch = require("node-fetch");
exports.handler = (event, context, callback) => {
  callback(null, {
    statusCode: 200,
    body: "Hello, World!"
  });

  fetch(
    "https://icanhazdadjoke.com/",
    { method: "GET", headers: { Accept: "application/json" } }
  )
    .then(res => res.json())
    .then(json => { console.log(json) })
}

That code results in Netlify functions logging this:

8:39:33 AM: hello invoked
8:39:33 AM: TypeError: o is not a function
    at t.handler (/var/task/hello.js:1:18480)
RequestId: 249caac6-9e6b-4c76-86a8-9b45d70d44a5 Process exited before completing request

When I run netlify-lamdba serve locally to test this, I get something similar when I hit http://localhost:9000/.netlify/functions/hello

Lambda server is listening on 9000
Request from ::1: GET /.netlify/functions/hello
Response with status 200 in 10 ms.
TypeError: o is not a function
    at Object.t.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/build/hello.js:1:18480)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/lib/serve.js:133:27
    at Layer.handle [as handle_request] (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:137:13)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at next (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:131:14)
    at Route.dispatch (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/layer.js:95:5)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:281:22
    at param (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:354:14)
    at param (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:365:14)
    at Function.process_params (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/router/index.js:410:3)

In fact, if I change that my hello.js file while the server is running locally, netlify-lambda seems to choke on something and just dies:

Hash: 80255c95f9b2a80ce027
Version: webpack 4.30.0
Time: 93ms
Built at: 04/17/2019 8:44:35 AM
              Asset      Size  Chunks             Chunk Names
           hello.js  18.1 KiB       0  [emitted]  hello
icg-slack-deploy.js  19.5 KiB       1  [emitted]  icg-slack-deploy
Entrypoint hello = hello.js
Entrypoint icg-slack-deploy = icg-slack-deploy.js
 [0] external "stream" 42 bytes {0} {1} [built]
 [1] external "zlib" 42 bytes {0} {1} [built]
 [2] external "url" 42 bytes {0} {1} [built]
 [3] external "http" 42 bytes {0} {1} [built]
 [4] external "https" 42 bytes {0} {1} [built]
 [5] ../node_modules/node-fetch/lib/index.mjs 39.6 KiB {0} {1} [built]
 [6] ./hello.js 349 bytes {0} [built]
 [7] ./icg-slack-deploy.js 2.19 KiB {1} [built]
 [8] ../node_modules/dotenv/lib/main.js 2.87 KiB {1} [built]
 [9] external "fs" 42 bytes {1} [built]
[10] external "path" 42 bytes {1} [built]
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE: address already in use :::9000
    at Server.setupListenHandle [as _listen2] (net.js:1270:14)
    at listenInCluster (net.js:1318:12)
    at Server.listen (net.js:1405:7)
    at Function.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/express/lib/application.js:618:24)
    at Object.exports.listen (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/lib/serve.js:179:7)
    at Watching.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/netlify-lambda/bin/cmd.js:38:26)
    at compiler.hooks.done.callAsync (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:98:9)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Watching._done (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:97:28)
    at compiler.emitRecords.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:72:19)
    at Compiler.emitRecords (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:449:39)
    at compiler.emitAssets.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Watching.js:53:20)
    at hooks.afterEmit.callAsync.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:435:14)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at asyncLib.forEachLimit.err (/Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/webpack/lib/Compiler.js:432:27)
    at /Users/jimnielsen/Sites/jim-nielsen.com/api/node_modules/neo-async/async.js:2813:7
Emitted 'error' event at:
    at emitErrorNT (net.js:1297:8)
    at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `netlify-lambda serve src`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jimnielsen/.npm/_logs/2019-04-17T14_44_35_788Z-debug.log

netlify dev seems to have a similar issue. If I run netlify dev and then hit http://localhost:8888/.netlify/functions/hello, it gives me a similar o is not a function (but netlify dev doesn't seem to choke when the server is running locally and I change the file, the server stays up and running with my updated file changes, but it still chokes on the fetch call)

Connected!

   ┌─────────────────────────────────────────────────┐
   │                                                 │
   │   ◈ Server now ready on http://localhost:8888   │
   │                                                 │
   └─────────────────────────────────────────────────┘

Request from ::ffff:127.0.0.1: GET /.netlify/functions/hello
Response with status 200 in 8 ms.
TypeError: o is not a function
    at Object.t.handler (/Users/jimnielsen/Sites/jim-nielsen.com/api/build/hello.js:1:18480)
    at /usr/local/lib/node_modules/netlify-cli/node_modules/netlify-dev-plugin/src/utils/serveFunctions.js:158:29
    at Layer.handle [as handle_request] (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/layer.js:95:5)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:137:13)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)
    at next (/usr/local/lib/node_modules/netlify-cli/node_modules/express/lib/router/route.js:131:14)

Hello, any solution for that o is not a function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests