-
Notifications
You must be signed in to change notification settings - Fork 293
AWS Lambda runs out of space with repeated invokations with zip=True
#83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
dschep
pushed a commit
that referenced
this issue
Jul 17, 2018
Updates the requirements on [fs-extra](https://github.com/jprichardson/node-fs-extra) to permit the latest version. <details> <summary>Changelog</summary> *Sourced from [fs-extra's changelog](https://github.com/jprichardson/node-fs-extra/blob/master/CHANGELOG.md).* > 7.0.0 / 2018-07-16 > ------------------ > > - **BREAKING:** Refine `copy*()` handling of symlinks to properly detect symlinks that point to the same file. ([#582](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/582)) > - Fix bug with copying write-protected directories ([#600](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/600)) > - Universalify `fs.lchmod()` ([#596](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/596)) > - Add `engines` field to `package.json` ([#580](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/580)) > > 6.0.1 / 2018-05-09 > ------------------ > > - Fix `fs.promises` `ExperimentalWarning` on Node v10.1.0 ([#578](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/578)) > > 6.0.0 / 2018-05-01 > ------------------ > > - Drop support for Node.js versions 4, 5, & 7 ([#564](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/564)) > - Rewrite `move` to use `fs.rename` where possible ([#549](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/549)) > - Don't convert relative paths to absolute paths for `filter` ([#554](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/554)) > - `copy*`'s behavior when `preserveTimestamps` is `false` has been OS-dependent since 5.0.0, but that's now explicitly noted in the docs ([#563](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/563)) > - Fix subdirectory detection for `copy*` & `move*` ([#541](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/541)) > - Handle case-insensitive paths correctly in `copy*` ([#568](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/568)) > > 5.0.0 / 2017-12-11 > ------------------ > > Significant refactor of `copy()` & `copySync()`, including breaking changes. No changes to other functions in this release. > > Huge thanks to **[[**manidlou**](https://github.com/manidlou)](https://github.com/manidlou)** for doing most of the work on this release. > > - The `filter` option can no longer be a RegExp (must be a function). This was deprecated since fs-extra v1.0.0. [#512](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/512) > - `copy()`'s `filter` option can now be a function that returns a Promise. [#518](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/518) > - `copy()` & `copySync()` now use `fs.copyFile()`/`fs.copyFileSync()` in environments that support it (currently Node 8.5.0+). Older Node versions still get the old implementation. [#505](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/505) > - Don't allow copying a directory into itself. [#83](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/issues/83) > - Handle copying between identical files. [#198](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/issues/198) > - Error out when copying an empty folder to a path that already exists. [#464](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/issues/464) > - Don't create `dest`'s parent if the `filter` function aborts the `copy()` operation. [#517](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/517) > - Fix `writeStream` not being closed if there was an error in `copy()`. [#516](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/516) > > 4.0.3 / 2017-12-05 > ------------------ > > - Fix wrong `chmod` values in `fs.remove()` [#501](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/501) > - Fix `TypeError` on systems that don't have some `fs` operations like `lchown` [#520](https://github-redirect.dependabot.com/jprichardson/node-fs-extra/pull/520) > > 4.0.2 / 2017-09-12 > ------------------ > > - Added `EOL` option to `writeJson*` & `outputJson*` (via upgrade to jsonfile v4) > - Added promise support to [`fs.copyFile()`](https://nodejs.org/api/fs.html#fs_fs_copyfile_src_dest_flags_callback) in Node 8.5+ ></table> ... (truncated) </details> <details> <summary>Commits</summary> - See full diff in [compare view](https://github.com/jprichardson/node-fs-extra/commits/7.0.0) </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com): - Update frequency (including time of day and day of week) - Automerge options (never/patch/minor, and dev/runtime dependencies) - Pull request limits (per update run and/or open at any time) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired) Finally, you can contact us by mentioning @dependabot. </details>
Is it possible that this could still be happening under certain circumstances? I've got scapy as a dependency, so I'm using the following serverless config:
and the corresponding
And though my serverless function deploys successfully, it dies early during runtime, with the very undescriptive logs
Note that this is on the first invocation, so I suppose this is probably a new issue. |
Yes, see #243. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This seems to be because the
zip
option extracts to a random tempfile, and when the lambda is kept warm, that quickly eats through the 512MB of available tempspace. Need to extract to a fixed path instead.The text was updated successfully, but these errors were encountered: