Skip to content

"Per-function requirements" modules can not contains more than one function #209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
amitm02 opened this issue Jun 19, 2018 · 6 comments
Closed
Labels

Comments

@amitm02
Copy link

amitm02 commented Jun 19, 2018

If i set in serverless.yml more than one function with the same module (e.g they both need the same requirements.txt),

e.g

package:
  individually: true

functions:
  func_a_1:
    handler: index.func1
    module: module_a
  func_a_2:
    handler: index.func2
    module: module_a
  func_b_1:
    handler: index.func1
    module: module_b

fun_a_2 is not treated (e.g coping vendor directory) because statement in the code:
!doneModules.includes(f.module) is equal to false after dealing with f_a_1

@dschep dschep added the bug label Jun 19, 2018
@dschep
Copy link
Contributor

dschep commented Jun 19, 2018

cc @cgrimal

@cgrimal
Copy link

cgrimal commented Jun 21, 2018

Hi guys!

I created a minimal example following your serverless.yml extract: 2 modules, with 2 functions using the same module (this was my original use case by the way) and a vendor folder. It created the artefacts as expected.
It is actually on purpose than the statement you noticed is false, as the vendors has already been copied in the requirements directory. Your functions func_a_1 and func_a_2 have the exact same artefacts sent to AWS, only the handler configuration will differ.

Since I can't reproduce, could you provide a minimal example demonstrating the bug?

Many thanks,
Clément

@bweigel
Copy link
Contributor

bweigel commented Nov 14, 2018

here is a minimal working example: https://github.com/bweigel/serverless_playground/tree/209_bug/sls_package

serverless.yml:

service: packaging_test

plugins:
  - serverless-python-requirements

provider:
  name: aws
  runtime: python3.6
  region: eu-central-1

package:
  individually: true
  exclude:
    - '**/*'

functions:
  function-one:
    handler: main.handler
    module: ./fn
    package:
      include:
        - 'fn/**'
  function-one-and-a-half:
    handler: main.handler2
    module: fn
    package:
      include:
        - 'fn/**'
  function-two:
    handler: main.handler
    module: fn2
    package:
      include:
        - 'fn2/**'
  • module fnrequirements.txt contains just dataclasses
  • module fn2requirements.txt contains just urllib3

However, the contents of the artifacts are vastly different and not as expected:

sls_209

I will try and fix this in #279

@bweigel
Copy link
Contributor

bweigel commented Nov 14, 2018

Looks like a caching bug that is caused by the cache-process (or rather if not using caching) using the incorrect workingReqsFolder.
See 0113376 for fix

bweigel pushed a commit to bweigel/serverless-python-requirements that referenced this issue Nov 25, 2018
@bweigel
Copy link
Contributor

bweigel commented Jan 4, 2019

upon further inspection, I guess @amitm02 's problem was with the vendor option, which is not heeded, when packaging two functions individually and both have the option enabled, but with different vendor-libraries. Only the first vendor-library will ever be packaged:

serverless.yml
...
package:
  individually: true
  exclude:
    - '**/*'
  include:
    - 'fn/**'

functions:
  function-one:
    handler: main.handler
    module: fn
    vendor: ./vendor
  function-two:
    handler: main.handler
    module: fn
    vendor: ./vendor2
serverless package
$ sls package
Serverless: Generated requirements from /home/bweigel/Projects/OSS/serverless_playground/sls_package/fn/requirements.txt in /home/bweigel/Projects/OSS/serverless_playground/sls_package/.serverless/requirements.txt...
Serverless: Copying from /home/bweigel/Projects/OSS/serverless_playground/sls_package/.serverless/requirements.txt into /home/bweigel/Projects/OSS/serverless_playground/sls_package/.serverless/fn/requirements.txt ...
Serverless: Installing requirements from /home/bweigel/Projects/OSS/serverless_playground/sls_package/.serverless/requirements/requirements.txt ...
Serverless: Copying vendor libraries from ./vendor to /home/bweigel/Projects/OSS/serverless_playground/sls_package/.serverless/requirements...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Excluding development dependencies...
Serverless: Injecting required Python packages to package...

$ zipinfo -1 .serverless/fn-packaging_test-dev-function-one.zip | sort
dataclasses-0.6.dist-info/
dataclasses-0.6.dist-info/INSTALLER
dataclasses-0.6.dist-info/METADATA
dataclasses-0.6.dist-info/RECORD
dataclasses-0.6.dist-info/top_level.txt
dataclasses-0.6.dist-info/WHEEL
dataclasses.py
__init__.py
main.py
other_module.py
__pycache__/
__pycache__/__init__.cpython-36.pyc
__pycache__/other_module.cpython-36.pyc
__pycache__/yam.cpython-36.pyc
requirements.txt
vendor_lib_a.py
yam.py

$ zipinfo -1 .serverless/fn-packaging_test-dev-function-two.zip | sort
dataclasses-0.6.dist-info/
dataclasses-0.6.dist-info/INSTALLER
dataclasses-0.6.dist-info/METADATA
dataclasses-0.6.dist-info/RECORD
dataclasses-0.6.dist-info/top_level.txt
dataclasses-0.6.dist-info/WHEEL
dataclasses.py
__init__.py
main.py
other_module.py
__pycache__/
__pycache__/__init__.cpython-36.pyc
__pycache__/other_module.cpython-36.pyc
__pycache__/yam.cpython-36.pyc
requirements.txt
vendor_lib_a.py
yam.py

bweigel pushed a commit to bweigel/serverless-python-requirements that referenced this issue Jan 4, 2019
@pgrzesik
Copy link
Contributor

Hey 👋 I'm closing this ticket as it looks like it's heavily outdated, we can of course reopen it if needed 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants