Packaging packages... setuptools style

[dfee]

I develop my projects as packages, and find it strange that this workflow isn't supported with serverless-python-requirements.

As an example, I might have a project xyz with a directory structure like this:

xyz/
  - serverless.yaml
  - setup.py
  - xyz/
    - __init__.py
    - app.py

And during development, I'll use pip install -e ., or python setup.py develop, or more probably pipenv install "-e .".

However, when processing the artifact requirements.txt that's produced by serverless-python-requirements I run into two problems:

1. the editable feature (-e ...) is not supported during the transformation to an uploadable zip file.
2. the reference to path . is not supported.

I've somewhat worked around this by disabling pipenv in the plugin options for serverless-python-requirements in my serverless.yaml file, and manage a custom requirements.txt file that is really just a pip freeze > requirements.txt and removing the -e . requirement.

What is the expected workflow here?

[dschep]

Experimenting a bit with it, I don't think it's currently possible with pipenv. I created this Pipfile but run into #85.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[dev-packages]
[packages]
xyz = {path = "."}

However, without pipenv, . in a requirements.txt works fine AFAICT.

[dfee]

@dschep yeah, looks like you've retraced my steps!

When you say that . in a requirements.txt "works fine", I assume you're packaging with usePipenv: false, right?

Are you also trying with . or -e . in your requirements.txt file? As described in the setuptools docs I need setuptools develop mode (which is what pip -e provides) or else I'll need to reinstall the project after every change.

[dschep]

Just ., but yeah, makes sense that you'd need -e . for development. And I just didn't have Pipfile when I was trying requirements.txt.

Possible workaround:

you could set usePipenv: false and use pipenv for your local dev, and use a requirements.txt for deploy, manually generated with

pipenv lock -r | sed -e 's/-e//' > requirements.txt

Possible fix:

turn editable requirements into non-editable

It might make sense for the plugin to filter the requirements and remove -e(as in the above workaround) since there's no need for editable packages in actual deployments to lambda.

add option to not use hashes

This would fix #85 allowing the Pipfile I provided above to work (tho.. is that editable or not?). Would be behind a custom.pythonRequirements.pipenvHashes=false option.

[dfee]

As you noticed, I tested the bugfix that #85 references pipenv #1380 and that solution doesn't seem to be totally working yet.

Anyway, what's interesting is that when pipenv produces the locked requirements file (on master), it actually just comments out the "-e ." entirely from the requirements.txt file. I.e.

# requirements
boto3==1.5.32 --hash=sha256:8db0b63e0bdf0b188803a4a793a0779369d7b08e45758d0fe0ac610d124bc019  --hash=sha256:0591da93d0c5e8fda7f032cd3e175e93ab0baf7b5174e07aee92b48baeb5582f
botocore==1.8.46 --hash=sha256:52bc09d2f4439bb89ec9606ad652dedeb5bfe2c4db56f27417f4d2e5e50e4e79  --hash=sha256:d798b9aa32e070f6483a9723fb3296da546a2d9d09f17bb62adc57eb5771aaa6
docutils==0.14 --hash=sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6  --hash=sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6  --hash=sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274
...
# -e .

I'm not sure that that's the right solution, either. But it would be the third alternative "get rid of editable requirement all-together". That's effectively the same work around I came to when I opened this issue, I'm just not clear on all the repercussions of doing this.

[pgrzesik]

Hey 👋 I'm closing this ticket as it looks like it's heavily outdated, we can of course reopen it if needed 👍