Skip to content

feat: Support dockerPrivateKey to specify path to SSH key #674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 8, 2022
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,20 @@ custom:
```

The `dockerSsh` option will mount your `$HOME/.ssh/id_rsa` and `$HOME/.ssh/known_hosts` as a
volume in the docker container. If your SSH key is password protected, you can use `ssh-agent`
because `$SSH_AUTH_SOCK` is also mounted & the env var set.
volume in the docker container.

In case you want to use a different key, you can specify the path (absolute) to it through `dockerPrivateKey` option:

```yaml
custom:
pythonRequirements:
dockerizePip: true
dockerSsh: true
dockerPrivateKey: /home/.ssh/id_ed25519
```

If your SSH key is password protected, you can use `ssh-agent`
because `$SSH_AUTH_SOCK` is also mounted & the env var is set.
It is important that the host of your private repositories has already been added in your
`$HOME/.ssh/known_hosts` file, as the install process will fail otherwise due to host authenticity
failure.
Expand Down Expand Up @@ -213,7 +225,7 @@ the names in `slimPatterns`

#### Option not to strip binaries

In some cases, stripping binaries leads to problems like "ELF load command address/offset not properly aligned", even when done in the Docker environment. You can still slim down the package without `*.so` files with
In some cases, stripping binaries leads to problems like "ELF load command address/offset not properly aligned", even when done in the Docker environment. You can still slim down the package without `*.so` files with:

```yaml
custom:
Expand Down Expand Up @@ -566,3 +578,4 @@ package:
- [@jacksgt](https://github.com/jacksgt) - Fixing pip issues
- [@lephuongbg](https://github.com/lephuongbg) - Fixing single function deployment
- [@rileypriddle](https://github.com/rileypriddle) - Introducing schema validation for `module` property
- [@martinezpl](https://github.com/martinezpl) - Fixing test issues, adding `dockerPrivateKey` option
7 changes: 5 additions & 2 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ const { installAllRequirements } = require('./lib/pip');
const { pipfileToRequirements } = require('./lib/pipenv');
const { pyprojectTomlToRequirements } = require('./lib/poetry');
const { cleanup, cleanupCache } = require('./lib/clean');

BbPromise.promisifyAll(fse);

/**
Expand Down Expand Up @@ -45,6 +44,7 @@ class ServerlessPythonRequirements {
: this.serverless.service.provider.runtime || 'python',
dockerizePip: false,
dockerSsh: false,
dockerPrivateKey: null,
dockerImage: null,
dockerFile: null,
dockerEnv: false,
Expand All @@ -71,7 +71,10 @@ class ServerlessPythonRequirements {
}
if (
!options.dockerizePip &&
(options.dockerSsh || options.dockerImage || options.dockerFile)
(options.dockerSsh ||
options.dockerImage ||
options.dockerFile ||
options.dockerPrivateKey)
) {
if (!this.warningLogged) {
if (this.log) {
Expand Down
8 changes: 6 additions & 2 deletions lib/pip.js
Original file line number Diff line number Diff line change
Expand Up @@ -275,12 +275,16 @@ async function installRequirements(targetFolder, pluginInstance) {

dockerCmd.push('docker', 'run', '--rm', '-v', `${bindPath}:/var/task:z`);
if (options.dockerSsh) {
const homePath = require('os').homedir();
const sshKeyPath =
options.dockerPrivateKey || `${homePath}/.ssh/id_rsa`;

// Mount necessary ssh files to work with private repos
dockerCmd.push(
'-v',
`${process.env.HOME}/.ssh/id_rsa:/root/.ssh/id_rsa:z`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check if that change to os.homedir() could be causing these issues with permissions on runner? AFAIK it still takes the result from process.env.HOME but nothing else in this PR looks like the cause of this problem and all other PRs are passing without problems 😕

`${sshKeyPath}:/root/.ssh/${sshKeyPath.split('/').splice(-1)[0]}:z`,
'-v',
`${process.env.HOME}/.ssh/known_hosts:/root/.ssh/known_hosts:z`,
`${homePath}/.ssh/known_hosts:/root/.ssh/known_hosts:z`,
'-v',
`${process.env.SSH_AUTH_SOCK}:/tmp/ssh_sock:z`,
'-e',
Expand Down
24 changes: 24 additions & 0 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,30 @@ const canUseDocker = () => {
// Skip if running on these platforms.
const brokenOn = (...platforms) => platforms.indexOf(process.platform) != -1;

test(
'can package private repo with dockerPrivateKey option',
async (t) => {
process.chdir('tests/base');
const path = npm(['pack', '../..']);
npm(['i', path]);
sls(['package'], {
env: {
dockerizePip: true,
dockerSsh: true,
dockerPrivateKey: `${__dirname}${sep}tests${sep}base${sep}id_ed25519`,
fileName: 'requirements-w-git-ssh.txt',
},
});
const zipfiles = await listZipFiles('.serverless/sls-py-req-test.zip');
t.true(
zipfiles.includes(`dockerPrivateKey_test/__init__.py`),
'private repo accessed with specified key'
);
t.end();
},
{ skip: !canUseDocker() || brokenOn('win32') }
);

test(
'default pythonBin can package flask with default options',
async (t) => {
Expand Down
7 changes: 7 additions & 0 deletions tests/base/id_ed25519
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
QyNTUxOQAAACAns3vFG1eJtPnqVRylDQsyu4QGpkvqlH/mj8N6eYIqXAAAAKDM80pnzPNK
ZwAAAAtzc2gtZWQyNTUxOQAAACAns3vFG1eJtPnqVRylDQsyu4QGpkvqlH/mj8N6eYIqXA
AAAECYfiON2DfcWuu1uDsUs5INIh1adTbdwSmWa1khvvAawyeze8UbV4m0+epVHKUNCzK7
hAamS+qUf+aPw3p5gipcAAAAG2Zpc2hlcm1hbm5ATWFyY2lucy1BaXIuaG9tZQEC
-----END OPENSSH PRIVATE KEY-----
2 changes: 2 additions & 0 deletions tests/base/requirements-w-git-ssh.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
flask
git+ssh://[email protected]/martinezpl/[email protected]#egg=dockerPrivateKey_test
4 changes: 4 additions & 0 deletions tests/base/serverless.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ custom:
pythonRequirements:
zip: ${env:zip, self:custom.defaults.zip}
dockerizePip: ${env:dockerizePip, self:custom.defaults.dockerizePip}
dockerSsh: ${env:dockerSsh, self:custom.defaults.dockerSsh}
dockerPrivateKey: ${env:dockerPrivateKey, self:custom.defaults.dockerPrivateKey}
slim: ${env:slim, self:custom.defaults.slim}
slimPatterns: ${file(./slimPatterns.yml):slimPatterns, self:custom.defaults.slimPatterns}
slimPatternsAppendDefaults: ${env:slimPatternsAppendDefaults, self:custom.defaults.slimPatternsAppendDefaults}
Expand All @@ -24,6 +26,8 @@ custom:
slimPatternsAppendDefaults: true
zip: false
dockerizePip: false
dockerSsh: false
dockerPrivateKey: ''
individually: false
useStaticCache: true
useDownloadCache: true
Expand Down