Serverless under Windows Subsystem for Linux

[nickweavers]

I'm currently working through the "AWS Lambda and the Serverless Framework - Hands On" course at www.udemy.com and was really impressed that I could install nodejs, npm and serverless with no issues and actually deploy apps into AWS. However, I just got to a section that uses an npm plugin for serverless called serverless-python-requirements (https://github.com/UnitedIncome/serverless-python-requirements) and while this installed okay, the subsequent deployment of the app ($sls deploy -v) failed with an error meaning I probably need to revert to a linux machine to continue. The plugin uses Docker which can't be run directly from WSL, but I have installed the Windows version and set "export DOCKER_HOST=tcp://127.0.0.1:2375" for docker access in my .bashrc
For the WSL bash terminal I can run docker info:

$docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 2
Server Version: 17.09.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.49-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.934GiB
Name: moby
ID: ALSG:GHDW:VGVS:NBZO:OQUP:ERCY:ADOP:CVSD:KTAE:5LZ7:JRJQ:4YEE
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 16
 Goroutines: 26
 System Time: 2017-12-15T22:41:37.2602756Z
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

While the plugin install went fine:

$sls plugin install -n serverless-python-requirements
Serverless: Creating an empty package.json file in your service directory
Serverless: Installing plugin "serverless-python-requirements@latest" (this might take a few seconds...)
Serverless: Successfully installed "serverless-python-requirements@latest"

What I see when i try to deploy with serverless is:

  Stack Trace --------------------------------------------

Error: The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Could not open requirements file: [Errno 2] No such file or directory: '.serverless/requirements.txt'

    at ServerlessPythonRequirements.installRequirements (/mnt/i/ddfs-serverless/python-s3-thumbnail/node_modules/serverless-python-requirements/lib/pip.js:97:11)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.spawn (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:384:17)
    at Deploy.BbPromise.bind.then.then (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:120:50)
From previous event:
    at Object.before:deploy:deploy [as hook] (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:110:10)
    at BbPromise.reduce (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:366:55)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.run (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:397:17)
    at variables.populateService.then (/usr/lib/node_modules/serverless/lib/Serverless.js:104:33)
    at runCallback (timers.js:773:18)
    at tryOnImmediate (timers.js:734:5)
    at processImmediate [as _immediateCallback] (timers.js:711:5)
From previous event:
    at Serverless.run (/usr/lib/node_modules/serverless/lib/Serverless.js:91:74)
    at serverless.init.then (/usr/lib/node_modules/serverless/bin/serverless:42:50)
    at <anonymous>

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     linux
     Node Version:           9.3.0
     Serverless Version:     1.24.1

Not sure why it issues the message Could not open requirements file: [Errno 2] No such file or directory: '.serverless/requirements.txt' since the file was found by the plugin installation and is definitely there and readable:

nickw@DESKTOP-OREBB4V:/i/ddfs-serverless/python-s3-thumbnail/.serverless
$cat requirements.txt
Pillow

Understandably the serverless-python-requirements is expecting a more conventional setup. Shame, though, WSL would (and I'm sure in time will be) be an awesome development environment for serverless apps.

[dschep]

See @heri16 and my conversation on #110, we've got ideas for a fix. Tho it looks like it's working for you and you just need to follow the windows steps in the readme.

[nickweavers]

Hi @dschep. I just checked my Windows Dockers settings and both drives I need are shared (C: and I:). I also checked that I had "Expose daemon on tcp://localhost:2375 without TLS" checked and that's all good too.

I think the fact that I can get a response from docker from my WSL bash window when I do $docker info must mean the WSL Docker Client and the Windows Docker Service are communicating okay and the hookup is good there.

nickw@DESKTOP-OREBB4V:/i/ddfs-serverless/python-s3-thumbnail
$docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
lambci/lambda       build-python2.7     db69ab76a426        3 weeks ago         1.57GB
d4w/nsenter         latest              9e4f13a0901e        14 months ago       83.8kB

I think the message "Error: The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory" is suspicious because under WSL there is no /.cache

nickw@DESKTOP-OREBB4V:/i/ddfs-serverless/python-s3-thumbnail
$ls -la /
total 80
drwxr-xr-x  0 root root   512 Dec 14 23:00 .
drwxr-xr-x  0 root root   512 Dec 14 23:00 ..
drwxr-xr-x  0 root root   512 Nov 16 01:19 bin
drwxr-xr-x  0 root root   512 Sep 22 17:15 boot
lrwxrwxrwx  1 root root     6 Dec 13 10:04 c -> /mnt/c
drwxr-xr-x  0 root root   512 Dec 16 00:12 dev
drwxr-xr-x  0 root root   512 Dec 14 23:00 etc
drwxr-xr-x  0 root root   512 Nov 16 01:11 home
lrwxrwxrwx  1 root root     6 Dec 13 10:04 i -> /mnt/i
-rwxr-xr-x  1 root root 79416 Jan  1  1970 init
drwxr-xr-x  0 root root   512 Dec 12 20:40 lib
drwxr-xr-x  0 root root   512 Sep 22 17:13 lib64
drwxr-xr-x  0 root root   512 Sep 22 17:12 media
drwxr-xr-x  0 root root   512 Nov 30 07:43 mnt
dr-xr-xr-x  9 root root     0 Dec 16 00:12 proc
drwx------  0 root root   512 Sep 22 17:15 root
drwxr-xr-x  0 root root   512 Dec 16 00:12 run
drwxr-xr-x  0 root root   512 Dec 14 22:51 sbin
drwxr-xr-x  0 root root   512 Aug 31 11:17 snap
drwxr-xr-x  0 root root   512 Sep 22 17:12 srv
dr-xr-xr-x 12 root root     0 Dec 16 00:12 sys
drwxrwxrwt  0 root root   512 Dec 16 00:26 tmp
drwxr-xr-x  0 root root   512 Dec 13 14:01 usr
drwxr-xr-x  0 root root   512 Nov 16 01:27 var

Also Could not open requirements file: [Errno 2] No such file or directory: '.serverless/requirements.txt' doesn't look quite right since

$cat .serverless/requirements.txt
Pillow 

Not sure if the fact that my code files are in the windows file system (so I can use a decent editor like Atom on them) and not under "C:\Users\nickw\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs" is confusing things over file owner/permissions.

I did add symlinks for /c->/mnt/c and /i->/mnt/i as I read that from windows docker it can translate /c to c: but not sure that is any help here.

[nickweavers]

Trying a more ambitious check for the wsl docker client working with the windows docker engine:

nickw@DESKTOP-OREBB4V:/i/ddfs-serverless/python-s3-thumbnail
$docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:be0cd392e45be79ffeffa6b05338b98ebb16c87b255f48e297ec7f98e123905c
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

I does look like the docker setup isn't the problem