[Bug]: 'Error: WebSocket close with status code 1006' behind nginx proxy #6023

zirui · 2023-02-16T08:51:38Z

Is there an existing issue for this?

I have searched the existing issues

OS/Web Information

Web Browser: edge
Local OS: macos
Remote OS: ubuntu
Remote Architecture: x86
code-server --version: 4.6.0

Steps to Reproduce

proxy with nginx:

  location /code/ {
      proxy_pass https://0.0.0.0:9997/;
      proxy_http_version 1.1;
      proxy_set_header Host $host;
      proxy_set_header Upgrade $http_upgrade;
      proxy_read_timeout 86400;
      proxy_set_header Connection $connection_upgrade;
      proxy_set_header Accept-Encoding gzip;
  }

run code-server

code-server  --cert MyCertificate.crt --cert-key MyKey.key --port 9997

visit from url: ${myhost}/code/

Expected

Code server opens up to home screen.

Actual

error log:

Logs

Screenshot/Video

No response

Does this issue happen in VS Code or GitHub Codespaces?

I cannot reproduce this in VS Code.
I cannot reproduce this in GitHub Codespaces.

Are you accessing code-server over HTTPS?

I am using HTTP.

Notes

Why does code-server create a websocket connection with port 80? And how can this issue be solved?

The network environment of the host machine:
The port 80 has been disabled by the administrator of the host machine, only a few ports are open.

The text was updated successfully, but these errors were encountered:

code-asher · 2023-02-16T21:06:23Z

It looks like it is using 80 because you are accessing the website over http. It might be better to let NGINX handle TLS, so remove --cert and --cert-key, change proxy_pass to http, give NGINX the certificate and key, then visit https://${myhost}/code/.

zirui · 2023-02-17T03:53:08Z

Thanks for your replay @code-asher
I'm sorry I made a mistake, the error above occurred when I visited using HTTP not HTTPS.
Then I tried the following based on your suggestion:

give NGINX the certificate and key

  ssl_certificate       MyCertificate.crt;
  ssl_certificate_key   MyKey.key ;

start code-server

code-server --log=debug --port 9997

visit using https:

https://${myhost}/code/

but it still went wrong.
error logs:

code-asher · 2023-02-17T18:22:16Z

Is this a self-signed certificate? Some browsers (like Chrome, not sure about Edge though) will have trouble connecting web sockets with self-signed certificates. If you see something like a gray or green lock then you should be OK but if not you may need to either get a certificate signed by a CA or trust the certificate on your local machine (you can do this manually or install mkcert to do it for you). You might also want to check NGINX's logs and code-server's output just in case it is an error on the server side rather than the browser.

zirui · 2023-02-18T09:05:08Z

Yes, it is a self-signed certificate generated by OpenSSL
I checked the nginx's logs, but didn't find any clues.
error.log:
no related logs
access.log:

10.10.6.9 - - [18/Feb/2023:16:58:06 +0800] "GET /code/stable-2062a59ca1a586d8a6e7bf483841085a94c440a4/static/extensions/git-base/dist/browser/extension.js HTTP/1.0" 404 29 "https://xxx.yy/code/stable-2062a59ca1a586d8a6e7bf483841085a94c440a4/static/out/vs/base/worker/workerMain.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36 Edg/110.0.1587.41"
10.10.6.9 - - [18/Feb/2023:16:58:06 +0800] "GET /code/stable-2062a59ca1a586d8a6e7bf483841085a94c440a4/static/extensions/emmet/dist/browser/emmetBrowserMain.js HTTP/1.0" 404 29 "https://xxx.yy/code/stable-2062a59ca1a586d8a6e7bf483841085a94c440a4/static/out/vs/base/worker/workerMain.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36 Edg/110.0.1587.41"

zirui · 2023-02-20T04:26:51Z

I have tried the following two methods:

create locally signed SSL certificates with mkcert
access through safari browser

but neither of them worked.

Is it because the 443 port of the host machine is not open? Is there an alternative solution? My machine has very strict network restrictions, and only a few ports are open.

the output logs of the Edge:
err.log

code-asher · 2023-02-27T22:28:58Z

Sorry for the delayed response. You will need port 443 open on the machine running NGINX. But you said you access via `https://${myhost}/code/` right? That means you are already accessing through port 443 so it should already be open. Both https:// and wss:// default to port 443. Interesting that there are no error logs from NGINX, that does suggest it might still be a certificate issue. Did you update NGINX to use the certificates generated by mkcert?

zirui · 2023-02-28T05:40:27Z

Sorry for the delayed response. You will need port 443 open on the machine running NGINX. But you said you access via https://${myhost}/code/ right? That means you are already accessing through port 443 so it should already be open. Both https:// and wss:// default to port 443. Interesting that there are no error logs from NGINX, that does suggest it might still be a certificate issue. Did you update NGINX to use the certificates generated by mkcert?

Hi,
After some test on websocket connections, I found the problem:
The security gateway on the host machine does not enable WebSocket when forwarding domain name, only HTTP/HTTPS requests will be forwarded.
Therefore, there are always WebSocket connection errors in the browser front-end:
[initial[xxx:443] socketFactory.connect() failed or timed out"

Due to security concerns, the administrator of the host machine will not allow WebSocket forwarding requests. Are there any alternative solutions?

code-asher · 2023-03-03T21:29:38Z

Ahh that makes sense! There is currently no way to use code-server without web sockets so you would need to figure out some other way to get the web sockets to connect. For example if you are able to use SSH you could access code-server through an SSH tunnel. ``` ssh -N -L 8080:127.0.0.1:8080 [user]@<instance-ip> ``` Then visit `localhost:8080`.

zirui · 2023-03-04T13:49:59Z

Ahh that makes sense! There is currently no way to use code-server without web sockets so you would need to figure out some other way to get the web sockets to connect. For example if you are able to use SSH you could access code-server through an SSH tunnel. ssh -N -L 8080:127.0.0.1:8080 [user]@<instance-ip> Then visit localhost:8080.

Unfortunately, the direct SSH connection to the remote host has also been blocked by the administrator (requiring connection through a jump host and do some authentication), so SSH tunnel won't work.
Anyway, thank you for your reply.

SimonTod · 2023-03-22T18:18:55Z

I can confirm that this issue is present since 4.10.1.
4.10.0 and below work fine.

code-asher · 2023-03-27T17:35:17Z

@SimonTod 4.10.1 added a security check that uses the host and origin headers; do you have code-server behind a reverse proxy and does it forward the host header?

TomBraun02 · 2023-04-13T08:27:59Z

In my installation with Nginx-Proxy the following works with 4.11.0

location / {
proxy_pass http://127.0.0.1:3443/;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header Accept-Encoding gzip;
proxy_set_header Origin https://$host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}

docker run -it --name code-server -p 127.0.0.1:3443:8080 -v "$HOME/.config:/home/coder/.config" -v "$PWD:/home/coder/project" -u "$(id -u):$(id -g)" -e "DOCKER_USER=$USER" codercom/code-server:latest

code-asher · 2023-04-13T17:37:44Z

That makes sense but do keep in mind that if you are always setting the origin to the host then you are bypassing the security check. However the exploit only happens on older browsers that do not support SameSite cookies and attacks across sub-domains that share the same root domain so if neither of these apply to you then there should be no problem.

maxorlovsky · 2023-04-18T17:06:06Z

@TomBraun02 solution worked for me, even though if it's not secure I would like to find a way to not bypass the check

code-asher · 2023-04-21T18:26:04Z

Just released 4.12.0 with some debug logs, running with `--log debug` should show why the requests are being blocked.

code-asher · 2023-04-24T19:33:05Z

Might need to use $http_host instead: #6166

SimonTod · 2023-05-24T17:12:23Z

@code-asher Thanks a lot.

I had this next line missing in my reverse proxy config

proxy_set_header Host $host;

and this next one had to be commented

# proxy_set_header X-Forwarded-Host $remote_addr;

code-asher · 2023-05-30T21:36:42Z

Ahh, that makes sense, remote_addr is the IP address I think.

HeveraletLaidCenx · 2023-06-23T19:03:00Z

I seem to be having a similar problem, at least in terms of the error shown( 1006 and WS 403 from the network tab), but I'm using Caddy v2 as a reverse proxy, my network topology may not be best practice (as shown in the diagram), but I hope it is appropriate to post here.

To debug, I run code-server with the flag --log debug, and I found that when I access from WAN, the log report something like:

code-server 4.13.0 2798322b03e7f446f59c5142215c11711ed7a427
Using user-data-dir ~/.local/share/code-server
Using config file ~/.config/code-server/config.yaml
HTTP server listening on http://127.0.0.1:8080/
    - Authentication is enabled
      - Using password from ~/.config/code-server/config.yaml
    - Not serving HTTPS



Extension host agent started.
redirecting from / to ./login
No uninstalled extensions found.
debug 0 active connections
got cookie domain {"host":"MyPublicDomain.com"}                                              
redirecting from /login to ./
host "192.168.LAN.CaddyHost" does not match origin "MyPublicDomain.com"; blocking request to /stable-b***5?reconnectionToken=a***a&reconnection=false&skipWebSocketFrames=false
Forbidden HttpError: Forbidden
    at ensureOrigin (/usr/lib/code-server/out/node/http.js:295:15)
    at wrapped (/usr/lib/code-server/out/node/wsRouter.js:64:24)
    at Layer.handle [as handle_request] (/usr/lib/code-server/node_modules/router/lib/layer.js:102:15)
    at next (/usr/lib/code-server/node_modules/router/lib/route.js:144:13)
    at Route.dispatch (/usr/lib/code-server/node_modules/router/lib/route.js:109:3)
    at handle (/usr/lib/code-server/node_modules/router/index.js:515:11)
    at Layer.handle [as handle_request] (/usr/lib/code-server/node_modules/router/lib/layer.js:102:15)
    at /usr/lib/code-server/node_modules/router/index.js:291:22
    at param (/usr/lib/code-server/node_modules/router/index.js:368:14)
    at param (/usr/lib/code-server/node_modules/router/index.js:379:14)
    at Function.process_params (/usr/lib/code-server/node_modules/router/index.js:424:3)
    at next (/usr/lib/code-server/node_modules/router/index.js:285:10)
    at Function.handle (/usr/lib/code-server/node_modules/router/index.js:184:3)
    at router (/usr/lib/code-server/node_modules/router/index.js:59:12)
    at Layer.handle [as handle_request] (/usr/lib/code-server/node_modules/router/lib/layer.js:102:15)
    at trim_prefix (/usr/lib/code-server/node_modules/router/index.js:330:13)
    at /usr/lib/code-server/node_modules/router/index.js:294:7
    at Function.process_params (/usr/lib/code-server/node_modules/router/index.js:349:12)
    at Immediate.next (/usr/lib/code-server/node_modules/router/index.js:285:10)
    at Immediate.<anonymous> (/usr/lib/code-server/node_modules/router/index.js:671:15)
    at processImmediate (node:internal/timers:466:21)

... many errors like the last 2 above loop

I check the log of my LAN caddy server, it shows something I think may be the problem:

request>headers>Origin: "https://MyPublicDomain.com"
request>headers>X-Forwarded-Host: "MyPublicDomain.com"
request>host: "192.168.LAN.CaddyHost"

I tried to find a solution in the doc of caddy and code-server's FAQ and issues, From my understanding of the problem, it seems that setting a trust proxy on the backend is the solution, but it looks like that was deprecated in code-server many versions ago. I'm not sure if I should continue to tweak the caddy configuration or if there's any way to fix this, if there is no way at all that such a network topology will work?

code-asher · 2023-06-23T21:48:19Z

I think you will have to set `trusted_proxies` in the LAN Caddy so it trusts the `X-Forwarded-*` headers from the WAN Caddy and passes them along rather than overriding them. https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#defaults

HeveraletLaidCenx · 2023-06-24T10:51:06Z

I think you will have to set trusted_proxies in the LAN Caddy so it trusts the X-Forwarded-* headers from the WAN Caddy and passes them along rather than overriding them. https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#defaults

Thanks for your help! After a few more debugging and testing, it finally worked, that's really saved my day!
Perhaps I should note some additional info here if it can help someone stuck in a similar situation:

Keep the network topology AS SIMPLE AS YOU CAN to avoid hard-to-locate errors. Or, if you have to set a relatively complex cases that are not directly connected to code-server using one layer Caddy's reverse_proxy, just like mine, then, switch on all debug logs available during the whole route to locate the main issue. For me, some lack of network knowledge I guess led me to not accurately understand the default behavior of how caddy deal with requests.

Besides the trusted_proxies asher mentioned above, I found:

after setting it, the Host in my LAN caddy's log changed to the WAN Router's WAN IP:port, which lead to a blank response even did not reach code-server at all.

To fix that, I edit the Caddyfile of my WAN caddy, replace the header_up Host {upstream_hostport} to header_up Host 192.168.LAN.CaddyHost:port.

After reloading, everything worked. Remember to switch off those debug to set a production state! Hope this helped XD.

code-asher · 2023-06-26T17:48:21Z

Whoops, ignore my last message if you saw it. I replied to the wrong email.

Glad you got it working!

Izooc · 2023-11-29T23:31:30Z

In my installation with Nginx-Proxy the following works with 4.11.0

location / {
proxy_pass http://127.0.0.1:3443/;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header Accept-Encoding gzip;
proxy_set_header Origin https://$host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}

docker run -it --name code-server -p 127.0.0.1:3443:8080 -v "$HOME/.config:/home/coder/.config" -v "$PWD:/home/coder/project" -u "$(id -u):$(id -g)" -e "DOCKER_USER=$USER" codercom/code-server:latest

This works for me (yay) but I was wondering if there is a way to do it securely? As someone in this thread mentioned it could pose a security risk. Or am I just being thick... Thanks for this tho 🙏

(edit)
I found the config for nginx on the docs and am using that, should be okay i think??

code-asher · 2023-12-04T18:20:21Z

Yeah the config in the docs is good. You want to avoid setting Origin in the proxy config, to prevent some cases of cross-site web socket hijacking.

code-server/CHANGELOG.md

Lines 195 to 197 in 3e8100b

    
           Added an origin check to web sockets to prevent cross-site hijacking attacks on 
        
           users using older or niche browser that do not support SameSite cookies and 
        
           attacks across sub-domains that share the same root domain.

izayoi-akira · 2023-12-13T09:57:47Z

In my installation with Nginx-Proxy the following works with 4.11.0
location / {
proxy_pass http://127.0.0.1:3443/;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header Accept-Encoding gzip;
proxy_set_header Origin https://$host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
docker run -it --name code-server -p 127.0.0.1:3443:8080 -v "$HOME/.config:/home/coder/.config" -v "$PWD:/home/coder/project" -u "$(id -u):$(id -g)" -e "DOCKER_USER=$USER" codercom/code-server:latest

This works for me (yay) but I was wondering if there is a way to do it securely? As someone in this thread mentioned it could pose a security risk. Or am I just being thick... Thanks for this tho 🙏

(edit) I found the config for nginx on the docs and am using that, should be okay i think??

Is there the docs link?🙏

code-asher · 2023-12-15T19:00:52Z

https://github.com/coder/code-server/blob/3e8100b70eedf1cd5d0c81c5e7cc35a55c0acc18/docs/guide.md#using-lets-encrypt-with-nginx

code-asher · 2024-07-12T23:42:01Z

Closing since I think we ended up not finding anything in code-server that needs to be changed.

zirui added bug Something isn't working triage This issue needs to be triaged by a maintainer labels Feb 16, 2023

code-asher mentioned this issue Apr 24, 2023

Account for hosts with/without ports in origin check #6166

Open

4 tasks

code-asher closed this as not planned Won't fix, can't repro, duplicate, stale Jul 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: 'Error: WebSocket close with status code 1006' behind nginx proxy #6023

[Bug]: 'Error: WebSocket close with status code 1006' behind nginx proxy #6023

zirui commented Feb 16, 2023 •

edited

Loading

code-asher commented Feb 16, 2023 •

edited

Loading

zirui commented Feb 17, 2023

code-asher commented Feb 17, 2023 via email

zirui commented Feb 18, 2023 •

edited

Loading

zirui commented Feb 20, 2023

code-asher commented Feb 27, 2023 via email

zirui commented Feb 28, 2023

code-asher commented Mar 3, 2023 via email

zirui commented Mar 4, 2023

SimonTod commented Mar 22, 2023

code-asher commented Mar 27, 2023

TomBraun02 commented Apr 13, 2023

code-asher commented Apr 13, 2023 via email

maxorlovsky commented Apr 18, 2023

code-asher commented Apr 21, 2023 via email

code-asher commented Apr 24, 2023

SimonTod commented May 24, 2023

code-asher commented May 30, 2023

HeveraletLaidCenx commented Jun 23, 2023

code-asher commented Jun 23, 2023 via email •

edited

Loading

HeveraletLaidCenx commented Jun 24, 2023

code-asher commented Jun 26, 2023

Izooc commented Nov 29, 2023 •

edited

Loading

code-asher commented Dec 4, 2023

izayoi-akira commented Dec 13, 2023

code-asher commented Dec 15, 2023

code-asher commented Jul 12, 2024

[Bug]: 'Error: WebSocket close with status code 1006' behind nginx proxy #6023

[Bug]: 'Error: WebSocket close with status code 1006' behind nginx proxy #6023

Comments

zirui commented Feb 16, 2023 • edited Loading

Is there an existing issue for this?

OS/Web Information

Steps to Reproduce

Expected

Actual

Logs

Screenshot/Video

Does this issue happen in VS Code or GitHub Codespaces?

Are you accessing code-server over HTTPS?

Notes

code-asher commented Feb 16, 2023 • edited Loading

zirui commented Feb 17, 2023

code-asher commented Feb 17, 2023 via email

zirui commented Feb 18, 2023 • edited Loading

zirui commented Feb 20, 2023

code-asher commented Feb 27, 2023 via email

zirui commented Feb 28, 2023

code-asher commented Mar 3, 2023 via email

zirui commented Mar 4, 2023

SimonTod commented Mar 22, 2023

code-asher commented Mar 27, 2023

TomBraun02 commented Apr 13, 2023

code-asher commented Apr 13, 2023 via email

maxorlovsky commented Apr 18, 2023

code-asher commented Apr 21, 2023 via email

code-asher commented Apr 24, 2023

SimonTod commented May 24, 2023

code-asher commented May 30, 2023

HeveraletLaidCenx commented Jun 23, 2023

code-asher commented Jun 23, 2023 via email • edited Loading

HeveraletLaidCenx commented Jun 24, 2023

code-asher commented Jun 26, 2023

Izooc commented Nov 29, 2023 • edited Loading

code-asher commented Dec 4, 2023

izayoi-akira commented Dec 13, 2023

code-asher commented Dec 15, 2023

code-asher commented Jul 12, 2024

zirui commented Feb 16, 2023 •

edited

Loading

code-asher commented Feb 16, 2023 •

edited

Loading

zirui commented Feb 18, 2023 •

edited

Loading

code-asher commented Jun 23, 2023 via email •

edited

Loading

Izooc commented Nov 29, 2023 •

edited

Loading