Serve custom 404 pages from Django

[humitos]

Currently, we are serving the 404.html from NGINX returning a Response that includes the X-Accel-Redirect header:

https://github.com/rtfd/readthedocs.org/blob/537f7b8d2e8d81f8243d1d4a562968e472bc6337/readthedocs/core/views/__init__.py#L208-L210

We found that even marking the response.status_code = 404, the response is 200.

$ curl -IL https://docs.readthedocs.io/notfound
HTTP/1.1 200 OK
Content-Length: 14167
Content-Type: text/html
Last-Modified: Mon, 22 Apr 2019 15:30:57 GMT
Accept-Ranges: bytes
ETag: "5cbdde31-3757"
Vary: Accept-Encoding
Server: nginx
X-Served: Nginx-Sendfile
X-Deity: web01
Strict-Transport-Security: max-age=1800; includeSubDomains
Date: Mon, 29 Apr 2019 07:59:21 GMT

To fix this, we have to serve this file directly from Django instead of using this header.

[saadmk11]

Working on it!

[saadmk11]

I looked into it. I'm a bit confused. can you tell me why this test passes:
self.assertEqual(response.status_code, 404)
https://github.com/rtfd/readthedocs.org/blob/537f7b8d2e8d81f8243d1d4a562968e472bc6337/readthedocs/rtd_tests/tests/test_doc_serving.py#L206-L224

and do we need to use the serve() or render() instead of _serve_file() method from django to serve this?

[humitos]

The test passes because Django is returning a 404 properly. Although, in production, that response is passed to NGINX who looks for the X-Accel-Redirect header and serve that file directly from disk. At this point, because NGINX found that file and was able to serve it properly, it adds the 200 in the response.

As we don't use NGINX at all for our tests, it succees.

and do we need to use the serve() or render() instead of _serve_file() method from django to serve this?

I suppose we need to use serve since it's a regular file, not a template.