Skip to content

Commit 35ec819

Browse files
humitoshumitos
committed
Merge tag '3.11.0' into relcorp
2 parents d068ca7 + 867a9e5 commit 35ec819

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+1844
-890
lines changed

‎.dockerignore

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
common
2+
logs
3+
bower_components
4+
5+
.git
6+
*.db*
7+
*.sql
8+
.tox
9+
.cache
10+
.pytest_cache
11+
celerybeat*
12+
debug.log
13+
14+
**/__pycache__
15+
16+
bower_components
17+
contrib
18+
datasets
19+
docs
20+
docker/azurite
21+
docker/search
22+
media/epub
23+
media/htmlzip
24+
media/json
25+
media/pdf
26+
node_modules
27+
private_cname_project
28+
private_cname_root
29+
private_web_root
30+
public_cname_project
31+
public_cname_root
32+
public_cnames
33+
public_symlinks
34+
public_web_root
35+
static
36+
spam-data
37+
user_builds

‎.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ contrib/supervisord.pid
2222
celerybeat-schedule.*
2323
deploy/.vagrant
2424
dist/*
25+
docker/azurite
26+
docker/search
2527
local_settings.py
2628
locks/**
2729
logs/*

‎CHANGELOG.rst

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,29 @@
1+
Version 3.11.0
2+
--------------
3+
4+
:Date: December 03, 2019
5+
6+
* `@davidfischer <http://github.com/davidfischer>`__: Use media availability instead of querying the filesystem (`#6428 <https://github.com/readthedocs/readthedocs.org/pull/6428>`__)
7+
* `@stsewd <http://github.com/stsewd>`__: Remove beta note about sharing by password and header auth (`#6426 <https://github.com/readthedocs/readthedocs.org/pull/6426>`__)
8+
* `@humitos <http://github.com/humitos>`__: Use trigger_build for update_repos command (`#6422 <https://github.com/readthedocs/readthedocs.org/pull/6422>`__)
9+
* `@humitos <http://github.com/humitos>`__: Add more supported field to APIv3 docs (`#6417 <https://github.com/readthedocs/readthedocs.org/pull/6417>`__)
10+
* `@humitos <http://github.com/humitos>`__: Add AuthenticationMiddleware to El Proxito tests (`#6416 <https://github.com/readthedocs/readthedocs.org/pull/6416>`__)
11+
* `@stsewd <http://github.com/stsewd>`__: Update docs on sharing (`#6410 <https://github.com/readthedocs/readthedocs.org/pull/6410>`__)
12+
* `@humitos <http://github.com/humitos>`__: Use WORKDIR to cd into a directory in Dockerfile (`#6409 <https://github.com/readthedocs/readthedocs.org/pull/6409>`__)
13+
* `@humitos <http://github.com/humitos>`__: Use /data inside Azurite container to persist data (`#6407 <https://github.com/readthedocs/readthedocs.org/pull/6407>`__)
14+
* `@humitos <http://github.com/humitos>`__: Serve non-html files from nginx (X-Accel-Redirect) (`#6404 <https://github.com/readthedocs/readthedocs.org/pull/6404>`__)
15+
* `@humitos <http://github.com/humitos>`__: Perform redirects at DB level (`#6398 <https://github.com/readthedocs/readthedocs.org/pull/6398>`__)
16+
* `@humitos <http://github.com/humitos>`__: Allow to extend El Proxito views from commercial (`#6397 <https://github.com/readthedocs/readthedocs.org/pull/6397>`__)
17+
* `@humitos <http://github.com/humitos>`__: Migrate El Proxito views to class-based views (`#6396 <https://github.com/readthedocs/readthedocs.org/pull/6396>`__)
18+
* `@agjohnson <http://github.com/agjohnson>`__: Fix CSS and how we were handling html in automation rule UI (`#6394 <https://github.com/readthedocs/readthedocs.org/pull/6394>`__)
19+
* `@ericholscher <http://github.com/ericholscher>`__: Release 3.10.0 (`#6391 <https://github.com/readthedocs/readthedocs.org/pull/6391>`__)
20+
* `@stsewd <http://github.com/stsewd>`__: Set privacy level explicitly (`#6390 <https://github.com/readthedocs/readthedocs.org/pull/6390>`__)
21+
* `@ericholscher <http://github.com/ericholscher>`__: Redirect index files in proxito instead of serving (`#6387 <https://github.com/readthedocs/readthedocs.org/pull/6387>`__)
22+
* `@humitos <http://github.com/humitos>`__: Fully working docker-compose file (`#6295 <https://github.com/readthedocs/readthedocs.org/pull/6295>`__)
23+
* `@saadmk11 <http://github.com/saadmk11>`__: Refactor Subproject validation to use it for Forms and API (`#6285 <https://github.com/readthedocs/readthedocs.org/pull/6285>`__)
24+
* `@saadmk11 <http://github.com/saadmk11>`__: Refactor Gold Views (`#6272 <https://github.com/readthedocs/readthedocs.org/pull/6272>`__)
25+
* `@stsewd <http://github.com/stsewd>`__: Add docs for automatin rules (`#6072 <https://github.com/readthedocs/readthedocs.org/pull/6072>`__)
26+
127
Version 3.10.0
228
--------------
329

‎docker-compose-search.yml

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# docker-compose-search.yml starts "search" service and override some
2+
# services to link them so they can use the "search" as well.
3+
version: '3'
4+
5+
volumes:
6+
search_data:
7+
8+
services:
9+
10+
web:
11+
links:
12+
- storage
13+
- database
14+
- cache
15+
- search
16+
17+
celery:
18+
links:
19+
- storage
20+
- database
21+
- cache
22+
- search
23+
depends_on:
24+
- storage
25+
- azure-cli
26+
- search
27+
28+
# You may need to increase the memory for vm
29+
# sudo sysctl -w vm.max_map_count=262144
30+
search:
31+
image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
32+
environment:
33+
- discovery.type=single-node
34+
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
35+
volumes:
36+
- search_data:/usr/share/elasticsearch/data

‎docker-compose.yml

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# docker-compose.yml file to launch a full instance of Read the Docs.
2+
#
3+
# Read the Docs' Core team is using docker-compose internally.
4+
# We do not offer support on this file and configuration at this point.
5+
# DO NOT USE DOCKER-COMPOSE SETUP FOR PRODUCTION OR CUSTOM INSTALLATION.
6+
7+
# GITHUB_TOKEN environment variable is required to build the images
8+
version: '3'
9+
10+
volumes:
11+
build-user-builds:
12+
azurite_data:
13+
postgres_data:
14+
postgres_backups_data:
15+
16+
services:
17+
server:
18+
# Image used for all the other services (proxito, web, celery, build)
19+
build:
20+
context: .
21+
dockerfile: ${PWD}/docker/Dockerfile
22+
args:
23+
GITHUB_TOKEN: ${GITHUB_TOKEN}
24+
25+
nginx:
26+
image: nginx
27+
ports:
28+
- "80:80"
29+
links:
30+
- web
31+
- proxito
32+
- storage
33+
volumes:
34+
- ${PWD}/docker/nginx:/etc/nginx/conf.d
35+
36+
proxito:
37+
image: readthedocsorg_server:latest
38+
volumes:
39+
- ${PWD}/docker/entrypoints/common.sh:/usr/src/app/docker/common.sh
40+
- ${PWD}/docker/entrypoints/proxito.sh:/usr/src/app/docker/proxito.sh
41+
- ${PWD}:/usr/src/app/checkouts/readthedocs.org
42+
- ${PWD}/../readthedocs-ext:/usr/src/app/checkouts/readthedocs-ext
43+
- ${PWD}/docker/settings/proxito.py:/usr/src/app/checkouts/readthedocs.org/readthedocs/settings/proxito.py
44+
links:
45+
- storage
46+
- database
47+
- cache
48+
environment:
49+
- DJANGO_SETTINGS_MODULE=readthedocs.settings.proxito
50+
51+
# Allow us to run `docker attach readthedocsorg_proxito_1` and get
52+
# control on STDIN and be able to debug our code with interactive pdb
53+
stdin_open: true
54+
tty: true
55+
56+
command: ["../../docker/proxito.sh"]
57+
58+
web:
59+
image: readthedocsorg_server:latest
60+
volumes:
61+
- ${PWD}/docker/scripts/createsuperuser.py:/usr/src/app/docker/createsuperuser.py
62+
- ${PWD}/docker/entrypoints/common.sh:/usr/src/app/docker/common.sh
63+
- ${PWD}/docker/entrypoints/web.sh:/usr/src/app/docker/web.sh
64+
- ${PWD}/../readthedocs-ext:/usr/src/app/checkouts/readthedocs-ext
65+
- ${PWD}:/usr/src/app/checkouts/readthedocs.org
66+
- ${PWD}/docker/settings/web.py:/usr/src/app/checkouts/readthedocs.org/readthedocs/settings/web.py
67+
links:
68+
- storage
69+
- database
70+
- cache
71+
depends_on:
72+
- storage
73+
- azure-cli
74+
environment:
75+
- DJANGO_SETTINGS_MODULE=readthedocs.settings.web
76+
stdin_open: true
77+
tty: true
78+
command: ["../../docker/web.sh"]
79+
80+
celery:
81+
image: readthedocsorg_server:latest
82+
volumes:
83+
- ${PWD}/docker/entrypoints/common.sh:/usr/src/app/docker/common.sh
84+
- ${PWD}/docker/entrypoints/celery.sh:/usr/src/app/docker/celery.sh
85+
- ${PWD}/../readthedocs-ext:/usr/src/app/checkouts/readthedocs-ext
86+
- ${PWD}:/usr/src/app/checkouts/readthedocs.org
87+
- ${PWD}/docker/settings/celery.py:/usr/src/app/checkouts/readthedocs.org/readthedocs/settings/celery.py
88+
links:
89+
- storage
90+
- database
91+
- cache
92+
depends_on:
93+
- storage
94+
- azure-cli
95+
environment:
96+
- DJANGO_SETTINGS_MODULE=readthedocs.settings.celery
97+
stdin_open: true
98+
tty: true
99+
command: ["../../docker/celery.sh"]
100+
101+
build:
102+
image: readthedocsorg_server:latest
103+
volumes:
104+
- ${PWD}/docker/entrypoints/common.sh:/usr/src/app/docker/common.sh
105+
- ${PWD}/docker/entrypoints/build.sh:/usr/src/app/docker/build.sh
106+
- ${PWD}/../readthedocs-ext:/usr/src/app/checkouts/readthedocs-ext
107+
- ${PWD}:/usr/src/app/checkouts/readthedocs.org
108+
- ${PWD}/docker/settings/build.py:/usr/src/app/checkouts/readthedocs.org/readthedocs/settings/build.py
109+
110+
# The python code at readthedocs/doc_builder/environments.py
111+
# mounts `self.project.doc_path`. We need to share this path
112+
# between the build container (git commands), and the container that
113+
# is created inside the build container (sphinx commands).
114+
# Because of this, we need to use a shared volume between them
115+
- build-user-builds:/usr/src/app/checkouts/readthedocs.org/user_builds
116+
117+
# Docker in Docker
118+
- /var/run/docker.sock:/var/run/docker.sock
119+
links:
120+
- web
121+
- storage
122+
- cache
123+
environment:
124+
- DJANGO_SETTINGS_MODULE=readthedocs.settings.build
125+
stdin_open: true
126+
tty: true
127+
command: ["../../docker/build.sh"]
128+
129+
storage:
130+
image: mcr.microsoft.com/azure-storage/azurite:latest
131+
volumes:
132+
- azurite_data:/data
133+
ports:
134+
- "10000:10000"
135+
command: ["azurite-blob", "--silent", "--blobPort", "10000", "--blobHost", "0.0.0.0", "--location", "/data"]
136+
137+
azure-cli:
138+
image: microsoft/azure-cli
139+
volumes:
140+
- ${PWD}/docker/scripts/create-containers.sh:/usr/local/bin/create-containers.sh
141+
links:
142+
- storage
143+
depends_on:
144+
- storage
145+
command: ["/usr/local/bin/create-containers.sh"]
146+
147+
cache:
148+
image: redis:3.2.7
149+
150+
database:
151+
image: postgres:11.1
152+
environment:
153+
- POSTGRES_USER=docs_user
154+
- POSTGRES_PASSWORD=docs_pwd
155+
- POSTGRES_DB=docs_db
156+
volumes:
157+
- postgres_data:/var/lib/postgresql/data
158+
- postgres_backups_data:/backups

‎docker/Dockerfile

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
FROM ubuntu:18.04
2+
3+
ARG GITHUB_TOKEN
4+
5+
ENV DEBIAN_FRONTEND noninteractive
6+
ENV LANG C.UTF-8
7+
8+
WORKDIR /usr/src/app
9+
10+
RUN apt-get -y update
11+
RUN apt-get -y install \
12+
curl \
13+
g++ \
14+
git-core \
15+
libevent-dev \
16+
libpq-dev \
17+
libxml2-dev \
18+
libxslt1-dev \
19+
build-essential \
20+
python3-pip \
21+
python3-dev \
22+
postgresql-client \
23+
libmysqlclient-dev \
24+
libfreetype6 \
25+
libjpeg-dev \
26+
sqlite
27+
28+
RUN pip3 install --upgrade pip
29+
30+
RUN mkdir checkouts
31+
WORKDIR /usr/src/app/checkouts
32+
33+
COPY requirements readthedocs.org/requirements
34+
35+
RUN pip install --no-cache-dir -r readthedocs.org/requirements/docker.txt
36+
37+
# We clone the -ext here only to install the requirements at this point.
38+
# -ext is also mounted as volume from the upper directory
39+
RUN git clone --single-branch --depth 1 https://${GITHUB_TOKEN}@github.com/readthedocs/readthedocs-ext
40+
RUN pip install --no-cache-dir -e readthedocs-ext
41+
42+
WORKDIR /usr/src/app/checkouts/readthedocs.org

‎docker/README.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Docker configuration files
2+
==========================
3+
4+
Read the Docs' Core team is using docker-compose internally.
5+
We do not offer support on this configuration at this point.
6+
7+
**DO NOT USE DOCKER-COMPOSE SETUP FOR PRODUCTION OR CUSTOM INSTALLATION**

‎docker/entrypoints/build.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#! /bin/sh
2+
3+
../../docker/common.sh
4+
5+
watchmedo auto-restart \
6+
--patterns="*.py" \
7+
--ignore-patterns="*.#*.py;./user_builds/*;./public_*;./private_*;*.pyo;*.pyc;*flycheck*.py;./media/*;./.tox/*" \
8+
--ignore-directories \
9+
--recursive \
10+
--signal=SIGTERM \
11+
--kill-after=5 \
12+
--interval=5 \
13+
-- \
14+
python3 -m celery worker -A readthedocs.worker -Ofair -c 2 -Q builder,celery,default,build01 -l DEBUG

‎docker/entrypoints/celery.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#! /bin/sh
2+
3+
../../docker/common.sh
4+
5+
watchmedo auto-restart \
6+
--patterns="*.py" \
7+
--ignore-patterns="*.#*.py;./user_builds/*;./public_*;./private_*;*.pyo;*.pyc;*flycheck*.py;./media/*;./.tox/*" \
8+
--ignore-directories \
9+
--recursive \
10+
--signal=SIGTERM \
11+
--kill-after=5 \
12+
--interval=5 \
13+
-- \
14+
python3 -m celery worker -A readthedocs.worker -Ofair -c 2 -Q web,web01,reindex -l DEBUG

‎docker/entrypoints/common.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#! /bin/sh
2+
3+
# This file is useful when you need to run a command on all the Python
4+
# instances and you don't want to rebuild the image. For example, when
5+
# working on a feature that requires a new dependency and you need to
6+
# call: "pip install newdependency==1.4.2". You can add that command
7+
# here, and it will be executed in all the instances at startup.

‎docker/entrypoints/proxito.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#! /bin/sh
2+
3+
../../docker/common.sh
4+
5+
python3 manage.py runserver 0.0.0.0:8000

‎docker/entrypoints/web.sh

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#! /bin/sh
2+
3+
../../docker/common.sh
4+
5+
python3 manage.py migrate
6+
cat ../../docker/createsuperuser.py | python3 manage.py shell
7+
8+
# collect static in background
9+
python3 manage.py collectstatic --no-input &
10+
11+
python3 manage.py loaddata test_data
12+
13+
python3 manage.py runserver 0.0.0.0:8000

0 commit comments

Comments
 (0)