Skip to content

enable local development #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 29, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions dev/docker/database/epicast/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# start with the `delphi_database` image
FROM delphi_database

# create the `epicast2` database
ENV MYSQL_DATABASE epicast2

# create the `epi` user account with a development-only password
ENV MYSQL_USER user
ENV MYSQL_PASSWORD pass

# provide DDL which will create empty tables at container startup
# note that files are executed in order of filename alphabetically, so here
# destination files are named such that table definitions are executed prior to
# data insertions
COPY repos/delphi/www-epicast/src/ddl/epicast2.sql /docker-entrypoint-initdb.d/0_epicast2.sql
COPY repos/delphi/www-epicast/src/ddl/development_data.sql /docker-entrypoint-initdb.d/1_development_data.sql
27 changes: 27 additions & 0 deletions dev/docker/database/epicast/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# `delphi_database_epicast`

This image extends Delphi's database by:

- adding the `epi` user account
- adding the `epicast2` database
- creating and minimally populating tables in `epicast2`

To start a container from this image, run:

```bash
docker run --rm -p 13306:3306 \
--network delphi-net --name delphi_database_epicast \
delphi_database_epicast
```

For debugging purposes, you can interactively connect to the database inside
the container using a `mysql` client (either installed locally or supplied via
a docker image) like this:

```bash
mysql --user=user --password=pass --port 13306 --host 127.0.0.1 epicast2
```

Note that using host `localhost` may fail on some platforms as mysql will
attempt, and fail, to use a Unix socket. Using `127.0.0.1`, which implies
TCP/IP, works instead.
27 changes: 27 additions & 0 deletions dev/docker/web/epicast/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# start with the `delphi_web` image
FROM delphi_web

# enable mod_rewrite, aka RewriteEngine
RUN a2enmod rewrite

# enable mod_headers
RUN a2enmod headers

# create an empty htpasswd file
RUN touch /var/www/passwords

# deploy the Epicast website (see `www-epicast/deploy.json`)
# deploy files in an order that tries to take advantage of build caching

COPY repos/delphi/www-epicast/site/images/* /var/www/html/images/
COPY repos/delphi/www-epicast/site/images/flags/* /var/www/html/images/flags/
COPY repos/delphi/www-epicast/site/data/* /var/www/html/data/

COPY repos/delphi/www-epicast/site/css/* /var/www/html/css/
COPY repos/delphi/www-epicast/site/js/*.js /var/www/html/js/
COPY repos/delphi/www-epicast/site/js/us-map /var/www/html/js/us-map
COPY repos/delphi/www-epicast/site/common/* /var/www/html/common/
COPY repos/delphi/www-epicast/site/*.php /var/www/html/

# point to the local development database (overwrites `common/settings.php`)
COPY repos/delphi/www-epicast/dev/docker/web/epicast/assets/settings.php /var/www/html/common/
24 changes: 24 additions & 0 deletions dev/docker/web/epicast/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# `delphi_web_epicast`

This image starts with Delphi's web server and adds the sources necessary for
hosting the Epicast website. It further extends Delphi's web server by:

- enabling the `mod_rewrite` extension
- enabling the `mod_headers` extension
- creating an empty `htpasswd` file

This image includes the file
[`settings.php`](assets/settings.php), which points to a local
container running the
[`delphi_database_epicast` image](../../database/epicast/README.md).

To start a container from this image, run:

```bash
docker run --rm -p 10080:80 \
--network delphi-net --name delphi_web_epicast \
delphi_web_epicast
```

You should be able to login and interact with the website (e.g. submitting
predictions) by visiting `http://localhost:10080/` in a web browser.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to make this work for a pandemic setup where, e.g., my development machine is on campus and I'm shelling into it from home? Are we talking X forwarding or is there a way to expose this address outside localhost?

Copy link
Contributor Author

@dfarrow0 dfarrow0 Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh interesting, that's a good use-case. no need to X-forward. you should be able to use a reverse ssh tunnel to forward localhost:10080 on your at-home machine to localhost:10080 on your at-work machine.

from your at-home machine, this command should work:

ssh -N -L localhost:10080:localhost:10080 your-username@at-work-machine

This works by listening on port 10080 at-home and tunneling connections to port 10080 at-work. (The first localhost:10080 is where to listen at-home, the second localhost:10080 is where to connect at-work.)

This is probably something that should go in the guide. Do you mind letting me know if this works for you?

edit: fix terrible typo, this should be a forward tunnel, not reverse

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and to be clear, after the ssh command above, you can then open http://localhost:10080/ on your at-home machine, and you should see your "local" (at-work) instance of the website.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the db and web server running in docker on my work machine, and I can load the site in lynx from there. Alas, the ssh tunnel fails:

$ ssh -N -R localhost:10080:localhost:10080 <me>@<work>
<me>@<work>'s password: 
Warning: remote port forwarding failed for listen port 10080
^C$ ssh -N -R 127.0.0.1:10080:127.0.0.1:10080 <me>@<work>
<me>@<work>'s password: 
Warning: remote port forwarding failed for listen port 10080

--which seems weird; I'm not a tunneling pro but I've done something like this before. I'll have more time to debug tomorrow morning.

Copy link
Contributor Author

@dfarrow0 dfarrow0 Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh my bad. this should be a forward tunnel, not reverse. just need to swap out -R with -L as in:

ssh -N -L 127.0.0.1:10080:127.0.0.1:10080 your-username@at-work-machine

(replaced localhost with 127.0.0.1 to be extra safe)

15 changes: 15 additions & 0 deletions dev/docker/web/epicast/assets/settings.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<?php

/* Settings and secrets for local development of the Epicast website. */

$dbHost = 'delphi_database_epicast';
$dbPort = 3306;
$dbUser = 'user';
$dbPass = 'pass';
$dbName = 'epicast2';

$epicastAdmin = array(
'name' => 'Delphi Developer',
'email' => 'fake_email_address',
);
?>
137 changes: 137 additions & 0 deletions src/ddl/development_data.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
/*
This file contains a minimally-useful set of values for development of the
Epicast website. Production tables contain much more data, which varies by
season and purpose (e.g. flu vs covid, wILI vs hospitalization, etc).

Edit this file as needed during development. This file is not used outside of
local testing.
*/

/* all age groups as of 2020-04-27 */
INSERT INTO `ec_fluv_age_groups` VALUES
(0,'rate_age_0','Group #1','0-4 years old'),
(0,'rate_age_1','Group #2','5-17 years old'),
(0,'rate_age_2','Group #3','18-49 years old'),
(0,'rate_age_3','Group #4','50-64 years old'),
(0,'rate_age_4','Group #5','65+ years old'),
(0,'rate_overall','Group #6','all ages');

/* all default preferences as of 2020-04-27 */
INSERT INTO `ec_fluv_defaults` VALUES
(0,'advanced_pandemic','1'),
(0,'email_notifications','1'),
(0,'email_reminders','0'),
(0,'skip_intro','0'),
(0,'survey_epi','2'),
(0,'survey_flu','2'),
(0,'survey_ph','2'),
(0,'survey_sml','2'),
(0,'survey_vir','2'),
(0,'_admin','0'),
(0,'_debug','0'),
(0,'advanced_leaderboard','0'),
(0,'advanced_initials','-'),
(0,'skip_instructions','0'),
(0,'_delphi','0'),
(0,'advanced_prior','0'),
(0,'allLocation','0'),
(0,'advanced_hospitalization','0'),
(0,'hidden_seasons','0');

/* all task groups as of 2020-04-27 */
INSERT INTO `ec_fluv_mturk_tasks` VALUES
(0,'1,13,14,17,19,20,23,32,38,41,56',2,201942,10),
(0,'1,21,22,24,26,27,33,39,42,46,57',2,201942,10),
(0,'1,15,18,34,48,50,51,54,58,59,61,64',3,201942,10),
(0,'1,25,28,29,37,40,44,47,55,60,63,65',1,201942,10),
(0,'1,12,16,31,35,36,43,45,49,52,53,62',1,201942,10),
(0,'1,2,3,4,5,6,7,8,9,10,11',0,201942,10);

/* all locations as of 2020-04-27 */
INSERT INTO `ec_fluv_regions` VALUES
(0,'nat','National','All U.S. States and Territories',313914040),
(0,'hhs1','HHS Region 1','CT, MA, ME, NH, RI, VT',14562704),
(0,'hhs2','HHS Region 2','NJ, NY',28434851),
(0,'hhs3','HHS Region 3','DE, MD, PA, VA, WV',30238794),
(0,'hhs4','HHS Region 4','AL, FL, GA, KY, MS, NC, SC, TN',62356916),
(0,'hhs5','HHS Region 5','IL, IN, MI, MN, OH, WI',51945711),
(0,'hhs6','HHS Region 6','AR, LA, NM, OK, TX',39510585),
(0,'hhs7','HHS Region 7','IA, KS, MO, NE',13837604),
(0,'hhs8','HHS Region 8','CO, MT, ND, SD, UT, WY',11157404),
(0,'hhs9','HHS Region 9','AZ, CA, HI, NV',48745929),
(0,'hhs10','HHS Region 10','AK, ID, OR, WA',13123542),
(0,'PA','Pennsylvania','PA',12780000),
(0,'GA','Georgia','GA',10310000),
(0,'DC','Washington DC','DC',672228),
(0,'TX','Texas','TX',27862596),
(0,'OR','Oregon','OR',4093465),
(0,'WY','Wyoming','WY',589713),
(0,'CT','Connecticut','CT',3583134),
(0,'ME','Maine','ME',1327472),
(0,'NH','New Hampshire','NH',1335832),
(0,'RI','Rhode Island','RI',1059080),
(0,'VT','Vermont','VT',624592),
(0,'NJ','New Jersey','NJ',8996351),
(0,'ny_minus_jfk','NY (excluding NYC)','ny_minus_jfk',11483000),
(0,'DE','Delaware','DE',965866),
(0,'MD','Maryland','MD',6068511),
(0,'VA','Virginia','VA',8492783),
(0,'WV','West Virginia','WV',1834882),
(0,'AL','Alabama','AL',4884115),
(0,'KY','Kentucky','KY',4436974),
(0,'MS','Mississippi','MS',2988726),
(0,'NC','North Carolina','NC',10146788),
(0,'SC','South Carolina','SC',4961119),
(0,'TN','Tennessee','TN',6651194),
(0,'IL','Illinois','IL',12801539),
(0,'IN','Indiana','IN',6633053),
(0,'MI','Michigan','MI',9928300),
(0,'MN','Minnesota','MN',5519952),
(0,'OH','Ohio','OH',11614373),
(0,'WI','Wisconsin','WI',5778708),
(0,'AR','Arkansas','AR',2988248),
(0,'NM','New Mexico','NM',2081015),
(0,'OK','Oklahoma','OK',3923561),
(0,'MA','Massachusetts','MA',6873018),
(0,'IA','Iowa','IA',3134693),
(0,'KS','Kansas','KS',2907289),
(0,'MO','Missouri','MO',6093000),
(0,'NE','Nebraska','NE',1907116),
(0,'CO','Colorado','CO',5540545),
(0,'MT','Montana','MT',1042520),
(0,'ND','North Dakota','ND',790701),
(0,'SD','South Dakota','SD',865454),
(0,'UT','Utah','UT',3051217),
(0,'AZ','Arizona','AZ',6931071),
(0,'CA','California','CA',39250017),
(0,'HI','Hawaii','HI',1428557),
(0,'NV','Nevada','NV',2940058),
(0,'ID','Idaho','ID',1683140),
(0,'AK','Alaska','AK',741204),
(0,'WA','Washington','WA',7288000),
(0,'LA','Louisiana','LA',4652581),
(0,'PR','Puerto Rico','PR',2906871),
(0,'VI','Virgin Islands','VI',107268),
(0,'JFK','New York City','JFK',8623000);

/* the "current" week (change as needed for development), should be consistent
with `ec_fluv_season` below */
INSERT INTO `ec_fluv_round` VALUES
(202016,'2020-04-27 08:00:00');

/* a high score for you (congratulations btw!) */
INSERT INTO `ec_fluv_scores` VALUES
(0,1,12345.678,123.456,'2020-01-02 03:04:05');

/* the "current" flu season (change as needed for development), should be
consistent with `ec_fluv_round` above */
INSERT INTO `ec_fluv_season` VALUES
(2019,201942,202035);

/* make yourself an admin */
INSERT INTO `ec_fluv_user_preferences` VALUES
(0,1,'_admin',1,'2019-09-25 12:36:34');

/* your account details */
INSERT INTO `ec_fluv_users` VALUES
(0,'00000000000000000000000000000000','Delphi Developer','fake_email_address',NULL,'2019-09-25 12:34:31','2019-09-25 12:34:31',NULL);
Loading