Merge branch 'kv_docs' of github.com:broadinstitute/cromwell into kv_docs

Author: katevoss

docs/Configuring.md

@@ -301,4 +301,35 @@ When running a job on the Config (Shared Filesystem) backend, Cromwell provides
           }
         }
       }
-```
+```
+
+**Workflow Log Directory**
+
+To change the directory where Cromwell writes workflow logs, change the directory location via the setting:
+
+```hocon
+workflow-options {
+    workflow-log-dir = "cromwell-workflow-logs"
+}
+```
+
+
+**Preserving Workflow Logs**
+
+By default Cromwell erases the per workflow logs when the workflow completes to reduce disk usage. You can change this behavior by setting the following value to `false`:
+
+```hocon
+workflow-options {
+    workflow-log-temporary = true
+}
+```
+
+
+**Exporting workflow logs via Sentry**
+
+Cromwell supports [Sentry](https://docs.sentry.io) for copying workflow logs. Sentry is a service that can be used to monitor exceptions reported in an application’s logs.
+
+To send the Cromwell logs to Sentry, enter your DSN URL into the configuration value:
+
+```hocon
+sentry.dsn = DSN_URL```

docs/Logging.md

@@ -1,7 +1,3 @@
-_For the Doc-A-Thon_  
-
-**NOTES TO EDITOR:**
-- There are [sections at the bottom](#todo-move-the-sections-below-to-the-configuration-page) that should be moved into configuration.
 
 **Questions to answer and things to consider:**
 
@@ -20,9 +16,9 @@ _For the Doc-A-Thon_
 
 ---
 
-## Logging properties
+## Logging Properties
 
-### Setting logging properties
+### Setting Logging Properties
 
 Cromwell accepts two properties for controlling logging. You can set these properties via a Java system property on the command line using `-D`:
 
@@ -37,15 +33,15 @@ export LOG_LEVEL=DEBUG
 java -jar cromwell.jar server
 ```
 
-If you set same property via a system property and an environment variable, the system property overrides the environment variable.
+*If you set same property via a system property, and an environment variable, the system property overrides the environment variable.*
 
 ### Log Format
 
-Cromwell outputs logs in one of two formats, either `pretty` or `standard`. You can change the format of the logs by setting the property `LOG_MODE`.
+Cromwell outputs log in one of two formats, either `pretty` or `standard`. You can change the format of the logs by setting the property to `LOG_MODE`.
 
-In `standard` mode, your logs will be written without ANSI escape code coloring, with a layout more appropriate for server logs.
+* In `standard` mode, your logs will be written without ANSI escape code coloring, with a layout more appropriate for server logs.
 
-In `pretty` mode, your logs are output is a colorful, easier to read output more appropriate for a single workflow run.
+* In `pretty` mode, your logs are output in a colorful, easier to read format, more appropriate for a single workflow run.
 
 The default mode for server is `standard`, while the default when running a single worklow is `pretty`. You can explicitly specify the format by running cromwell with:
 
@@ -55,7 +51,7 @@ java -DLOG_MODE=pretty -jar cromwell.jar server
 
 ### Log Level
 
-By default Cromwell outputs messages at a `LOG_LEVEL` of `INFO`. Sometimes, you may want more or less information logged. For example, while debugging an issue you may want to increase the amount information in the logs temporarily. Or, in some situations, the standard level may be too verbose, and you may only want Cromwell to log warnings and errors.
+By default, Cromwell outputs messages at a `LOG_LEVEL` of `INFO`. Sometimes, you may want more or less information logged. For example, while debugging an issue you may want to increase the amount information in the logs temporarily. Alternatively, the standard level may be too verbose, and you may only want Cromwell to log warnings and errors.
 
 You can set the level via the property `LOG_LEVEL` to any one of the values: `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, or `OFF`. The default log level is `INFO`.
 
@@ -65,44 +61,12 @@ java -DLOG_LEVEL=DEBUG -jar cromwell.jar server
 
 ## Workflow Logs
 
-While a workflow is running, Cromwell generates a log file specifically for the workflow. After the workflow completes, to clear up local disk space, cromwell deletes the local copy of this log file. See the [configuration](Configuring) section on Logs for more information on preventing cromwell from deleting each workflow log.
+While a workflow is running, Cromwell generates a log file specifically for the workflow. After the workflow completes, to clear up local disk space, Cromwell deletes the local copy of this log file. See the [configuration](Configuring) section on logs for more information on preventing cromwell from deleting each workflow log.
 
-After the workflow completes, but before Cromwell deletes the files, you can configure Cromwell to copy the workflow logs to various locations. Normally, you'll want to copy the log to a remote bucket or directory. To specify the remote directory to copy the logs to use the separate [workflow option](WorkflowOptions) `final_workflow_log_dir`. Workflow logs may also be copied via [Sentry](https://docs.sentry.io) by setting the [configuration](Configuring) value `sentry.dsn`.
+Before Cromwell deletes teh files and before the workflow completes, you can configure Cromwell to copy the workflow logs to various locations. Normally, you'll want to copy the log to a remote bucket or directory. To specify the remote directory to copy the logs to use the separate [workflow option](WorkflowOptions) `final_workflow_log_dir`. Workflow logs may also be copied via [Sentry](https://docs.sentry.io) by setting the [configuration](Configuring) value `sentry.dsn`.
 
 ## Call Logs
 
 As each call in a workflow runs, it generates output to the standard output and standard error. This output is stored per call in call log files. Additionally, depending on the backend, specific per call backand logs may be generated.
 
 All of these call logs may be copied at the end of a workflow to a remote directory. Configure this directory by setting the [workflow option](WorkflowOptions) `final_call_logs_dir`.
-
-# TODO MOVE THE SECTIONS BELOW TO THE CONFIGURATION PAGE
-
-### Workflow Log Directory
-
-To change the directory where cromwell writes workflow logs, change the directory location via the setting:
-
-```hocon
-workflow-options {
-    workflow-log-dir = "cromwell-workflow-logs"
-}
-```
-
-### Preserving Workflow Logs
-
-By default Cromwell erases the per workflow logs when the workflow completes to reduce disk usage. You can change this behavior by setting the following value to `false`:
-
-```hocon
-workflow-options {
-    workflow-log-temporary = true
-}
-```
-
-### Exporting workflow logs via Sentry
-
-Cromwell supports [Sentry](https://docs.sentry.io) for copying workflow logs. Sentry is a service that can be used to monitor exceptions reported in an application's logs.
-
-To send the cromwell logs to Sentry, enter your DSN URL into the configuration value:
-
-```hocon
-sentry.dsn = DSN_URL
-```

docs/Security.md

@@ -21,18 +21,19 @@ Making sure that cromwell is set to run “local” or “JES” or other workfl
 
  - Cromwell is NOT on its own a security appliance!
  - Only YOU are responsible for your own security! 
+ - Please be sure to check with your security team before setting up your Cromwell server
  - Some recommendations and suggestions on security can be found below
 
 
-**Security by sysadmin**
-
 __Warning!__
 
-__This section is community-contributed. It is intended as helpful guidance only, and is not endorsed by the Broad Institute.__
+__This is intended as helpful guidance only, and is not endorsed by the Broad Institute.__
+
+Cromwell running in server mode accepts all connections on the configured webservice port.  Without taking additional measures to protect your Cromwell server, this can leave your Cromwell server and therefore all information stored by and accessible by your Cromwell server vulnerable to anyone who is able to access the server.  For instance, an unprotected Cromwell server running against the [Google Cloud backend](backends/Google) would leave you vulnerable to outside users running workflows that will cost you money, and potentially access the data files you use within your workflows!
 
-Cromwell running in server mode accepts all connections on the configured webservice port. The simplest way to restrict access is by putting an authenticating proxy server in between users and the cromwell server:
- 1. Configure a firewall rule on the cromwell server host to deny access to the webservice port (e.g. 8000) from all addresses except a secure proxy host.
- 1. Configure `<YourFavoriteWebProxy>` on the proxy host with `<YourFavoriteAuthMechanism>`, to proxy authenticated traffic from the world to the cromwell server. Using Apache `httpd` web server for example with basic htpassword file-based authentication, the configuration might look something like:
+The simplest way to restrict access is by putting an authenticating proxy server in between users and the Cromwell server:
+ 1. Configure a firewall rule on the Cromwell server host to deny access to the webservice port (e.g. 8000) from all addresses except a secure proxy host.
+ 1. Configure `<YourFavoriteWebProxy>` on the proxy host with `<YourFavoriteAuthMechanism>`, to proxy authenticated traffic from the world to the Cromwell server. Using Apache `httpd` web server for example with basic `htpassword` file-based authentication, the configuration might look something like:
 
  ```Apache
 <Location /cromwell>
@@ -46,11 +47,11 @@ Cromwell running in server mode accepts all connections on the configured webser
 </Location>
 ```
 
- 1. That's it. Users now hit `http://my.proxy.org/cromwell` with authenticated requests, and they're forwarded to port 8000 on the cromwell server host.
+ 1. Users now hit `http://my.proxy.org/cromwell` with authenticated requests, and they're forwarded to port 8000 on the Cromwell server host. 
 
-**Multi-tenant**
+**Multiple Servers on one Host**
 
-The above scheme extends easily to multiple cromwell instances, for use by different groups within an organization for example. If the instances are running on the same host then each instance should be run as its own dedicated service account user, e.g. `cromwell1`, `cromwell2` etc. so that processes running under one cromwell instance cannot access the files of another; different webservice ports must also be configured. If persistent database storage is being used then each instance should be configured with its own database and database user. The proxy configuration above is extended simply by adding another `Location`:
+The above scheme extends easily to multiple Cromwell instances, for use by different groups within an organization, for example. If multiple instances are running on the same host, then each instance should be run as its own dedicated user (such as service account when using a [Google Cloud backend](backends/Google)), e.g. `cromwell1`, `cromwell2` etc so that processes running under one Cromwell instance cannot access the files of another. Different webservice ports must also be configured separately in order to not clash. If persistent database storage is being used, then each instance must be configured with its own database. The proxy configuration above is extended simply by adding another `Location`:
 
 ```Apache
 <Location /cromwell1>
@@ -64,3 +65,14 @@ The above scheme extends easily to multiple cromwell instances, for use by diffe
 </Location>
 ```
 
+**Multiple Tenants in one Cromwell Server**
+
+Even with a proxy in place, a single Cromwell server does not provide _authorization_ for individual users hitting the endpoints.  This allows, for instance, one _authenticated_ user to view the metadata for a workflow run by another _authenticated_ user.  Due to this limitation, it is important that all users of this Cromwell server be trusted.  
+
+With the exception of the use of the `refresh_token` or `google_compute_service_account` scheme in the [Configuration](Configuring#authentication) for the [Google Cloud backend](backends/Google)), Cromwell servers are setup to access files for use in workflows using a single set of credentials.  This means that all users of these Cromwell servers will have access to the same files that any other user has access to.  The `refresh_token` or `google_compute_service_account` scheme is the only way to ensure data is protected among multiple users of a Cromwell server, however the aforementioned caveats of authorization for endpoints still applies.  
+
+**Protecting Secrets**
+
+Various parts of the Cromwell server [Configuration](Configuring) contain sensitive information, e.g. username and password for your persistent database.  It is strongly recommended to protect the configuration files from any untrusted users, for instance by limiting who can access your Cromwell server host or using a technology such as [HashiCorp Vault](https://www.vaultproject.io/).  
+
+Additionally, the contents of the Cromwell database can contain sensitive information, so it is recommended to limit the access of the database only to trusted users.