Add new Jenkins deployment workflow

[korlaxxalrok]

Description

This PR adds new Jenkins functionality to simplify indicator deploys for the new repo development workflow.

It will work like this:

• A merge to main:

  • Creates a venv for each indicator, handles package installation and setup,
    and then creates a package (at the moment a tar.gz).
  • Deploys each indicator to the staging environment.

• A merge to prod:

  • Deploys each indicator to the production environment.

Changelog

Itemize code/test/documentation changes and files added/removed.

• Add new Jenkinsfile
• Add new supporting shell scripts

[korlaxxalrok]

@chinandrew @krivard We should convene to make sure that we have all the indicators in main before we merge this. I'll need to make a couple of changes as well as I have a test directory configured in a few places as a fail safe.

[chinandrew]

@chinandrew @krivard We should convene to make sure that we have all the indicators in main before we merge this. I'll need to make a couple of changes as well as I have a test directory configured in a few places as a fail safe.

Which indicators are outstanding? DV, quidel [covidtest], fb-survey (in R), and maybe one other?

[korlaxxalrok]

@chinandrew I think initially we might only need to make sure that we account for what is running in production, so that would be:

changehc
google_health
jhu
quidel_covidtest
safegraph
usafacts

Nothing else is scheduled in Automation yet so wouldn't be much of a problem if initially we unpacked the code into the runtime dir.

For the initial merge, I think something like this makes sense:

• Update test directories
• Publish PR
• Take a backup of the Jenkins artifact dir
• Take a backup of the staging runtime dir
• Merge

We can give it some time to settle in, and then get it over to prod (I want to take a backup of the prod runtime dir before we do that).

[korlaxxalrok]

Ok, looks like this now:

• Update test directories
• Publish PR
• Take a backup of the Jenkins artifact dir
• Take a backup of the staging runtime dir
• Merge

[korlaxxalrok]

@krivard I can't accept those changes because I've already made them as further commits outside of the PR convo and they are now 'Outdated'. I think I broke the PR workflow a little bit.

At any rate, I think we can merge this.

• Issue is filed.
• Directory backups are taken.
• Production directories are configured.

When we merge to main this will build+package and then deploy to staging. This is mostly inconsequential at the moment, but will be good to see in action.

[krivard]

lgtm, apologies for the weird changes confusion