Evaluate batchtools

[dshemetov]

• Use batchtools to do covid forecast evaluation
• Use batchtools to do flu forecast evaluation
• Confirm that arrow/parquet interacts well with it

[dsweber2]

see if chunk jobs will let us do forecaster dependency tree They will not. If we want to do sequential jobs, we're probably going to want to use tags to say 1st wave (independent forecasters), second wave (ensembles), third wave (ensembles containing ensembles), etc.

[dsweber2]

Major thing left to test is what happens with parallelization.

[dsweber2]

A limitation is that summarizeExperiments won't print jobs with parameters that are lists. Those jobs will run fine however

[dshemetov]

Ended up going with targets because it supports a notion of job dependence and allows for parameter sweeps across lists of complex objects (such as different trainers).

• Built out a prototype job with file targets
• Built out a prototype forecaster target
• Built out a prototype ensemble forecaster target
• Built out a parameter sweep job using tarchetypes