Skip to content
This repository was archived by the owner on Jan 16, 2025. It is now read-only.

Commit 060daac

Browse files
ScottGuymernpalm
ScottGuymer
and
npalm
authored
feat: Add hooks for prebuilt images (AMI), including amazon linux packer example (#1444)
* Initial creation of runner image * Refactored startup script and added it to the per-boot folder * Make the runner location a variable So we can pass the runner version in at packer build time if we want to update the runner version. * Retrieve external config setting via tags Retrieve the required config via the instance tags so we dont have to pass in and set environment on the instance in an awkward way. * Enable tag based config Give the instance the permission to query its own tags and set the correct tags on the instance. * Add a CI job * Fix the CI build * Fix the formatting * Retain user_data provisioning and remove duplication refactored to make sure user_data continues to work with minimal breaking changes. Use a single set of scripts shared between image and user_data provisioning. * Fix interpolation issues in template file * fix build * Fix formatting * minor tweaks and fixes * Fixes from testing * Enable docker on boot * Add in output of start time for the runner * Scoop up the runner log * Add a powershell build script for windows users * Fix formatting * Use SSM parameters for configuration Its best practice to use SSM parameters for configuration of the runners. In adding this i have also added parameter path based config so its easy to extend in the future. * Make the SSM policy more specific * Update .github/workflows/packer-build.yml Co-authored-by: Niek Palm <[email protected]> * Added condition to the describe tags policy * Dont use templatefile on the tags policy Because of the use of ${} in the policy terraform is trying to replace it. * Added an option to turn off userdata scripting * Added/updated documentation * Revert policy as it has no effect on the permissions * Add reference to prebuilt images in the main readme * Add an example of deploying with prebuilt images * Update readme * Use current user as ami_owner * Update example to 5 secs * Updated ami name to include the arch * Fixed log file variable * Added explicit info about required settings to the readme * Change userdata_enabled to enabled_userdata Keep within existing naming convention Co-authored-by: Niek Palm <[email protected]>
1 parent 5809fee commit 060daac

32 files changed

+691
-86
lines changed

Diff for: ‎.ci/build.ps1

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
$TOP_DIR=$(git rev-parse --show-toplevel)
2+
$OUTPUT_DIR="$TOP_DIR/lambda_output"
3+
4+
New-Item "$OUTPUT_DIR" -ItemType Directory -ErrorAction SilentlyContinue
5+
6+
$env:DOCKER_BUILDKIT=1
7+
docker build --no-cache --target=final --output=type=local,dest="$OUTPUT_DIR" -f "$TOP_DIR/.ci/Dockerfile" "$TOP_DIR"
8+

Diff for: ‎.editorconfig

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[*]
2+
end_of_line = lf

Diff for: ‎.github/workflows/packer-build.yml

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: "Packer checks"
2+
on:
3+
push:
4+
branches:
5+
- master
6+
- develop
7+
pull_request:
8+
paths:
9+
- "images/**"
10+
- ".github/workflows/packer-build.yml"
11+
12+
env:
13+
AWS_REGION: eu-west-1
14+
15+
jobs:
16+
verify_packer:
17+
name: Verify packer
18+
runs-on: ubuntu-latest
19+
container:
20+
image: hashicorp/packer:1.7.8
21+
defaults:
22+
run:
23+
working-directory: images/linux-amzn2
24+
steps:
25+
- name: "Checkout"
26+
uses: actions/checkout@v2
27+
28+
- name: packer init
29+
run: packer init .
30+
31+
- name: check terraform formatting
32+
run: packer fmt -recursive -check=true .
33+
34+
- name: packer validate
35+
run: packer validate .

Diff for: ‎.github/workflows/terraform.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ jobs:
4343
fail-fast: false
4444
matrix:
4545
terraform: [0.14.3, 0.15.5, 1.0.8]
46-
example: ["default", "ubuntu"]
46+
example: ["default", "ubuntu", "prebuilt"]
4747
defaults:
4848
run:
4949
working-directory: examples/${{ matrix.example }}

Diff for: ‎README.md

+11-4
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ This [Terraform](https://www.terraform.io/) module creates the required infrastr
1616
- [Install app](#install-app)
1717
- [Encryption](#encryption)
1818
- [Idle runners](#idle-runners)
19+
- [Prebuilt Images](#prebuilt-images)
1920
- [Examples](#examples)
2021
- [Sub modules](#sub-modules)
2122
- [ARM64 configuration for submodules](#arm64-configuration-for-submodules)
@@ -265,6 +266,10 @@ idle_config = [{
265266
}]
266267
```
267268

269+
### Prebuilt Images
270+
271+
This module also allows you to run agents from a prebuilt AMI to gain faster startup times. You can find more information in [the image README.md](/images/README.md)
272+
268273
#### Supported config <!-- omit in toc -->
269274

270275
Cron expressions are parsed by [cron-parser](https://github.com/harrisiirak/cron-parser#readme). The supported syntax.
@@ -289,6 +294,7 @@ Examples are located in the [examples](./examples) directory. The following exam
289294

290295
- _[Default](examples/default/README.md)_: The default example of the module
291296
- _[Permissions boundary](examples/permissions-boundary/README.md)_: Example usages of permissions boundaries.
297+
- _[Prebuilt Images](examples/prebuilt/README.md)_: Example usages of deploying runners with a custom prebuilt image.
292298

293299
## Sub modules
294300

@@ -346,10 +352,10 @@ In case the setup does not work as intended follow the trace of events:
346352

347353
| Name | Source | Version |
348354
|------|--------|---------|
349-
| <a name="module_runner_binaries"></a> [runner\_binaries](#module\_runner\_binaries) | ./modules/runner-binaries-syncer | n/a |
350-
| <a name="module_runners"></a> [runners](#module\_runners) | ./modules/runners | n/a |
351-
| <a name="module_ssm"></a> [ssm](#module\_ssm) | ./modules/ssm | n/a |
352-
| <a name="module_webhook"></a> [webhook](#module\_webhook) | ./modules/webhook | n/a |
355+
| <a name="module_runner_binaries"></a> [runner\_binaries](#module\_runner\_binaries) | ./modules/runner-binaries-syncer | |
356+
| <a name="module_runners"></a> [runners](#module\_runners) | ./modules/runners | |
357+
| <a name="module_ssm"></a> [ssm](#module\_ssm) | ./modules/ssm | |
358+
| <a name="module_webhook"></a> [webhook](#module\_webhook) | ./modules/webhook | |
353359

354360
## Resources
355361

@@ -422,6 +428,7 @@ In case the setup does not work as intended follow the trace of events:
422428
| <a name="input_syncer_lambda_s3_key"></a> [syncer\_lambda\_s3\_key](#input\_syncer\_lambda\_s3\_key) | S3 key for syncer lambda function. Required if using S3 bucket to specify lambdas. | `any` | `null` | no |
423429
| <a name="input_syncer_lambda_s3_object_version"></a> [syncer\_lambda\_s3\_object\_version](#input\_syncer\_lambda\_s3\_object\_version) | S3 object version for syncer lambda function. Useful if S3 versioning is enabled on source bucket. | `any` | `null` | no |
424430
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no |
431+
| <a name="input_enabled_userdata"></a> [enabled_userdata](#input\_enabled_userdata) | Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI | `bool` | `true` | no |
425432
| <a name="input_userdata_post_install"></a> [userdata\_post\_install](#input\_userdata\_post\_install) | Script to be ran after the GitHub Actions runner is installed on the EC2 instances | `string` | `""` | no |
426433
| <a name="input_userdata_pre_install"></a> [userdata\_pre\_install](#input\_userdata\_pre\_install) | Script to be ran before the GitHub Actions runner is installed on the EC2 instances | `string` | `""` | no |
427434
| <a name="input_userdata_template"></a> [userdata\_template](#input\_userdata\_template) | Alternative user-data template, replacing the default template. By providing your own user\_data you have to take care of installing all required software, including the action runner. Variables userdata\_pre/post\_install are ignored. | `string` | `null` | no |

Diff for: ‎examples/prebuilt/.terraform.lock.hcl

+60
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Diff for: ‎examples/prebuilt/README.md

+83
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Action runners deployment with prebuilt image
2+
3+
This module shows how to create GitHub action runners using a prebuilt AMI for the runners
4+
5+
## Usages
6+
7+
Steps for the full setup, such as creating a GitHub app can be found in the root module's [README](../../README.md).
8+
9+
### Lambdas
10+
11+
You can either download the released lambda code or build them locally yourself.
12+
13+
First download the Lambda releases from GitHub. Ensure you have set the version in `lambdas-download/main.tf` for running the example. The version needs to be set to a GitHub release version, see https://github.com/philips-labs/terraform-aws-github-runner/releases
14+
15+
```bash
16+
cd lambdas-download
17+
terraform init
18+
terraform apply
19+
cd ..
20+
```
21+
22+
Alternatively you can build the lambdas locally with Node or Docker, there is a simple build script in `<root>/.ci/build.sh`. In the `main.tf` you need to specify the build location for all of the zip files.
23+
24+
```hcl
25+
webhook_lambda_zip = "../../lambda_output/webhook.zip"
26+
runner_binaries_syncer_lambda_zip = "../../lambda_output/runner-binaries-syncer.zip"
27+
runners_lambda_zip = "../../lambda_output/runners.zip"
28+
```
29+
30+
### GitHub App Configuration
31+
32+
Before running Terraform, ensure the GitHub app is configured. See the [configuration details](../../README.md#usages) for more details.
33+
34+
### Packer Image
35+
36+
You will need to build your image. This example deployment uses the image example in `/images/linux-amz2`. You must build this image with packer in your AWS account first. Once you have built this you need to provider your owner ID as a variable
37+
38+
## Deploy
39+
40+
To use your image in the terraform modules you will need to set some values on the module.
41+
42+
Assuming you have built the `linux-amzn2` image which has a pre-defined AMI name in the following format `github-runner-amzn2-x86_64-YYYYMMDDhhmm` you can use the following values.
43+
44+
```hcl
45+
46+
module "runners" {
47+
...
48+
# set the name of the ami to use
49+
ami_filter = { name = ["github-runner-amzn2-x86_64-2021*"] }
50+
# provide the owner id of
51+
ami_owners = ["<your owner id>"]
52+
53+
enabled_userdata = false
54+
...
55+
}
56+
```
57+
58+
If your owner is the same as the account you are logging into then you can use `aws_caller_identity` to retrieve it dynamically.
59+
60+
```hcl
61+
data "aws_caller_identity" "current" {}
62+
63+
module "runners" {
64+
...
65+
ami_owners = [data.aws_caller_identity.current.account_id]
66+
...
67+
}
68+
```
69+
70+
You can then deploy the terraform
71+
72+
```bash
73+
terraform init
74+
terraform apply
75+
```
76+
77+
You can receive the webhook details by running:
78+
79+
```bash
80+
terraform output -raw webhook_secret
81+
```
82+
83+
Be-aware some shells will print some end of line character `%`.

Diff for: ‎examples/prebuilt/lambdas-download/main.tf

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
locals {
2+
version = "<REPLACE_BY_GITHUB_RELEASE_VERSION>"
3+
}
4+
5+
module "lambdas" {
6+
source = "../../../modules/download-lambda"
7+
lambdas = [
8+
{
9+
name = "webhook"
10+
tag = local.version
11+
},
12+
{
13+
name = "runners"
14+
tag = local.version
15+
},
16+
{
17+
name = "runner-binaries-syncer"
18+
tag = local.version
19+
}
20+
]
21+
}
22+
23+
output "files" {
24+
value = module.lambdas.files
25+
}

Diff for: ‎examples/prebuilt/main.tf

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
locals {
2+
environment = "prebuilt"
3+
aws_region = "eu-west-1"
4+
}
5+
6+
resource "random_password" "random" {
7+
length = 28
8+
}
9+
10+
data "aws_caller_identity" "current" {}
11+
12+
module "runners" {
13+
source = "../../"
14+
create_service_linked_role_spot = true
15+
aws_region = local.aws_region
16+
vpc_id = module.vpc.vpc_id
17+
subnet_ids = module.vpc.private_subnets
18+
19+
environment = local.environment
20+
21+
github_app = {
22+
key_base64 = var.github_app_key_base64
23+
id = var.github_app_id
24+
webhook_secret = random_password.random.result
25+
}
26+
27+
webhook_lambda_zip = "../../lambda_output/webhook.zip"
28+
runner_binaries_syncer_lambda_zip = "../../lambda_output/runner-binaries-syncer.zip"
29+
runners_lambda_zip = "../../lambda_output/runners.zip"
30+
31+
runner_extra_labels = "default,example"
32+
33+
# configure your pre-built AMI
34+
enabled_userdata = false
35+
ami_filter = { name = ["github-runner-amzn2-x86_64-2021*"] }
36+
ami_owners = [data.aws_caller_identity.current.account_id]
37+
38+
# enable access to the runners via SSM
39+
enable_ssm_on_runners = true
40+
41+
# override delay of events in seconds
42+
delay_webhook_event = 5
43+
44+
# override scaling down
45+
scale_down_schedule_expression = "cron(* * * * ? *)"
46+
}

Diff for: ‎examples/prebuilt/outputs.tf

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
output "runners" {
2+
value = {
3+
lambda_syncer_name = module.runners.binaries_syncer.lambda.function_name
4+
}
5+
}
6+
7+
output "webhook_endpoint" {
8+
value = module.runners.webhook.endpoint
9+
}
10+
11+
output "webhook_secret" {
12+
sensitive = true
13+
value = random_password.random.result
14+
}
15+

Diff for: ‎examples/prebuilt/providers.tf

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
provider "aws" {
2+
region = local.aws_region
3+
}

Diff for: ‎examples/prebuilt/variables.tf

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
2+
variable "github_app_key_base64" {}
3+
4+
variable "github_app_id" {}

Diff for: ‎examples/prebuilt/versions.tf

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
terraform {
2+
required_providers {
3+
aws = {
4+
source = "hashicorp/aws"
5+
version = ">= 3.27"
6+
}
7+
local = {
8+
source = "hashicorp/local"
9+
}
10+
random = {
11+
source = "hashicorp/random"
12+
}
13+
}
14+
required_version = ">= 0.14"
15+
}

Diff for: ‎examples/prebuilt/vpc.tf

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
module "vpc" {
2+
source = "git::https://github.com/philips-software/terraform-aws-vpc.git?ref=2.2.0"
3+
4+
environment = local.environment
5+
aws_region = local.aws_region
6+
create_private_hosted_zone = false
7+
}

0 commit comments

Comments
 (0)