✨ Add `flatcar-sysext` template to use regular Flatcar images #1776

tormath1 · 2023-12-07T11:55:14Z

What this PR does / why we need it: In this PR, we add a new template: flatcar-sysext. This template has two main benefits:

consume upstream images (directly from Flatcar release servers without going through the image-builder requirement)
automatically download and install Kubernetes patch releases (disabled by default for now to mimic the current CAPI/CAPO behavior)

Special notes for your reviewer: This can solve the question of providing CAPO image for Flatcar - as a user can directly consume the upstream image.

Some resources:

CAPO demo during the office hours: https://www.youtube.com/watch?v=RwKl-PtSjjw
For more context, here's a demo I did a few months ago during the Flatcar office hours: https://www.youtube.com/watch?v=omppbFNxSDU&t=210s
https://github.com/flatcar/sysext-bakery/ (where the sysext images are produced and stored)
https://www.flatcar.org/docs/latest/provisioning/sysext/
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysupdate.html

TODOs:

squashed commits
if necessary:
- includes documentation
- adds unit tests

/hold

netlify · 2023-12-07T11:55:20Z

✅ Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

Name	Link
🔨 Latest commit	`5a39a20`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-openstack/deploys/65802789086f9700076eab9f
😎 Deploy Preview	https://deploy-preview-1776--kubernetes-sigs-cluster-api-openstack.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

tormath1 · 2023-12-07T13:49:35Z

/retest

(cloud-init is failing to provision the devstack instance)

mdbooth

I've left some comments, but as you're our flatcar maintainer I'm happy for you to merge this whenever you see fit.

mdbooth · 2023-12-12T09:11:27Z

docs/book/src/clusteropenstack/configuration.md

-* Build the image with the [image-builder](https://image-builder.sigs.k8s.io/capi/providers/openstack.html): `make OEM_ID=openstack build-qemu-flatcar`
-* Export the name of the uploaded image: `export OPENSTACK_FLATCAR_IMAGE_NAME=flatcar-stable-3374.2.5-kube-v1.25.6`
-* When generating the cluster configuration, use the following Cluster API [flavor](https://cluster-api.sigs.k8s.io/clusterctl/commands/generate-cluster.html?#flavors): `--flavor flatcar` (_NOTE_: Don't forget to refer to the [external-cloud-provider](https://cluster-api-openstack.sigs.k8s.io/topics/external-cloud-provider.html) section)
+Flatcar comes in two [flavor][flavor] variant:


nit: variants.

mdbooth · 2023-12-12T09:22:21Z

kustomize/v1alpha7/flatcar-sysext/patch-flatcar.yaml

+          # Fixme(lentzi90): This is here just to override the value set in the default
+          # kustomization. It will be replaced with a value that works for flatcar in
+          # https://github.com/kubernetes-sigs/cluster-api-provider-openstack/pull/1564
+          provider-id: null


Are we able to update this now?

See: #1564 (comment)

mdbooth · 2023-12-12T09:49:59Z

kustomize/v1alpha7/flatcar-sysext/patch-flatcar.yaml

+                      ExecStartPost=/usr/bin/sh -c "if ! cmp --silent /tmp/kubernetes /tmp/kubernetes-new; then touch /run/reboot-required; fi"
+              - name: update-engine.service
+                # Set this to 'false' if you want to enable Flatcar auto-update
+                masked: true


Do you want a config variable for this? Maybe ${FLATCAR_DISABLE_AUTO_UPDATE:=true}?

If you do I'd try to minimise the number of config variables you add here, and below as they're presumably highly correlated.

Oh thanks. I did not know it was possible to have this kind of optional variable:

clusterctl generate cluster ... --list-variables ... Optional Variables: - FLATCAR_DISABLE_AUTO_UPDATE (defaults to "true")

NOTE: It's fine to only have the update-engine.service to be customizable as the usage of locksmithd is not recommended to manage the reboot of the instance (kured is a better choice).

mdbooth · 2023-12-12T09:53:39Z

kustomize/v1alpha7/flatcar-sysext/patch-flatcar.yaml

+      - mv /etc/kubeadm.yml.tmp /etc/kubeadm.yml
+---
+apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+kind: KubeadmConfigTemplate


I feel like we should be able to DRY this.

k8s-ci-robot · 2023-12-12T09:56:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mdbooth, tormath1

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mdbooth]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This template allows to use a plain Flatcar image from the Flatcar release server without building with the image-builder. This template will consume Kubernetes systemd-sysext image from the flatcar/sysext-bakery release artifacts. This template will emit a /run/reboot-required file flag when a new Kubernetes release has been downloaded and that we need to coordinate the reboot of the node using Kured for example. Signed-off-by: Mathieu Tortuyaux <[email protected]> Co-authored-by: Kai Lüke <[email protected]>

This test consumes the flatcar-sysext template to deploy a workload cluster. It uses a plain Flatcar OpenStack image from the release servers as it does not need to rely on the image-builder built image. Signed-off-by: Mathieu Tortuyaux <[email protected]>

Signed-off-by: Mathieu Tortuyaux <[email protected]> Co-authored-by: Kai Lüke <[email protected]>

tormath1 · 2023-12-18T14:18:07Z

/retest

First fail was a devstack issue - now it's the default scenario:

Workload cluster (default) [It] It should be creatable and deletable

tormath1 · 2023-12-18T16:46:27Z

/unhold

EmilienM · 2023-12-19T13:41:44Z

hack/ci/cloud-init/controller.yaml.tpl

@@ -64,7 +64,8 @@
    IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/cirros/2022-12-05/cirros-0.6.1-x86_64-disk.img,"
    IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/ubuntu/2023-09-29/ubuntu-2204-kube-v1.27.2.img,"
    IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/ubuntu/2023-09-29/ubuntu-2204-kube-v1.28.2.img,"
-    IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/flatcar/flatcar-stable-3602.2.0-kube-v1.28.2.img"
+    IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/flatcar/flatcar-stable-3602.2.0-kube-v1.28.2.img,"
+    IMAGE_URLS+="https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_openstack_image.img"


I don't have enough experience to tell if that's ok to include an image from another endpoint than other images in the list. I'm sure there is a good reason to have them stored on google vs others. I'll let the maintainers to comment on this one.

Thanks @EmilienM for your review. This specific image comes directly from Flatcar releases servers without passing through the image-builder.
Of course, we can mirror the image to the GCS bucket but it would lost a bit the purpose of this new template.

mdbooth · 2024-01-02T09:51:19Z

/lgtm

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 7, 2023

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 7, 2023

k8s-ci-robot requested review from dulek and seanschneeweiss December 7, 2023 11:55

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Dec 7, 2023

tormath1 force-pushed the tormath1/sysext branch 2 times, most recently from b63aa28 to 0dd9a8b Compare December 7, 2023 16:28

mdbooth approved these changes Dec 12, 2023

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 12, 2023

ader1990 mentioned this pull request Dec 13, 2023

Add Flatcar systemd-sysext how to tinkerbell/cluster-api-provider-tinkerbell#333

Open

tormath1 force-pushed the tormath1/sysext branch from 0dd9a8b to f50a95b Compare December 18, 2023 09:14

tormath1 and others added 3 commits December 18, 2023 12:05

test: add flatcar-sysext test

d85b5fc

This test consumes the flatcar-sysext template to deploy a workload cluster. It uses a plain Flatcar OpenStack image from the release servers as it does not need to rely on the image-builder built image. Signed-off-by: Mathieu Tortuyaux <[email protected]>

doc: mention Flatcar sysext template

5a39a20

Signed-off-by: Mathieu Tortuyaux <[email protected]> Co-authored-by: Kai Lüke <[email protected]>

tormath1 force-pushed the tormath1/sysext branch from f50a95b to 5a39a20 Compare December 18, 2023 11:05

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 18, 2023

EmilienM reviewed Dec 19, 2023

View reviewed changes

k8s-ci-robot assigned mdbooth Jan 2, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 2, 2024

k8s-ci-robot merged commit 7edf6a1 into kubernetes-sigs:main Jan 2, 2024

tormath1 deleted the tormath1/sysext branch March 12, 2024 12:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

✨ Add `flatcar-sysext` template to use regular Flatcar images #1776

✨ Add `flatcar-sysext` template to use regular Flatcar images #1776

Uh oh!

tormath1 commented Dec 7, 2023 •

edited

Loading

Uh oh!

netlify bot commented Dec 7, 2023 •

edited

Loading

Uh oh!

tormath1 commented Dec 7, 2023

Uh oh!

mdbooth left a comment

Uh oh!

mdbooth Dec 12, 2023

Uh oh!

mdbooth Dec 12, 2023

Uh oh!

tormath1 Dec 19, 2023

Uh oh!

mdbooth Dec 12, 2023

Uh oh!

tormath1 Dec 18, 2023 •

edited

Loading

Uh oh!

mdbooth Dec 12, 2023

Uh oh!

k8s-ci-robot commented Dec 12, 2023

Uh oh!

tormath1 commented Dec 18, 2023

Uh oh!

tormath1 commented Dec 18, 2023

Uh oh!

EmilienM Dec 19, 2023

Uh oh!

tormath1 Dec 19, 2023

Uh oh!

mdbooth commented Jan 2, 2024

Uh oh!

Uh oh!

✨ Add flatcar-sysext template to use regular Flatcar images #1776

✨ Add flatcar-sysext template to use regular Flatcar images #1776

Uh oh!

Conversation

tormath1 commented Dec 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Dec 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

Uh oh!

tormath1 commented Dec 7, 2023

Uh oh!

mdbooth left a comment

Choose a reason for hiding this comment

Uh oh!

mdbooth Dec 12, 2023

Choose a reason for hiding this comment

Uh oh!

mdbooth Dec 12, 2023

Choose a reason for hiding this comment

Uh oh!

tormath1 Dec 19, 2023

Choose a reason for hiding this comment

Uh oh!

mdbooth Dec 12, 2023

Choose a reason for hiding this comment

Uh oh!

tormath1 Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdbooth Dec 12, 2023

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Dec 12, 2023

Uh oh!

tormath1 commented Dec 18, 2023

Uh oh!

tormath1 commented Dec 18, 2023

Uh oh!

EmilienM Dec 19, 2023

Choose a reason for hiding this comment

Uh oh!

tormath1 Dec 19, 2023

Choose a reason for hiding this comment

Uh oh!

mdbooth commented Jan 2, 2024

Uh oh!

Uh oh!

✨ Add `flatcar-sysext` template to use regular Flatcar images #1776

✨ Add `flatcar-sysext` template to use regular Flatcar images #1776

tormath1 commented Dec 7, 2023 •

edited

Loading

netlify bot commented Dec 7, 2023 •

edited

Loading

tormath1 Dec 18, 2023 •

edited

Loading