Update CI to use OpenStack Bobcat & Ubuntu 22.04

Author: EmilienM

README.md

@@ -58,12 +58,12 @@ This provider's versions are able to install and manage the following versions o
 
 This provider's versions are able to install Kubernetes to the following versions of OpenStack:
 
-|                                    | Queens | Rocky | Stein | Train | Ussuri | Victoria | Wallaby | Xena | Yoga |
-|------------------------------------| ------ | ----- | ----- | ----- | ------ | -------- | ------- | ---- | ---- |
-| OpenStack Provider v1alpha5 (v0.6) | +      | +     | +     | +     | +      | ✓        | ✓       | ✓    | ★    |
-| OpenStack Provider v1alpha6 (v0.7) | +      | +     | +     | +     | +      | ✓        | ✓       | ✓    | ★    |
-| OpenStack Provider v1alpha7 (v0.9) |        | +     | +     | +     | +      | ✓        | ✓       | ✓    | ★    |
-| OpenStack Provider v1alpha8        |        | +     | +     | +     | +      | ✓        | ✓       | ✓    | ★    |
+|                                    | Queens | Rocky | Stein | Train | Ussuri | Victoria | Wallaby | Xena | Yoga | Bobcat |
+|------------------------------------| ------ | ----- | ----- | ----- | ------ | -------- | ------- | ---- | ---- | ------ |
+| OpenStack Provider v1alpha5 (v0.6) | +      | +     | +     | +     | +      | ✓        | ✓       | ✓    | ✓    | ★      |
+| OpenStack Provider v1alpha6 (v0.7) | +      | +     | +     | +     | +      | ✓        | ✓       | ✓    | ✓    | ★      |
+| OpenStack Provider v1alpha7 (v0.9) |        | +     | +     | +     | +      | ✓        | ✓       | ✓    | ✓    | ★      |
+| OpenStack Provider v1alpha8        |        | +     | +     | +     | +      | ✓        | ✓       | ✓    | ✓    | ★      |
 
 Test status:
 

docs/book/src/development/ci.md

@@ -28,7 +28,7 @@ The entry point for the creation of the test DevStack is `hack/ci/create_devstac
 
 ### DevStack OS
 
-In GCE, DevStack is installed on a community-maintained Ubuntu 20.04 LTS cloud image. The cloud-init config is also intended to work on CentOS 8, and this is known to work as of 2021-01-12. However, note that this is not regularly tested. See the comment in `hack/ci/gce-project.sh` for how to deploy on CentOS.
+In GCE, DevStack is installed on a community-maintained Ubuntu 22.04 LTS cloud image. The cloud-init config is also intended to work on CentOS 8, and this is known to work as of 2021-01-12. However, note that this is not regularly tested. See the comment in `hack/ci/gce-project.sh` for how to deploy on CentOS.
 
 It is convenient to the project to have a viable second OS option as it gives us an option to work around issues which only affect one or the other. This is most likely when enabling new DevStack features, but may also include infrastructure issues. Consequently, when making changes to cloud-init, try not to use features specific to Ubuntu or CentOS. DevStack already supports both operating systems, so we just need to be careful in our peripheral configuration, for example by using cloud-init's `packages` module rather than manually invoking `apt-get` or `yum`. Fortunately package names tend to be consistent across the two distributions.
 
@@ -40,13 +40,13 @@ We configure a 2 node DevStack. `controller` is running:
 * Nova: all services, including compute
 * Glance: all services
 * Octavia: all services
-* Neutron: all services with ML2/OVS, including L3 agent
+* Neutron: all services with ML2/OVN
 * Cinder: all services, including volume with default LVM/iSCSI backend
 
 `worker` is running:
 
 * Nova: compute only
-* Neutron: agent only (not L3 agent)
+* Neutron: OVN agents only
 * Cinder: volume only with default LVM/iSCSI backend
 
 `controller` is using the `n2-standard-16` machine type with 16 vCPUs and 64 GB RAM. `worker` is using the `n2-standard-8` machine type with 8 vCPUs and 32 GB RAM. Each job has a quota limit of 24 vCPUs.
@@ -66,6 +66,8 @@ In addition, DevStack will create a floating IP network using CIDR defined in `F
 
 The effect of this is that instances created on either `controller` or `worker` can get a floating ip from the `public` network. Traffic using this floating IP will be routed via `controller` and externally via NAT.
 
+We are configuring OVN to provide default DNS servers if a subnet is created without specifying DNS servers. This can be overridden in `OPENSTACK_DNS_NAMESERVERS`.
+
 ### Availability zones
 
 We are running `nova compute` and `cinder volume` on each of `controller` and `worker`. Each `nova compute` and `cinder volume` are configured to be in their own availability zone. The names of the availability zones are defined in `OPENSTACK_FAILURE_DOMAIN` and `OPENSTACK_FAILURE_DOMAIN_ALT` in `test/e2e/data/e2e_conf.yaml`, with the services running on `controller` being in `OPENSTACK_FAILURE_DOMAIN` and the services running on `worker` being in `OPENSTACK_FAILURE_DOMAIN_ALT`.

hack/ci/aws-project.sh

@@ -23,8 +23,8 @@ function cloud_init {
   AWS_ZONE=${AWS_ZONE:-"eu-central-1a"}
   # AMIs:
   # * capa-ami-ubuntu-20.04-1.20.4-00-1613898574 id: ami-0120656d38c206057
-  # * ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20210223 id: ami-0767046d1677be5a0
-  AWS_AMI=${AWS_AMI:-"ami-0767046d1677be5a0"}
+  # * ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-20231207 id: ami-05d47d29a4c2d19e1
+  AWS_AMI=${AWS_AMI:-"ami-05d47d29a4c2d19e1"}
   # Choose via: https://eu-central-1.console.aws.amazon.com/ec2/v2/home?region=eu-central-1#InstanceTypes:
   AWS_MACHINE_TYPE=${AWS_MACHINE_TYPE:-"c5.metal"}
   AWS_NETWORK_NAME=${AWS_NETWORK_NAME:-"${CLUSTER_NAME}-mynetwork"}

hack/ci/cloud-init/controller.yaml.tpl

@@ -12,9 +12,6 @@
     VERBOSE=True
     LOG_COLOR=True
 
-    # Neutron
-    enable_plugin neutron https://github.com/openstack/neutron stable/${OPENSTACK_RELEASE}
-
     # Octavia
     enable_plugin octavia https://github.com/openstack/octavia stable/${OPENSTACK_RELEASE}
     enable_plugin octavia-dashboard https://github.com/openstack/octavia-dashboard stable/${OPENSTACK_RELEASE}
@@ -35,9 +32,17 @@
     ENABLED_SERVICES+=,g-api
 
     # Neutron
-    ENABLED_SERVICES+=,neutron-api,neutron-agent,neutron-dhcp,neutron-l3,neutron-trunk
+    enable_plugin neutron https://github.com/openstack/neutron stable/${OPENSTACK_RELEASE}
+    ENABLED_SERVICES+=,q-svc,neutron-trunk,ovn-controller,ovs-vswitchd,ovn-northd,ovsdb-server,q-ovn-metadata-agent
+    
+    DISABLED_SERVICES=q-agt,q-dhcp,q-l3,q-meta,q-metering
+    PUBLIC_BRIDGE_MTU=${MTU}
+    ENABLE_CHASSIS_AS_GW="True"
+    OVN_DBS_LOG_LEVEL="dbg"
+    Q_ML2_PLUGIN_MECHANISM_DRIVERS="ovn,logger"
+    OVN_L3_CREATE_PUBLIC_NETWORK="True"
+    Q_AGENT="ovn"
 
-    ENABLED_SERVICES+=,neutron-metadata-agent,neutron-qos
     # Octavia
     ENABLED_SERVICES+=,octavia,o-api,o-cw,o-hm,o-hk,o-da
 
@@ -50,6 +55,7 @@
 
     # Additional services
     ENABLED_SERVICES+=${OPENSTACK_ADDITIONAL_SERVICES}
+    DISABLED_SERVICES+=${OPENSTACK_DISABLED_SERVICES}
 
     # Don't download default images, just our test images
     DOWNLOAD_DEFAULT_IMAGES=False
@@ -87,13 +93,18 @@
     [DEFAULT]
     storage_availability_zone = ${PRIMARY_AZ}
 
-    [[post-config|/$NEUTRON_CORE_PLUGIN_CONF]]
-    [ml2]
-    path_mtu = ${MTU}
-
     [[post-config|$NEUTRON_CONF]]
     [DEFAULT]
+    global_physnet_mtu = ${MTU}
     service_plugins = trunk,router
+
+    # The following are required for OVN to set default DNS when a subnet is
+    # created without specifying DNS servers.
+    # Not specifying these will result in the default DNS servers being set to
+    # 127.0.0.53 which might be problematic in some environments.
+    [[post-config|/$Q_PLUGIN_CONF_FILE]]
+    [ovn]
+    dns_servers = ${OPENSTACK_DNS_NAMESERVERS}
 - path: /tmp/register-worker.sh
   permissions: "0755"
   content: |

hack/ci/cloud-init/worker.yaml.tpl

@@ -25,15 +25,23 @@
     DATABASE_TYPE=mysql
     DATABASE_HOST=$SERVICE_HOST
 
-    # Neutron
-    enable_plugin neutron https://github.com/openstack/neutron stable/${OPENSTACK_RELEASE}
-
     # Nova
-    ENABLED_SERVICES=n-cpu,placement-client,c-vol,neutron-agent
+    ENABLED_SERVICES=n-cpu,placement-client,c-vol
     VOLUME_BACKING_FILE_SIZE=100G
 
+    # Neutron
+    enable_plugin neutron https://github.com/openstack/neutron stable/${OPENSTACK_RELEASE}
+    ENABLED_SERVICES+=,ovn-controller,ovs-vswitchd,ovsdb-server,q-fake,q-ovn-metadata-agent
+    DISABLED_SERVICES=q-svc,horizon,ovn-northd,q-agt,q-dhcp,q-l3,q-meta,q-metering,q-vpn
+    PUBLIC_BRIDGE_MTU=${MTU}
+    ENABLE_CHASSIS_AS_GW="False"
+    OVN_DBS_LOG_LEVEL="dbg"
+    Q_ML2_PLUGIN_MECHANISM_DRIVERS="ovn,logger"
+    Q_AGENT="ovn"
+
     # Additional services
     ENABLED_SERVICES+=${OPENSTACK_ADDITIONAL_SERVICES}
+    DISABLED_SERVICES+=${OPENSTACK_DISABLED_SERVICES}
 
     [[post-config|$NOVA_CONF]]
     [DEFAULT]
@@ -43,9 +51,9 @@
     [DEFAULT]
     storage_availability_zone = ${SECONDARY_AZ}
 
-    [[post-config|/$NEUTRON_CORE_PLUGIN_CONF]]
-    [ml2]
-    path_mtu = ${MTU}
+    [[post-config|$NEUTRON_CONF]]
+    [DEFAULT]
+    global_physnet_mtu = ${MTU}
 - path: /root/devstack.sh
   permissions: "0755"
   content: |

hack/ci/create_devstack.sh

@@ -31,13 +31,16 @@ source "${scriptdir}/${RESOURCE_TYPE}.sh"
 
 CLUSTER_NAME=${CLUSTER_NAME:-"capo-e2e"}
 
-OPENSTACK_RELEASE=${OPENSTACK_RELEASE:-"yoga"}
+OPENSTACK_RELEASE=${OPENSTACK_RELEASE:-"2023.2"}
 OPENSTACK_ENABLE_HORIZON=${OPENSTACK_ENABLE_HORIZON:-"false"}
 
 # Devstack will create a provider network using this range
 # We create a route to it with sshuttle
 FLOATING_RANGE=${FLOATING_RANGE:-"172.24.4.0/24"}
 
+# That will be the default DNS server for the Neutron subnets
+OPENSTACK_DNS_NAMESERVERS=${OPENSTACK_DNS_NAMESERVERS:-"8.8.8.8"}
+
 # Servers will be directly attached to the private network
 # We create a route to it with sshuttle
 PRIVATE_NETWORK_CIDR=${PRIVATE_NETWORK_CIDR:-"10.0.3.0/24"}
@@ -86,10 +89,6 @@ function ensure_openstack_client {
         # We explicitly pin to the stable branch version of openstackclient.
         curl -L https://releases.openstack.org/constraints/upper/${OPENSTACK_RELEASE} -o /tmp/openstack-constraints
 
-        # Hack for yoga only: wrapt <1.14 doesn't support python 3.11
-        [ "${OPENSTACK_RELEASE}" == "yoga" ] || exit 1 # Delete this hack
-        sed -i "s/^wrapt===1\.13.*/wrapt===1.14.1/" /tmp/openstack-constraints
-
         pip install -c /tmp/openstack-constraints \
                 python-openstackclient python-cinderclient \
                 python-glanceclient python-keystoneclient \
@@ -213,10 +212,11 @@ function create_devstack {
         HOST_IP="$ip" \
         CONTROLLER_IP="$CONTROLLER_IP" \
         FLOATING_RANGE="$FLOATING_RANGE" \
+	OPENSTACK_DNS_NAMESERVERS="$OPENSTACK_DNS_NAMESERVERS" \
         MTU="$(get_mtu)" \
         PRIMARY_AZ="$PRIMARY_AZ" SECONDARY_AZ="$SECONDARY_AZ" \
             envsubst '${SSH_PUBLIC_KEY} ${OPENSTACK_ADDITIONAL_SERVICES}
-                    ${OPENSTACK_RELEASE} ${HOST_IP} ${FLOATING_RANGE}
+                    ${OPENSTACK_RELEASE} ${HOST_IP} ${FLOATING_RANGE} ${OPENSTACK_DNS_NAMESERVERS}
                     ${CONTROLLER_IP} ${MTU} ${PRIMARY_AZ} ${SECONDARY_AZ}' \
                 < "./hack/ci/cloud-init/${tpl}.yaml.tpl" >> "$cloud_init"
     done

hack/ci/gce-project.sh

@@ -95,7 +95,7 @@ function create_vm {
         --zone "$GCP_ZONE" \
         --enable-nested-virtualization \
         --image-project ubuntu-os-cloud \
-        --image-family ubuntu-2004-lts \
+        --image-family ubuntu-2204-lts \
         --boot-disk-size 200G \
         --boot-disk-type pd-ssd \
         --can-ip-forward \

hack/ci/openstack.sh

@@ -30,7 +30,7 @@ function cloud_init {
     OPENSTACK_SUBNET_NAME=${OPENSTACK_SUBNET_NAME:-${CLUSTER_NAME}-subnet}
     OPENSTACK_SECGROUP_NAME=${OPENSTACK_SECGROUP_NAME:-${CLUSTER_NAME}-secgroup}
     OPENSTACK_ROUTER_NAME=${OPENSTACK_ROUTER_NAME:-${CLUSTER_NAME}-router}
-    OPENSTACK_IMAGE_NAME=${OPENSTACK_IMAGE_NAME:-ubuntu-2004-lts}
+    OPENSTACK_IMAGE_NAME=${OPENSTACK_IMAGE_NAME:-ubuntu-2204-lts}
 
     OPENSTACK_FLAVOR=${OPENSTACK_FLAVOR:-m1.xlarge}
     OPENSTACK_FLAVOR_controller=${OPENSTACK_FLAVOR_controller:-$OPENSTACK_FLAVOR}
@@ -88,9 +88,9 @@ function init_infrastructure() {
     # We don't tag the image with the cluster name as we expect it to be shared
     if ! imageid=$(openstack image show "$OPENSTACK_IMAGE_NAME" -f value -c id 2>/dev/null)
     then
-        curl -o /tmp/ubuntu-2004.qcow2 https://cloud-images.ubuntu.com/releases/focal/release/ubuntu-20.04-server-cloudimg-amd64.img
-        imageid=$(openstack image create --disk-format qcow2 --file /tmp/ubuntu-2004.qcow2 "$OPENSTACK_IMAGE_NAME" -f value -c id)
-        rm /tmp/ubuntu-2004.qcow2
+        curl -o /tmp/ubuntu-2204.qcow2 https://cloud-images.ubuntu.com/releases/jammy/release/ubuntu-22.04-server-cloudimg-amd64.img
+        imageid=$(openstack image create --disk-format qcow2 --file /tmp/ubuntu-2204.qcow2 "$OPENSTACK_IMAGE_NAME" -f value -c id)
+        rm /tmp/ubuntu-2204.qcow2
     fi
 }
 

scripts/ci-conformance.sh

@@ -47,7 +47,7 @@ trap cleanup EXIT
 
 apt-get update -y
 # Install requests module explicitly for HTTP calls.
-# libffi required for pip install cffi (yoga dependency)
+# libffi required for pip install cffi (bobcat dependency)
 apt-get install -y python3-requests libffi-dev
 rm -rf /var/lib/apt/lists/*
 

scripts/ci-e2e.sh

@@ -48,7 +48,7 @@ trap cleanup EXIT
 
 apt-get update -y
 # Install requests module explicitly for HTTP calls.
-# libffi required for pip install cffi (yoga dependency)
+# libffi required for pip install cffi (bobcat dependency)
 apt-get install -y python3-requests libffi-dev
 rm -rf /var/lib/apt/lists/*