Add project support #356

vishesh92 · 2024-05-09T07:06:43Z

Issue #, if available:
Fixes #217
Description of changes:
Allow specifying project name along with account & domains in CloudStackClusterSpec.

Testing performed: Created and tested an env in default project and a new test project.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

k8s-ci-robot · 2024-05-09T07:06:50Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vishesh92

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vishesh92]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

netlify · 2024-05-09T07:06:59Z

✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!

Name	Link
🔨 Latest commit	`28faaac`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-cloudstack/deploys/664cfc758711f20008778d06
😎 Deploy Preview	https://deploy-preview-356--kubernetes-sigs-cluster-api-cloudstack.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

vishesh92 · 2024-05-09T07:07:17Z

/ok-to-test

rohityadavcloud · 2024-05-20T07:02:21Z

@weizhouapache @g-gaston @hrak pl review and advise, this attempts to solve a long-standing issue that users are not able to use CAPC in their projects.

g-gaston

Thanks for the change!

I would like to discuss a bit the UX (the implementation looks sound). I was expecting to see the projectID field in the API (talking about CAPD CRDs).

It seems to me like this is a config that the user might want specify per cluster, at leats that's what I extract from the original issue. It feels like a cluster attribute (create all resources for this cluster in this project). This seems to point to the [CloudStackCluster](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api-provider-cloudstack/infrastructure.cluster.x-k8s.io/CloudStackCluster/[email protected]#spec-failureDomains-account). And that intuition gets re-enforced by the presence of account in the CloudStackCluster (I think it used to be at the top level and it's now part of the failure domain).

So, would it make sense to move this field from the credentials secret to the CloudStackCluster object? Are there technical reasons why this is not possible?

weizhouapache · 2024-05-20T17:07:33Z

So, would it make sense to move this field from the credentials secret to the CloudStackCluster object? Are there technical reasons why this is not possible?

It makes sense to me

rohityadavcloud · 2024-05-20T17:30:05Z

@g-gaston I had the same thought, but @vishesh92 advised to me that since in the credentials we don't specify account ID we can keep project ID there and it would require lesser refactorings. Ideally all configuration params including projectID should be part of CloudStackCluster

@vishesh92 could you proceed to address the review comments, and move projectID to cloudstack cluster object?

weizhouapache · 2024-05-20T18:03:05Z

@vishesh92
Can you also consider the project resource limitations?

weizhouapache

thanks for the update @vishesh92

Is it possible to add a e2e test for project, similar as test/e2e/data/infrastructure-cloudstack/v1beta2/cluster-template-subdomain/cloudstack-cluster.yaml ?

pkg/cloud/user_credentials.go

docs/book/src/getting-started.md

pkg/cloud/user_credentials.go

blueorangutan · 2024-05-21T13:31:53Z

Test Results : (tid-404)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr356-sl-404.zip



Summarizing 2 Failures:
  [FAIL] When testing resource cleanup [AfterEach] Should create a new network when the specified network does not exist
  /jenkins/workspace/capc-e2e-new/test/e2e/resource_cleanup.go:101
  [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
  /jenkins/workspace/capc-e2e-new/test/e2e/common.go:332

Ran 28 of 29 Specs in 9906.986 seconds
FAIL! -- 26 Passed | 2 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (9906.99s)
FAIL

weizhouapache · 2024-05-21T14:49:39Z

/run-e2e -c 4.19

blueorangutan · 2024-05-21T14:50:02Z

@weizhouapache a jenkins job has been kicked to run test with following paramaters:

kubernetes version: 1.27.2
CloudStack version: 4.19
hypervisor: kvm
template: ubuntu-2004-kube
Kubernetes upgrade from: 1.26.5 to 1.27.2

blueorangutan · 2024-05-21T16:12:58Z

Setting up environment failed

blueorangutan · 2024-05-21T16:58:11Z

Tests were aborted.

blueorangutan · 2024-05-21T16:58:37Z

Setting up environment failed

blueorangutan · 2024-05-21T16:59:06Z

Setting up environment failed

blueorangutan · 2024-05-22T10:00:47Z

Test Results : (tid-418)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr356-sl-418.zip



Summarizing 4 Failures:
  [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
  /root/go/pkg/mod/sigs.k8s.io/cluster-api/[email protected]/framework/machinedeployment_helpers.go:127
  [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
  /jenkins/workspace/capc-e2e-new/test/e2e/common.go:348
  [FAIL] When testing project [AfterEach] Should create a cluster in a project
  /jenkins/workspace/capc-e2e-new/test/e2e/project.go:103
  [TIMEDOUT] When testing app deployment to the workload cluster with network interruption [ToxiProxy] [It] Should be able to create a cluster despite a network interruption during that process
  /jenkins/workspace/capc-e2e-new/test/e2e/network_interruption_toxi.go:77

Ran 29 of 30 Specs in 10818.560 seconds
FAIL! - Suite Timeout Elapsed -- 25 Passed | 4 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (10818.56s)
FAIL

vishesh92 · 2024-05-23T06:54:12Z

@g-gaston I have made the changes and moved the project to CloudStackCluster. Please review again.

blueorangutan · 2024-05-23T10:16:39Z

Test Results : (tid-424)
Environment: kvm Rocky8(x3), Advanced Networking with Management Server Rocky8
Kubernetes Version: v1.27.2
Kubernetes Version upgrade from: v1.26.5
Kubernetes Version upgrade to: v1.27.2
CloudStack Version: 4.19
Template: ubuntu-2004-kube
E2E Test Run Logs: https://github.com/blueorangutan/capc-prs/releases/download/capc-pr-ci-cd/capc-e2e-artifacts-pr356-sl-424.zip



Summarizing 3 Failures:
  [FAIL] When testing affinity group [It] Should have host affinity group when affinity is pro
  /jenkins/workspace/capc-e2e-new/test/e2e/common.go:348
  [FAIL] When testing Kubernetes version upgrades [It] Should successfully upgrade kubernetes versions when there is a change in relevant fields
  /root/go/pkg/mod/sigs.k8s.io/cluster-api/[email protected]/framework/machinedeployment_helpers.go:127
  [TIMEDOUT] When testing MachineDeployment rolling upgrades [It] Should successfully upgrade Machines upon changes in relevant MachineDeployment fields
  /jenkins/workspace/capc-e2e-new/test/e2e/md_rollout.go:60

Ran 29 of 30 Specs in 10817.946 seconds
FAIL! - Suite Timeout Elapsed -- 26 Passed | 3 Failed | 0 Pending | 1 Skipped
--- FAIL: TestE2E (10817.95s)
FAIL

weizhouapache · 2024-05-28T12:38:52Z

/lgtm

g-gaston · 2024-05-30T19:59:55Z

Sorry I didn't get to review, high level lgtm, thanks again!

vishesh92 added 3 commits May 8, 2024 22:06

Add support for project

133f7cc

Fix tests

8129de8

Update docs

64d390e

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 9, 2024

k8s-ci-robot requested review from jweite-amazon and weizhouapache May 9, 2024 07:06

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 9, 2024

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 9, 2024

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label May 9, 2024

rohityadavcloud added this to the v0.5.0 milestone May 17, 2024

rohityadavcloud requested review from hrak and g-gaston May 20, 2024 07:00

rohityadavcloud added the release:must-have label May 20, 2024

rohityadavcloud assigned vishesh92 May 20, 2024

g-gaston reviewed May 20, 2024

View reviewed changes

vishesh92 added 2 commits May 21, 2024 14:49

Move project to CloudStackCluster Spec

659ff9a

Add project resource limit checks

b61ae4c

vishesh92 force-pushed the add-project-support branch from cc95d4a to b61ae4c Compare May 21, 2024 09:46

fixup unit tests

5eea6e8

weizhouapache reviewed May 21, 2024

View reviewed changes

pkg/cloud/user_credentials.go Outdated Show resolved Hide resolved

pkg/cloud/user_credentials.go Show resolved Hide resolved

Fixup

eba72cf

vishesh92 force-pushed the add-project-support branch from f719a1e to eba72cf Compare May 21, 2024 12:26

weizhouapache reviewed May 21, 2024

View reviewed changes

docs/book/src/getting-started.md Outdated Show resolved Hide resolved

pkg/cloud/user_credentials.go Show resolved Hide resolved

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 21, 2024

vishesh92 force-pushed the add-project-support branch 2 times, most recently from a5c90b4 to 87a8e78 Compare May 21, 2024 14:01

vishesh92 force-pushed the add-project-support branch 2 times, most recently from 21ce310 to 37a3996 Compare May 21, 2024 16:49

Add e2e test for project

28faaac

vishesh92 force-pushed the add-project-support branch from 37a3996 to 28faaac Compare May 21, 2024 19:56

weizhouapache mentioned this pull request May 22, 2024

VMs with same host-affinity group are deployed to different hosts apache/cloudstack#9110

Closed

kubernetes-sigs deleted a comment from blueorangutan May 23, 2024

k8s-ci-robot assigned weizhouapache May 28, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 28, 2024

k8s-ci-robot merged commit 514e4c3 into kubernetes-sigs:main May 28, 2024
10 checks passed

vishesh92 deleted the add-project-support branch May 29, 2024 05:59

sp1999 mentioned this pull request Sep 7, 2024

Bump kubernetes-sigs/cluster-api-provider-cloudstack to latest release aws/eks-anywhere-build-tooling#3409

Merged

Add project support #356

Add project support #356

Uh oh!

Conversation

vishesh92 commented May 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented May 9, 2024

Uh oh!

netlify bot commented May 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!

Uh oh!

vishesh92 commented May 9, 2024

Uh oh!

rohityadavcloud commented May 20, 2024

Uh oh!

g-gaston left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented May 20, 2024

Uh oh!

rohityadavcloud commented May 20, 2024

Uh oh!

weizhouapache commented May 20, 2024

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

weizhouapache commented May 21, 2024

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

blueorangutan commented May 21, 2024

Uh oh!

blueorangutan commented May 22, 2024

Uh oh!

vishesh92 commented May 23, 2024

Uh oh!

blueorangutan commented May 23, 2024

Uh oh!

weizhouapache commented May 28, 2024

Uh oh!

Uh oh!

g-gaston commented May 30, 2024

Uh oh!

Uh oh!

vishesh92 commented May 9, 2024 •

edited

Loading

netlify bot commented May 9, 2024 •

edited

Loading