🌱 Add prometheus to e2e workflow #1928
Conversation
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1928 +/- ##
==========================================
- Coverage 69.17% 69.11% -0.06%
==========================================
Files 79 79
Lines 7027 7027
==========================================
- Hits 4861 4857 -4
- Misses 1885 1888 +3
- Partials 281 282 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
So, this adds a new e2e that isn't run downstream? There's no workflow to run it in upstream CI? |
Yes - it currently only adds prometheus to the environment when you run the new e2e target, so I've kept the workflow the same for now. If you'd prefer I'm happy to make that change here - it doesn't add much overhead to the e2e at all. |
dtfranz
force-pushed
the
prometheus-e2e
branch
2 times, most recently
from
6c32ce6 to
b51c814
Compare
Makefile
Outdated
| prometheus: LATEST := $(shell curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name) | ||
| prometheus: ## Deploy Prometheus into 'prometheus' namespace | ||
| trap 'echo "Cleaning up $(TMPDIR)"; rm -rf "$(TMPDIR)"' EXIT; \ | ||
| kubectl create ns $(PROMETHEUS_NAMESPACE); \ | ||
| curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$(LATEST)/kustomization.yaml" > "$(TMPDIR)/kustomization.yaml" ; \ | ||
| curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$(LATEST)/bundle.yaml" > "$(TMPDIR)/bundle.yaml" ; \ | ||
| (cd $(TMPDIR) && $(KUSTOMIZE) edit set namespace $(PROMETHEUS_NAMESPACE)) && kubectl create -k "$(TMPDIR)" ; \ | ||
| kubectl wait --for=condition=Ready pods -n $(PROMETHEUS_NAMESPACE) -l app.kubernetes.io/name=prometheus-operator |
There was a problem hiding this comment.
Did you consider adding the prometheus setup/teardown into the e2e's test_main.go? https://github.com/openshift/operator-framework-operator-controller/blob/355dcf40d5fcb4d9e93ab312a376c4e80d46ba76/test/e2e/e2e_suite_test.go#L31-L40
That would keep the Makefile a bit cleaner, and it would be a bit easier to deal with the tmpdir cleanup.
There was a problem hiding this comment.
I think this would go against our convention of setting up and tearing down everything in the Makefile. If people want to make changes to the e2e environment, they're going to be looking for things in here, and not in the go test files IMO.
Wouldn't that also add prometheus to every test run? I was hoping not to do that. I assume there's probably ways to turn it off/on in the go test calls, but then that adds more complication, and then we'll need to add additional targets in the Makefile for that too.
If your goal is just to keep the Makefile clean, I would be happy to compromise by adding a script instead and calling it from the Makefile. It's mainly just that putting it into the go test files doesn't feel right to me.
There was a problem hiding this comment.
I think this would go against our convention of setting up and tearing down everything in the Makefile. If people want to make changes to the e2e environment, they're going to be looking for things in here, and not in the go test files IMO.
That's fair according to how everything else works. If we do want to reorganize and move setup/teardown out of Makefile, that does feel like a separate PR.
Wouldn't that also add prometheus to every test run? I was hoping not to do that. I assume there's probably ways to turn it off/on in the go test calls, but then that adds more complication, and then we'll need to add additional targets in the Makefile for that too.
That could be managed with an environment variable. However, once we get to the point where prometheus and alert manager are actually used in tests, I assume we'll want prometheus unconditionally enabled.
If your goal is just to keep the Makefile clean, I would be happy to compromise by adding a script instead and calling it from the Makefile. It's mainly just that putting it into the go test files doesn't feel right to me.
No need for a script for this PR. If folks are worried about the Makefile sprawl, let's tackle that separately.
|
Finally getting back to this. @joelanford @tmshort Do you still have any remaining concerns? Since you last looked, I've updated the Makefile script to use a set prometheus version instead of latest. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: dtfranz <[email protected]>
Description
Closes #1902
Adds an additional e2e Makefile target which runs the e2e with prometheus deployed. This will allow us to check for alerts and potential performance degradation after e2e runs.
Starting out with just the minimal install as opposed to the entire
kube-prometheusstack, which has far more than we need right now.Reviewer Checklist