Skip to content

DEVPROD-16438 - migrate perf.send to cedar_report endpoint #1679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+33 −2
Open

DEVPROD-16438 - migrate perf.send to cedar_report endpoint #1679

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3dffda
migrate perf.send to cedar_report endpoint
MAhmadShah Apr 11, 2025
64df760
PR feedback
MAhmadShah Apr 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 40 additions & 8 deletions .evergreen/.evg.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ functions:
. ./activate-authawsvenv.sh
python ./lib/aws_assign_instance_profile.py
fi

"run tests":
- command: shell.exec
type: test
Expand All @@ -277,9 +277,9 @@ functions:
AZUREKMS_KEY_NAME: ${testazurekms_keyname}
script: |
${PREPARE_SHELL}

. ${DRIVERS_TOOLS}/.evergreen/csfle/set-temp-creds.sh

export AWS_TEMP_ACCESS_KEY_ID=$CSFLE_AWS_TEMP_ACCESS_KEY_ID
export AWS_TEMP_SECRET_ACCESS_KEY=$CSFLE_AWS_TEMP_SECRET_ACCESS_KEY
export AWS_TEMP_SESSION_TOKEN=$CSFLE_AWS_TEMP_SESSION_TOKEN
Expand Down Expand Up @@ -375,7 +375,7 @@ functions:
STREAM_TYPE="netty" AUTH="${AUTH}" SSL="${SSL}" NETTY_SSL_PROVIDER="${NETTY_SSL_PROVIDER}" MONGODB_URI="${MONGODB_URI}" \
TOPOLOGY="${TOPOLOGY}" COMPRESSOR="${COMPRESSOR}" JAVA_VERSION="${JAVA_VERSION}" \
AZUREKMS_KEY_VAULT_ENDPOINT=${testazurekms_keyvaultendpoint} AZUREKMS_KEY_NAME=${testazurekms_keyname} \
.evergreen/run-tests.sh
.evergreen/run-tests.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few random line break / whitespace changes. Could we revert those to help preserve the git history on those lines?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for sure, it must be my IDE


"run plain auth test":
- command: shell.exec
Expand Down Expand Up @@ -683,11 +683,11 @@ functions:
script: |
${PREPARE_SHELL}
. ${DRIVERS_TOOLS}/.evergreen/csfle/set-temp-creds.sh

export AWS_TEMP_ACCESS_KEY_ID=$CSFLE_AWS_TEMP_ACCESS_KEY_ID
export AWS_TEMP_SECRET_ACCESS_KEY=$CSFLE_AWS_TEMP_SECRET_ACCESS_KEY
export AWS_TEMP_SESSION_TOKEN=$CSFLE_AWS_TEMP_SESSION_TOKEN

MONGODB_URI="${MONGODB_URI}" JAVA_VERSION="${JAVA_VERSION}" .evergreen/run-csfle-tests-with-mongocryptd.sh

"trace artifacts":
Expand Down Expand Up @@ -777,9 +777,41 @@ functions:
PROJECT_DIRECTORY=${PROJECT_DIRECTORY} .evergreen/run-perf-tests.sh

"send dashboard data":
- command: perf.send
- command: shell.exec
params:
file: src/results.json
script: |
if [ "${requester}" == "commit" ]; then
echo "is_mainline: true" >> expansion.yml
else
echo "is_mainline: false" >> expansion.yml
fi

echo "parsed_order_id: $(echo "${revision_order_id}" | awk -F'_' '{print $NF}')" >> expansion.yml
Comment on lines +782 to +789
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we move this script into a single shell.exec command at line 793? It looks like we might be able to skip expansions.update, which could help keep the CI file a bit more concise.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try this originally, but evergreen was not handling the variable assignment without them being written to the expansions unfortunately, the variables were not being retained after assignment (showing up as null when I recalled them).

- command: expansions.update
params:
file: expansion.yml
- command: shell.exec
params:
script: |
response=$(curl -s -w "\nHTTP_STATUS:%{http_code}" -X 'POST' \
"https://performance-monitoring-api.corp.mongodb.com/raw_perf_results/cedar_report?project=${project_id}&version=${version_id}&variant=${build_variant}&order=${parsed_order_id}&task_name=${task_name}&task_id=${task_id}&execution=${execution}&mainline=${is_mainline}" \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d @src/results.json)

http_status=$(echo "$response" | grep "HTTP_STATUS" | awk -F':' '{print $2}')
response_body=$(echo "$response" | sed '/HTTP_STATUS/d')

# We want to throw an error if the data was not successfully submitted
if [ "$http_status" -ne 200 ]; then
echo "Error: Received HTTP status $http_status"
echo "Response Body: $response_body"
exit 1
fi

echo "Response Body: $response_body"
echo "HTTP Status: $http_status"
add_expansions_to_env: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since all expansions appear to be available during task execution and via expansions.update, do we need this step?


"run graalvm native image app":
- command: shell.exec
Expand Down