Add support of macrobenchmark result analysis in fireci #4323

yifanyang · 2022-11-14T19:58:17Z

#4285 was reverted in #4312 as it broke the release process (#4285 requires python 3.10, release infra supports only 3.8).

This pull request adds #4285 back:

re-implement to be compatible with python 3.8
add type linting with mypy to CI checks

Needs to submit together with:

https://team-review.git.corp.google.com/c/firebase-android-core/fireescape-release-plumbing/+/1644031

google-oss-bot · 2022-11-14T19:58:47Z

	1 Warning
⚠️	Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

google-oss-bot · 2022-11-14T20:21:44Z

Size Report ¹

Affected Products

base
Type Base (c1adf5b) Merge (d76c438) Diff
apk (aggressive) ? 8.39 kB ? (?)
apk (release) ? 8.65 kB ? (?)
firebase-abt
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 13.8 kB ? (?)
apk (aggressive) ? 85.8 kB ? (?)
apk (release) ? 688 kB ? (?)
firebase-annotations
Type Base (c1adf5b) Merge (d76c438) Diff
apk (aggressive) ? 8.39 kB ? (?)
apk (release) ? 8.89 kB ? (?)
firebase-appcheck
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 37.6 kB ? (?)
apk (aggressive) ? 332 kB ? (?)
apk (release) ? 978 kB ? (?)
firebase-appcheck-debug
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 11.6 kB ? (?)
apk (aggressive) ? 332 kB ? (?)
apk (release) ? 980 kB ? (?)
firebase-appcheck-debug-testing
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.26 kB ? (?)
apk (aggressive) ? 334 kB ? (?)
apk (release) ? 1.02 MB ? (?)
firebase-appcheck-interop
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.02 kB ? (?)
apk (aggressive) ? 329 kB ? (?)
apk (release) ? 966 kB ? (?)
firebase-appcheck-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.98 kB ? (?)
apk (aggressive) ? 356 kB ? (?)
apk (release) ? 1.92 MB ? (?)
firebase-appcheck-playintegrity
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 10.1 kB ? (?)
apk (aggressive) ? 332 kB ? (?)
apk (release) ? 991 kB ? (?)
firebase-appcheck-safetynet
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 12.1 kB ? (?)
apk (aggressive) ? 332 kB ? (?)
apk (release) ? 992 kB ? (?)
firebase-appdistribution
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 140 kB ? (?)
apk (aggressive) ? 777 kB ? (?)
apk (release) ? 2.00 MB ? (?)
firebase-appdistribution-api
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 14.2 kB ? (?)
apk (aggressive) ? 85.8 kB ? (?)
apk (release) ? 689 kB ? (?)
firebase-appdistribution-api-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.83 kB ? (?)
apk (aggressive) ? 110 kB ? (?)
apk (release) ? 1.63 MB ? (?)
firebase-common
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 50.1 kB ? (?)
apk (aggressive) ? 85.1 kB ? (?)
apk (release) ? 683 kB ? (?)
firebase-common-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.02 kB ? (?)
apk (aggressive) ? 109 kB ? (?)
apk (release) ? 1.62 MB ? (?)
firebase-components
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 42.8 kB ? (?)
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 31.9 kB ? (?)
firebase-config
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 63.0 kB ? (?)
apk (aggressive) ? 94.4 kB ? (?)
apk (release) ? 736 kB ? (?)
firebase-config-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.58 kB ? (?)
apk (aggressive) ? 119 kB ? (?)
apk (release) ? 1.67 MB ? (?)
firebase-crashlytics
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 341 kB ? (?)
apk (aggressive) ? 217 kB ? (?)
apk (release) ? 896 kB ? (?)
firebase-crashlytics-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.57 kB ? (?)
apk (aggressive) ? 237 kB ? (?)
apk (release) ? 1.83 MB ? (?)

`firebase-crashlytics-ndk`

Type	Base (`c1adf5b`)	Merge (`d76c438`)	Diff
aar	?	1.68 MB	? (?)
apk (aggressive / arm64-v8a)	?	1.34 MB	? (?)
apk (aggressive / armeabi-v7a)	?	846 kB	? (?)
apk (aggressive / x86)	?	1.38 MB	? (?)
apk (aggressive / x86_64)	?	1.45 MB	? (?)
apk (release / arm64-v8a)	?	2.02 MB	? (?)
apk (release / armeabi-v7a)	?	1.52 MB	? (?)
apk (release / x86)	?	2.06 MB	? (?)
apk (release / x86_64)	?	2.12 MB	? (?)

firebase-database
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 488 kB ? (?)
apk (aggressive) ? 330 kB ? (?)
apk (release) ? 1.14 MB ? (?)
firebase-database-collection
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 33.6 kB ? (?)
apk (aggressive) ? 312 kB ? (?)
apk (release) ? 941 kB ? (?)
firebase-database-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 22.5 kB ? (?)
apk (aggressive) ? 354 kB ? (?)
apk (release) ? 2.08 MB ? (?)
firebase-datatransport
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 4.94 kB ? (?)
apk (aggressive) ? 136 kB ? (?)
apk (release) ? 771 kB ? (?)
firebase-decoders-json
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 7.67 kB ? (?)
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 15.3 kB ? (?)
firebase-dynamic-links
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 54.6 kB ? (?)
apk (aggressive) ? 331 kB ? (?)
apk (release) ? 979 kB ? (?)
firebase-dynamic-links-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 8.28 kB ? (?)
apk (aggressive) ? 355 kB ? (?)
apk (release) ? 1.92 MB ? (?)
firebase-dynamic-module-support
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.67 kB ? (?)
apk (aggressive) ? 94.3 kB ? (?)
apk (release) ? 735 kB ? (?)
firebase-encoders
Type Base (c1adf5b) Merge (d76c438) Diff
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 15.3 kB ? (?)
firebase-encoders-json
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 10.7 kB ? (?)
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 20.1 kB ? (?)
firebase-encoders-proto
Type Base (c1adf5b) Merge (d76c438) Diff
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 21.6 kB ? (?)
firebase-encoders-reflective
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 7.80 kB ? (?)
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 22.2 kB ? (?)
firebase-firestore
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 1.31 MB ? (?)
apk (aggressive) ? 492 kB ? (?)
apk (release) ? 3.35 MB ? (?)
firebase-firestore-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 13.5 kB ? (?)
apk (aggressive) ? 512 kB ? (?)
apk (release) ? 4.29 MB ? (?)
firebase-functions
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 25.8 kB ? (?)
apk (aggressive) ? 375 kB ? (?)
apk (release) ? 1.23 MB ? (?)
firebase-functions-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.69 kB ? (?)
apk (aggressive) ? 395 kB ? (?)
apk (release) ? 2.17 MB ? (?)
firebase-inappmessaging
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 470 kB ? (?)
apk (aggressive) ? 673 kB ? (?)
apk (release) ? 3.38 MB ? (?)
firebase-inappmessaging-display
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 201 kB ? (?)
apk (aggressive) ? 1.51 MB ? (?)
apk (release) ? 4.64 MB ? (?)
firebase-inappmessaging-display-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 17.6 kB ? (?)
apk (aggressive) ? 1.53 MB ? (?)
apk (release) ? 5.58 MB ? (?)
firebase-inappmessaging-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.03 kB ? (?)
apk (aggressive) ? 694 kB ? (?)
apk (release) ? 4.31 MB ? (?)
firebase-installations
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 54.9 kB ? (?)
apk (aggressive) ? 86.5 kB ? (?)
apk (release) ? 706 kB ? (?)
firebase-installations-interop
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 8.05 kB ? (?)
apk (aggressive) ? 65.0 kB ? (?)
apk (release) ? 651 kB ? (?)
firebase-installations-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 5.00 kB ? (?)
apk (aggressive) ? 111 kB ? (?)
apk (release) ? 1.64 MB ? (?)
firebase-messaging
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 142 kB ? (?)
apk (aggressive) ? 440 kB ? (?)
apk (release) ? 1.14 MB ? (?)
firebase-messaging-directboot
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 8.75 kB ? (?)
apk (aggressive) ? 440 kB ? (?)
apk (release) ? 1.14 MB ? (?)
firebase-messaging-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.10 kB ? (?)
apk (aggressive) ? 460 kB ? (?)
apk (release) ? 2.08 MB ? (?)
firebase-ml-modeldownloader
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 87.5 kB ? (?)
apk (aggressive) ? 143 kB ? (?)
apk (release) ? 823 kB ? (?)
firebase-ml-modeldownloader-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.10 kB ? (?)
apk (aggressive) ? 163 kB ? (?)
apk (release) ? 1.76 MB ? (?)
firebase-perf
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 313 kB ? (?)
apk (aggressive) ? 1.03 MB ? (?)
apk (release) ? 2.48 MB ? (?)
firebase-perf-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 16.0 kB ? (?)
apk (aggressive) ? 1.05 MB ? (?)
apk (release) ? 3.41 MB ? (?)
firebase-segmentation
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 20.0 kB ? (?)
apk (aggressive) ? 87.1 kB ? (?)
apk (release) ? 712 kB ? (?)
firebase-storage
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 116 kB ? (?)
apk (aggressive) ? 330 kB ? (?)
apk (release) ? 1.01 MB ? (?)
firebase-storage-ktx
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 13.1 kB ? (?)
apk (aggressive) ? 354 kB ? (?)
apk (release) ? 1.95 MB ? (?)
protolite-well-known-types
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 999 kB ? (?)
apk (aggressive) ? 134 kB ? (?)
apk (release) ? 663 kB ? (?)
transport-api
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 6.74 kB ? (?)
apk (aggressive) ? 8.68 kB ? (?)
apk (release) ? 14.9 kB ? (?)
transport-backend-cct
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 53.6 kB ? (?)
apk (aggressive) ? 58.2 kB ? (?)
apk (release) ? 105 kB ? (?)
transport-runtime
Type Base (c1adf5b) Merge (d76c438) Diff
aar ? 180 kB ? (?)
apk (aggressive) ? 44.0 kB ? (?)
apk (release) ? 83.5 kB ? (?)

Test Logs

Merge (d76c438): https://github.com/firebase/firebase-android-sdk/actions/runs/3464729960

https://storage.googleapis.com/firebase-sdk-metric-reports/FkLIVua5UY.html

vkryachko · 2022-11-14T20:37:30Z

.github/workflows/fireci.yml

@@ -18,8 +18,10 @@ jobs:
      - uses: actions/[email protected]
      - uses: actions/setup-python@v2
        with:
-          python-version: '3.9'
+          python-version: '3.8'


so have we been running on 3.8 for release build and 3.9 for GHA workflows?

I believe so.

We didn't specify a python version in the CI dockerfile. From the cloud build log for the latest image (registry, latest build), I think it was 3.8 that has been run for release.

I tried update the docker file. Building with 3.9 works (log), but 3.10 (log) and 3.11 (log) seem not.

github-actions · 2022-11-14T21:02:58Z

Unit Test Results

  395 files ±0   395 suites ±0 18m 49s ⏱️ - 1m 47s
4 730 tests ±0 4 708 ✔️ ±0 22 💤 ±0 0 ❌ ±0
4 746 runs ±0 4 724 ✔️ ±0 22 💤 ±0 0 ❌ ±0

Results for commit e7c108b. ± Comparison against base commit c1adf5b.

♻️ This comment has been updated with latest results.

Add support of macrobenchmark result analysis in fireci (#4285)

e7c108b

google-oss-bot added the size/XXL label Nov 14, 2022

yifanyang requested review from rlazo and vkryachko November 14, 2022 20:16

vkryachko reviewed Nov 14, 2022

View reviewed changes

vkryachko approved these changes Nov 15, 2022

View reviewed changes

rlazo approved these changes Nov 15, 2022

View reviewed changes

yifanyang merged commit 66b067d into master Nov 15, 2022

yifanyang deleted the yifany/macrobenchmark branch November 15, 2022 19:29

davidmotson pushed a commit that referenced this pull request Nov 28, 2022

Add support of macrobenchmark result analysis in fireci (#4323)

1331531

firebase locked and limited conversation to collaborators Dec 16, 2022

Add support of macrobenchmark result analysis in fireci #4323

Add support of macrobenchmark result analysis in fireci #4323

Uh oh!

Conversation

yifanyang commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

google-oss-bot commented Nov 14, 2022

Uh oh!

google-oss-bot commented Nov 14, 2022

Size Report 1

Affected Products

base

firebase-abt

firebase-annotations

firebase-appcheck

firebase-appcheck-debug

firebase-appcheck-debug-testing

firebase-appcheck-interop

firebase-appcheck-ktx

firebase-appcheck-playintegrity

firebase-appcheck-safetynet

firebase-appdistribution

firebase-appdistribution-api

firebase-appdistribution-api-ktx

firebase-common

firebase-common-ktx

firebase-components

firebase-config

firebase-config-ktx

firebase-crashlytics

firebase-crashlytics-ktx

firebase-crashlytics-ndk

firebase-database

firebase-database-collection

firebase-database-ktx

firebase-datatransport

firebase-decoders-json

firebase-dynamic-links

firebase-dynamic-links-ktx

firebase-dynamic-module-support

firebase-encoders

firebase-encoders-json

firebase-encoders-proto

firebase-encoders-reflective

firebase-firestore

firebase-firestore-ktx

firebase-functions

firebase-functions-ktx

firebase-inappmessaging

firebase-inappmessaging-display

firebase-inappmessaging-display-ktx

firebase-inappmessaging-ktx

firebase-installations

firebase-installations-interop

firebase-installations-ktx

firebase-messaging

firebase-messaging-directboot

firebase-messaging-ktx

firebase-ml-modeldownloader

firebase-ml-modeldownloader-ktx

firebase-perf

firebase-perf-ktx

firebase-segmentation

firebase-storage

firebase-storage-ktx

protolite-well-known-types

transport-api

transport-backend-cct

transport-runtime

Test Logs

Uh oh!

vkryachko Nov 14, 2022

Choose a reason for hiding this comment

Uh oh!

yifanyang Nov 14, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

yifanyang commented Nov 14, 2022 •

edited

Loading

Size Report ¹

`base`

`firebase-abt`

`firebase-annotations`

`firebase-appcheck`

`firebase-appcheck-debug`

`firebase-appcheck-debug-testing`

`firebase-appcheck-interop`

`firebase-appcheck-ktx`

`firebase-appcheck-playintegrity`

`firebase-appcheck-safetynet`

`firebase-appdistribution`

`firebase-appdistribution-api`

`firebase-appdistribution-api-ktx`

`firebase-common`

`firebase-common-ktx`

`firebase-components`

`firebase-config`

`firebase-config-ktx`

`firebase-crashlytics`

`firebase-crashlytics-ktx`

`firebase-crashlytics-ndk`

`firebase-database`

`firebase-database-collection`

`firebase-database-ktx`

`firebase-datatransport`

`firebase-decoders-json`

`firebase-dynamic-links`

`firebase-dynamic-links-ktx`

`firebase-dynamic-module-support`

`firebase-encoders`

`firebase-encoders-json`

`firebase-encoders-proto`

`firebase-encoders-reflective`

`firebase-firestore`

`firebase-firestore-ktx`

`firebase-functions`

`firebase-functions-ktx`

`firebase-inappmessaging`

`firebase-inappmessaging-display`

`firebase-inappmessaging-display-ktx`

`firebase-inappmessaging-ktx`

`firebase-installations`

`firebase-installations-interop`

`firebase-installations-ktx`

`firebase-messaging`

`firebase-messaging-directboot`

`firebase-messaging-ktx`

`firebase-ml-modeldownloader`

`firebase-ml-modeldownloader-ktx`

`firebase-perf`

`firebase-perf-ktx`

`firebase-segmentation`

`firebase-storage`

`firebase-storage-ktx`

`protolite-well-known-types`

`transport-api`

`transport-backend-cct`

`transport-runtime`

github-actions bot commented Nov 14, 2022 •

edited

Loading