Skip to content

test: unit test multiple node versions in separate jobs #23714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 35 additions & 7 deletions .circleci/dynamic_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ parameters:
var_1: &cache_key v1-angular_devkit-14.19-{{ checksum "yarn.lock" }}
var_1_win: &cache_key_win v1-angular_devkit-win-16.10-{{ checksum "yarn.lock" }}
var_3: &default_nodeversion '14.19'
var_3_major: &default_nodeversion_major '14'
# The major version of node toolchains. See tools/toolchain_info.bzl
# NOTE: entries in this array may be repeated elsewhere in the file, find them before adding more
var_3_all_major: &all_nodeversion_major ['14', '16']
# Workspace initially persisted by the `setup` job, and then enhanced by `setup-and-build-win`.
# https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs
# https://circleci.com/blog/deep-diving-into-circleci-workspaces/
Expand Down Expand Up @@ -287,19 +291,39 @@ jobs:
- custom_attach_workspace
- run: yarn bazel build //tests/legacy-cli/...

test:
unit-test:
executor: test-executor
resource_class: xlarge
parameters:
nodeversion:
type: string
default: *default_nodeversion_major
steps:
- custom_attach_workspace
- browser-tools/install-chrome
- setup_bazel_rbe
- run: sudo cp .circleci/bazel.rc /etc/bazel.bazelrc
- run:
command: yarn bazel:test
# This timeout provides time for the actual tests to timeout and report status
# instead of CircleCI stopping the job without test failure information.
no_output_timeout: 40m
- when:
# The default nodeversion runs all *excluding* other versions
condition:
equal: [*default_nodeversion_major, << parameters.nodeversion >>]
steps:
- run:
command: yarn bazel test --test_tag_filters=-node16,-node<< parameters.nodeversion >>-broken //packages/...
Copy link
Contributor Author

@jbedard jbedard Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ugly condition (and opposite below) is so this one catches all node14 and non-versioned, while the other is all specifically for node16. Could add more in the future by adding to the array at the top of the file and add the -node## to this line.

# This timeout provides time for the actual tests to timeout and report status
# instead of CircleCI stopping the job without test failure information.
no_output_timeout: 40m
- when:
# Non-default nodeversion runs only that specific nodeversion
condition:
not:
equal: [*default_nodeversion_major, << parameters.nodeversion >>]
steps:
- run:
command: yarn bazel test --test_tag_filters=node<< parameters.nodeversion >>,-node<< parameters.nodeversion >>-broken //packages/...
# This timeout provides time for the actual tests to timeout and report status
# instead of CircleCI stopping the job without test failure information.
no_output_timeout: 40m
- fail_fast

snapshot_publish:
Expand Down Expand Up @@ -446,7 +470,11 @@ workflows:
# These jobs only really depend on Setup, but the build job is very quick to run (~35s) and
# will catch any build errors before proceeding to the more lengthy and resource intensive
# Bazel jobs.
- test:
- unit-test:
name: test-node<< matrix.nodeversion >>
matrix:
parameters:
nodeversion: *all_nodeversion_major
requires:
- build

Expand Down
1 change: 1 addition & 0 deletions packages/angular/cli/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ ts_library(
jasmine_node_test(
name = "angular-cli_test_" + toolchain_name,
srcs = [":angular-cli_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
)
for toolchain_name, toolchain in zip(
Expand Down
1 change: 1 addition & 0 deletions packages/angular/pwa/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ ts_library(
jasmine_node_test(
name = "pwa_test_" + toolchain_name,
srcs = [":pwa_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
)
for toolchain_name, toolchain in zip(
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/architect/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ ts_library(
jasmine_node_test(
name = "architect_test_" + toolchain_name,
srcs = [":architect_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
)
for toolchain_name, toolchain in zip(
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/benchmark/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ ts_library(
jasmine_node_test(
name = "benchmark_test_" + toolchain_name,
srcs = [":benchmark_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"@npm//jasmine",
Expand Down
12 changes: 9 additions & 3 deletions packages/angular_devkit/build_angular/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ ts_library(
jasmine_node_test(
name = "build_angular_test_" + toolchain_name,
srcs = [":build_angular_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
)
for toolchain_name, toolchain in zip(
Expand Down Expand Up @@ -295,8 +296,10 @@ LARGE_SPECS = {
"@npm//puppeteer",
"@npm//ts-node",
],
# NB: does not run on rbe because webdriver manager uses an absolute path to chromedriver
"tags": ["no-remote-exec"],
"tags": [
# TODO: node crashes with an internal error on node16
"node16-broken",
],
},
"dev-server": {
"shards": 10,
Expand Down Expand Up @@ -400,7 +403,10 @@ LARGE_SPECS = {
# These tests are resource intensive and should not be over-parallized as they will
# compete for the resources of other parallel tests slowing everything down.
# Ask Bazel to allocate multiple CPUs for these tests with "cpu:n" tag.
tags = ["cpu:2"] + LARGE_SPECS[spec].get("tags", []),
tags = [
"cpu:2",
toolchain_name,
] + LARGE_SPECS[spec].get("tags", []),
toolchain = toolchain,
deps = [":build_angular_" + spec + "_test_lib"],
)
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/build_webpack/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ ts_library(
jasmine_node_test(
name = "build_webpack_test_" + toolchain_name,
srcs = [":build_webpack_test_lib"],
tags = [toolchain_name],
# Turns off nodejs require patches and turns on the linker, which sets up up node_modules
# so that standard node module resolution work.
templated_args = ["--nobazel_patch_module_resolver"],
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/core/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ ts_library(
jasmine_node_test(
name = "core_test_" + toolchain_name,
srcs = [":core_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
# @node_module: ajv
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/core/node/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ ts_library(
jasmine_node_test(
name = "node_test_" + toolchain_name,
srcs = [":node_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"@npm//chokidar",
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/schematics/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ ts_library(
jasmine_node_test(
name = "schematics_test_" + toolchain_name,
srcs = [":schematics_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"@npm//jasmine",
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/schematics/tools/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ ts_library(
jasmine_node_test(
name = "tools_test_" + toolchain_name,
srcs = [":tools_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"@npm//jasmine",
Expand Down
1 change: 1 addition & 0 deletions packages/angular_devkit/schematics_cli/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ ts_library(
jasmine_node_test(
name = "schematics_cli_test_" + toolchain_name,
srcs = [":schematics_cli_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
)
for toolchain_name, toolchain in zip(
Expand Down
1 change: 1 addition & 0 deletions packages/ngtools/webpack/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ ts_library(
jasmine_node_test(
name = "webpack_test_" + toolchain_name,
srcs = [":webpack_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"@npm//jasmine",
Expand Down
2 changes: 2 additions & 0 deletions packages/schematics/angular/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ ts_library(
jasmine_node_test(
name = "no_typescript_runtime_dep_test_" + toolchain_name,
srcs = ["no_typescript_runtime_dep_spec.js"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
":angular",
Expand Down Expand Up @@ -130,6 +131,7 @@ ts_library(
jasmine_node_test(
name = "angular_test_" + toolchain_name,
srcs = [":angular_test_lib"],
tags = [toolchain_name],
toolchain = toolchain,
deps = [
"//packages/schematics/angular/third_party/github.com/Microsoft/TypeScript",
Expand Down
14 changes: 6 additions & 8 deletions tools/toolchain_info.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@
# the order will match against the order in the TOOLCHAIN_VERSION list.
TOOLCHAINS_NAMES = [
"node14",
# TODO enable one we know more why there is a memory usage increase and app-shell tests work with Node.js 16.
# "node16",
"node16",
]

# this is the list of toolchains that should be used and are registered with nodejs_register_toolchains in the WORKSPACE file
Expand All @@ -15,12 +14,11 @@ TOOLCHAINS_VERSIONS = [
"@bazel_tools//src/conditions:darwin": "@node14_darwin_amd64//:node_toolchain",
"@bazel_tools//src/conditions:windows": "@node14_windows_amd64//:node_toolchain",
}),
# TODO enable one we know more why there is a memory usage increase and app-shell tests work with Node.js 16.
# select({
# "@bazel_tools//src/conditions:linux_x86_64": "@node16_linux_amd64//:node_toolchain",
# "@bazel_tools//src/conditions:darwin": "@node16_darwin_amd64//:node_toolchain",
# "@bazel_tools//src/conditions:windows": "@node16_windows_amd64//:node_toolchain",
# }),
select({
"@bazel_tools//src/conditions:linux_x86_64": "@node16_linux_amd64//:node_toolchain",
"@bazel_tools//src/conditions:darwin": "@node16_darwin_amd64//:node_toolchain",
"@bazel_tools//src/conditions:windows": "@node16_windows_amd64//:node_toolchain",
}),
]

# A default toolchain for use when only one is necessary
Expand Down