Provide testing infrastructure for RuntimeHints #27981
Assignees
Labels
in: test
Issues in the test module
theme: aot
An issue related to Ahead-of-time processing
type: enhancement
A general enhancement
Milestone
Comments
bclozel
added
type: enhancement
bclozel
added a commit
that referenced
this issue
Jun 30, 2022
With the introduction of `RuntimeHints`, we can now contribute reflection, resources and proxies hints that describe the expected runtime behavior of the application. While this can be verified at runtime with smoke tests, managing such tests and compiling to native there is not very efficient. This commit introduces the new `RuntimeHintsAgent`, a Java agent that instruments JDK methods related to `RuntimeHints`. It is different from the GraalVM agent, which aims at collecting all the required hints for the runtime behavior of an application and dump those in the expected format. Here, the `RuntimeHintsAgent` can collect the related invocations only for a delimited scope (typically, a lambda within a test) and later check those against a `RuntimeHints` instance. In the case of testing `RuntimeHintsRegistrar` implementations, the process is reversed: instead of manually checking for registered hints in a `RuntimeHints` instance, tests should exercise the use cases and then check that the recorded behavior is in line with the prepared hints. This first commit adds the agent infrastructure that collects the invocations for all relevant JDK methods. See gh-27981
bclozel
added a commit
that referenced
this issue
Jun 30, 2022
This commit adds the supporting testing infrastructure using the
`RuntimeHintsAgent`. Given that the agent is loaded by the JVM running
the test suite, we can then use it to record method invocations at
runtime and check whether the prepared `RuntimeHints` match the expected
behavior.
This commit contributes the `RuntimeHintsRecorder`. With this, we can
record relevant method invocations for a given lambda, focusing on a
specific part of the code behavior. This returns a
`RuntimeHintsInvocations` instance, which is an AssertJ assert provider.
From there, we can perform assertions on the recorded invocations and
check that a given collection of hints cover the reflection, resources
and proxies needs at runtime.
This also ships the `@EnabledIfRuntimeHintsAgent` opinionated
annotation: this applies the `RuntimeHintsAgentCondition` JUnit
extension that detects whether the `RuntimeHintsAgent` is loaded by the
current JVM. Tests annotated with this will be skipped if the agent is
not present. This annotation is also tagged with a JUnit `@Tag` to
gather such tests in a specific `"RuntimeHintsTests"` test suite.
In the Spring Framework build, we have chosen to isolate such tests and
not load the agent for the main test suite ("RuntimeHintsTests" tests
are excluded from the main suite). While the agent's intent is to be as
transparent as possible, there are security and access considerations
that could interefere with other tests.
With this approach, we can then create a separate test suite and run
agent tests in a dedicated JVM.
Note that projects using this infrastructure can choose to use the
condition by itself in a custom annotation.
Here is an example of this testing infrastructure:
```
@EnabledIfRuntimeHintsAgent
class MyTestCases {
@test
void hintsForMethodsReflectionShouldMatch() {
RuntimeHints hints = new RuntimeHints();
hints.reflection().registerType(String.class,
hint -> hint.withMembers(MemberCategory.INTROSPECT_PUBLIC_METHODS));
RuntimeHintsInvocations invocations = RuntimeHintsRecorder.record(() -> {
Method[] methods = String.class.getMethods();
});
assertThat(invocations).match(hints);
}
}
```
See gh-27981
bclozel
added a commit
that referenced
this issue
Jun 30, 2022
This commit configures the `RuntimeHintsAgent` in the Spring Framework test suite. Instead of applying the agent to the entire test suite, and possibly interfering with other tests, this commit adds a new custom Gradle plugin that does the following: * create a new test task named `"runtimeHintsTest"` * run this task with the runtime hints java agent * only execute tests tagged with `"RuntimeHintsTests"` See gh-27981
bclozel
added a commit
that referenced
this issue
Jun 30, 2022
This commit adds new tests for the `ApplicationContextAotGenerator`, this time leveraging the `RuntimeHintsAgent` that checks the need for runtime hints at runtime. See gh-27981
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
#27829 introduced
RuntimeHints- a new infrastructure to register the need for runtime reflection, resources, proxying, and serialization on components.While this infrastructure will help Spring modules to register such concerns, we don't have the testing infrastructure to ensure that existing registrations are enough to cover actual runtime behavior. Since most of those calls are done on JDK APIs, we should provide testing support that:
The text was updated successfully, but these errors were encountered: