Add support for generating code to send MuleSoft Dataweave Transformations to TriggerMesh #183

cab105 · 2022-06-30T14:03:03Z

I introduced a new runtime flag for spring-boot-migrator called sbm.muleTriggerMeshTransformEnabled that will modify the generated mule-to-boot recipe for handling DataWeave transformations to stream the transformation to TriggerMesh's Dataweave Transformer.

The generated code works best when the service runs inside a Kubernetes cluster leveraging Knative's Sink Binding to associate the the newly migrated service to the TriggerMesh Dataweave transformation, however the service can run on it's own by setting the K_SINK environment variable to the exposed transformation service.

I created a gist to demonstrate what an integration within TriggerMesh would look like.

fabapp2 · 2022-07-07T09:10:11Z

@cab105 Thank you for your contribution!

I had three comments. Feel free to ping me if you want to discuss in person?

...ava/org/springframework/sbm/mule/actions/javadsl/translators/dwl/DwlTransformTranslator.java

cab105 · 2022-07-07T16:24:18Z

@cab105 Thank you for your contribution!

I had three comments. Feel free to ping me if you want to discuss in person?

Thank you for taking a look at @fabapp2 . Will address the rest template, tabs, and test issues, then resubmit.

sanagaraj-pivotal · 2022-07-11T14:55:42Z

Could we get an integration test for this feature, please refer this test for info

fabapp2 · 2022-08-15T18:52:37Z

Hi @cab105

Are you facing issues with this? If you like we can pair to get the last few bits added and this PR merged?

cab105 · 2022-08-16T01:37:10Z

The codebase has been pretty unstable due to the springboot 3 work, but does look like it has now solidified, and am identifying issues with my original test code that need to be addressed. Once that is resolved, then we should be good to go.

cab105 · 2022-08-25T02:26:42Z

Resync from upstream main branch, added unit tests, and migrated the primary dataweave transformation into a template file.

cab105 · 2022-09-06T20:18:24Z

So I've been able to resolve some of my outstanding issues, and have fixed a couple of issues in SBM proper (#385 and #373). Given the current Mule integration test that was cited in the comments is disabled due to #351, I'm wondering if there's anything else that would be required for this PR (save for a resync with upstream)?

…it test scenario

cab105 · 2022-09-21T17:37:10Z

...hell/src/test/java/org/springframework/sbm/MigrateSimpleMuleAppDataweaveIntegrationTest.java

+import static org.assertj.core.api.Assertions.assertThat;
+
+@TestMethodOrder(MethodOrderer.MethodName.class)
+//@Disabled("Temporary disabled before CI will be fixed with docker in docker issue: #351")


Looking at the base test, this annotation is in effect, but left it commented out to facilitate verification for review.

Not sure if I get you here, but the method order is obsolete as there's only one test and no shared state, or?
You can remove the commented @Disabled, the test is green 🚀

...hell/src/test/java/org/springframework/sbm/MigrateSimpleMuleAppDataweaveIntegrationTest.java

cab105 · 2022-09-21T17:39:44Z

...hell/src/test/java/org/springframework/sbm/MigrateSimpleMuleAppDataweaveIntegrationTest.java

+        IntegrationTestBaseClass.beforeAll();
+
+        // Will need to ensure this is set globally for the test
+        System.setProperty("sbm.muleTriggerMeshTransformEnabled", "true");


I looked into overriding @SpringProperties, but this would not get picked up by the transformer that would swap out the stub code for the code that will send the transformation out to the TriggerMesh Dataweave transformer. If there's a cleaner way to do this, then would appreciate hearing it.

I think we should rework this approach and set the flag in the action of a dedicated recipe, e.g. migrate-mule-to-boot-tm and then just call it from here.

cab105 · 2022-09-21T17:43:58Z

...ava/org/springframework/sbm/mule/actions/javadsl/translators/dwl/DwlTransformTranslator.java

@@ -60,13 +91,19 @@ public DslSnippet translate(
            String flowName,
            Map<Class, MuleComponentToSpringIntegrationDslTranslator> translatorsMap
    ) {
+        // Ugly hack to work around an inability to inject a sbm property into the mulesoft parser.
+        String isTmTransformationEnabled = System.getProperty("sbm.muleTriggerMeshTransformEnabled");


Required to work around case when running unit test and @Autowired is unable to bring in the SBM properties.

Technically speaking, this should work

@Autowired private SbmApplicationProperties p; public void m() { p.getPropertyXyz(); }

But in this case an annotation on type level @ConditionalOnProperty(name = "sbm.muleTriggerMeshTransformEnabled", havingValue = "true") of a dedicated implementation would would be better.

fabapp2

Hi @cab105 !
Thanks for your effort! 🚀

I think the hacky ;) bits will disapear as soon as the application property is removed and the action gets parametrized per recipe, which feels somehow naturally (now).
Everything else lgtm!

...-recipes-mule-to-boot/src/main/java/org/springframework/sbm/mule/actions/JavaDSLAction2.java

...hell/src/test/java/org/springframework/sbm/MigrateSimpleMuleAppDataweaveIntegrationTest.java

fabapp2 · 2022-09-21T21:35:25Z

...hell/src/test/java/org/springframework/sbm/MigrateSimpleMuleAppDataweaveIntegrationTest.java

+        IntegrationTestBaseClass.beforeAll();
+
+        // Will need to ensure this is set globally for the test
+        System.setProperty("sbm.muleTriggerMeshTransformEnabled", "true");


I think we should rework this approach and set the flag in the action of a dedicated recipe, e.g. migrate-mule-to-boot-tm and then just call it from here.

fabapp2 · 2022-09-21T21:39:39Z

...s/sbm-recipes-mule-to-boot/src/main/java/org/springframework/sbm/mule/MigrateMuleToBoot.java

+        String description = "Migrate Mulesoft 3.9 to Spring Boot.";
+
+        // Flag to enable TriggerMesh ransformation mode
+        if (sbmProperties.isMuleTriggerMeshTransformEnabled()) {


We should better create two bean methods here, calling one factory method that creates the recipe twice with the flag true and false. Taht would then finally allow to remove the application property :)

fabapp2

I just realize that there must be some way to toggle these translators if boot creates them...
but it should not be an application/system property when possible...
it should come from the recipe/action...
this could be tricky...

fabapp2 · 2022-09-21T22:58:11Z

...ava/org/springframework/sbm/mule/actions/javadsl/translators/dwl/DwlTransformTranslator.java

@@ -60,13 +91,19 @@ public DslSnippet translate(
            String flowName,
            Map<Class, MuleComponentToSpringIntegrationDslTranslator> translatorsMap
    ) {
+        // Ugly hack to work around an inability to inject a sbm property into the mulesoft parser.
+        String isTmTransformationEnabled = System.getProperty("sbm.muleTriggerMeshTransformEnabled");


Technically speaking, this should work

@Autowired private SbmApplicationProperties p; public void m() { p.getPropertyXyz(); }

But in this case an annotation on type level @ConditionalOnProperty(name = "sbm.muleTriggerMeshTransformEnabled", havingValue = "true") of a dedicated implementation would would be better.

cab105 · 2022-09-21T23:34:54Z

I just realize that there must be some way to toggle these translators if boot creates them...
but it should not be an application/system property when possible...
it should come from the recipe/action...
this could be tricky...

It is. I originally used your approach of @Autowired on SbmApplicationProperties, but the unit test doesn't wire in the SbmApplicationProperties resulting in the tests to use the default behavior, and why I explicitly checked/set the System properties where I did. Granted if there was an easier way to pass a flag through the rewrite engine, then I'd be happy, but it appeared to require a lot more refactoring than I felt comfortable to pass the single flag down to the DwlTransformTranslator class.

fabapp2 · 2022-09-22T11:47:50Z

I just realize that there must be some way to toggle these translators if boot creates them...
but it should not be an application/system property when possible...
it should come from the recipe/action...
this could be tricky...

It is. I originally used your approach of @Autowired on SbmApplicationProperties, but the unit test doesn't wire in the SbmApplicationProperties resulting in the tests to use the default behavior, and why I explicitly checked/set the System properties where I did. Granted if there was an easier way to pass a flag through the rewrite engine, then I'd be happy, but it appeared to require a lot more refactoring than I felt comfortable to pass the single flag down to the DwlTransformTranslator class.

The problem is that the test is not a @SpringBootTest and the SbmApplicationProperties would need to be added to the TestProjectContext test harness. But due to the inheritance design it is not (yet) possible.

What about...
you leave the property as it is, for now, fix the few comments left and we finally merge this back in?
And I will think about a way to remove the need for the application property.
Not 💯 sure yet how this will look like though...

fabapp2 added the Mule-3.9 label Jul 7, 2022

fabapp2 suggested changes Jul 7, 2022

View reviewed changes

cab105 force-pushed the tm-transformation branch from f265415 to da0fef5 Compare July 17, 2022 03:10

cab105 force-pushed the tm-transformation branch 2 times, most recently from 711359f to 78f2ee8 Compare August 7, 2022 22:20

cab105 force-pushed the tm-transformation branch from a376ef0 to 31c1533 Compare August 25, 2022 02:23

cab105 force-pushed the tm-transformation branch from b451850 to acc51b9 Compare September 1, 2022 15:56

cab105 force-pushed the tm-transformation branch from acc51b9 to ab033a9 Compare September 6, 2022 20:21

cab105 requested a review from fabapp2 September 6, 2022 20:22

cab105 force-pushed the tm-transformation branch from ab033a9 to d31c0d3 Compare September 8, 2022 20:50

cab105 force-pushed the tm-transformation branch from f1fac5d to fa7bf11 Compare September 21, 2022 01:16

cab105 added 7 commits September 20, 2022 18:17

Add initial TriggerMesh transformation support

37d850d

Strip out LinkedMultiValueMap as it is no longer necessary

320e1df

Add triggermesh parameter and clean up mule to boot integration

4a48fa2

Address PR feedback and merge conflicts

622a7cc

Add unit test and migrate transformation class to a template

0f55515

Re-add autowired template configuration and if case to work around un…

3ffb5b5

…it test scenario

Add integration tests

7fa4a62

cab105 force-pushed the tm-transformation branch from fa7bf11 to 7fa4a62 Compare September 21, 2022 01:18

cab105 commented Sep 21, 2022

View reviewed changes

fabapp2 suggested changes Sep 21, 2022

View reviewed changes

removed flag set to default, formatted test data

5df5d14

fabapp2 reviewed Sep 21, 2022

View reviewed changes

Address PR comments

750d3d3

fabapp2 approved these changes Sep 23, 2022

View reviewed changes

Merge branch 'main' into tm-transformation

15141be

fabapp2 added the type: enhancement New feature or request label Sep 23, 2022

fabapp2 assigned cab105 Sep 23, 2022

fabapp2 added this to the v0.12.0 milestone Sep 23, 2022

fabapp2 linked an issue Sep 23, 2022 that may be closed by this pull request

Supports TriggerMesh for MuleSoft Dataweave Transformations #434

Closed

Merge branch 'main' into tm-transformation

b1fc898

fabapp2 merged commit caab9e0 into spring-projects-experimental:main Sep 23, 2022

cab105 deleted the tm-transformation branch September 23, 2022 18:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for generating code to send MuleSoft Dataweave Transformations to TriggerMesh #183

Add support for generating code to send MuleSoft Dataweave Transformations to TriggerMesh #183

cab105 commented Jun 30, 2022

fabapp2 commented Jul 7, 2022

cab105 commented Jul 7, 2022 •

edited

Loading

sanagaraj-pivotal commented Jul 11, 2022

fabapp2 commented Aug 15, 2022

cab105 commented Aug 16, 2022

cab105 commented Aug 25, 2022

cab105 commented Sep 6, 2022

cab105 Sep 21, 2022

fabapp2 Sep 21, 2022

cab105 Sep 21, 2022

fabapp2 Sep 21, 2022

cab105 Sep 21, 2022

fabapp2 Sep 21, 2022

fabapp2 left a comment

fabapp2 Sep 21, 2022

fabapp2 Sep 21, 2022

fabapp2 left a comment

fabapp2 Sep 21, 2022

cab105 commented Sep 21, 2022

fabapp2 commented Sep 22, 2022

Add support for generating code to send MuleSoft Dataweave Transformations to TriggerMesh #183

Add support for generating code to send MuleSoft Dataweave Transformations to TriggerMesh #183

Conversation

cab105 commented Jun 30, 2022

fabapp2 commented Jul 7, 2022

cab105 commented Jul 7, 2022 • edited Loading

sanagaraj-pivotal commented Jul 11, 2022

fabapp2 commented Aug 15, 2022

cab105 commented Aug 16, 2022

cab105 commented Aug 25, 2022

cab105 commented Sep 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabapp2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabapp2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cab105 commented Sep 21, 2022

fabapp2 commented Sep 22, 2022

cab105 commented Jul 7, 2022 •

edited

Loading