|
1 | | -# Topcoder - Submission Legacy Processor Application - Verification |
| 1 | +# Topcoder - Legacy Submission Processor Application - Verification |
2 | 2 | ------------ |
| 3 | +> NOTE: ALL COMMANDS BELOW EXECUTE UNDER ```<legacy-sub-procecssor>``` directory |
3 | 4 |
|
4 | | -Please check [docs/Validation.md](/docs/Validation.md) and [docs/Verification_with_DB.md](/docs/Verification_with_DB.md). |
5 | | - |
6 | | -Please note currently you have to verify application with database or please check [docs/Verification_with_DB.md](/docs/Verification_with_DB.md) only. |
7 | | - |
8 | | -I would recommend you to verify and test with docker otherwise you need to check related Dockerfile to have better understanding about how to setup environment properly(not recommend). |
| 5 | +## Run Kafka and Create Topic |
9 | 6 |
|
| 7 | +Build Kafka image: |
| 8 | +```bash |
| 9 | +docker-compose build kafka |
| 10 | +``` |
10 | 11 |
|
11 | | -## Topcoder - Marathon Match - Legacy Processor |
12 | | -Please verify under linux or osx and I test under Ubuntu 18.04 and OSX 12 and windows may have issues to verify with docker. |
| 12 | +Run Kafka server: |
| 13 | +```bash |
| 14 | +docker-compose up -d kafka |
| 15 | +docker exec -ti kafka bash -c "kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic submission.notification.create" |
| 16 | +docker exec -ti kafka bash -c "kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic submission.notification.update" |
| 17 | +``` |
13 | 18 |
|
14 | | -Please check README.md and ensure you could run tests in docker environment successfully and you can check coverage folder to ensure |
15 | | -`src/services/MarathonSubmissionService.js`,`src/services/NonMarathonSubmissionService.js`,`src/services/LegacySubmissionIdService.js` are fully tested. |
| 19 | +## Run Informix and Insert Test Data |
| 20 | +Make sure you're running a clean database (you can take down and remove iif_innovator_c container and then up it again) |
| 21 | +```bash |
| 22 | +export DB_SERVER_NAME=informix |
16 | 23 |
|
17 | | -## Setup data in direct |
18 | | -Only necessary MM challenge related test data updated in `./test/sql/test.sql` and you can also setup complete test data using direct application. |
| 24 | +docker kill iif_innovator_c |
| 25 | +docker rm iif_innovator_c |
19 | 26 |
|
20 | | -You can follow [docs/Verification_with_DB.md](/docs/Verification_with_DB.md) to setup MM challenge in direct app, choose Marathon Match under Data menu during creating challenge and create new project with billing account if error to save mm challenge, in last step you have to create as draft challenge,add user as Submitter and get match submission phase id. |
| 27 | +docker-compose up -d tc-informix |
21 | 28 |
|
22 | | -Currently tc-direct have [issue to save Match Round ID](https://github.com/appirio-tech/direct-app/issues/341) or you may see such logs from command `docker-compose logs tc-direct` if you want to save in page and refresh page. |
23 | | -```bash |
24 | | - | 07:48:37,097 ERROR [ExceptionMappingInterceptor] Invalid action class configuration that references an unknown class named [saveDraftContestAction] |
25 | | -tc-direct_1 | java.lang.RuntimeException: Invalid action class configuration that references an unknown class named [saveDraftContestAction] |
| 29 | +docker logs iif_innovator_c |
| 30 | +# When you see log like following then informix is started: |
| 31 | +# starting rest listener on port 27018 |
| 32 | +# starting mongo listener on port 27017 |
26 | 33 | ``` |
27 | 34 |
|
28 | | -So you have to run such sql and replace <mm challenge id> with your new created mm challenge id. |
| 35 | +**Then insert test data (which will be used by Unit Tests step and Verification step)**: |
29 | 36 | ```bash |
30 | | -database tcs_catalog; |
31 | | -INSERT INTO informix.project_info |
32 | | -(project_id, project_info_type_id, value, create_user, create_date, modify_user, modify_date) |
33 | | -VALUES(<mm challenge id>, 56, 2001, '132456', current, '132456', current); |
| 37 | +docker cp test/sql/test.sql iif_innovator_c:/ |
| 38 | +docker exec -ti iif_innovator_c bash -c "source /home/informix/ifx_informixoltp_tcp.env && dbaccess - /test.sql" |
34 | 39 | ``` |
35 | 40 |
|
36 | | -Even you run sql direct app will still fail to show details page with such error |
37 | | -```bash |
38 | | - java.lang.NullPointerException |
39 | | -tc-direct_1 | at com.topcoder.direct.services.view.action.analytics.longcontest.MarathonMatchHelper.getMarathonMatchDetails(MarathonMatchHelper.java:115) |
40 | | -tc-direct_1 | at com.topcoder.direct.services.view.action.contest.launch.GetContestAction.executeAction(GetContestAction.java:476) |
41 | | -tc-direct_1 | at com.topcoder.direct.services.view.action.BaseDirectStrutsAction.execute(BaseDirectStrutsAction.java:305) |
42 | | -tc-direct_1 | at sun.reflect.GeneratedMethodAccessor553.invoke(Unknown Source) |
43 | | - |
| 41 | +## Build Application Docker Image |
| 42 | +We only need to do this once |
| 43 | +```bash |
| 44 | +export DB_SERVER_NAME=informix |
| 45 | +docker-compose build lsp-app |
44 | 46 | ``` |
45 | 47 |
|
46 | | -Please check https://github.com/appirio-tech/direct-app/blob/dev/src/java/main/com/topcoder/direct/services/view/action/contest/launch/GetContestAction.java#L502 |
47 | | - |
48 | | -Even latest direct codes will comment out related codes to solve this issue so just leave direct page there. |
49 | | - |
50 | | -## Run Legacy Submission Proc. app |
51 | | - |
52 | | -Make sure related services started and test data prepared and start app with `NODE_ENV=mock` to mock challenge api otherwise new created mm challenge will still consider as non mm challenge |
| 48 | +## Install App dependencies |
53 | 49 | ```bash |
54 | | -export NODE_ENV=mock |
55 | 50 | export DB_SERVER_NAME=informix |
56 | | -docker-compose up lsp-app |
| 51 | +rm -rf node_modules && docker-compose run lsp-app-install |
57 | 52 | ``` |
58 | 53 |
|
59 | | -## Send Test data |
60 | | -From previous data setup I got: |
61 | | -- challengeId = 40005570 |
62 | | -- memberId = 132458 (user) |
63 | | -- submissionPhaseId = 100024 |
64 | | - |
65 | | -Let's send mm submission with example = 0 event to kafka: |
| 54 | +**Note**, if the ***legacy-processor-module*** is changed locally (e.g. during local dev and not pushed to git yet), then you need to delete it from *node_modules* and copy the local changed one to *node_modules*: |
66 | 55 |
|
67 | 56 | ```bash |
68 | | -docker exec -ti lsp-app bash -c "npm run produce-test-event mm 40005570 132458 100024 0" |
| 57 | +rm -rf ./node_modules/legacy-processor-module |
| 58 | +cp -rf <path/to/legacy-processor-module> ./node_modules |
| 59 | +# e.g cp -rf ../legacy-processor-module ./node_modules |
69 | 60 | ``` |
70 | 61 |
|
71 | | -Let's send mm submission with example = 1 event to kafka: |
| 62 | +## Standard Code Style |
| 63 | + |
| 64 | +- Check code style `npm run lint` |
| 65 | +- Check code style with option to fix the errors `npm run lint:fix` |
72 | 66 |
|
| 67 | +## Run Unit Tests |
| 68 | +- Stop `legacy-sub-processor` application if it was running: `docker stop lsp-app` |
| 69 | +- Make sure kafka container running with topic created and informix container running with test data inserted |
| 70 | +- Run unit tests: |
73 | 71 | ```bash |
74 | | -docker exec -ti lsp-app bash -c "npm run produce-test-event mm 40005570 132458 100024 1" |
| 72 | +docker-compose run lsp-app-test |
75 | 73 | ``` |
76 | 74 |
|
77 | | -or you can run sample mm submission message directly(valid if run `test/sql/test.sql`) |
| 75 | +## Verify with Test Data |
| 76 | +Deploy first: |
78 | 77 | ```bash |
79 | | -docker exec -ti lsp-app bash -c "npm run produce-test-event 9" |
| 78 | +export DB_SERVER_NAME=informix |
| 79 | +docker-compose up lsp-app |
80 | 80 | ``` |
81 | 81 |
|
82 | | - |
83 | | -Please note currently processor will call challenge api to check whether challenge is MM challenge and default server api.topcoder-dev.com may not exist mm challenge created in local direct application. |
84 | | -So when we start app with NODE_ENV=mock it will use mock challenge api configurations and start mock challenge api server. |
| 82 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event different-topic"` and verify that the app doesn't consume this message (no log) |
| 83 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event null-message"` and verify that the app skips this message (log: `Skipped null or empty event`) |
| 84 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event empty-message"` and verify that the app skips this message (log: `Skipped null or empty event`) |
| 85 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event invalid-json"` and verify that the app skips this message (log: `Skipped Invalid message JSON`) |
| 86 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event empty-json"` and verify that the app skips this message (log: `Skipped the message topic "undefined" doesn't match the Kafka topic submission.notification.create`) |
| 87 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event invalid-payload"` and verify that the app skips this message (log: `Skipped invalid event, reasons: "timestamp" must be...`) |
| 88 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event wrong-topic"` and verify that the app skips this message (log: `Skipped the message topic "wrong-topic" doesn't match the Kafka topic submission.notification.create`) |
| 89 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event wrong-originator"` and verify that the app skips this message (log: `Skipped event from topic wrong-originator`) |
| 90 | + |
| 91 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event submission"` and verify that the app makes call to the Submission API successfully (log: `Successfully processed non MM message - Patched to the Submission API: id 111, patch: {"legacySubmissionId":60000}`) and the Mock API log (`docker logs mock-api`) like `Patch /submissions/111 with {"legacySubmissionId":60000}`. |
| 92 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event final-fix"` and verify that the app has log like `final fix upload, only insert upload`, and it should only insert into `upload` table, but not `submission`/`resource_submission` table. |
| 93 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event not-allow-multiple"` and verify that the app has log like `delete previous submission for challengeId...`. |
| 94 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event update-url"` and verify that the app has log like `Successfully processed non MM message - Submission url updated...`. |
| 95 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event no-challenge-props"` and verify that the app has error log like `Error: null or empty result get challenge properties...`. |
| 96 | +- Run `docker exec -ti lsp-app bash -c "npm run produce-test-event mm-submission"` and verify that the app skips this message (log: `Skipped event for subTrack: DEVELOP_MARATHON_MATCH`). |
85 | 97 |
|
86 | 98 | ## Verify Database |
87 | | -Open your database explorer (**DBeaver** application, for instance). Connect to database informixoltp |
88 | | -Check table: `long_component_state`, `long_submission` or run below sql |
89 | | -```bash |
90 | | -select lcs.status_id,lcs.points, lcs.example_submission_number,lcs.submission_number,ls.* from informixoltp:long_submission ls, informixoltp:long_component_state lcs where ls.long_component_state_id=lcs.long_component_state_id |
91 | | -``` |
| 99 | +Open your database explorer (**DBeaver** application, for instance). Connect to database tcs_catalog |
| 100 | +Check table: `upload`, `submission` and `resource_submission` |
| 101 | + |
| 102 | +## Cleanup |
| 103 | +After verification, run `docker-compose down` to take down and remove containers. |
0 commit comments