V5 not support new Challenge UUID #179
Comments
|
It definitely does support UUIDs. The error that you are getting seems to be coming from Elasticsearch. Need to check what's up there... |
|
I think he's talking about Challenge UUIDs, not submission UUIDs. I don't
think the submission api is set up to use v5 challenge api, right?
…On Fri, Jun 5, 2020 at 1:30 PM Mithun Kamath ***@***.***> wrote:
It definitely does support UUIDs. The error that you are getting seems to
be coming from Elasticsearch. Need to check what's up there...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#179 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJGMXFBWDZOE3Z5BG2IFD3RVETSZANCNFSM4NUEZHGA>
.
--
James Cori
Lead Software Engineer
[email protected] / 413.961.9044 / @jamesmcori
<https://twitter.com/jamesmcori>
<[email protected]>www.topcoder.com <http://topcoder.com/> / Topcoder
Blog: topcoder.com/blog / twitter: @topcoder
<https://twitter.com/topcoder>201 S Capitol Ave, Indianapolis, IN 46225
|
|
It actually is. Here's the schema:
The test data that we used in the past too had UUIDs (we changed them to legacy to keep it realistic)... hence curious as to why elasticsearch complained. Will check and get back with my findings... |
|
@callmekatootie what's ETA on this? |
|
Full error log:
Important bit in the error message above: Cause of issue seems to be that the Checked the mappings and indeed - @rootelement Could you kindly advice - I think in order to support v5 challenge ids, we would need to create a new index, update the data type for challengeId to string, copy the data over to the new index and drop the old index. |
|
@sushilshinde This could take some time depending on the approach that we will take. The fix is not straightforward and thus an ETA would be difficult to provide. |
|
@callmekatootie couldn't we just set up a new index, update the document model to a string and then run this loading script against it ? |
|
Yes, we could use that loading script as well. So is it confirmed that we move ahead with creation of a new index? |
|
@callmekatootie yes, please do that |
|
@callmekatootie what do you want the index to be? I can create it in dev and prod, then when the type is updated, we can just run that script. |
|
How does |
|
FYI - current value is |
|
Perhaps I name the environment variable as |
|
@rootelement So current status: Pull Request with the changes: #184 Let me know if I can proceed to merge with
|
I don't see any option to create a new index, so we should just change the name and try it. I've added |
|
@callmekatootie slack me directly if you want to work on this |
|
@rootelement The error says:
What is your ES endpoint url look like... is it in http perhaps? |
#179 Update data type for challengeId in the index, by creating a new index
|
@luizrrodrigues this is fixed in dev. Please test there. We're scheduling the hotfix to deploy to prod hopefully tomorrow |
That is right. They would need to use the same id (legacy or v5) that was used during submission creation. But you can let me know the expectations here. |
Sure. Let us get back to you. |
|
@SathyaJayabal , this shouldn't be an issue, because going forward after this release, everything should be working in UUIDs. So if the submission was created, it will have a UUID, and if we're retrieving old data, the system will look up the submission by that UUID and convert it to legacyId on the backend to get any related data. The only gray area will be if a submission was created in testing before this change and a legacyId was used, and now you're referring to the same submission by a UUID. |
|
@SathyaJayabal We're discussing if there's an easier option. Give us about an hour to decide |
* #179 - Support v5 challenge ids but ensure that we store legacy in db * Fix test
This has numbers now. I think this was originally part of the sync issue. These numbers should be updated by the challenge-processor-es when a submission is created or a user registers. When i click that link, i see numbers in registrants and submissions. Please confirm this is still an issue. cc: @callmekatootie |
|
@luizrrodrigues The other issues that you have reported are now resolved. Could you kindly verify at your end and let me know if you still face any issues. @sushilshinde @SathyaJayabal You can now pass either v5 or legacy challenge id for your queries. Submission API will convert the v5 challenge id to legacy and respond accordingly. The FE should now be able to work with both. |
|
@callmekatootie @rootelement cool, thanks. I'll check shortly and report here. |
|
Ok, we're updating to store the legacyId in the db. So if you pass in the uuid, it's translated, if you pass in the legacyId, it's used. @callmekatootie has pushed to dev and his tests pass. You should be able to use either. |
|
It looks like not working well. Tests:
|
|
@luizrrodrigues when you upload a submission, don't pass |
|
It's storing a v5 uuid on the object, but the ES processor is looking up that v5 uuid in v4 and not finding it. cc: @callmekatootie The error from the submission-processor-es: {
"path":"/submission-index/submission/b63d88bb-bb4b-4333-ba01-7c0cce2fc4aa/_create",
"query":{
},
"body":"{\"resource\":\"submission\",\"id\":\"b63d88bb-bb4b-4333-ba01-7c0cce2fc4aa\",\"type\":\"Contest Submission\",\"url\":\"https://s3.amazonaws.com/topcoder-dev-submissions-dmz/ce7c03ff-ccd1-4c8a-b02f-a6a8dc27739f-40152905-SUBMISSION_ZIP-1593569314304.zip\",\"memberId\":40152905,\"challengeId\":30056662,\"created\":\"2020-07-01T02:09:15.021Z\",\"updated\":\"2020-07-01T02:09:15.021Z\",\"createdBy\":\"dan_developer\",\"updatedBy\":\"dan_developer\",\"submissionPhaseId\":\"ae38efe8-680e-446f-83f9-184153b57097\",\"fileType\":\"zip\",\"isFileSubmission\":false}",
"statusCode":400,
"response":"{\"error\":{\"root_cause\":[{\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse [submissionPhaseId]\"}],\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse [submissionPhaseId]\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"For input string: \\\"ae38efe8-680e-446f-83f9-184153b57097\\\"\"}},\"status\":400}"
} |
|
From the ES index: So, if you are passing, it needs to be a number or do not pass it - and the api will retrieve it. I am guessing the new submission phase id is uuid. I will update the validation in the api to NOT accept uuids for submission phase id but do let me know if this is needed. |
|
@callmekatootie Yes, only have UUID in phases. So if don't pass |
|
You don't have to pass it - submission api will retrieve it, based on the challenge id. |
|
Okay, cool. I'll update and test. |
|
@callmekatootie @rootelement Cool, looks good now. |
|
@callmekatootie, after this fix, I was able to verify the following on beta(prod backend)
But I was not able to verify the 2 and 3 in test env(dev backend) Do you have any idea, why there is a difference between the two envs? |
|
@SathyaJayabal I am afraid not. I don't expect the issue to be with submission api - it could be with the dev instances of Online Review and the Email processor. If there's any additional information that you could provide (say - an example request that is failing in dev instance) I could look into it further |
|
@callmekatootie, thanks. Like you said the issue could be with the dev instances of the OR or submissions or email processor. |
|
Parent ticket (topcoder-platform/community-app#4586) closed and thus closing this ticket too |
Issue: submissions_api not support new challenge
UUID, only acceptlegacyId:@rootelement I can't assign in this repo.
The text was updated successfully, but these errors were encountered: