Add Attachments to the challenge specification

[maxceem]

As API already supports it, we have to support adding attachments for the challenge specification.

FYI @vikasrohit

[rootelement]

Challenge API supports Attachments. You'll need to upload through the UI via the Filestack widget into the S3 bucket, then submit that s3 URL as the attachment URL. Then the challenge api will handle security and retrieval of the attachment.

[maxceem]

Thanks, @rootelement. This is how we implement it now in Work Manager. I only had an issue with creating attachments, as it didn't allow S3 bucket that I used see #737 (comment). But I guess you've already whitelisted the bucket for me, as I contacted Thomas for this. Would give some time for the deployment and try it again.

submission-staging-dev
topcoder-dev-media
tc-challenge-v5-dev

[maxceem]

@vikasrohit file uploading is done via PR #931

See demo video: https://monosnap.com/file/I4B4UsRVCtJBQv1wzraQwwvkcvibNF

Though there is some issue with config, so I use the incorrect FileStack/AWS bucket config, because it doesn't work for me with the correct one. As was clarified by @rootelement (on Slack) we have to upload challenge attachments to tc-challenge-v5-dev bucket. But if I config this bucket I get 403 error during uploading files to AWS via FileStack:

But if I only change the bucket to submission-staging-dev the same code works for me, and I can upload files to AWS via FileStack:

But Challenge API doesn't allow creating attachments with submission-staging-dev because this bucket is not whitelisted. So to be able to test I had to set the bucket tc-challenge-v5-dev for attachments when creating them in Challenge API:

So as result I upload files to submission-staging-dev but tell Challenge API that file is uploaded to tc-challenge-v5-dev so it accepts my POST requests 🤯

This might lead to the next issues I have:

• When I reload the challenge page, not all uploaded attachments are returned by Chalelnge API (even though all POST request were successful)
• I cannot download attachments as the server returns Internal server error. This is logical as I misconfigured buckets.
• I cannot remove attachments as the server returns Internal server error. This could be also logical if the server tries to clean file and cannot find it on the misconfigured bucket.

So we have to find out why we cannot upload files to tc-challenge-v5-dev AWS, while can upload to submission-staging-dev. There must be something with FileStack or AWS configuration.

[vikasrohit]

@rootelement @mtwomey can you guys help us here?

[rootelement]

I've reached out to Guna to fix the s3 bucket. @vikasrohit this is exactly like how the file upload in community app works. The idea is using the Filestack Dev key to upload to the tc-challenge-v5-dev bucket, then the challenge api should be able to read from it. I'm unclear how the frontend of filestack works, but if you're using this filestack api key in dev: AzFINuQoqTmqw0QEoaw9az you should be able to write to the tc-challenge-v5-dev bucket.

Matt and I will look into uploading to a workflow so things are virus scanned automatically.

I will confirm here when Guna has fixed the security issue.

[vikasrohit]

@rootelement Do we have confirmation for S3 bucket? @maxceem can you please give it a try in case it has been fixed?

[maxceem]

@vikasrohit it's still forbidden for tc-challenge-v5-dev bucket:

[maxceem]

@vikasrohit Uploading attachments work well now end-to-end now.

There is only one issue with Challenge API when adding multiple attachments at once:

• we send multiple POST /challenges/:id/attachments requests in parallel and all of them are successful
• but after I reload the challenge GET /challenge/:id returns only one attachment

See demo video https://monosnap.com/file/mVqe0igPbDIa68bUepKwRujaTU0mQN

Example challenge 29d3d66f-d87f-402f-882f-258235ac7a6f on DEV.

I've logged an issue on Challenge API topcoder-platform/challenge-api#348.
FYI @ThomasKranitsas

[vikasrohit]

Can we merge it now @maxceem ?

Sorry @maxceem I didn't read your message, it was a stale UI which didn't show me your latest message.

I guess, as it has issues right now, I would like to wait further to get it resolved.

[vikasrohit]

@maxceem I am merging PR now as it seems the issue topcoder-platform/challenge-api#348 is fixed.

[maxceem]

@vikasrohit looks like something is not configured properly on DEV. When I'm trying to add attachments I'm getting errors from AWS services I guess:

[maxceem]

@vikasrohit locally the issue above is not reproducible and works good for me:

The endpoint in challenge API has changed, so we would have to update UI code first before attachments creation would work.

So I've reverted the PR #1035 for now, just in case we couldn't update it before the Jan 15th release.

[maxceem]

Uploading attachments on DEV works good. But on PROD there seems to be some config issue as uploading fails during uploading files to AWS S3 due to 403 permission errors:

start was good:

upload was good:

but actual uploading to S3 failed because of authentication issue 403:

[vikasrohit]

Moving it out to next milestone after Project Defaults shape up project as we are not able to get it working in production. We need latest challenge api in production to get it working.

fyi @acshields

[acshields]

@vikasrohit - noted.

[maxceem]

@vikasrohit looks like Challenge API has been deployed to PROD topcoder-platform/challenge-api@master...develop so we might give it another try.

[vikasrohit]

Yep, we can. But I would like to work on it only when we have its consumers i.e. some app is using the attachments.

[vikasrohit]

@acshields @rootelement do you know if Community/Forums app is already have code or plans to use the challenge Attachments?

[rootelement]

I believe Community App will read attachments. I think that needs to be tested. Forums uses their own attachments mechanism, but i don't believe they're reading challenge-api attachments at all. The security on the endpoint for attachments requires you're registered on the challenge as a submitter (or above... copilot, manager, observer) to get the attachments.

[vikasrohit]

Okay, thanks for the updates @rootelement. @acshields let us know how you want to prioritize it, IMO, we should do it once we have confirmation on the usage of attachments.

[acshields]

@vikasrohit - for my clarity, when you say confirmation on the usage of attachments, are you referring to the forums usage of attachments?

[vikasrohit]

I meant any kind of usage of the challenge attachments. I am not sure where exactly we want to consume the attachments of the challenge, but what I want to say is that we should retest this feature and deploy to prod when we know there is some consumer of this feature whether it is Community app or forums.

[vikasrohit]

@acshields moving it out from bug fix cycle.

[acshields]

@vikasrohit - noted. Right now, I have this as an aspect of the new UI implementation for WM. If that project pitch gets accepted for the next round, we could group this into that effort.

[yoution]

@phead198708 please assign to me

[yoution]

@vikasrohit @acshields can you provide the new UI implementation?
currently the ui is like so

[vikasrohit]

Sorry @yoution I guess this issue is not ready yet.