[$350] Duplicate tickets #234

jmgasper · 2019-11-21T09:57:27Z

@afrisalyp - As you noted last week, we can sometimes get duplicate tickets, like what I saw here with the timeouts:

We need to make sure that this can't happen. I'm open to suggestions.

jmgasper · 2019-11-21T09:57:41Z

Contest https://www.topcoder-dev.com/challenges/30107517 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

jmgasper · 2019-11-21T09:57:48Z

Contest https://www.topcoder-dev.com/challenges/30107518 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

jmgasper · 2019-11-21T10:21:52Z

@afrisalyp - Another one where duplicates were created:

#235

afrisalyp · 2019-11-21T14:02:41Z

@jmgasper
Does the prod deployment run multiple instances of the processor?
If yes, did you put different KAFKA_GROUP_ID value for each of them?

So far, I can reproduce the problem if I run multiple instances of the processor. Ideally, all of the instances must have the same KAFKA_GROUP_ID value. But I got some errors. So I ran multiple processors but with different KAFKA_GROUP_ID. This deployment makes the duplicate issue occurs.

So I believe this is the main problem that causes this duplicate issue.
I'm working to fix it so the processors can run with the same KAFKA_GROUP_ID.

jmgasper · 2019-11-21T19:50:29Z

@afrisalyp - I'm checking on that, but I'm not sure that's the issue. For instance, I closed out a ticket that got migrated over, and it's attempting to create a new challenge, instead of using the existing one. Can you check why that may be?

Relevant logs:

debug: received message from kafka: {"topic":"topcoder-x","originator":"topcoder-x-receiver","timestamp":"2019-11-21T19:26:37.869Z","mime-type":"application/json","payload":{"value":"{\"event\":\"issue.closed\",\"provider\":\"github\",\"data\":{\"issue\":{\"number\":47,\"body\":\"Steps: Select `Test-Project` project -> Select `Scope` tab\\r\\n\\r\\n- Current result: The `Scope` tab doesn't highlight\\r\\n- Expected result: The `Scope` tab should highlight\\r\\n\\r\\n<img width=\\\"886\\\" alt=\\\"Screen Shot 2019-11-21 at 13 57 43\\\" src=\\\"https://user-images.githubusercontent.com/5815182/69314514-ef58e780-0c66-11ea-94ba-b7a240a4f46d.png\\\">\\r\\n\",\"title\":\"[$25] - The selected tab doesn't highlight\",\"labels\":[\"tcx_FixAccepted\",\"tcx_OpenForPickup\",\"tcx_ReadyForReview\"],\"assignees\":[{\"id\":5815182}],\"owner\":{\"id\":5815182}},\"repository\":{\"id\":221569487,\"name\":\"ABB_TaaS\",\"full_name\":\"topcoderinc/ABB_TaaS\"},\"assignee\":{\"id\":5815182}}}"}}
debug: ENTER process
debug: input arguments
debug: { event:
   { event: 'issue.closed',
     provider: 'github',
     data: { issue: [Object], repository: [Object], assignee: [Object] } } }
debug: ENTER getRepositoryCopilotOrOwner
debug: input arguments
debug: { provider: 'github', repoFullName: 'topcoderinc/ABB_TaaS' }
debug: EXIT getRepositoryCopilotOrOwner
debug: output arguments
debug: { accessToken: '<removed>',
  userProviderId: 21790,
  topcoderUsername: 'ghostar' }
debug: ENTER getUsernameById
debug: input arguments
debug: { copilot:
   { accessToken: '<removed>',
     userProviderId: 21790,
     topcoderUsername: 'ghostar' },
  userId: 5815182 }
debug: EXIT getUsernameById
debug: output arguments
debug: 'suppermancool'
debug: existing project was found with id 24535 for repository topcoderinc/ABB_TaaS
debug: existing project was found with id 24535 for repository topcoderinc/ABB_TaaS
debug: existing project was found with id 24535 for repository topcoderinc/ABB_TaaS
debug: existing project was found with id 24535 for repository topcoderinc/ABB_TaaS
debug: existing project was found with id 24535 for repository topcoderinc/ABB_TaaS
debug: EndPoint: POST /challenges,  POST parameters: {"param":{"confidentialityType":"PUBLIC","technologies":[],"subTrack":"FIRST_2_FINISH","name":"- The selected tab doesn't highlight","reviewType":"COMMUNITY","milestoneId":1,"detailedRequirements":"<p>Steps: Select <code>Test-Project</code> project -&gt; Select <code>Scope</code> tab</p>\n<ul>\n<li>Current result: The <code>Scope</code> tab doesn't highlight</li>\n<li>Expected result: The <code>Scope</code> tab should highlight</li>\n</ul>\n<p>&lt;img width=&quot;886&quot; alt=&quot;Screen Shot 2019-11-21 at 13 57 43&quot; src=&quot;https://user-images.githubusercontent.com/5815182/69314514-ef58e780-0c66-11ea-94ba-b7a240a4f46d.png&quot;&gt;</p>\n","submissionGuidelines":"Git issue link: https://github.com/topcoderinc/ABB_TaaS/issues/47","registrationStartsAt":"2019-11-21T19:26:39.653Z","registrationEndsAt":"2019-11-26T19:26:39.653Z","submissionEndsAt":"2019-11-26T19:26:39.653Z","platforms":[],"finalDeliverableTypes":[],"prizes":[25],"projectId":24535,"task":true}}, Status Code:null,
    Error: 'Failed to create challenge.', Details: {"status":504,"response":{"req":{"method":"POST","url":"http://api.topcoder.com/v4/challenges","data":{"param":{"confidentialityType":"PUBLIC","technologies":[],"subTrack":"FIRST_2_FINISH","name":"- The selected tab doesn't highlight","reviewType":"COMMUNITY","milestoneId":1,"detailedRequirements":"<p>Steps: Select <code>Test-Project</code> project -&gt; Select <code>Scope</code> tab</p>\n<ul>\n<li>Current result: The <code>Scope</code> tab doesn't highlight</li>\n<li>Expected result: The <code>Scope</code> tab should highlight</li>\n</ul>\n<p>&lt;img width=&quot;886&quot; alt=&quot;Screen Shot 2019-11-21 at 13 57 43&quot; src=&quot;https://user-images.githubusercontent.com/5815182/69314514-ef58e780-0c66-11ea-94ba-b7a240a4f46d.png&quot;&gt;</p>\n","submissionGuidelines":"Git issue link: https://github.com/topcoderinc/ABB_TaaS/issues/47","registrationStartsAt":"2019-11-21T19:26:39.653Z","registrationEndsAt":"2019-11-26T19:26:39.653Z","submissionEndsAt":"2019-11-26T19:26:39.653Z","platforms":[],"finalDeliverableTypes":[],"prizes":[25],"projectId":24535,"task":true}},"headers":{"user-agent":"node-superagent/3.7.0","authorization":"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6Ik1rWTNNamsxTWpNeU5Ua3dRalkzTmtKR00wRkZPRVl3TmtJd1FqRXlNVUk0TUVFNE9UQkZOZyJ9.eyJpc3MiOiJodHRwczovL3RvcGNvZGVyLmF1dGgwLmNvbS8iLCJzdWIiOiJVWmZOcVkyRDFjaUlZbWFpZ01ielZVbVBYdVJaZ0syaUBjbGllbnRzIiwiYXVkIjoiaHR0cHM6Ly93d3cudG9wY29kZXIuY29tLyIsImlhdCI6MTU3NDMxNzI4NywiZXhwIjoxNTc0NDAzNjg3LCJhenAiOiJVWmZOcVkyRDFjaUlZbWFpZ01ielZVbVBYdVJaZ0syaSIsInNjb3BlIjoid3JpdGU6Y2hhbGxlbmdlcyByZWFkOmNoYWxsZW5nZXMgYWxsOmNoYWxsZW5nZXMgcmVhZDpncm91cHMgY3JlYXRlOnN1Ym1pc3Npb24gdXBkYXRlOnN1Ym1pc3Npb24gcmVhZDpzdWJtaXNzaW9uIHJlYWQ6YnVzX3RvcGljcyB3cml0ZTpidXNfYXBpIHJlYWQ6cm9sZXMgcmVhZDp1c2VyX3Byb2ZpbGVzIHJlYWQ6cHJvamVjdCByZWFkOnByb2plY3QtdXNlciByZWFkOnByb2plY3QtcGVybWlzc2lvbiIsImd0eSI6ImNsaWVudC1jcmVkZW50aWFscyJ9.SPryIMQo_Kl_lXF9vpC3Mx-1p9IlxQqfDuaIlACp1UMpgDyPi4LX2Qu5W1sMeHrbTzXKL2lgkqXn0zXxUejhY9BPSNB_RzCKiC9j1rTO2TNw7pemEMuF0FPkYoVEnjKtAC1_TJdbC3AL9vZ1gV7rkvmhk-ABCpZNS0bOTQqC5Sg7xNT4vdrdeUU5CW2rH43f0NVfbn2Ddzs8tAm_ImEqKsGQ6B3xV1khCJ5pak6aTXmCuklXElokY6-ox9vv3MASsLtkxL2v692o2R2cotA9kkUun8GzyjCfYpJz0jP4Pvw5-akLE6ojkJ3XSmh_C6RD8WNMsqG_FqHmocXliNM2ZA","content-type":"application/json"}},"header":{"date":"Thu, 21 Nov 2019 19:31:39 GMT","content-length":"0","connection":"close","server":"nginx","access-control-allow-headers":"Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since","access-control-allow-credentials":"true","access-control-allow-methods":"GET, POST, OPTIONS, DELETE, PUT, PATCH"},"status":504,"text":""}}

jmgasper · 2019-11-21T20:04:20Z

@afrisalyp - It just seems like it's trying to create a new ticket for every action, like label assigned, user assigned, etc...

jmgasper · 2019-11-21T20:12:44Z

For this particular ticket, I see multiple DynamoDB entries. I'm not sure why the initial ticket wasn't used for assigning:

        {
            "projectId": {
                "S": "4eeb7236-6721-4bc0-99b1-fabda68f2dee-1574306795229"
            },
            "challengeId": {
                "N": "30107517"
            },
            "number": {
                "N": "234"
            },
            "repositoryId": {
                "N": "135514613"
            },
            "labels": {
                "SS": [
                    "tcx_OpenForPickup"
                ]
            },
            "updatedAt": {
                "N": "1574330261094"
            },
            "status": {
                "S": "challenge_creation_successful"
            },
            "provider": {
                "S": "github"
            },
            "id": {
                "S": "1639cd81-dec0-47d5-9fb6-964439f4f712-1574330249210"
            },
            "prizes": {
                "NS": [
                    "100"
                ]
            },
            "title": {
                "S": "Duplicate tickets"
            },
            "body": {
                "S": "<p>@afrisalyp - As you noted last week, we can sometimes get duplicate tickets, like what I saw here with the timeouts:</p>\n<p>https://github.com/topcoder-platform/topcoder-x-ui/issues/233</p>\n<p>We need to make sure that this can't  happen.  I'm open to suggestions.</p>\n"
            }
        },

        {
            "projectId": {
                "S": "4eeb7236-6721-4bc0-99b1-fabda68f2dee-1574306795229"
            },
            "challengeId": {
                "N": "30107518"
            },
            "number": {
                "N": "234"
            },
            "repositoryId": {
                "N": "135514613"
            },
            "labels": {
                "SS": [
                    "tcx_OpenForPickup"
                ]
            },
            "updatedAt": {
                "N": "1574330268656"
            },
            "status": {
                "S": "challenge_creation_successful"
            },
            "provider": {
                "S": "github"
            },
            "id": {
                "S": "baf11f94-919e-4ed4-a50d-319fc0c6b90c-1574330257114"
            },
            "prizes": {
                "NS": [
                    "100"
                ]
            },
            "title": {
                "S": "Duplicate tickets"
            },
            "body": {
                "S": "<p>@afrisalyp - As you noted last week, we can sometimes get duplicate tickets, like what I saw here with the timeouts:</p>\n<p>https://github.com/topcoder-platform/topcoder-x-ui/issues/233</p>\n<p>We need to make sure that this can't  happen.  I'm open to suggestions.</p>\n"
            }
        },

jmgasper · 2019-11-21T20:13:03Z

@afrisalyp - Got confirmation that there is only one processor instance running.

afrisalyp · 2019-11-22T00:32:53Z

@jmgasper
Does the issue also appear in the dev environment?

jmgasper · 2019-11-22T00:48:05Z

@afrisalyp - I don't think we've seen it in dev, no. I'll test to make sure.

jmgasper · 2019-11-22T00:49:15Z

@afrisalyp - Is dev loading for you?

https://x.topcoder-dev.com/

afrisalyp · 2019-11-22T00:59:51Z

@jmgasper
No, I got blank screen.

jmgasper · 2019-11-22T07:12:02Z

@afrisalyp - Here's a sample from a test ticket I just logged.

debug: received message from kafka: {"topic":"topcoder-x","originator":"topcoder-x-receiver","timestamp":"2019-11-22T07:10:05.524Z","mime-type":"application/json","payload":{"value":"{\"event\":\"issue.labelUpdated\",\"provider\":\"github\",\"data\":{\"issue\":{\"number\":241,\"body\":\"asdfasdf asdf asdf \",\"title\":\"[$5] Testing 1,2,3\",\"labels\":[\"tcx_OpenForPickup\"],\"assignees\":[],\"owner\":{\"id\":21790}},\"repository\":{\"id\":135514613,\"name\":\"topcoder-x-ui\",\"full_name\":\"topcoder-platform/topcoder-x-ui\"},\"labels\":[\"tcx_OpenForPickup\"]}}"}}
debug: ENTER process
debug: input arguments
debug: { event:
   { event: 'issue.labelUpdated',
     provider: 'github',
     data: { issue: [Object], repository: [Object], labels: [Array] } } }
debug:  topics ======= [{"offset":25335,"messageSize":509,"message":{"crc":-1905224099,"magicByte":0,"attributes":{"_raw":0,"codec":0},"key":null,"value":{"type":"Buffer","data":[123,34,116,111,112,105,99,34,58,34,116,111,112,99,111,100,101,114,45,120,34,44,34,111,114,105,103,105,110,97,116,111,114,34,58,34,116,111,112,99,111,100,101,114,45,120,45,114,101,99,101,105,118,101,114,34,44,34,116,105,109,101,115,116,97,109,112,34,58,34,50,48,49,57,45,49,49,45,50,50,84,48,55,58,49,48,58,48,53,46,52,48,50,90,34,44,34,109,105,109,101,45,116,121,112,101,34,58,34,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111,110,34,44,34,112,97,121,108,111,97,100,34,58,123,34,118,97,108,117,101,34,58,34,123,92,34,101,118,101,110,116,92,34,58,92,34,105,115,115,117,101,46,99,114,101,97,116,101,100,92,34,44,92,34,112,114,111,118,105,100,101,114,92,34,58,92,34,103,105,116,104,117,98,92,34,44,92,34,100,97,116,97,92,34,58,123,92,34,105,115,115,117,101,92,34,58,123,92,34,110,117,109,98,101,114,92,34,58,50,52,49,44,92,34,98,111,100,121,92,34,58,92,34,97,115,100,102,97,115,100,102,32,97,115,100,102,32,97,115,100,102,32,92,34,44,92,34,116,105,116,108,101,92,34,58,92,34,91,36,53,93,32,84,101,115,116,105,110,103,32,49,44,50,44,51,92,34,44,92,34,108,97,98,101,108,115,92,34,58,91,92,34,116,99,120,95,79,112,101,110,70,111,114,80,105,99,107,117,112,92,34,93,44,92,34,97,115,115,105,103,110,101,101,115,92,34,58,91,93,44,92,34,111,119,110,101,114,92,34,58,123,92,34,105,100,92,34,58,50,49,55,57,48,125,125,44,92,34,114,101,112,111,115,105,116,111,114,121,92,34,58,123,92,34,105,100,92,34,58,49,51,53,53,49,52,54,49,51,44,92,34,110,97,109,101,92,34,58,92,34,116,111,112,99,111,100,101,114,45,120,45,117,105,92,34,44,92,34,102,117,108,108,95,110,97,109,101,92,34,58,92,34,116,111,112,99,111,100,101,114,45,112,108,97,116,102,111,114,109,47,116,111,112,99,111,100,101,114,45,120,45,117,105,92,34,125,125,125,34,125,125]}}}]
debug: received message from kafka: {"topic":"topcoder-x","originator":"topcoder-x-receiver","timestamp":"2019-11-22T07:10:05.402Z","mime-type":"application/json","payload":{"value":"{\"event\":\"issue.created\",\"provider\":\"github\",\"data\":{\"issue\":{\"number\":241,\"body\":\"asdfasdf asdf asdf \",\"title\":\"[$5] Testing 1,2,3\",\"labels\":[\"tcx_OpenForPickup\"],\"assignees\":[],\"owner\":{\"id\":21790}},\"repository\":{\"id\":135514613,\"name\":\"topcoder-x-ui\",\"full_name\":\"topcoder-platform/topcoder-x-ui\"}}}"}}
debug: ENTER process
debug: input arguments
debug: { event:
   { event: 'issue.created',
     provider: 'github',
     data: { issue: [Object], repository: [Object] } } }
debug: ENTER getRepositoryCopilotOrOwner
debug: input arguments
debug: { provider: 'github',
  repoFullName: 'topcoder-platform/topcoder-x-ui' }
debug: ENTER getRepositoryCopilotOrOwner
debug: input arguments
debug: { provider: 'github',
  repoFullName: 'topcoder-platform/topcoder-x-ui' }
debug: EXIT getRepositoryCopilotOrOwner
debug: output arguments
debug: { accessToken: '<removed>',
  userProviderId: 21790,
  topcoderUsername: 'ghostar' }
debug: EXIT getRepositoryCopilotOrOwner
debug: output arguments
debug: { accessToken: '<removed>',
  userProviderId: 21790,
  topcoderUsername: 'ghostar' }
debug: DB Issue number: 241
debug: DB Issue provider: github
debug: DB Issue repository: 135514613
error: Error happened in process
TypeError: Cannot read property 'id' of null
    at handleIssueLabelUpdated (/topcoder-x-processor/services/IssueService.js:580:47)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:229:7)
error:  message=Cannot read property 'id' of null, stack=TypeError: Cannot read property 'id' of null
    at handleIssueLabelUpdated (/topcoder-x-processor/services/IssueService.js:580:47)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:229:7), logged=true
debug: existing project was found with id 17159 for repository topcoder-platform/topcoder-x-ui

jmgasper · 2019-11-22T07:12:45Z

Seems like we get label updated before issue created, but that was never an issue in the past.

afrisalyp · 2019-11-22T09:59:07Z

@jmgasper
The new test ticket seems to create a new challenge after an assignment or unassignment consistently.
Can you please provide the logs when the duplication created occurs? And also please provide the DB records of the ticket if it's possible.

afrisalyp · 2019-11-22T10:03:24Z

This issue seems affect the prod only. The dev and my local deployment don't.

jmgasper · 2019-11-22T10:34:42Z

@afrisalyp - Yeah, agreed. We didn't get this in dev in my testing.

jmgasper · 2019-11-22T10:37:14Z

I sent logs targeting this test issue: #242 , and a full dump of the issues table via email. If you want to do extra logging anywhere, feel free to open a merge request with the additional logging and I'll get it deployed.

jmgasper · 2019-11-22T19:54:20Z

@afrisalyp - Any ideas? Any additional logging we can add that would be helpful?

afrisalyp · 2019-11-22T23:11:22Z

@jmgasper
A merge request added. I put some logs to see how the DB query works.
Please test the scenario so we get duplicate issue.
Also, let me try to change the Kafka Group Consumer to Simple Consumer and we'll see if it has an impact.
Thanks.

jmgasper · 2019-11-22T23:16:30Z

@afrisalyp - Thanks. Deploying now.

jmgasper · 2019-11-24T01:45:46Z

@afrisalyp - Where are the different scanOne calls made? Are they using slightly different DB configs or queries somehow?

Seems like when ensureChallengeExists works, the next call doesn't and vice versa.

jmgasper · 2019-11-24T01:57:11Z

And maybe we have to add handling for an empty result? Seems like we don't have that currently.

afrisalyp · 2019-11-24T02:11:23Z

@jmgasper
The log is a process from handleIssueLabelUpdated which the event can be received before issue.created. So maybe the issue is still empty. And yeah, We can handle this.

But I think we need to investigate the log related to handleIssueAssignment. That's the process when the duplication occurs.

afrisalyp · 2019-11-24T02:15:32Z

@afrisalyp - We can't do that config update. That's not needed, and Topcoder explicitly didn't want it configured like that, just FYI. Is that config update needed?

I'm wondering how's the prod deployed. How's the connection between the topcoder-x app and Dynamodb instance?
The config is mandatory if the service want to connect to the dynamodb instance in aws cloud. But it doesn't necessary if the dynamodb is locally deployed.

afrisalyp · 2019-11-24T02:26:46Z

@afrisalyp - Where are the different scanOne calls made? Are they using slightly different DB configs or queries somehow?

I put consistent function when the app query to the DB. It makes sure that we get consistent data. If the dynamoDB is deployed with replications, the DB query can sometimes get un-updated data record without consistent function.

jmgasper · 2019-11-24T02:59:19Z

@afrisalyp - The environment is set up separately for AWS - this was discussed quite a bit with Topcoder when they made the change. That shouldn't affect the app randomly though, should it? If that was wrong, the app just wouldn't work at all.

afrisalyp · 2019-11-24T03:29:20Z

@jmgasper - I added handler if the dbissue is null or empty.
We'll put it in the reschedule mechanism.
Is that acceptable?

This also reated with #232

afrisalyp · 2019-11-24T03:35:18Z

@afrisalyp - The environment is set up separately for AWS - this was discussed quite a bit with Topcoder when they made the change. That shouldn't affect the app randomly though, should it? If that was wrong, the app just wouldn't work at all.

I'm not sure. But since I don't experience the duplication issue in my local env and also we don't see it happens in the dev env so far, I just think about the prod deployment.
Or, maybe something missed in the migration. The DB is just migrated from mongodb, isn't it?.

jmgasper · 2019-11-24T03:45:34Z

@afrisalyp - Yeah, the data was migrated, but I don't see anything weird in DynamoDB, do you? I'm also asking Topcoder about differences in how the DynamoDB tables are set up in dev vs. prod. I agree that it's really strange that this issue is only affecting dev.

jmgasper · 2019-11-24T04:01:25Z

The changes to handle the null record seem to have improved things slightly.

jmgasper · 2019-11-26T10:57:25Z

Contest https://www.topcoder.com/challenges/30108143 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

jmgasper · 2019-11-26T10:57:26Z

This ticket was not processed for payment. If you would like to process it for payment, please reopen it, add the tcx_FixAccepted label, and then close it again

This is an automated message for ghostar via Topcoder X

jmgasper · 2019-11-26T10:59:10Z

Contest https://www.topcoder.com/challenges/30108143 has been updated - it has been assigned to afrisalyp.

This is an automated message for ghostar via Topcoder X

jmgasper · 2019-11-26T11:00:19Z

Payment task has been updated: https://software.topcoder-dev.com/review/actions/ViewProjectDetails?pid=30108143

This is an automated message for ghostar via Topcoder X

jmgasper added the tcx_OpenForPickup label Nov 21, 2019

jmgasper assigned afrisalyp Nov 21, 2019

jmgasper changed the title ~~[$100] Duplicate tickets~~ [$150] Duplicate tickets Nov 21, 2019

afrisalyp added tcx_Assigned and removed tcx_Assigned labels Nov 22, 2019

jmgasper changed the title ~~[$150] Duplicate tickets~~ [$250] Duplicate tickets Nov 22, 2019

afrisalyp mentioned this issue Nov 22, 2019

Add logs to debug duplicate issue. topcoder-archive/topcoder-platform-topcoder-x-processor#26

Merged

jmgasper added tcx_FixAccepted and removed tcx_OpenForPickup labels Nov 22, 2019

afrisalyp mentioned this issue Nov 24, 2019

Handle if dbissue is null. topcoder-archive/topcoder-platform-topcoder-x-processor#32

Merged

jmgasper added tcx_OpenForPickup and removed tcx_Feedback labels Nov 26, 2019

jmgasper closed this as completed Nov 26, 2019

jmgasper reopened this Nov 26, 2019

jmgasper assigned afrisalyp and unassigned afrisalyp Nov 26, 2019

jmgasper unassigned afrisalyp Nov 26, 2019

jmgasper assigned afrisalyp Nov 26, 2019

jmgasper added tcx_Assigned tcx_FixAccepted and removed tcx_OpenForPickup tcx_Assigned labels Nov 26, 2019

jmgasper closed this as completed Nov 26, 2019

jmgasper added tcx_Assigned tcx_Paid labels Nov 26, 2019

[$350] Duplicate tickets #234

[$350] Duplicate tickets #234

Comments

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

afrisalyp commented Nov 21, 2019

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

jmgasper commented Nov 21, 2019

afrisalyp commented Nov 22, 2019

jmgasper commented Nov 22, 2019

jmgasper commented Nov 22, 2019

afrisalyp commented Nov 22, 2019

jmgasper commented Nov 22, 2019

jmgasper commented Nov 22, 2019

afrisalyp commented Nov 22, 2019

afrisalyp commented Nov 22, 2019

jmgasper commented Nov 22, 2019

jmgasper commented Nov 22, 2019

jmgasper commented Nov 22, 2019

afrisalyp commented Nov 22, 2019

jmgasper commented Nov 22, 2019

jmgasper commented Nov 24, 2019

jmgasper commented Nov 24, 2019

afrisalyp commented Nov 24, 2019

afrisalyp commented Nov 24, 2019

afrisalyp commented Nov 24, 2019

jmgasper commented Nov 24, 2019

afrisalyp commented Nov 24, 2019

afrisalyp commented Nov 24, 2019

jmgasper commented Nov 24, 2019

jmgasper commented Nov 24, 2019

jmgasper commented Nov 26, 2019

jmgasper commented Nov 26, 2019

jmgasper commented Nov 26, 2019

jmgasper commented Nov 26, 2019