Optimise DynamoDB usage

[jmgasper]

We're starting to see problems (timeouts and other problems) with our DynamoDB usage, and a lot of that stems from how we migrated from MongoDB, I think. Stuff that worked well in Mongo may not be working well with DynamoDB.

Specifically:

1. We use scanOne a lot when query may be better
2. Indices may not be properly defined
3. https://github.com/topcoder-platform/topcoder-x-ui/blob/develop/src/common/db-helper.js#L85 -- We're using all() here, which is likely not the best approach.

We need to look into how we use DynamoDB and optimise it.

[afrisalyp]

@jmgasper

1. Seems we still need the all() feature, it will make sure that we will continue the operation to find the data if we got operation limit.

A single Query operation will read up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression. If LastEvaluatedKey is present in the response, you will need to paginate the result set. For more information, see Paginating the Results in the Amazon DynamoDB Developer Guide.

This statement is from https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html

We add the add() feature in the past to fix the confusing query problem. #256 (comment)

2. To prevent the operation scans all document, we need to use queryOne instead of query here https://github.com/topcoder-platform/topcoder-x-processor/blob/develop/utils/db-helper.js#L59. it will stop the operation if we have found one data.

3. For scanOne, we need to add options {one: true} or we can convert it to queryOne, but we need to add indexes.

Any thought on this?

[jmgasper]

1. Ok, we can leave that in
2. That'll be fine, thanks
3. Let's do queryOne and add the indices, thanks!