Skip to content

feat: add phase management api #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
495e4e6
fix: only sanitize html
rakibansary May 1, 2023
40018d7
Merge branch 'dev' into feature/edit-task-payment
rakibansary May 5, 2023
8064a4e
fix: enable read only root file system
rakibansary May 5, 2023
4114f71
fix: yarn not needed
rakibansary May 5, 2023
4acdf36
Merge branch 'dev' into feature/edit-task-payment
rakibansary May 8, 2023
6ce398e
feat: add interfaces for fetching phase facts
rakibansary May 15, 2023
e0448e0
ci: deploy to dev
rakibansary May 16, 2023
61436de
fix: cyclic dependency
rakibansary May 16, 2023
9b028f0
fix: success undefined
rakibansary May 16, 2023
582b77f
fix: success undefined
rakibansary May 16, 2023
d5aa181
feat: index challenge after advancing phase
rakibansary May 16, 2023
096334e
fix: track, type is not needed
rakibansary May 16, 2023
f1f38d6
fix: use id from updated challenge
rakibansary May 16, 2023
b3587de
fix: phase must be in open state before it can be closed
rakibansary May 16, 2023
711c2af
fix: include operation in next
rakibansary May 21, 2023
75bbf8a
fix: phase name is sanitized
rakibansary May 21, 2023
229474d
fix: use normalized phase name
rakibansary May 21, 2023
49a8e52
fix: include additional facts in open phase operation
rakibansary May 21, 2023
d02fb6a
fix: submission phase closure check
rakibansary May 23, 2023
6715751
Merge branch 'dev' into feature/phase-advance
rakibansary May 23, 2023
31cec39
feat: add origin control
eisbilir Jun 17, 2023
43fe131
ci: deploy to dev
eisbilir Jun 17, 2023
267670f
chore: use updated versions
rakibansary Jun 17, 2023
39c4ed1
Merge remote-tracking branch 'origin/fix/plat-2985' into feature/phas…
rakibansary Jun 17, 2023
d9b55d1
wip: iterative review phase management
rakibansary Jun 19, 2023
0b1ae00
fix: incorrect constraint name check
rakibansary Jun 22, 2023
da0be50
wip: open next iterative review phase when appropriate
rakibansary Jun 26, 2023
724f458
fix: typo
rakibansary Jun 26, 2023
f761e06
fix: better handling of constraints
rakibansary Jun 26, 2023
6255831
fix: incorrect filter
rakibansary Jun 26, 2023
cb574a0
fix: remove - from incoming phase name
rakibansary Jul 3, 2023
d4bf9f1
feat: complete challenge on receiving winning submission
rakibansary Jul 3, 2023
4d3d794
chore: add comment
rakibansary Jul 3, 2023
b4d7b02
fix: handle closing challenge when winner is found
rakibansary Jul 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ install_deploysuite: &install_deploysuite
cp ./../buildscript/master_deploy.sh .
cp ./../buildscript/buildenv.sh .
cp ./../buildscript/awsconfiguration.sh .

restore_cache_settings_for_build: &restore_cache_settings_for_build
key: docker-node-modules-{{ checksum "yarn.lock" }}

Expand All @@ -32,8 +33,14 @@ builddeploy_steps: &builddeploy_steps
- run: *install_deploysuite
- restore_cache: *restore_cache_settings_for_build
- run:
name: "Authenticate with CodeArtifact and build docker image"
command: "./awsconfiguration.sh ${CODEARTIFACT_ENV}\nsource awsenvconf\naws codeartifact login --tool npm --repository topcoder-framework --domain topcoder --domain-owner $AWS_ACCOUNT_ID --region $AWS_REGION --namespace @topcoder-framework\ncp ~/.npmrc .\nrm -f awsenvconf \n./build.sh ${APPNAME}\n"
name: "Authenticate with CodeArtifact and build Docker image"
command: |
./awsconfiguration.sh ${CODEARTIFACT_ENV}
source awsenvconf
aws codeartifact login --tool npm --repository topcoder-framework --domain topcoder --domain-owner $AWS_ACCOUNT_ID --region $AWS_REGION --namespace @topcoder-framework
cp ~/.npmrc .
rm -f awsenvconf
./build.sh ${APPNAME}
- save_cache: *save_cache_settings
- deploy:
name: Running MasterScript.
Expand Down Expand Up @@ -83,7 +90,7 @@ workflows:
branches:
only:
- dev
- fix/floating-point-arhithmetic
- feature/phase-advance

- "build-qa":
context: org-global
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,5 @@ typings/
.next
ecr-login.sh
.npmrc

test.js
4 changes: 4 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"cors": "^2.7.1",
"decimal.js": "^10.4.3",
"deep-equal": "^2.2.0",
"dompurify": "^3.0.2",
"dotenv": "^8.2.0",
"elasticsearch": "^16.7.3",
"express": "^4.15.4",
Expand All @@ -62,8 +63,11 @@
"http-aws-es": "^6.0.0",
"http-status-codes": "^1.3.0",
"joi": "^14.0.0",
"jsdom": "^21.1.2",
"json-rules-engine": "^6.1.2",
"jsonwebtoken": "^8.3.0",
"lodash": "^4.17.19",
"markdown-it": "^13.0.1",
"moment": "^2.24.0",
"node-cache": "^5.1.2",
"swagger-ui-express": "^4.1.3",
Expand Down
11 changes: 8 additions & 3 deletions src/common/helper.js
Original file line number Diff line number Diff line change
Expand Up @@ -226,13 +226,18 @@ async function deleteFromS3(bucket, key) {
* @param {String} challengeId the challenge id
* @returns {Promise<Array>} the challenge resources
*/
async function getChallengeResources(challengeId) {
async function getChallengeResources(challengeId, roleId = null) {
const token = await m2mHelper.getM2MToken();
const perPage = 100;
let page = 1;
let result = [];
while (true) {
const url = `${config.RESOURCES_API_URL}?challengeId=${challengeId}&perPage=${perPage}&page=${page}`;
const url = `${
config.RESOURCES_API_URL
}?challengeId=${challengeId}&perPage=${perPage}&page=${page}${
roleId ? `&roleId=${roleId}` : ""
}`;

const res = await axios.get(url, {
headers: { Authorization: `Bearer ${token}` },
});
Expand Down Expand Up @@ -1020,7 +1025,7 @@ async function getGroupById(groupId) {
/**
* Get challenge submissions
* @param {String} challengeId the challenge id
* @returns {Array} the submission
* @returns {Promise<Array>} the submission
*/
async function getChallengeSubmissions(challengeId) {
const token = await m2mHelper.getM2MToken();
Expand Down
6 changes: 5 additions & 1 deletion src/common/phase-helper.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const phaseDomain = new PhaseDomain(GRPC_CHALLENGE_SERVER_HOST, GRPC_CHALLENGE_S
class ChallengePhaseHelper {
phaseDefinitionMap = {};
timelineTemplateMap = {};

async populatePhasesForChallengeCreation(phases, startDate, timelineTemplateId) {
if (_.isUndefined(timelineTemplateId)) {
throw new errors.BadRequestError(`Invalid timeline template ID: ${timelineTemplateId}`);
Expand Down Expand Up @@ -176,7 +177,10 @@ class ChallengePhaseHelper {

handlePhasesAfterCancelling(phases) {
return _.map(phases, (phase) => {
const shouldClosePhase = _.includes(["Registration", "Submission", "Checkpoint Submission"], phase.name);
const shouldClosePhase = _.includes(
["Registration", "Submission", "Checkpoint Submission"],
phase.name
);
return {
...phase,
isOpen: shouldClosePhase ? false : phase.isOpen,
Expand Down
10 changes: 10 additions & 0 deletions src/controllers/ChallengeController.js
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,15 @@ async function deleteChallenge(req, res) {
res.send(result);
}

/**
* Advance phase
* @param {Object} req the request
* @param {Object} res the response
*/
async function advancePhase(req, res) {
res.send(await service.advancePhase(req.authUser, req.params.challengeId, req.body));
}

module.exports = {
searchChallenges,
createChallenge,
Expand All @@ -124,4 +133,5 @@ module.exports = {
deleteChallenge,
getChallengeStatistics,
sendNotifications,
advancePhase,
};
249 changes: 249 additions & 0 deletions src/phase-management/PhaseAdvancer.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
const config = require("config");
const { Engine } = require("json-rules-engine");

const rulesJSON = require("./phase-rules.json");
const errors = require("../common/errors");

const helper = require("../common/helper");

// Helper functions

// TODO: Move these to a common place

const normalizeName = (name) => name.replace(/ /g, "");

const shouldCheckConstraint = (operation, phase, constraintName, rules) => {
const normalizedConstraintName = normalizeName(constraintName);
return (
operation === "close" &&
phase.constraints &&
rules.constraintRules[phase.name]?.includes(normalizedConstraintName)
);
};

const parseDate = (dateString) => {
const date = new Date(dateString).getTime();
return isNaN(date) ? null : date;
};

// End of helper functions

class PhaseAdvancer {
#rules = rulesJSON;

#factGenerators = {
Registration: async (challengeId) => ({
registrantCount: await this.#getRegistrantCount(challengeId),
}),
Submission: async (challengeId) => ({
submissionCount: await this.#getSubmissionCount(challengeId),
hasActiveUnreviewedSubmissions: await this.#hasActiveUnreviewedSubmissions(challengeId),
}),
Review: async (challengeId) => ({
allSubmissionsReviewed: await this.#areAllSubmissionsReviewed(challengeId),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another fact that checks if all test cases are uploaded If there is a development reviewer for challenge. But this could be something we are not doing anymore.

}),
"Iterative Review": async (challengeId) => ({
hasActiveUnreviewedSubmissions: await this.#hasActiveUnreviewedSubmissions(challengeId),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasActiveUnreviewedSubmissions is for opening the phase.
for closing the phase, the fact should check if the earliest active submission has a committed review. there can be other active unreviewed submissions in the queue but they are ignored.

}),
"Appeals Response": async (challengeId) => ({
allAppealsResolved: await this.#areAllAppealsResolved(challengeId),
}),
};

async #generateFacts(challengeId, phases, phase, operation) {
const facts = {
name: phase.name,
isOpen: phase.isOpen,
isClosed: !phase.isOpen && phase.actualEndDate != null,
isPastScheduledStartTime: this.#isPastTime(phase.scheduledStartDate),
isPastScheduledEndTime: this.#isPastTime(phase.scheduledEndDate),
isPostMortemOpen: phases.find((p) => p.name === "Post-Mortem")?.isOpen,
hasPredecessor: phase.predecessorId != null,
isPredecessorPhaseClosed:
phase.predecessorId != null
? this.#isPastTime(phases.find((p) => p.phaseId === phase.predecessorId)?.actualEndDate)
: true,
nextPhase: phases.find((p) => p.predecessor === phase.phaseId)?.name,
};

if (operation === "close" && this.#factGenerators[phase.name]) {
const additionalFacts = await this.#factGenerators[phase.name](challengeId);
Object.assign(facts, additionalFacts);
}

return facts;
}

async advancePhase(challengeId, phases, operation, phaseName) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to identify correct phase if phase name is "Iterative Review"

const phase = phases.find((phase) => phase.name === phaseName);

if (!phase) {
throw new errors.BadRequestError(`Phase ${phaseName} not found`);
}

const essentialRules = this.#rules[`${operation}Rules`][normalizeName(phase.name)]
? this.#rules[`${operation}Rules`][normalizeName(phase.name)].map((rule) => ({
name: rule.name,
conditions: rule.conditions,
event: rule.event,
}))
: [];

const constraintRules =
phase.constraints
?.filter((constraint) =>
shouldCheckConstraint(operation, phase, constraint.name, this.#rules)
)
.map((constraint) => ({
name: `Constraint: ${constraint.name}`,
fact: normalizeName(constraint.name),
operator: "greaterOrEqual",
value: constraint.value,
})) || [];

const rules = [...essentialRules, ...constraintRules];
const facts = await this.#generateFacts(challengeId, phases, phase, operation);

console.log("rules", JSON.stringify(rules, null, 2));
console.log("facts", JSON.stringify(facts, null, 2));

for (const rule of rules) {
const ruleExecutionResult = await this.#executeRule(rule, facts);

if (!ruleExecutionResult.success) {
return {
success: false,
message: `Cannot ${operation} phase ${phase.name} for challenge ${challengeId}`,
detail: `Rule ${rule.name} failed`,
failureReasons: ruleExecutionResult.failureReasons,
};
}
}

if (operation === "open") {
await this.#open(challengeId, phases, phase);
} else if (operation === "close") {
await this.#close(challengeId, phases, phase);
}

const next = operation === "close" ? phases.filter((p) => p.predecessor === phase.phaseId) : [];

return {
success: true,
message: `Successfully ${operation}d phase ${phase.name} for challenge ${challengeId}`,
updatedPhases: phases,
next,
};
}

async #open(challengeId, phases, phase) {
console.log(`Opening phase ${phase.name} for challenge ${challengeId}`);

phase.isOpen = true;
const actualStartDate = new Date();
phase.actualStartDate = actualStartDate.toISOString();
phase.scheduledEndDate = new Date(
actualStartDate.getTime() + phase.duration * 1000
).toISOString();

const scheduledStartDate = parseDate(phase.scheduledStartDate);
const delta = scheduledStartDate - actualStartDate; // in milliseconds

if (delta !== 0) {
this.#updateSubsequentPhases(phases, phase, -delta);
}

console.log(`Updated phases: ${JSON.stringify(phases, null, 2)}`);
}

async #close(challengeId, phases, phase) {
console.log(`Closing phase ${phase.name} for challenge ${challengeId}`);

phase.isOpen = false;
const actualEndDate = new Date();
phase.actualEndDate = actualEndDate.toISOString();

const scheduledEndDate = parseDate(phase.scheduledEndDate);
const delta = scheduledEndDate - actualEndDate;

if (delta !== 0) {
this.#updateSubsequentPhases(phases, phase, -delta);
}

console.log(`Updated phases: ${JSON.stringify(phases, null, 2)}`);
}

#isPastTime(dateString) {
if (dateString == null) {
return false;
}
const date = new Date(dateString);
const now = new Date();
return date <= now;
}

async #getRegistrantCount(challengeId) {
console.log(`Getting registrant count for challenge ${challengeId}`);
// TODO: getChallengeResources loops through all pages, which is not necessary here, we can just get the first page and total count from header[X-Total]
const submitters = await helper.getChallengeResources(challengeId, config.SUBMITTER_ROLE_ID);
return submitters.length;
}

async #getSubmissionCount(challengeId) {
console.log(`Getting submission count for challenge ${challengeId}`);
// TODO: getChallengeSubmissions loops through all pages, which is not necessary here, we can just get the first page and total count from header[X-Total]
const submissions = await helper.getChallengeSubmissions(challengeId);
return submissions.length;
}

async #areAllSubmissionsReviewed(challengeId) {
console.log(`Getting review count for challenge ${challengeId}`);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: also we need to check if enough review is present per submission according to "Reviewer Count" phase constraint.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will double check - however areAllSubmissionReviewed is an essential rule while checking if all submissions received the minimum requisite reviews would be a constraint check

return Promise.resolve(false);
}

async #areAllAppealsResolved(challengeId) {
console.log(`Checking if all appeals are resolved for challenge ${challengeId}`);
return Promise.resolve(false);
}

async #hasActiveUnreviewedSubmissions(challengeId) {
console.log(`Checking if there are active unreviewed submissions for challenge ${challengeId}`);
return Promise.resolve(false);
}

async #executeRule(rule, facts) {
const ruleEngine = new Engine();
ruleEngine.addRule(rule);

const result = await ruleEngine.run(facts);

const failureReasons = result.failureResults.map((failureResult) => ({
rule: rule.name,
failedConditions: failureResult.conditions.all
.filter((condition) => !condition.result)
.map((condition) => ({
fact: condition.fact,
operator: condition.operator,
value: condition.value,
})),
}));

return {
success: result.events.length > 0,
failureReasons,
};
}

// prettier-ignore
#updateSubsequentPhases(phases, currentPhase, delta) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is F2F, after submission phase is closed we shouldn't update iterative phases.

let nextPhase = phases.find((phase) => phase.predecessor === currentPhase.phaseId);

while (nextPhase) {
nextPhase.scheduledStartDate = new Date(new Date(nextPhase.scheduledStartDate).getTime() + delta).toISOString();
nextPhase.scheduledEndDate = new Date(new Date(nextPhase.scheduledEndDate).getTime() + delta).toISOString();
nextPhase = phases.find((phase) => phase.predecessor === nextPhase.phaseId);
}
}
}

module.exports = new PhaseAdvancer();
Loading