Skip to content

Feature/discourse migration #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Apr 23, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2531b5d
email support
gondzo Mar 23, 2018
28af61a
merge dev
gondzo Mar 23, 2018
3ca321f
send notification emails
gondzo Mar 24, 2018
ea7165a
add enable emails flag
gondzo Mar 24, 2018
692efab
Picking env variable from circle ci config
Apr 4, 2018
1750fc0
Fixed printf command to replace new values
Apr 4, 2018
35f9f09
bool => string
Apr 4, 2018
1617c0f
Fixed local development carry over which was causing notifications er…
Apr 4, 2018
3bbd5af
Fixed usage of base url env variable
Apr 4, 2018
e5d7686
One more fix for v4 base url
Apr 4, 2018
18c7411
One more fix for the env variable for base api url
Apr 4, 2018
41e37cc
debugging
Apr 4, 2018
3de5643
Upgrading event bus to V5
Apr 4, 2018
dc1996d
Removing projectId from email service payload
Apr 5, 2018
80c22bc
Merge branch 'dev' into feature/discourseMigration
Apr 6, 2018
3112c16
debugging
Apr 6, 2018
d69e10e
Supporting v5 format.
Apr 9, 2018
9bb179c
added action keyword in the topic name as per v5 specs
Apr 9, 2018
9189708
Handling mail aliases as per gmail format
Apr 10, 2018
9efc6a9
Lint fixes
Apr 10, 2018
edb80b0
Added dev mode feature for sending email notifications to fixed email…
Apr 10, 2018
8dbd1d4
logging message api url as it is still not picking up new topic
Apr 10, 2018
a774320
moved log statement to correct location
Apr 10, 2018
59ab433
Fixed the issue with missing MESSAGE_API_BASE_URL env variable
Apr 10, 2018
e3c0a57
Fixed the message api base url param at both places.
Apr 10, 2018
0a8ea8f
fixing parsing of dev mode env variable
Apr 10, 2018
36c6e0d
Added projectId for email templates
Apr 12, 2018
db305ef
Added topicId and postId as well to the email placeholder parameters
Apr 12, 2018
3d2ed89
Extracted constants for kafka topics
Apr 13, 2018
3168984
Merge branch 'dev' into feature/discourseMigration
Apr 16, 2018
3d2aa96
Updated message API URL as per v5 changes
Apr 17, 2018
4a8b467
Fixed issue in sending email notifications. Redundancy is the cause. …
Apr 17, 2018
a9c058d
Updated package-lock
Apr 23, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ workflows:
- "build-dev":
filters:
branches:
only: dev
only: [dev, 'feature/discourseMigration']
- "build-prod":
filters:
branches:
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,12 @@ The following parameters can be set in config files or in env variables:
- KAFKA_CLIENT_CERT_KEY: Kafka connection private key, optional;
if not provided, then SSL connection is not used, direct insecure connection is used;
if provided, it can be either path to private key file or private key content

- BUS_API_BASE_URL: Bus API url
- BUS_API_AUTH_TOKEN: Bus API auth token
- REPLY_EMAIL_PREFIX: prefix of the genereated reply email address
- REPLY_EMAIL_DOMAIN: email domain
- DEFAULT_REPLY_EMAIL: default reply to email address, for example [email protected]
- MENTION_EMAIL: recipient email used for email.project.post.mention event

Configuration for the connect notification server is at `connect/config.js`.
The following parameters can be set in config files or in env variables:
Expand Down
17 changes: 16 additions & 1 deletion config/default.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,20 @@ module.exports = {
KAFKA_CLIENT_CERT: process.env.KAFKA_CLIENT_CERT ? process.env.KAFKA_CLIENT_CERT.replace('\\n', '\n') : null,
KAFKA_CLIENT_CERT_KEY: process.env.KAFKA_CLIENT_CERT_KEY ?
process.env.KAFKA_CLIENT_CERT_KEY.replace('\\n', '\n') : null,
API_CONTEXT_PATH: process.env.API_CONTEXT_PATH || '/v5/notifications',

BUS_API_AUTH_TOKEN: process.env.BUS_API_AUTH_TOKEN,
MENTION_EMAIL: process.env.MENTION_EMAIL,
REPLY_EMAIL_PREFIX: process.env.REPLY_EMAIL_PREFIX,
REPLY_EMAIL_DOMAIN: process.env.REPLY_EMAIL_DOMAIN,

TC_ADMIN_TOKEN: process.env.TC_ADMIN_TOKEN,
TC_API_BASE_URL: process.env.TC_API_BASE_URL || 'https://api.topcoder-dev.com',
TC_API_V3_BASE_URL: process.env.TC_API_V3_BASE_URL || 'https://api.topcoder-dev.com/v3',
TC_API_V4_BASE_URL: process.env.TC_API_V4_BASE_URL || 'https://api.topcoder-dev.com/v4',
TC_API_V5_BASE_URL: process.env.TC_API_V5_BASE_URL || 'https://api.topcoder-dev.com/v5',
MESSAGE_API_BASE_URL: process.env.MESSAGE_API_BASE_URL || 'https://api.topcoder-dev.com/v4',
ENABLE_EMAILS: process.env.ENABLE_EMAILS || true,
ENABLE_DEV_MODE: process.env.ENABLE_DEV_MODE || true,
DEV_MODE_EMAIL: process.env.DEV_MODE_EMAIL,
API_CONTEXT_PATH: process.env.API_CONTEXT_PATH || '/v5/notifications',
};
1 change: 1 addition & 0 deletions connect/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
module.exports = {
TC_API_V3_BASE_URL: process.env.TC_API_V3_BASE_URL || 'https://api.topcoder-dev.com/v3',
TC_API_V4_BASE_URL: process.env.TC_API_V4_BASE_URL || 'https://api.topcoder-dev.com/v4',
MESSAGE_API_BASE_URL: process.env.MESSAGE_API_BASE_URL || 'https://api.topcoder-dev.com/v4',
// eslint-disable-next-line max-len
TC_ADMIN_TOKEN: process.env.TC_ADMIN_TOKEN,

Expand Down
21 changes: 11 additions & 10 deletions connect/connectNotificationServer.js
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@ const getNotificationsForMentionedUser = (eventConfig, content) => {
}

let notifications = [];
// eslint-disable-next-line
const regexUserHandle = /title=\"@([a-zA-Z0-9-_.{}\[\]]+)\"/g;
let handles=[];
let matches = regexUserHandle.exec(content);
console.log(content)
const handles = [];
let matches = regexUserHandle.exec(content);
while (matches) {
let handle = matches[1].toString();
const handle = matches[1].toString();
notifications.push({
userHandle: handle,
newType: 'notifications.connect.project.post.mention',
Expand All @@ -85,13 +85,13 @@ const getNotificationsForMentionedUser = (eventConfig, content) => {
// only one per userHandle
notifications = _.uniqBy(notifications, 'userHandle');

return new Promise((resolve)=>{
service.getUsersByHandle(handles).then((users)=>{
_.map(notifications,(notification)=>{
notification.userId = _.find(users,{handle:notification.userHandle}).userId;
return new Promise((resolve) => {
service.getUsersByHandle(handles).then((users) => {
_.map(notifications, (notification) => {
notification.userId = _.find(users, { handle: notification.userHandle }).userId.toString();
});
resolve(notifications);
})
});
});
};

Expand Down Expand Up @@ -307,7 +307,7 @@ const handler = (topic, message, callback) => {
project,
})
)).then((notifications) => {
allNotifications = _.filter(notifications,notification=>notification.userId!=message.initiatorUserId);
allNotifications = _.filter(notifications, notification => notification.userId !== message.initiatorUserId);

// now let's retrieve some additional data

Expand All @@ -326,6 +326,7 @@ const handler = (topic, message, callback) => {
if (users.length) {
notification.contents.userHandle = users[0].handle;
notification.contents.userFullName = `${users[0].firstName} ${users[0].lastName}`;
notification.contents.userEmail = users[0].email;
}
});
callback(null, allNotifications);
Expand Down
11 changes: 6 additions & 5 deletions connect/service.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ const getRoleMembers = (roleId) => request
const getUsersById = (ids) => {
const query = _.map(ids, (id) => 'userId:' + id).join(' OR ');
return request
.get(`${config.TC_API_V3_BASE_URL}/members/_search?fields=userId,handle,firstName,lastName&query=${query}`)
.get(`${config.TC_API_V3_BASE_URL}/members/_search?fields=userId,email,handle,firstName,lastName&query=${query}`)
.set('accept', 'application/json')
.set('authorization', `Bearer ${config.TC_ADMIN_TOKEN}`)
.then((res) => {
Expand All @@ -78,7 +78,6 @@ const getUsersById = (ids) => {
}

const users = _.get(res, 'body.result.content');

return users;
}).catch((err) => {
const errorDetails = _.get(err, 'response.body.result.content.message');
Expand Down Expand Up @@ -126,8 +125,8 @@ const getUsersByHandle = (handles) => {
*
* @return {Promise} promise resolved to topic details
*/
const getTopic = (topicId) => request
.get(`${config.TC_API_V4_BASE_URL}/topics/${topicId}`)
const getTopic = (topicId, logger) => request
.get(`${config.MESSAGE_API_BASE_URL}/topics/${topicId}/read`)
.set('accept', 'application/json')
.set('authorization', `Bearer ${config.TC_ADMIN_TOKEN}`)
.then((res) => {
Expand All @@ -137,14 +136,16 @@ const getTopic = (topicId) => request

return _.get(res, 'body.result.content');
}).catch((err) => {
if (logger) {
logger.error(err, `Error while calling ${config.MESSAGE_API_BASE_URL}/topics/${topicId}/read`);
}
const errorDetails = _.get(err, 'response.body.result.content.message');
throw new Error(
`Failed to get topic details of topic id: ${topicId}.` +
(errorDetails ? ' Server response: ' + errorDetails : '')
);
});


module.exports = {
getProject,
getRoleMembers,
Expand Down
49 changes: 48 additions & 1 deletion deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,19 @@ TC_ADMIN_TOKEN=$(eval "echo \$${ENV}_TC_ADMIN_TOKEN")
LOG_LEVEL=$(eval "echo \$${ENV}_LOG_LEVEL")
PORT=$(eval "echo \$${ENV}_PORT")

# email notifications config
ENABLE_EMAILS=$(eval "echo \$${ENV}_ENABLE_EMAILS")
BUS_API_AUTH_TOKEN=$(eval "echo \$${ENV}_BUS_API_AUTH_TOKEN")
MENTION_EMAIL=$(eval "echo \$${ENV}_MENTION_EMAIL")
REPLY_EMAIL_PREFIX=$(eval "echo \$${ENV}_REPLY_EMAIL_PREFIX")
REPLY_EMAIL_DOMAIN=$(eval "echo \$${ENV}_REPLY_EMAIL_DOMAIN")
ENABLE_DEV_MODE=$(eval "echo \$${ENV}_ENABLE_DEV_MODE")
DEV_MODE_EMAIL=$(eval "echo \$${ENV}_DEV_MODE_EMAIL")

TC_API_V3_BASE_URL=$(eval "echo \$${ENV}_TC_API_V3_BASE_URL")
TC_API_V4_BASE_URL=$(eval "echo \$${ENV}_TC_API_V4_BASE_URL")
TC_API_V5_BASE_URL=$(eval "echo \$${ENV}_TC_API_V5_BASE_URL")
MESSAGE_API_BASE_URL=$(eval "echo \$${ENV}_MESSAGE_API_BASE_URL")

DB_USER=$(eval "echo \$${ENV}_DB_USER")
DB_PASSWORD=$(eval "echo \$${ENV}_DB_PASSWORD")
Expand Down Expand Up @@ -159,10 +170,46 @@ make_task_def(){
"name": "TC_API_V4_BASE_URL",
"value": "%s"
},
{
"name": "TC_API_V5_BASE_URL",
"value": "%s"
},
{
"name": "MESSAGE_API_BASE_URL",
"value": "%s"
},
{
"name": "TC_ADMIN_TOKEN",
"value": "%s"
},
{
"name": "ENABLE_EMAILS",
"value": "%s"
},
{
"name": "MENTION_EMAIL",
"value": "%s"
},
{
"name": "REPLY_EMAIL_PREFIX",
"value": "%s"
},
{
"name": "REPLY_EMAIL_DOMAIN",
"value": "%s"
},
{
"name": "ENABLE_DEV_MODE",
"value": "%s"
},
{
"name": "DEV_MODE_EMAIL",
"value": "%s"
},
{
"name": "BUS_API_AUTH_TOKEN",
"value": "%s"
},
{
"name": "LOG_LEVEL",
"value": "%s"
Expand Down Expand Up @@ -198,7 +245,7 @@ make_task_def(){
}
]'

task_def=$(printf "$task_template" $AWS_ECS_CONTAINER_NAME $AWS_ACCOUNT_ID $AWS_REGION $AWS_REPOSITORY $TAG $ENV "$KAFKA_CLIENT_CERT" "$KAFKA_CLIENT_CERT_KEY" $KAFKA_GROUP_ID "$KAFKA_TOPIC_IGNORE_PREFIX" $KAFKA_URL $DATABASE_URL $AUTHSECRET "$AUTHDOMAIN" "$JWKSURI" $TC_API_BASE_URL $TC_API_V3_BASE_URL $TC_API_V4_BASE_URL $TC_ADMIN_TOKEN $LOG_LEVEL $VALID_ISSUERS $PORT "$API_CONTEXT_PATH" $AWS_ECS_CLUSTER $AWS_REGION $AWS_ECS_CLUSTER $ENV)
task_def=$(printf "$task_template" $AWS_ECS_CONTAINER_NAME $AWS_ACCOUNT_ID $AWS_REGION $AWS_REPOSITORY $TAG $ENV "$KAFKA_CLIENT_CERT" "$KAFKA_CLIENT_CERT_KEY" $KAFKA_GROUP_ID "$KAFKA_TOPIC_IGNORE_PREFIX" $KAFKA_URL $DATABASE_URL $AUTHSECRET "$AUTHDOMAIN" "$JWKSURI" $TC_API_BASE_URL $TC_API_V3_BASE_URL $TC_API_V4_BASE_URL $TC_API_V5_BASE_URL $MESSAGE_API_BASE_URL $TC_ADMIN_TOKEN $ENABLE_EMAILS $MENTION_EMAIL $REPLY_EMAIL_PREFIX $REPLY_EMAIL_DOMAIN $ENABLE_DEV_MODE $DEV_MODE_EMAIL $BUS_API_AUTH_TOKEN $LOG_LEVEL $VALID_ISSUERS $PORT "$API_CONTEXT_PATH" $AWS_ECS_CLUSTER $AWS_REGION $AWS_ECS_CLUSTER $ENV)
}

register_definition() {
Expand Down
Loading