Skip to content
This repository was archived by the owner on Mar 13, 2025. It is now read-only.

Commit 93448dd

Browse files
jmgasperjmgasper
committed
July 2019 release challenge
1 parent 5871613 commit 93448dd

13 files changed

+207
-48
lines changed

‎DynamoDBMigration.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
## Migrate the data to Dynamo
2+
3+
Prerequisite is to turn off any throttling on the DynamoDB, temporarily
4+
5+
1. Download the migration-tools.zip from here: https://github.com/topcoder-platform/topcoder-x-ui/issues/140#issuecomment-497663806
6+
2. Unzip
7+
3. Modify `migration-tools/config/default.js` to match prod
8+
* MONGODB_URL: Will provide in Slack
9+
* AWS_ACCESS_KEY_ID
10+
* AWS_SECRET_ACCESS_KEY
11+
* AWS_REGION
12+
* IS_LOCAL = 'false'
13+
4. `npm i`
14+
5. `npm run migrate-data`
15+
16+
#### Tables created
17+
18+
* `Topcoder_X.CopilotPayment`
19+
* `Topcoder_X.Issue`
20+
* `Topcoder_X.OwnerUserGroup`
21+
* `Topcoder_X.OwnerUserTeam`
22+
* `Topcoder_X.Project`
23+
* `Topcoder_X.User`
24+
* `Topcoder_X.UserMapping`
25+
26+
27+
## Deploy latest code
28+
29+
* Deploy the latest code from `master` to Heroku:
30+
* topcoder-x-receiver
31+
* topcoder-x-processor
32+
* topcoder-x-ui
33+
34+
Update the AWS Dynamo config values in Heroku:
35+
36+
* AWS_ACCESS_KEY_ID
37+
* AWS_SECRET_ACCESS_KEY
38+
* IS_LOCAL='false'
39+
* AWS_DYNAMODB_REGION
40+
* AWS_REGION

‎TopcoderXDeploy.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ KAFKA_CLIENT_CERT: <cert string>
5959
6060
KAFKA_CLIENT_CERT_KEY: <cert string>
6161
62-
KAFKA_HOST: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
62+
KAFKA_URL: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
6363
6464
TOPIC: topcoder-x
6565
```
@@ -70,7 +70,7 @@ So it would be updated from this:
7070

7171
```
7272
KAFKA_OPTIONS: {
73-
connectionString: process.env.KAFKA_HOST || 'localhost:9092',
73+
connectionString: process.env.KAFKA_URL || 'localhost:9092',
7474
ssl: {
7575
cert: process.env.KAFKA_CLIENT_CERT || fs.readFileSync('./kafka_client.cer'), // eslint-disable-line no-sync
7676
key: process.env.KAFKA_CLIENT_CERT_KEY || fs.readFileSync('./kafka_client.key'), // eslint-disable-line no-sync
@@ -82,7 +82,7 @@ To this:
8282

8383
```
8484
KAFKA_OPTIONS: {
85-
connectionString: process.env.KAFKA_HOST || 'localhost:9092'
85+
connectionString: process.env.KAFKA_URL || 'localhost:9092'
8686
},
8787
```
8888

@@ -168,7 +168,7 @@ EMAIL_SENDER_ADDRESS: [email protected]
168168
ISSUE_BID_EMAIL_RECEIVER: [email protected]
169169
KAFKA_CLIENT_CERT: <cert>
170170
KAFKA_CLIENT_CERT_KEY: <key>
171-
KAFKA_HOST: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
171+
KAFKA_URL: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
172172
LOG_LEVEL: debug
173173
MAILGUN_API_KEY: key-5ebe7a0fae37a9008721ec0bfe5bdd95
174174
MAILGUN_DOMAIN: sandbox3fcf4920781449f2a5293f8ef18e4bb6.mailgun.org
@@ -195,7 +195,7 @@ Justins-Mac-Pro:~ justingasper$ heroku config --app topcoder-x-receiver-dev
195195
=== topcoder-x-receiver-dev Config Vars
196196
KAFKA_CLIENT_CERT: <cert>
197197
KAFKA_CLIENT_CERT_KEY: <key>
198-
KAFKA_HOST: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
198+
KAFKA_URL: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
199199
LOG_LEVEL: debug
200200
NODE_ENV: development
201201
NODE_TLS_REJECT_UNAUTHORIZED: 0
@@ -219,7 +219,7 @@ GITLAB_CLIENT_SECRET: 70367d8255e160828ae47f35ff71723202afc7be1f29a4c7319bc82c4
219219
HOOK_BASE_URL: https://topcoder-x-receiver-dev.herokuapp.com
220220
KAFKA_CLIENT_CERT: <cert>
221221
KAFKA_CLIENT_CERT_KEY: <key>
222-
KAFKA_HOST: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
222+
KAFKA_URL: silver-craft-01.srvs.cloudkafka.com:9093,silver-craft-01.srvs.cloudkafka.com:9094
223223
NPM_CONFIG_PRODUCTION: false
224224
SESSION_SECRET: kjsdfkj34857
225225
TC_LOGIN_URL: https://accounts.topcoder-dev.com/member

‎configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ The following config parameters are supported, they are defined in `src/config.j
1818
| GITLAB_API_BASE_URL | The Gitlab API base URL | https://gitlab.com|
1919
|TOPIC | The Kafka topic where events are published. This must be the same as the configured value for topcoder-x-processor| |
2020
|KAFKA_OPTIONS | Kafka connection options| |
21-
|KAFKA_HOST | The Kafka host to connect to| localhost:9092 |
21+
|KAFKA_URL | The Kafka host to connect to| localhost:9092 |
2222
|KAFKA_CLIENT_CERT | The Kafka SSL certificate to use when connecting| Read from kafka_client.cer file, but this can be set as a string like it is on Heroku |
2323
|KAFKA_CLIENT_CERT_KEY | The Kafka SSL certificate key to use when connecting| Read from kafka_client.key file, but this can be set as a string like it is on Heroku|
2424
| HOOK_BASE_URL | The base URL of the topcoder-x-receiver, used when adding webhooks automatically to repositories | |

‎package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,7 @@
134134
"jshint": "~2.9.4",
135135
"jshint-stylish": "~1.0.0",
136136
"mocha": "^3.5.0",
137+
"natives": "^1.1.6",
137138
"protractor": "~5.1.1",
138139
"require-dir": "~0.3.2",
139140
"supertest": "^3.0.0",

‎src/common/constants.js

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
const DEFAULT_PER_PAGE = 30;
1414

1515
// The max page size
16-
const MAX_PER_PAGE = 100;
16+
const MAX_PER_PAGE = Number.MAX_SAFE_INTEGER;
1717

1818
// The service error
1919
const SERVICE_ERROR = 'SERVICE_ERROR';
@@ -35,8 +35,8 @@ const USER_TYPES = {
3535
// The default page size for Gitlab API
3636
const GITLAB_DEFAULT_PER_PAGE = 20;
3737

38-
// The max page size for Gitlab API
39-
const GITLAB_MAX_PER_PAGE = 100;
38+
// The max page size for Gitlab API. Increased to max num for local search purpose.
39+
const GITLAB_MAX_PER_PAGE = Number.MAX_SAFE_INTEGER;
4040

4141
// The gitlab default group member access level,
4242
// the access level can be: 10 - GUEST, 20 - REPORTER, 30 - DEVELOPER, 40 - MASTER, 50 - OWNER

‎src/config.js

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ module.exports = {
2626
// kafka configuration
2727
TOPIC: process.env.TOPIC || 'tc-x-events',
2828
KAFKA_OPTIONS: {
29-
connectionString: process.env.KAFKA_HOST || 'localhost:9092',
29+
connectionString: process.env.KAFKA_URL || 'localhost:9092',
3030
ssl: {
3131
cert: process.env.KAFKA_CLIENT_CERT || fs.readFileSync('./kafka_client.cer'), // eslint-disable-line no-sync
3232
key: process.env.KAFKA_CLIENT_CERT_KEY || fs.readFileSync('./kafka_client.key'), // eslint-disable-line no-sync
@@ -50,10 +50,10 @@ module.exports = {
5050
HELP_LINK: process.env.HELP_LINK || 'https://github.com/topcoder-platform/topcoder-x-ui/wiki',
5151
ADMINISTRATOR_ROLES: process.env.ADMINISTRATOR_ROLES || ['administrator', 'admin'],
5252
DYNAMODB: {
53-
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID || '',
54-
AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY || '',
55-
AWS_REGION: process.env.AWS_REGION || '',
56-
IS_LOCAL: process.env.IS_LOCAL || 'false',
53+
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID,
54+
AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY,
55+
AWS_REGION: process.env.AWS_REGION,
56+
IS_LOCAL: process.env.IS_LOCAL
5757
},
5858
TOPCODER_VALUES: {
5959
dev: {

‎src/front/src/app/git-access-control/access-control.html

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,19 @@ <h2>Git Access Control</h2>
1616
<br/>
1717
<div ng-if="settings.github==true">
1818
<div ng-show="tableConfig.github.initialized">
19-
<table ng-if="tableConfig.github.items.length!=0" class="footable table table-stripped toggle-arrow-tiny">
19+
<div ng-if="tableConfig.github.items.length!=0 || tableConfig.github.allItems.length!=0" class="row">
20+
<div class="col-lg-3">
21+
<div class="input-group custom-search-form">
22+
<input ng-model="searchText" type="text" class="form-control" placeholder="Find teams" ng-change="onSearchChange('github', this)">
23+
<span class="input-group-btn">
24+
<button class="btn btn-default" type="button" ng-click="onSearchIconClicked('github')">
25+
<span class="glyphicon glyphicon-search"></span>
26+
</button>
27+
</span>
28+
</div>
29+
</div>
30+
</div>
31+
<table ng-if="tableConfig.github.items.length!=0 || tableConfig.github.allItems.length!=0" class="footable table table-stripped toggle-arrow-tiny">
2032
<thead>
2133
<tr>
2234
<th class="col-lg-2" data-sort-ignore="true">Team Name</th>
@@ -72,7 +84,7 @@ <h2>Git Access Control</h2>
7284
</tfoot>
7385
</table>
7486
</div>
75-
<div ng-if="tableConfig.github.items.length==0 && tableConfig.github.initialized">
87+
<div ng-if="tableConfig.github.items.length==0 && tableConfig.github.allItems.length==0 && tableConfig.github.initialized">
7688
You don't appear to have any Github teams available to manage through Topcoder X. Topcoder X is used to give Topcoder
7789
members access to specific teams using your ownership credentials. If you
7890
don't have any teams, Topcoder X won't be able to manage access for you.
@@ -91,8 +103,20 @@ <h2>Git Access Control</h2>
91103
<uib-tab index="1" heading="Gitlab" select="tabChanged('gitlab')">
92104
<br/>
93105
<div ng-if="settings.gitlab==true">
94-
<div ng-show="tableConfig.gitlab.initialized">
95-
<table ng-show="tableConfig.gitlab.items.length!=0" class="footable table table-stripped toggle-arrow-tiny">
106+
<div ng-show="tableConfig.gitlab.initialized">
107+
<div ng-if="tableConfig.gitlab.items.length!=0 || tableConfig.gitlab.allItems.length!=0" class="row">
108+
<div class="col-lg-3">
109+
<div class="input-group custom-search-form">
110+
<input ng-model="searchText" type="text" class="form-control" placeholder="Find groups" ng-change="onSearchChange('gitlab', this)">
111+
<span class="input-group-btn">
112+
<button class="btn btn-default" type="button" ng-click="onSearchIconClicked('gitlab')">
113+
<span class="glyphicon glyphicon-search"></span>
114+
</button>
115+
</span>
116+
</div>
117+
</div>
118+
</div>
119+
<table ng-show="tableConfig.gitlab.items.length!=0 || tableConfig.gitlab.allItems.length!=0" class="footable table table-stripped toggle-arrow-tiny">
96120
<thead>
97121
<tr>
98122
<th class="col-lg-2" data-sort-ignore="true">Group Name</th>
@@ -148,7 +172,7 @@ <h2>Git Access Control</h2>
148172
</tfoot>
149173
</table>
150174
</div>
151-
<div ng-if="tableConfig.gitlab.items.length==0 && tableConfig.gitlab.initialized">
175+
<div ng-if="tableConfig.gitlab.items.length==0 && tableConfig.gitlab.allItems.length==0 && tableConfig.gitlab.initialized">
152176
You don't appear to have any Gitlab groups available to manage through Topcoder X. Topcoder X is used to give Topcoder
153177
members access to specific group using your ownership credentials. If you
154178
don't have any group, Topcoder X won't be able to manage access for you.

‎src/front/src/app/git-access-control/gitAccessControl.controller.js

Lines changed: 66 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,26 +5,31 @@ angular.module('topcoderX').controller('GitAccessController', ['currentUser', '$
55
$scope.settings = {};
66
var vm = this;
77
$scope.isLoaded = false;
8+
$scope.isLoadingSearchData = false;
89
$scope.tableConfig = {
910
github: {
1011
pageNumber: 1,
1112
pageSize: 10,
1213
isLoading: false,
1314
items: [],
15+
allItems: [],
1416
totalPages: 1,
1517
searchMethod: GitAccessControlService.getGithubOwnerTeams,
1618
initialized: false,
1719
accessLinkMethod: GitAccessControlService.getGithubShareableLink,
20+
query: ''
1821
},
1922
gitlab: {
2023
pageNumber: 1,
2124
pageSize: 10,
2225
isLoading: false,
2326
items: [],
27+
allItems: [],
2428
totalPages: 1,
2529
searchMethod: GitAccessControlService.getGitlabOwnerGroups,
2630
initialized: false,
2731
accessLinkMethod: GitAccessControlService.getGitlabShareableLink,
32+
query: ''
2833
}
2934
}
3035

@@ -89,7 +94,14 @@ angular.module('topcoderX').controller('GitAccessController', ['currentUser', '$
8994
return false;
9095
}
9196
$scope.tableConfig[provider].pageNumber = pageNumber;
92-
_getOwnerList(provider);
97+
98+
var config = $scope.tableConfig[provider];
99+
if (config.query.length === 0) {
100+
_getOwnerList(provider);
101+
}
102+
else {
103+
_searchLocal(config, config.query, provider);
104+
}
93105
};
94106
// get the number array that shows the pagination bar
95107
$scope.getPageArray = function (provider) {
@@ -120,4 +132,56 @@ angular.module('topcoderX').controller('GitAccessController', ['currentUser', '$
120132
_getOwnerList(provider);
121133
}
122134
}
123-
}]);
135+
136+
$scope.onSearchChange = function (provider, obj) {
137+
var config = $scope.tableConfig[provider];
138+
config.query = obj.searchText;
139+
if (config.allItems.length === 0 && !$scope.isLoadingSearchData) {
140+
$scope.isLoadingSearchData = true;
141+
config.searchMethod.apply(vm, [1, Number.MAX_SAFE_INTEGER, true]).then(function (res) {
142+
config.allItems = provider === 'github' ? res.data.teams : res.data.groups;
143+
$log.log(config.allItems);
144+
$scope.isLoadingSearchData = false;
145+
_searchLocal(config, obj.searchText, provider);
146+
if (config.pageNumber > 1) {
147+
$scope.changePage(1, provider);
148+
}
149+
}).catch(function (err) {
150+
_handleError(err, 'An error occurred while getting the data for ' + provider + '.');
151+
$scope.isLoadingSearchData = false;
152+
});
153+
}
154+
else if (config.allItems.length > 0) {
155+
_searchLocal(config, obj.searchText, provider);
156+
if (config.pageNumber > 1) {
157+
$scope.changePage(1, provider);
158+
}
159+
}
160+
161+
if (config.query.length === 0) {
162+
$scope.changePage(1, provider);
163+
}
164+
};
165+
166+
$scope.onSearchIconClicked = function (provider) {
167+
var config = $scope.tableConfig[provider];
168+
if (config.pageNumber > 1) {
169+
$scope.changePage(1, provider);
170+
}
171+
if (config.query.length > 0) {
172+
_searchLocal(config, config.query, provider);
173+
}
174+
};
175+
176+
function _searchLocal(config, query, provider) {
177+
config.items = config.allItems.filter(function(value) {
178+
if (provider === 'github') {
179+
return value['name'].toLowerCase().includes(query.toLowerCase());
180+
}
181+
return value['full_name'].toLowerCase().includes(query.toLowerCase());
182+
})
183+
config.totalPages = Math.ceil(config.items.length / config.pageSize);
184+
config.items = config.items
185+
.slice((config.pageNumber - 1)*config.pageSize, (config.pageNumber - 1)*config.pageSize + config.pageSize);
186+
}
187+
}]);

‎src/front/src/app/main/main.controller.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,4 +138,5 @@ angular.module('topcoderX')
138138
$scope.tableConfig[provider].pageNumber = 1;
139139
_search(provider);
140140
};
141+
141142
}]);

‎src/front/src/app/upsertproject/upsertproject.controller.js

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,14 @@ angular.module('topcoderX').controller('ProjectController', ['currentUser', '$sc
5050

5151
// function to add hooks to the current project.
5252
$scope.addHooks = function () {
53-
ProjectService.createHooks({ projectId: $scope.project.id }).then(function () {
54-
Alert.info('Webhook Added Successfully', $scope);
53+
ProjectService.createHooks({ projectId: $scope.project.id }).then(function (result) {
54+
if (result && result.data.updated === true) {
55+
Alert.info('Existing Webhook Updated Successfully', $scope);
56+
}
57+
else {
58+
Alert.info('Webhook Added Successfully', $scope);
59+
}
60+
5561
}).catch(function (error) {
5662
Alert.error(error.data.message, $scope);
5763
});

‎src/models/Project.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ const schema = new Schema({
3737
owner: {type: String, required: true},
3838
secretWebhookKey: {type: String, required: true},
3939
copilot: {type: String, required: false},
40+
registeredWebhookId: {type: String, required: false}
4041
});
4142

4243
module.exports = schema;

‎src/services/GithubService.js

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,15 +87,16 @@ ensureOwnerUser.schema = Joi.object().keys({
8787
* @param {Number} perPage the page size (default to be constants.DEFAULT_PER_PAGE). Must be within range [1, constants.MAX_PER_PAGE]
8888
* @returns {Promise} the promise result
8989
*/
90-
async function listOwnerUserTeams(token, page = 1, perPage = constants.DEFAULT_PER_PAGE) {
90+
async function listOwnerUserTeams(token, page = 1, perPage = constants.DEFAULT_PER_PAGE) {
9191
try {
9292
const github = new GitHub({token});
9393
const user = github.getUser();
94+
9495
const response = await user._request('GET', '/user/teams', {
9596
page,
9697
per_page: perPage,
9798
});
98-
99+
99100
const result = {
100101
page,
101102
perPage,

0 commit comments

Comments
 (0)