Skip to content

Commit 16a51ef

Browse files
committed
Merge pull request #48 from adriaanm/master
cleanup
2 parents d7c7e55 + f0607e1 commit 16a51ef

File tree

10 files changed

+354
-278
lines changed

10 files changed

+354
-278
lines changed

.chef/Vagrantfile

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,20 @@
11
# TODO: this kind of works, but the cookbook won't run in virtual box:
2-
# - (scatter-gather workaround)
32
# - chef-solo can't access the vault on the chef server (passwords etc)
4-
# to make this work, you need to populate a cookbooks/ directory next to this file,
5-
# as documented in the README.md (use knife)
3+
# vagrant box add utopic-daily https://cloud-images.ubuntu.com/vagrant/utopic/current/utopic-server-cloudimg-amd64-vagrant-disk1.box
4+
# vagrant init utopic-daily
5+
# vagrant up
6+
# centos: vagrant box add chef/centos-7.0
67
Vagrant.configure("2") do |config|
7-
config.vm.box = "hashicorp/precise64"
8+
config.vm.box = "utopic-daily"
89
config.vm.provision :chef_solo do |chef|
10+
chef.cookbooks_path = ["~/git/cookbooks"]
11+
chef.node_name = "jenkins-master"
912
chef.add_recipe("scala-jenkins-infra::master-init")
13+
chef.add_recipe("scala-jenkins-infra::_master-config-proxy")
14+
end
15+
config.vm.network "public_network"
16+
config.vm.provider "virtualbox" do |v|
17+
v.memory = 4096
18+
v.cpus = 2
1019
end
11-
config.vm.network :forwarded_port, guest: 80, host: 11180
1220
end

README.md

Lines changed: 93 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -105,31 +105,33 @@ aws iam create-instance-profile --instance-profile-name JenkinsMaster
105105
aws iam create-instance-profile --instance-profile-name JenkinsWorkerPublish
106106
aws iam create-instance-profile --instance-profile-name JenkinsWorker
107107
108-
aws iam create-role --role-name jenkins-master --assume-role-policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/ec2-role-trust-policy.json
109-
aws iam create-role --role-name jenkins-worker --assume-role-policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/ec2-role-trust-policy.json
110-
aws iam create-role --role-name jenkins-worker-publish --assume-role-policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/ec2-role-trust-policy.json
108+
aws iam create-role --role-name jenkins-master --assume-role-policy-document file://$PWD/chef/ec2-role-trust-policy.json
109+
aws iam create-role --role-name jenkins-worker --assume-role-policy-document file://$PWD/chef/ec2-role-trust-policy.json
110+
aws iam create-role --role-name jenkins-worker-publish --assume-role-policy-document file://$PWD/chef/ec2-role-trust-policy.json
111111
112112
aws iam add-role-to-instance-profile --instance-profile-name JenkinsMaster --role-name jenkins-master
113113
aws iam add-role-to-instance-profile --instance-profile-name JenkinsWorker --role-name jenkins-worker
114114
aws iam add-role-to-instance-profile --instance-profile-name JenkinsWorkerPublish --role-name jenkins-worker-publish
115115
```
116116

117117
### Attach policies to roles:
118+
NOTE: if you get syntax errors, check the policy doc URL
118119

119120
```
120-
aws iam put-role-policy --role-name jenkins-master --policy-name jenkins-ec2-start-stop --policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/jenkins-ec2-start-stop.json
121-
aws iam put-role-policy --role-name jenkins-master --policy-name jenkins-dynamodb --policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/dynamodb.json
122-
123-
// TODO: once https://github.com/sbt/sbt-s3/issues/14 is fixed, remove s3credentials from nodes and use IAM profile instead
124-
aws iam put-role-policy --role-name jenkins-worker-publish --policy-name jenkins-s3-upload --policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/jenkins-s3-upload.json
125-
126-
aws iam put-role-policy --role-name jenkins-worker --policy-name jenkins-ebs-create-vol --policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/ebs-create-vol.json
121+
aws iam put-role-policy --role-name jenkins-master --policy-name jenkins-ec2-start-stop --policy-document file://$PWD/chef/jenkins-ec2-start-stop.json
122+
aws iam put-role-policy --role-name jenkins-master --policy-name jenkins-dynamodb --policy-document file://$PWD/chef/dynamodb.json
123+
aws iam put-role-policy --role-name jenkins-master --policy-name jenkins-ebs-create-vol --policy-document file://$PWD/chef/ebs-create-vol.json
124+
```
127125

128-
aws iam put-role-policy --role-name jenkins-worker-publish --policy-name jenkins-ebs-create-vol --policy-document file:///Users/adriaan/git/scala-jenkins-infra/chef/ebs-create-vol.json
126+
```
127+
aws iam put-role-policy --role-name jenkins-worker --policy-name jenkins-ebs-create-vol --policy-document file://$PWD/chef/ebs-create-vol.json
129128
```
130129

131-
NOTE: if you get syntax errors, check the policy doc URL
132-
pass JenkinsWorker as the iam profile to knife bootstrap
130+
TODO: once https://github.com/sbt/sbt-s3/issues/14 is fixed, remove s3credentials from nodes (use IAM profile below instead)
131+
```
132+
aws iam put-role-policy --role-name jenkins-worker-publish --policy-name jenkins-s3-upload --policy-document file://$PWD/chef/jenkins-s3-upload.json
133+
aws iam put-role-policy --role-name jenkins-worker-publish --policy-name jenkins-ebs-create-vol --policy-document file://$PWD/chef/ebs-create-vol.json
134+
```
133135

134136

135137
## Create an Elastic IP for each node
@@ -157,35 +159,70 @@ If your username on chef.io does not match the local username on your machine, y
157159
export CHEF_USER="[username]"
158160
```
159161

160-
You can then generate and download your private key on https://www.chef.io/account/password. Put it to `.chef/config/$CHEF_USER.pem`, then you can use knife without further config. See `.chef/knife.rb` for key locations.
162+
You can then generate and download your private key on https://www.chef.io/account/password. Put it to `$PWD/.chef/config/$CHEF_USER.pem`, then you can use knife without further config. See `$PWD/.chef/knife.rb` for key locations.
161163

162164
Test if knife works correctly by running `knife cookbook list`.
163165

164-
Obtain the organization validation key from Adriaan and put it to `.chef/config/$CHEF_ORG-validator.pem`. (Q: When is this key used exactly? https://docs.chef.io/chef_private_keys.html says it's when a new node runs `chef-client` for the first time.)
166+
Obtain the organization validation key from Adriaan and put it to `$PWD/.chef/config/$CHEF_ORG-validator.pem`. (Q: When is this key used exactly? https://docs.chef.io/chef_private_keys.html says it's when a new node runs `chef-client` for the first time.)
167+
168+
## Clone scala-jenkins-infra cookbook and its dependencies
165169

166-
## Get cookbooks
170+
I think you can safely ignore `ERROR: IOError: Cannot open or read **/metadata.rb!` in the below
167171

168172
```
169-
git init .chef/cookbooks
170-
cd .chef/cookbooks
173+
cd ~/git/cookbooks
174+
git init .
171175
g commit --allow-empty -m"Initial"
172-
```
173-
174-
- knife cookbook site install wix 1.0.2 # newer versions don't work for me; also installs windows
175-
- knife cookbook site install aws
176-
- knife cookbook site install git
177-
- knife cookbook site install git_user
178-
- knife cookbook site install partial_search
179-
180-
- move to unreleased versions on github:
181-
- knife cookbook github install opscode-cookbooks/windows # fix nosuchmethoderror (#150)
182-
- knife cookbook github install adriaanm/java/windows-jdk1.6 # jdk 1.6 installer barfs on re-install -- wipe its INSTALLDIR
183-
- knife cookbook github install adriaanm/jenkins/fix305 # ssl fail on windows
184-
- knife cookbook github install adriaanm/scala-jenkins-infra
185-
- knife cookbook github install adriaanm/chef-sbt
186-
- knife cookbook github install gildegoma/chef-sbt-extras
187176
188-
- knife cookbook upload --all
177+
hub clone scala/scala-jenkins-infra
178+
cd scala-jenkins-infra
179+
ln -sh ~/git/cookbooks $PWD/.chef/
180+
181+
knife site install cron
182+
knife site install logrotate
183+
knife site install chef_handler
184+
knife site install windows
185+
knife site install chef-client
186+
knife site install aws
187+
knife site install delayed_evaluator
188+
knife site install ebs
189+
knife site install java
190+
knife site install apt
191+
knife site install packagecloud
192+
knife site install runit
193+
knife site install yum
194+
knife site install jenkins
195+
knife site install 7-zip
196+
knife site install ark
197+
knife site install artifactory
198+
knife site install build-essential
199+
knife site install dmg
200+
knife site install yum-epel
201+
knife site install git
202+
knife site install user
203+
knife site install partial_search
204+
knife site install ssh_known_hosts
205+
knife site install git_user
206+
knife site install chef-sbt
207+
knife site install sbt-extras
208+
```
209+
210+
### Switch to unreleased versions from github
211+
```
212+
//fixed: knife cookbook github install opscode-cookbooks/windows # fix nosuchmethoderror (#150)
213+
//knife cookbook github install adriaanm/jenkins/fix305 # ssl fail on windows -- fix pending: https://github.com/opscode-cookbooks/jenkins/pull/313
214+
knife cookbook github install b-dean/jenkins/http_ca_fixes # pending fix for above ^^^
215+
216+
knife cookbook github install adriaanm/java/windows-jdk1.6 # jdk 1.6 installer barfs on re-install -- wipe its INSTALLDIR
217+
knife cookbook github install adriaanm/chef-sbt
218+
knife cookbook github install gildegoma/chef-sbt-extras
219+
knife cookbook github install adriaanm/artifactory
220+
```
221+
222+
### Upload cookbooks to chef server
223+
```
224+
knife cookbook upload --all
225+
```
189226

190227
## Cache installers locally
191228
- they are tricky to access, might disappear,...
@@ -205,23 +242,23 @@ NOTE: the JSON must not have a field "id"!!!
205242
### Chef user with keypair for jenkins cli access
206243
```
207244
eval "$(chef shell-init zsh)" # use chef's ruby, which has the net/ssh gem
208-
ruby chef/keypair.rb > ~/Desktop/chef-secrets/config/keypair.json
209-
ruby chef/keypair.rb > ~/Desktop/chef-secrets/config/scabot-keypair.json
245+
ruby chef/keypair.rb > $PWD/.chef/keypair.json
246+
ruby chef/keypair.rb > $PWD/.chef/scabot-keypair.json
210247
211-
# extract private key to ~/Desktop/chef-secrets/config/scabot.pem
248+
# extract private key to $PWD/.chef/scabot.pem
212249
213250
knife vault create master scala-jenkins-keypair \
214-
--json ~/Desktop/chef-secrets/config/keypair.json \
251+
--json $PWD/.chef/keypair.json \
215252
--search 'name:jenkins*' \
216253
--admins adriaan
217254
218255
knife vault create master scabot-keypair \
219-
--json ~/Desktop/chef-secrets/config/scabot-keypair.json \
256+
--json $PWD/.chef/scabot-keypair.json \
220257
--search 'name:jenkins-master' \
221258
--admins adriaan
222259
223260
knife vault create master scabot \
224-
--json ~/Desktop/chef-secrets/config/scabot.json \
261+
--json $PWD/.chef/scabot.json \
225262
--search 'name:jenkins-master' \
226263
--admins adriaan
227264
@@ -267,12 +304,12 @@ knife vault create worker-publish s3-downloads \
267304
--admins adriaan
268305
269306
knife vault create worker-publish chara-keypair \
270-
--json chara-keypair.json \
307+
--json $PWD/.chef/config/chara-keypair.json \
271308
--search 'name:jenkins-worker-ubuntu-publish' \
272309
--admins adriaan
273310
274311
knife vault create worker-publish gnupg \
275-
--json /Users/adriaan/Desktop/chef-secrets/gnupg.json \
312+
--json $PWD/.chef/config/gnupg.json \
276313
--search 'name:jenkins-worker-ubuntu-publish' \
277314
--admins adriaan
278315
@@ -301,28 +338,28 @@ Note that the IPs are stable by allocating elastic IPs and associating them to n
301338
## ~/.ssh/config
302339
```
303340
Host jenkins-worker-ubuntu-publish
304-
IdentityFile ~/Desktop/chef-secrets/config/chef.pem
341+
IdentityFile $PWD/.chef/config/chef.pem
305342
User ubuntu
306343
307344
Host jenkins-worker-behemoth-1
308-
IdentityFile ~/Desktop/chef-secrets/config/chef.pem
345+
IdentityFile $PWD/.chef/config/chef.pem
309346
User ec2-user
310347
311348
Host jenkins-worker-behemoth-2
312-
IdentityFile ~/Desktop/chef-secrets/config/chef.pem
349+
IdentityFile $PWD/.chef/config/chef.pem
313350
User ec2-user
314351
315352
Host jenkins-master
316-
IdentityFile ~/Desktop/chef-secrets/config/chef.pem
353+
IdentityFile $PWD/.chef/config/chef.pem
317354
User ec2-user
318355
319356
Host scabot
320357
HostName jenkins-master
321-
IdentityFile ~/Desktop/chef-secrets/config/scabot.pem
358+
IdentityFile $PWD/.chef/scabot.pem
322359
User scabot
323360
324361
Host jenkins-worker-windows-publish
325-
IdentityFile ~/Desktop/chef-secrets/jenkins-chef
362+
IdentityFile $PWD/.chef/config/chef.pem
326363
User jenkins
327364
```
328365

@@ -363,7 +400,7 @@ knife ec2 server create -N jenkins-master \
363400
--region us-west-1 --flavor t2.small -I ami-4b6f650e \
364401
-G Master --ssh-user ec2-user \
365402
--iam-profile JenkinsMaster \
366-
--identity-file .chef/config/chef.pem \
403+
--identity-file $PWD/.chef/config/chef.pem \
367404
--run-list "scala-jenkins-infra::master-init"
368405
369406
knife ec2 server create -N jenkins-worker-windows-publish \
@@ -375,7 +412,7 @@ knife ec2 server create -N jenkins-worker-windows-publish \
375412
--security-group-ids sg-1dec3d78 \
376413
--subnet subnet-4bb3b80d --associate-eip 54.183.156.89 \
377414
--server-connect-attribute public_ip_address \
378-
--identity-file .chef/config/chef.pem \
415+
--identity-file $PWD/.chef/config/chef.pem \
379416
--run-list "scala-jenkins-infra::worker-init"
380417
381418
@@ -389,7 +426,7 @@ knife ec2 server create -N jenkins-worker-ubuntu-publish \
389426
--security-group-ids sg-ecb06389 \
390427
--subnet subnet-4bb3b80d --associate-eip 54.67.33.167 \
391428
--server-connect-attribute public_ip_address \
392-
--identity-file .chef/config/chef.pem \
429+
--identity-file $PWD/.chef/config/chef.pem \
393430
--run-list "scala-jenkins-infra::worker-init"
394431
395432
echo NOTE: Make sure to first remove the ips in $behemothIp from your ~/.ssh/known_hosts. Also remove the corresponding worker from the chef server (can be only one with the same name).
@@ -404,7 +441,7 @@ do knife ec2 server create -N jenkins-worker-behemoth-$behemoth \
404441
--security-group-ids sg-ecb06389 \
405442
--subnet subnet-4bb3b80d --associate-eip ${behemothIp[$behemoth]} \
406443
--server-connect-attribute public_ip_address \
407-
--identity-file .chef/config/chef.pem \
444+
--identity-file $PWD/.chef/config/chef.pem \
408445
--run-list "scala-jenkins-infra::worker-init"
409446
done
410447
@@ -446,7 +483,7 @@ done
446483

447484
- windows:
448485
```
449-
PASS=$(aws ec2 get-password-data --instance-id i-f67c0a35 --priv-launch-key ~/Desktop/chef-secrets/config/chef.pem | jq .PasswordData | xargs echo)
486+
PASS=$(aws ec2 get-password-data --instance-id i-f67c0a35 --priv-launch-key $PWD/.chef/config/chef.pem | jq .PasswordData | xargs echo)
450487
knife winrm jenkins-worker-windows-publish chef-client -m -P $PASS
451488
```
452489

@@ -470,7 +507,7 @@ Workaround: make sure EC2 instance names are unique.
470507

471508
http://blog.gravitystorm.co.uk/2013/09/13/using-vagrant-to-test-chef-cookbooks/:
472509

473-
See `.chef/Vagrantfile` -- make sure you first populated `.chef/cookbooks/` using knife,
510+
See `$PWD/.chef/Vagrantfile` -- make sure you first populated `$PWD/.chef/cookbooks/` using knife,
474511
as [documented above](#get-cookbooks)
475512

476513
## If connections hang
@@ -506,15 +543,15 @@ $ openssl req -text -noout -in scala-ci.csr
506543

507544
## Retry bootstrap
508545
```
509-
knife bootstrap -c .chef/knife.rb jenkins-worker-ubuntu-publish --ssh-user ubuntu --sudo -c .chef/knife.rb -N jenkins-worker-ubuntu-publish -r "scala-jenkins-infra::worker-init"
546+
knife bootstrap -c $PWD/.chef/knife.rb jenkins-worker-ubuntu-publish --ssh-user ubuntu --sudo -c $PWD/.chef/knife.rb -N jenkins-worker-ubuntu-publish -r "scala-jenkins-infra::worker-init"
510547
```
511548

512549
## WinRM troubles?
513550
If it appears stuck at "Waiting for remote response before bootstrap.", the userdata didn't make it across
514551
(check C:\Program Files\Amazon\Ec2ConfigService\Logs) we need to enable unencrypted authentication:
515552

516553
```
517-
aws ec2 get-password-data --instance-id $INST --priv-launch-key ~/git/scala-jenkins-infra/.chef/config/chef.pem
554+
aws ec2 get-password-data --instance-id $INST --priv-launch-key $PWD/.chef/config/chef.pem
518555
519556
cord $IP, log in using password above and open a command line:
520557

0 commit comments

Comments
 (0)