Skip to content

Commit 89be114

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #78 from pohly/prow-update-master
master: update release-tools
2 parents e9aafaf + 5cfc84c commit 89be114

17 files changed

+487
-36
lines changed

‎release-tools/.prow.sh

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,23 @@
11
#! /bin/bash -e
2+
3+
# Copyright 2021 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
28
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
317
# This is for testing csi-release-tools itself in Prow. All other
418
# repos use prow.sh for that, but as csi-release-tools isn't a normal
519
# repo with some Go code in it, it has a custom Prow test script.
620

721
./verify-shellcheck.sh "$(pwd)"
22+
./verify-spelling.sh "$(pwd)"
23+
./verify-boilerplate.sh "$(pwd)"

‎release-tools/README.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,17 @@ is the recommended way of maintaining a copy of the rules inside the
4444
changes also locally, test them and then push them back to the shared
4545
repository at a later time.
4646

47+
We no longer care about importing the full commit history, so `--squash` should be used
48+
when submitting a `release-tools` update. Also make sure that the PR for that
49+
contains the automatically generated commit message in the PR description.
50+
It contains the list of individual commits that were squashed. The script from
51+
https://github.com/kubernetes-csi/csi-release-tools/issues/7 can create such
52+
PRs automatically.
53+
4754
Cheat sheet:
4855

49-
- `git subtree add --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - add release tools to a repo which does not have them yet (only once)
50-
- `git subtree pull --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - update local copy to latest upstream (whenever upstream changes)
56+
- `git subtree add --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - add release tools to a repo which does not have them yet (only once)
57+
- `git subtree pull --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - update local copy to latest upstream (whenever upstream changes)
5158
- edit, `git commit`, `git subtree push --prefix=release-tools [email protected]:<user>/csi-release-tools.git <my-new-or-existing-branch>` - push to a new branch before submitting a PR
5259

5360
verify-shellcheck.sh

‎release-tools/SIDECAR_RELEASE_PROCESS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The release manager must:
1111
kubernetes/org to request membership
1212
* Be a top level approver for the repository. To become a top level approver,
1313
the candidate must demonstrate ownership and deep knowledge of the repository
14-
through active maintainence, responding to and fixing issues, reviewing PRs,
14+
through active maintenance, responding to and fixing issues, reviewing PRs,
1515
test triage.
1616
* Be part of the maintainers or admin group for the repository. admin is a
1717
superset of maintainers, only maintainers level is required for cutting a

‎release-tools/boilerplate/boilerplate.Dockerfile.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎release-tools/boilerplate/boilerplate.Makefile.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎release-tools/boilerplate/boilerplate.bzl.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎release-tools/boilerplate/boilerplate.go.txt

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
/*
2+
Copyright YEAR The Kubernetes Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/

‎release-tools/boilerplate/boilerplate.py

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2019 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
from __future__ import print_function
18+
19+
import argparse
20+
import difflib
21+
import glob
22+
import os
23+
import re
24+
import sys
25+
from datetime import date
26+
27+
parser = argparse.ArgumentParser()
28+
parser.add_argument(
29+
"filenames",
30+
help="list of files to check, all files if unspecified",
31+
nargs='*')
32+
33+
# Rootdir defaults to the directory **above** the repo-infra dir.
34+
rootdir = os.path.dirname(__file__) + "./../../"
35+
rootdir = os.path.abspath(rootdir)
36+
parser.add_argument(
37+
"--rootdir", default=rootdir, help="root directory to examine")
38+
39+
default_boilerplate_dir = os.path.abspath(os.path.dirname(__file__))
40+
41+
parser.add_argument(
42+
"--boilerplate-dir", default=default_boilerplate_dir)
43+
44+
parser.add_argument(
45+
"-v", "--verbose",
46+
help="give verbose output regarding why a file does not pass",
47+
action="store_true")
48+
49+
args = parser.parse_args()
50+
51+
verbose_out = sys.stderr if args.verbose else open("/dev/null", "w")
52+
53+
def get_refs():
54+
refs = {}
55+
56+
for path in glob.glob(os.path.join(args.boilerplate_dir, "boilerplate.*.txt")):
57+
extension = os.path.basename(path).split(".")[1]
58+
59+
ref_file = open(path, 'r')
60+
ref = ref_file.read().splitlines()
61+
ref_file.close()
62+
refs[extension] = ref
63+
64+
return refs
65+
66+
def file_passes(filename, refs, regexs):
67+
try:
68+
f = open(filename, 'r')
69+
except Exception as exc:
70+
print("Unable to open %s: %s" % (filename, exc), file=verbose_out)
71+
return False
72+
73+
data = f.read()
74+
f.close()
75+
76+
basename = os.path.basename(filename)
77+
extension = file_extension(filename)
78+
if extension != "":
79+
ref = refs[extension]
80+
else:
81+
ref = refs[basename]
82+
83+
# remove build tags from the top of Go files
84+
if extension == "go":
85+
p = regexs["go_build_constraints"]
86+
(data, found) = p.subn("", data, 1)
87+
88+
# remove shebang from the top of shell files
89+
if extension == "sh" or extension == "py":
90+
p = regexs["shebang"]
91+
(data, found) = p.subn("", data, 1)
92+
93+
data = data.splitlines()
94+
95+
# if our test file is smaller than the reference it surely fails!
96+
if len(ref) > len(data):
97+
print('File %s smaller than reference (%d < %d)' %
98+
(filename, len(data), len(ref)),
99+
file=verbose_out)
100+
return False
101+
102+
# trim our file to the same number of lines as the reference file
103+
data = data[:len(ref)]
104+
105+
p = regexs["year"]
106+
for d in data:
107+
if p.search(d):
108+
print('File %s is missing the year' % filename, file=verbose_out)
109+
return False
110+
111+
# Replace all occurrences of the regex "CURRENT_YEAR|...|2016|2015|2014" with "YEAR"
112+
p = regexs["date"]
113+
for i, d in enumerate(data):
114+
(data[i], found) = p.subn('YEAR', d)
115+
if found != 0:
116+
break
117+
118+
# if we don't match the reference at this point, fail
119+
if ref != data:
120+
print("Header in %s does not match reference, diff:" % filename, file=verbose_out)
121+
if args.verbose:
122+
print(file=verbose_out)
123+
for line in difflib.unified_diff(ref, data, 'reference', filename, lineterm=''):
124+
print(line, file=verbose_out)
125+
print(file=verbose_out)
126+
return False
127+
128+
return True
129+
130+
def file_extension(filename):
131+
return os.path.splitext(filename)[1].split(".")[-1].lower()
132+
133+
skipped_dirs = ['Godeps', 'third_party', '_gopath', '_output', '.git',
134+
'cluster/env.sh', 'vendor', 'test/e2e/generated/bindata.go',
135+
'repo-infra/verify/boilerplate/test', '.glide']
136+
137+
def normalize_files(files):
138+
newfiles = []
139+
for pathname in files:
140+
if any(x in pathname for x in skipped_dirs):
141+
continue
142+
newfiles.append(pathname)
143+
return newfiles
144+
145+
def get_files(extensions):
146+
files = []
147+
if len(args.filenames) > 0:
148+
files = args.filenames
149+
else:
150+
for root, dirs, walkfiles in os.walk(args.rootdir):
151+
# don't visit certain dirs. This is just a performance improvement
152+
# as we would prune these later in normalize_files(). But doing it
153+
# cuts down the amount of filesystem walking we do and cuts down
154+
# the size of the file list
155+
for d in skipped_dirs:
156+
if d in dirs:
157+
dirs.remove(d)
158+
159+
for name in walkfiles:
160+
pathname = os.path.join(root, name)
161+
files.append(pathname)
162+
163+
files = normalize_files(files)
164+
165+
outfiles = []
166+
for pathname in files:
167+
basename = os.path.basename(pathname)
168+
extension = file_extension(pathname)
169+
if extension in extensions or basename in extensions:
170+
outfiles.append(pathname)
171+
return outfiles
172+
173+
def get_regexs():
174+
regexs = {}
175+
# Search for "YEAR" which exists in the boilerplate, but shouldn't in the real thing
176+
regexs["year"] = re.compile( 'YEAR' )
177+
# dates can be 2014, 2015, 2016, ..., CURRENT_YEAR, company holder names can be anything
178+
years = range(2014, date.today().year + 1)
179+
regexs["date"] = re.compile( '(%s)' % "|".join(map(lambda l: str(l), years)) )
180+
# strip // +build \n\n build constraints
181+
regexs["go_build_constraints"] = re.compile(r"^(// \+build.*\n)+\n", re.MULTILINE)
182+
# strip #!.* from shell scripts
183+
regexs["shebang"] = re.compile(r"^(#!.*\n)\n*", re.MULTILINE)
184+
return regexs
185+
186+
187+
188+
def main():
189+
regexs = get_regexs()
190+
refs = get_refs()
191+
filenames = get_files(refs.keys())
192+
193+
for filename in filenames:
194+
if not file_passes(filename, refs, regexs):
195+
print(filename, file=sys.stdout)
196+
197+
return 0
198+
199+
if __name__ == "__main__":
200+
sys.exit(main())

‎release-tools/boilerplate/boilerplate.py.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎release-tools/boilerplate/boilerplate.sh.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright YEAR The Kubernetes Authors.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎release-tools/build.make

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020

2121
# This is the default. It can be overridden in the main Makefile after
2222
# including build.make.
23-
REGISTRY_NAME=quay.io/k8scsi
23+
REGISTRY_NAME?=quay.io/k8scsi
2424

2525
# Can be set to -mod=vendor to ensure that the "vendor" directory is used.
2626
GOFLAGS_VENDOR=
@@ -275,3 +275,16 @@ test-shellcheck:
275275
.PHONY: check-go-version-%
276276
check-go-version-%:
277277
./release-tools/verify-go-version.sh "$*"
278+
279+
# Test for spelling errors.
280+
.PHONY: test-spelling
281+
test-spelling:
282+
@ echo; echo "### $@:"
283+
@ ./release-tools/verify-spelling.sh "$(pwd)"
284+
285+
# Test the boilerplates of the files.
286+
.PHONY: test-boilerplate
287+
test-boilerplate:
288+
@ echo; echo "### $@:"
289+
@ ./release-tools/verify-boilerplate.sh "$(pwd)"
290+

‎release-tools/cloudbuild.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,19 @@
11
#! /bin/bash
22

3+
# Copyright 2021 The Kubernetes Authors.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
317
# shellcheck disable=SC1091
418
. release-tools/prow.sh
519

0 commit comments

Comments
 (0)