PYTHON-3071 [DevOps] Merge and improve resync_specs.sh #839
Conversation
.evergreen/resync-specs.sh
Outdated
| echo "Optional flags:" | ||
| echo " -b is used to add a string to the blocklist for that next run. Can be used" | ||
| echo " any number of times on a single command to block multiple patterns." | ||
| echo " Utilizes ." |
There was a problem hiding this comment.
Yeah, sorry. I realized there are a few errors like that. I will push a commit soon with fixes.
|
Should this script be in the |
|
If it is meant to be used locally, we should add a section to the contributing guide. |
I'm not sure it fits in the tools folder. Those tools are things we ship to pypi whereas this is an internal dev script. Perhaps we should just shove it in the test/ folder? |
|
I've used a |
That sounds good to me. Though I also think it works fine in .evergreen too. |
.evergreen/resync-specs.sh
Outdated
| #!/bin/bash | ||
| # exit when any command fails | ||
| set -e | ||
| PYMONGO=$(dirname $PWD) |
There was a problem hiding this comment.
PWD is the current working directory which isn't necessarily the repo root. Instead we should use the path to this file itself:
PYMONGO=$(dirname $(dirname $0))
There was a problem hiding this comment.
If you run it inside the .evergreen folder as ./resync-specs.sh then it fails because $(dirname $(dirname $0)) resolves to ..
#!/bin/bash
# exit when any command fails
set -ex
PYMONGO=$(dirname $(dirname $0))
echo "$PYMONGO"
...
+ PYMONGO=.
+ echo .
There was a problem hiding this comment.
Sorry this should work:
PYMONGO=$(dirname $(cd $(dirname $0); pwd))
.evergreen/resync-specs.sh
Outdated
| cd $SPECS/source | ||
| make | ||
| cd - | ||
| # cpjson2 unified-test-format/tests/invalid unified-test-format/invalid |
CONTRIBUTING.rst
Outdated
| folders will be copied from and to. It supports | ||
| filtering using regex patterns, and it will show you which files it ignored | ||
| when using this feature so you can ensure you are not leaving any important | ||
| tests behind. |
There was a problem hiding this comment.
It would be more helpful to document how to setup and use the script. Like this (but fixed up so it actually works):
git clone specs repo
export MDB_SPECS=path/to/specs
cd mongodb-python-driver
.evergreen/resync-specs.sh crud bson-corpus
.evergreen/resync-specs.sh
Outdated
| #!/bin/bash | ||
| # exit when any command fails | ||
| set -e | ||
| PYMONGO=$(dirname $PWD) |
There was a problem hiding this comment.
Sorry this should work:
PYMONGO=$(dirname $(cd $(dirname $0); pwd))
…driver/.evergreen
There was a problem hiding this comment.
I ran shellcheck on this file and it showed 3 warnings:
In test.sh line 4:
PYMONGO=$(dirname $(cd $(dirname $0); pwd))
^----------------------^ SC2046 (warning): Quote this to prevent word splitting.
^-----------^ SC2046 (warning): Quote this to prevent word splitting.
^-- SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
PYMONGO=$(dirname $(cd $(dirname "$0"); pwd))
In test.sh line 30:
b) BLOCKLIST+="|"$OPTARG""
^-----^ SC2027 (warning): The surrounding quotes actually unquote this. Remove or escape them.
This is the merged and improved version of resync_specs.sh. I added a help message for basic usage, in addition to the ability to specify a commit or branch for the specs repo, a blocklist feature to ignore files that match regex patterns, and also helpful diagnostic messages that print the specific files ignored by your usage of the blocklist.