Skip to content

DOC: Simplify the gbq integration testing procedure for contributors #14541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions ci/travis_encrypt_gbq.sh
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
#!/bin/bash

GBQ_JSON_FILE=$1
GBQ_PROJECT_ID=$2

if [[ $# -ne 2 ]]; then
if [[ $# -ne 1 ]]; then
echo -e "Too few arguments.\nUsage: ./travis_encrypt_gbq.sh "\
"<gbq-json-credentials-file> <gbq-project-id>"
"<gbq-json-credentials-file>"
exit 1
fi

Expand All @@ -23,9 +22,9 @@ echo "Encrypting $GBQ_JSON_FILE..."
read -d "\n" TRAVIS_KEY TRAVIS_IV <<<$(travis encrypt-file $GBQ_JSON_FILE \
travis_gbq.json.enc -f | grep -o "\w*_iv\|\w*_key");

echo "Adding your secure key and project id to travis_gbq_config.txt ..."
echo -e "TRAVIS_IV_ENV=$TRAVIS_IV\nTRAVIS_KEY_ENV=$TRAVIS_KEY\n"\
"GBQ_PROJECT_ID='$GBQ_PROJECT_ID'" > travis_gbq_config.txt
echo "Adding your secure key to travis_gbq_config.txt ..."
echo -e "TRAVIS_IV_ENV=$TRAVIS_IV\nTRAVIS_KEY_ENV=$TRAVIS_KEY"\
> travis_gbq_config.txt

echo "Done. Removing file $GBQ_JSON_FILE"
rm $GBQ_JSON_FILE
Expand Down
1 change: 0 additions & 1 deletion ci/travis_gbq_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
TRAVIS_IV_ENV=encrypted_1d9d7b1f171b_iv
TRAVIS_KEY_ENV=encrypted_1d9d7b1f171b_key
GBQ_PROJECT_ID='pandas-travis'
6 changes: 4 additions & 2 deletions ci/travis_process_gbq_encryption.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@

source ci/travis_gbq_config.txt

if [[ -n ${!TRAVIS_IV_ENV} ]]; then
if [[ -n ${SERVICE_ACCOUNT_KEY} ]]; then
echo "${SERVICE_ACCOUNT_KEY}" > ci/travis_gbq.json;
elif [[ -n ${!TRAVIS_IV_ENV} ]]; then
openssl aes-256-cbc -K ${!TRAVIS_KEY_ENV} -iv ${!TRAVIS_IV_ENV} \
-in ci/travis_gbq.json.enc -out ci/travis_gbq.json -d;
export GBQ_PROJECT_ID=$GBQ_PROJECT_ID;
export GBQ_PROJECT_ID='pandas-travis';
echo 'Successfully decrypted gbq credentials'
fi

30 changes: 16 additions & 14 deletions doc/source/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -643,20 +643,22 @@ on Travis-CI and are only accessible from the pandas-dev/pandas repository. The
credentials won't be available on forks of pandas. Here are the steps to run
gbq integration tests on a forked repository:

#. First, complete all the steps in the `Encrypting Files Prerequisites
<https://docs.travis-ci.com/user/encrypting-files/>`__ section.
#. Sign into `Travis <https://travis-ci.org/>`__ using your GitHub account.
#. Enable your forked repository of pandas for testing in `Travis
<https://travis-ci.org/profile/>`__.
#. Run the following command from terminal where the current working directory
is the ``ci`` folder::

./travis_encrypt_gbq.sh <gbq-json-credentials-file> <gbq-project-id>

#. Create a new branch from the branch used in your pull request. Commit the
encrypted file called ``travis_gbq.json.enc`` as well as the file
``travis_gbq_config.txt``, in an otherwise empty commit. DO NOT commit the
``*.json`` file which contains your unencrypted private key.
#. Go to `Travis CI <https://travis-ci.org/>`__ and sign in with your GitHub
account.
#. Click on the ``+`` icon next to the ``My Repositories`` list and enable
Travis builds for your fork.
#. Click on the gear icon to edit your travis build, and add two environment
variables:

- ``GBQ_PROJECT_ID`` with the value being the ID of your BigQuery project.

- ``SERVICE_ACCOUNT_KEY`` with the value being the contents of the JSON key
that you downloaded for your service account. Use single quotes around
your JSON key to ensure that it is treated as a string.

For both environment variables, keep the "Display value in build log" option
DISABLED. These variables contain sensitive data and you do not want their
contents being exposed in build logs.
#. Your branch should be tested automatically once it is pushed. You can check
the status by visiting your Travis branches page which exists at the
following location: https://travis-ci.org/your-user-name/pandas/branches .
Expand Down