fix missing variable when building the image on cloudbuild #761

cpanato · 2021-04-21T13:56:54Z

What type of PR is this?
/kind bug

What this PR does / why we need it:
When we built the image k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.1 we missed one parameter that needs to be set during the compilation process

This PR fix that, making it similar to how we build the image for windows.

built on my gcp account and now we can run the image without the error we saw before

$ docker run -it k8s.gcr.io/cloud-provider-gcp/gcp-compute-persistent-disk-csi-driver:v1.2.1
F0421 13:50:23.955027       1 main.go:71] version must be set at compile time

# after the fix
$ docker run -it gcr.io/cpanato-general/gcp-compute-persistent-disk-csi-driver:v99.99.99
W0421 13:50:31.576613       1 gce.go:137] GOOGLE_APPLICATION_CREDENTIALS env var not set
F0421 13:50:31.576650       1 main.go:103] Failed to get cloud provider: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

/assign @msau42 @mattcary @saikat-royc

PR for release-1.2: #760

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

cpanato · 2021-04-21T13:58:11Z

on the master branch, the cloudbuild is failing. I will debug and fix in a followup PR

cpanato · 2021-04-21T14:04:08Z

when running the cloudbuilld


#18 [linux/arm64 mad-hack 1/2] FROM k8s.gcr.io/build-image/debian-base-amd64...
#18 sha256:96459fd64d20bd24c773c6e36ee5eca5096e86580da0c38817f76a2b0d09853b 23.29MB / 23.29MB 1.2s done
#18 extracting sha256:96459fd64d20bd24c773c6e36ee5eca5096e86580da0c38817f76a2b0d09853b 8.2s done
#18 DONE 9.5s

#24 [linux/arm64 mad-hack 2/2] RUN clean-install udev
#24 0.290 .buildkit_qemu_emulator: /bin/sh: Invalid ELF image for this architecture
#24 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c clean-install udev]: exit code: 255

#26 [linux/arm64 builder 1/4] FROM docker.io/library/golang:1.13.15@sha256:8...
#26 extracting sha256:d5517ee72007172d5b814636405254dea459120ce08f85777bb287d106a6a240 5.0s done
#26 DONE 9.1s

#19 [linux/arm64 builder 2/4] WORKDIR /go/src/sigs.k8s.io/gcp-compute-persis...
#19 CANCELED

#9 [linux/amd64 builder 1/4] FROM docker.io/library/golang:1.13.15@sha256:8...
#9 extracting sha256:d6ff36c9ec4822c9ff8953560f7ba41653b348a9c1136755e653575f58fbded7 3.7s done
#9 DONE 8.6s

#10 [linux/amd64 builder 2/4] WORKDIR /go/src/sigs.k8s.io/gcp-compute-persis...
#10 CANCELED

#16 [linux/amd64 mad-hack 2/2] RUN clean-install udev
#16 0.521 Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
#16 0.583 Get:2 http://deb.debian.org/debian buster InRelease [121 kB]
#16 0.591 Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
#16 CANCELED
------
 > [linux/arm64 mad-hack 2/2] RUN clean-install udev:
------
failed to solve: rpc error: code = Unknown desc = executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c clean-install udev]: exit code: 255
make: *** [Makefile:85: build-and-push-container-linux] Error 1
ERROR
ERROR: build step 0 "gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f" failed: step exited with non-zero status: 2
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ERROR: (gcloud.builds.submit) build 043356f7-1fd0-4269-b613-8bf8959426a8 completed with status "FAILURE"

cpanato · 2021-04-21T14:09:08Z

ahhh!!
because of this https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/master/Dockerfile#L21 we are building linux/arm64 and trying to load an amd64 😄

I will fix it in a follow up if that is fine

cpanato · 2021-04-21T15:40:39Z

when running the cloudbuild i see this error

#12 49.99 Fetched 5171 kB in 0s (11.3 MB/s)
#12 50.28 Selecting previously unselected package libuuid1:arm64.
(Reading database ... 3887 files and directories currently installed.)
#12 50.30 Preparing to unpack .../libuuid1_2.33.1-0.1_arm64.deb ...
#12 50.34 Unpacking libuuid1:arm64 (2.33.1-0.1) ...
#12 51.00 Error while loading /usr/sbin/rm: No such file or directory
#12 51.00 dpkg: error while cleaning up:
#12 51.00  rm command for cleanup subprocess returned error exit status 1
#12 51.15 Setting up libuuid1:arm64 (2.33.1-0.1) ...
#12 51.36 Error while loading /usr/sbin/rm: No such file or directory
#12 51.37 dpkg: error processing archive /var/cache/apt/archives/libblkid1_2.33.1-0.1_arm64.deb (--unpack):
#12 51.37  rm command for cleanup subprocess returned error exit status 1
#12 51.40 Errors were encountered while processing:
#12 51.40  /var/cache/apt/archives/libblkid1_2.33.1-0.1_arm64.deb
#12 51.53 E: Sub-process /usr/bin/dpkg returned an error code (1)
#12 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c clean-install udev]: exit code: 100

it is in the clean-install udev part for arm64

cpanato · 2021-04-21T15:56:54Z

for reference: kubernetes/release#1983

mattcary · 2021-04-21T16:15:14Z

/assign @sarandia

Ziyuan, can you look at the arm issue?

k8s-ci-robot · 2021-04-21T16:15:15Z

@mattcary: GitHub didn't allow me to assign the following users: sarandia.

Note that only kubernetes-sigs members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @sarandia

Ziyuan, can you look at the arm issue?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cpanato · 2021-04-21T16:16:36Z

@mattcary i'm checking and i have a running cloudbuild now :)

sarandia · 2021-04-21T16:24:33Z

@cpanato

when running the cloudbuild i see this error

#12 49.99 Fetched 5171 kB in 0s (11.3 MB/s)
#12 50.28 Selecting previously unselected package libuuid1:arm64.
(Reading database ... 3887 files and directories currently installed.)
#12 50.30 Preparing to unpack .../libuuid1_2.33.1-0.1_arm64.deb ...
#12 50.34 Unpacking libuuid1:arm64 (2.33.1-0.1) ...
#12 51.00 Error while loading /usr/sbin/rm: No such file or directory
#12 51.00 dpkg: error while cleaning up:
#12 51.00  rm command for cleanup subprocess returned error exit status 1
#12 51.15 Setting up libuuid1:arm64 (2.33.1-0.1) ...
#12 51.36 Error while loading /usr/sbin/rm: No such file or directory
#12 51.37 dpkg: error processing archive /var/cache/apt/archives/libblkid1_2.33.1-0.1_arm64.deb (--unpack):
#12 51.37  rm command for cleanup subprocess returned error exit status 1
#12 51.40 Errors were encountered while processing:
#12 51.40  /var/cache/apt/archives/libblkid1_2.33.1-0.1_arm64.deb
#12 51.53 E: Sub-process /usr/bin/dpkg returned an error code (1)
#12 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c clean-install udev]: exit code: 100

it is in the clean-install udev part for arm64

The commit in PR #751 should resolve this. Did it work?

cpanato · 2021-04-21T16:29:26Z

@sarandia not yet, working in a fix, running cloudbuild now to validate, will push soon

sarandia · 2021-04-21T16:30:36Z

#752 introduced hard-coded image paths with the -amd64 suffix. That's why this build was working locally for me and not working here at HEAD. Thanks @cpanato for taking a look into this.

cpanato · 2021-04-21T16:39:29Z

now it is failing in the last stage

#20 90.01 Selecting previously unselected package python3.7.
#20 90.02 Preparing to unpack .../7-python3.7_3.7.3-2+deb10u3_arm64.deb ...
#20 90.03 Unpacking python3.7 (3.7.3-2+deb10u3) ...
#20 90.66 Selecting previously unselected package libpython3-stdlib:arm64.
#20 90.66 Preparing to unpack .../8-libpython3-stdlib_3.7.3-1_arm64.deb ...
#20 90.67 Unpacking libpython3-stdlib:arm64 (3.7.3-1) ...
#20 90.99 Setting up python3-minimal (3.7.3-1) ...
#20 95.92 Traceback (most recent call last):
#20 95.92   File "/usr/bin/py3compile", line 290, in <module>
#20 95.93     main()
#20 95.93   File "/usr/bin/py3compile", line 251, in main
#20 95.93     pkg_files = tuple(dpf.from_package(options.package))
#20 95.93   File "/usr/share/python3/debpython/files.py", line 53, in from_package
#20 95.93     raise Exception("cannot get content of %s" % package_name)
#20 95.93 Exception: cannot get content of python3-minimal
#20 95.99 dpkg: error processing package python3-minimal (--configure):
#20 95.99  installed python3-minimal package post-installation script subprocess returned error exit status 1
#20 96.01 Errors were encountered while processing:
#20 96.02  python3-minimal
#20 96.25 E: Sub-process /usr/bin/dpkg returned an error code (1)
#20 ERROR: executor failed running [/dev/.buildkit_qemu_emulator /bin/sh -c ln -s /bin/rm /usr/sbin/rm   && clean-install util-linux e2fsprogs mount ca-certificates udev xfsprogs]: exit code: 100

I will debug that and try to fix it tomorrow.

sarandia · 2021-04-21T17:37:36Z

I think we need to merge my PR #751 first, which contains some bug fixes to the Makefile. @cpanato

cpanato · 2021-04-22T08:54:27Z

build that in my account and now it works!

skopeo inspect docker://gcr.io/cpanato-general/gcp-compute-persistent-disk-csi-driver@sha256:d3b4d7f397f27e9c873847a27f91386e461633ae98562bcf7523bec3d9d9e708 --raw
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 956,
         "digest": "sha256:2fe23c50cef3bb3848373083c0649413d6cbcb1ae4f18df983deee1269ea21a9",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.18363.1500"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 956,
         "digest": "sha256:e1e4f404d28eb1e6e38f300027e8761ac19938eb69e199131fb257d7b1eb8b03",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.19041.928"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 956,
         "digest": "sha256:61f2bfc9812adf9291af205af57ec009aa2f7979726b0cbee4b912397da68d47",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.19042.928"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1162,
         "digest": "sha256:e165f2e8578a8ee5393f7c0f970f372e040670b54cd4418177f1e0966863cf85",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1162,
         "digest": "sha256:3574771154aab96c5ec8e2b8c1ea8ceefbc330f0aa4d289b82aa6e42a4fc14e7",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 956,
         "digest": "sha256:dc3f667d6f709d3efd26282eee2415a772882fa0465c0fb2a4f76eac763190fc",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.17763.1879"
         }
      }
   ]
}

cpanato · 2021-04-22T10:54:41Z

previous run

W0422 10:05:18.266] F0422 10:05:18.266115    6100 main.go:187] Failed to run integration test: failed to install CSI Driver: failed to install kustomize: installing kustomize

/test pull-gcp-compute-persistent-disk-csi-driver-kubernetes-integration

cpanato · 2021-04-22T12:25:41Z

looks like the same flaky error as other PRs that I was following

/test pull-gcp-compute-persistent-disk-csi-driver-kubernetes-integration

cpanato · 2021-04-22T12:26:59Z

@msau42 @sarandia @mattcary this PR is ready for review and this applies to the master branch, for the release-1.2 branch there is another PR (because in the release-1.2 we don't have this multi-platform for Linux yet) - #760

thanks!

mattcary · 2021-04-22T15:18:42Z

That /sbin/rm thing is weird...

Thx for updating the debian base.

/lgtm

jingxu97 · 2021-04-22T21:19:33Z

Dockerfile

 WORKDIR /go/src/sigs.k8s.io/gcp-compute-persistent-disk-csi-driver
 ADD . .
-RUN make gce-pd-driver
+# RUN mkdir -p bin \


remove this line and the following?

oh my bad! :(
will remove that in a follow up pr

jingxu97 · 2021-04-22T21:23:57Z

/approve

k8s-ci-robot · 2021-04-22T21:24:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, jingxu97

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jingxu97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mauriciopoppe · 2021-04-30T00:13:04Z

Makefile

-	STAGINGVERSION=${REV}
-endif
-
-GCFLAGS=""


GCFLAGS was removed too, is this ok?

They aren't used anymore, but this will allow them to be set from an env. That was my understanding at any rate.

the value of GCFLAGS was set conditionally based on this flag before:

GCFLAGS="" ifdef GCE_PD_CSI_DEBUG GCFLAGS="all=-N -l" endif

GCE_PD_CSI_DEBUG is set in Dockerfile.debug

ohh, my fault! i might remove that by mistake :(

mattcary · 2021-04-30T00:57:47Z

Oh, sorry, I missed that. My bad, I'll put it back in.

k8s-ci-robot assigned mattcary Apr 21, 2021

k8s-ci-robot added the release-note-none Denotes a PR that doesn't merit a release note. label Apr 21, 2021

k8s-ci-robot assigned msau42 and saikat-royc Apr 21, 2021

k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 21, 2021

k8s-ci-robot requested review from jingxu97 and saikat-royc April 21, 2021 13:57

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 21, 2021

cpanato mentioned this pull request Apr 21, 2021

tests/e2e: replace deprecated registry to use the k8s community kubernetes/kubernetes#100747

Closed

fix missing variable when building the image on cloudbuild

e47ce9a

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 22, 2021

Fix cloudbuild build

7ce26e6

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 22, 2021

jingxu97 reviewed Apr 22, 2021

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 22, 2021

k8s-ci-robot merged commit 3bc7660 into kubernetes-sigs:master Apr 22, 2021

cpanato deleted the fix-image branch April 23, 2021 08:03

cpanato mentioned this pull request Apr 23, 2021

remove comments #763

Closed

mauriciopoppe reviewed Apr 30, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix missing variable when building the image on cloudbuild #761

fix missing variable when building the image on cloudbuild #761

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021 •

edited

Loading

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

mattcary commented Apr 21, 2021

k8s-ci-robot commented Apr 21, 2021

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021 •

edited

Loading

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021

cpanato commented Apr 22, 2021 •

edited

Loading

cpanato commented Apr 22, 2021 •

edited

Loading

cpanato commented Apr 22, 2021

cpanato commented Apr 22, 2021

mattcary commented Apr 22, 2021

jingxu97 Apr 22, 2021 •

edited

Loading

cpanato Apr 23, 2021

jingxu97 commented Apr 22, 2021

k8s-ci-robot commented Apr 22, 2021

mauriciopoppe Apr 30, 2021

mattcary Apr 30, 2021

mauriciopoppe Apr 30, 2021

cpanato Apr 30, 2021

mattcary commented Apr 30, 2021

fix missing variable when building the image on cloudbuild #761

fix missing variable when building the image on cloudbuild #761

Conversation

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021 • edited Loading

cpanato commented Apr 21, 2021

cpanato commented Apr 21, 2021

mattcary commented Apr 21, 2021

k8s-ci-robot commented Apr 21, 2021

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021 • edited Loading

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021

cpanato commented Apr 21, 2021

sarandia commented Apr 21, 2021

cpanato commented Apr 22, 2021 • edited Loading

cpanato commented Apr 22, 2021 • edited Loading

cpanato commented Apr 22, 2021

cpanato commented Apr 22, 2021

mattcary commented Apr 22, 2021

jingxu97 Apr 22, 2021 • edited Loading

Choose a reason for hiding this comment

cpanato Apr 23, 2021

Choose a reason for hiding this comment

jingxu97 commented Apr 22, 2021

k8s-ci-robot commented Apr 22, 2021

mauriciopoppe Apr 30, 2021

Choose a reason for hiding this comment

mattcary Apr 30, 2021

Choose a reason for hiding this comment

mauriciopoppe Apr 30, 2021

Choose a reason for hiding this comment

cpanato Apr 30, 2021

Choose a reason for hiding this comment

mattcary commented Apr 30, 2021

cpanato commented Apr 21, 2021 •

edited

Loading

sarandia commented Apr 21, 2021 •

edited

Loading

cpanato commented Apr 22, 2021 •

edited

Loading

cpanato commented Apr 22, 2021 •

edited

Loading

jingxu97 Apr 22, 2021 •

edited

Loading