Skip to content

[Merged by Bors] - Update to latest stackable images #176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.

## [Unreleased]

### Changed

- Updated stackable image versions ([#176])

[#176]: https://github.com/stackabletech/spark-k8s-operator/pull/176

## [0.6.0] - 2022-11-07

### Changed
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/examples/example-encapsulated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: spark-pi
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0 # <1>
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0 # <1>
mode: cluster
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: /stackable/spark/examples/jars/spark-examples_2.12-3.3.0.jar # <2>
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/examples/example-sparkapp-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: s3a://stackable-spark-k8s-jars/jobs/ny-tlc-report-1.1.0.jar # <3>
mainClass: tech.stackable.demo.spark.NYTLCReport
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: s3a://stackable-spark-k8s-jars/jobs/ny_tlc_report.py # <1>
args:
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/examples/example-sparkapp-image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ metadata:
spec:
version: "1.0"
image: docker.stackable.tech/stackable/ny-tlc-report:0.1.0 # <1>
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: local:///stackable/spark/jobs/ny_tlc_report.py # <2>
args:
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/examples/example-sparkapp-pvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: s3a://stackable-spark-k8s-jars/jobs/ny-tlc-report-1.0-SNAPSHOT.jar # <1>
mainClass: org.example.App # <2>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: example-sparkapp-s3-private
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: s3a://my-bucket/spark-examples_2.12-3.3.0.jar # <1>
mainClass: org.apache.spark.examples.SparkPi # <2>
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/examples/example-sparkapp-streaming.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.1.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: local:///stackable/spark/examples/src/main/python/streaming/hdfs_wordcount.py
args:
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/usage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ Below are listed the CRD fields that can be defined by the user:
|User-supplied image containing spark-job dependencies that will be copied to the specified volume mount

|`spec.sparkImage`
| Spark image which will be deployed to driver and executor pods, which must contain spark environment needed by the job e.g. `docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0`
| Spark image which will be deployed to driver and executor pods, which must contain spark environment needed by the job e.g. `docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0`

|`spec.sparkImagePullPolicy`
| Optional Enum (one of `Always`, `IfNotPresent` or `Never`) that determines the pull policy of the spark job image
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: local:///stackable/spark/examples/src/main/python/pi.py
driver:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: local:///stackable/spark/examples/src/main/python/pi.py
driver:
Expand Down
2 changes: 1 addition & 1 deletion examples/ny-tlc-report-external-dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
namespace: default
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
# Always | IfNotPresent | Never
sparkImagePullPolicy: IfNotPresent
mode: cluster
Expand Down
2 changes: 1 addition & 1 deletion examples/ny-tlc-report-image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ spec:
version: "1.0"
# everything under /jobs will be copied to /stackable/spark/jobs
image: docker.stackable.tech/stackable/ny-tlc-report:0.1.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/pyspark-k8s:3.3.0-stackable0.3.0
sparkImagePullPolicy: IfNotPresent
mode: cluster
mainApplicationFile: local:///stackable/spark/jobs/ny_tlc_report.py
Expand Down
2 changes: 1 addition & 1 deletion examples/ny-tlc-report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ metadata:
name: spark-ny-cm
spec:
version: "1.0"
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.2.0
sparkImage: docker.stackable.tech/stackable/spark-k8s:3.3.0-stackable0.3.0
mode: cluster
mainApplicationFile: s3a://my-bucket/ny-tlc-report-1.1.0-3.3.0.jar
mainClass: tech.stackable.demo.spark.NYTLCReport
Expand Down
2 changes: 1 addition & 1 deletion kind/assert-pvc-jars.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ spec:
claimName: pvc-ksv
containers:
- name: assert-pvc-jars
image: docker.stackable.tech/stackable/tools:0.2.0-stackable0
image: docker.stackable.tech/stackable/tools:0.2.0-stackable0.4.0
env:
- name: DEST_DIR
value: "/dependencies/jars"
Expand Down
2 changes: 1 addition & 1 deletion kind/kind-pvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ spec:
claimName: pvc-ksv
containers:
- name: aws-deps
image: docker.stackable.tech/stackable/tools:0.2.0-stackable0
image: docker.stackable.tech/stackable/tools:0.2.0-stackable0.4.0
env:
- name: DEST_DIR
value: "/dependencies/jars"
Expand Down
4 changes: 2 additions & 2 deletions tests/test-definition.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# To run these tests on OpenShift you have ensure that:
# To run these tests on OpenShift you have to ensure that:
# 1. The "openshift" dimension below is set to "true"
# 2. At least one node in the cluster is labeled with "node: 1"
#
Expand All @@ -13,7 +13,7 @@ dimensions:
- 3.3.0
- name: stackable
values:
- 0.2.0
- 0.3.0
- name: ny-tlc-report
values:
- 0.1.0
Expand Down