Skip to content

[Merged by Bors] - Add support for Spark 3.4.0 #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 14 commits into from
Closed

[Merged by Bors] - Add support for Spark 3.4.0 #243

wants to merge 14 commits into from

Conversation

sbernauer
Copy link
Member

@sbernauer sbernauer commented May 22, 2023 •
edited
Loading

Description

For #242

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes
# Author
- [ ] Changes are OpenShift compatible
- [ ] CRD changes approved
- [ ] Helm chart can be installed and deployed operator works
- [ ] Integration tests passed (for non trivial changes)

# Reviewer
- [ ] Code contains useful comments
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)

# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

sbernauer added 8 commits May 22, 2023 16:22
@sbernauer
Fix Spark driver not getting imagePullPolicy set
b7bec6b 
Reported in https://stackable-workspace.slack.com/archives/C037Q9DPF4P/p1682326403038869
@sbernauer
fix tests
e128f3e
@sbernauer
Revert "Fix Spark driver not getting imagePullPolicy set"
0c14413 
This reverts commit b7bec6b.

The commit succeed in telling spark to set the pull policy on the driver main container. Vector container is still wrong.

Everything is a mess.
Everything is optional (event the image and the Spark version!!!).
Wrong image pull secrets and pull policy everywhere.
No ResolvedProductImage (and if used totally messed up).
Needs complete overhaul!
@sbernauer
Rename spark examples
80e9462
@sbernauer
WIP: Switch images to sandbox
3b38d1f
@sbernauer
Remove comment
548b15f
@sbernauer
Fix tests to support test multiple Spark versions
d4aa0b6
@sbernauer
Remove uneeded file
bbc1df1
@sbernauer sbernauer self-assigned this May 23, 2023
@maltesander maltesander self-requested a review May 23, 2023 08:51
maltesander
maltesander reviewed May 23, 2023
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. Will test a little now.

@sbernauer sbernauer requested a review from maltesander May 23, 2023 10:09
maltesander
maltesander requested changes May 23, 2023
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many references to 3.3.0-stackable0.3.0 instead e.g. 3.4.0-stackable0.0.0-dev left over.

@sbernauer
Copy link
Member Author

Started test https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/spark-k8s-operator-it-custom/86/

@sbernauer
Copy link
Member Author

Bumped stackable versions.
Test passed as well 🚀

@sbernauer sbernauer requested a review from maltesander May 23, 2023 13:19
maltesander
maltesander approved these changes May 23, 2023
View reviewed changes
Copy link
Member

@maltesander maltesander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sbernauer
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request May 23, 2023
@sbernauer
Add support for Spark 3.4.0 (#243)
f4441de 
# Description

For #242
@bors
Copy link

bors bot commented May 23, 2023

Pull request successfully merged into main.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot changed the title Add support for Spark 3.4.0 [Merged by Bors] - Add support for Spark 3.4.0 May 23, 2023
@bors bors bot closed this May 23, 2023
@bors bors bot deleted the spark-3.4.0 branch May 23, 2023 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maltesander maltesander maltesander approved these changes

Assignees

@sbernauer sbernauer

Labels
None yet
Projects
Stackable Engineering
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sbernauer @maltesander