Skip to content
This repository was archived by the owner on Feb 16, 2024. It is now read-only.

[Merged by Bors] - Spark anomaly detection demo #155

Closed
wants to merge 26 commits into from

Conversation

adwk67
Copy link
Member

@adwk67 adwk67 commented Oct 28, 2022

Description

A demo using spark-k8s to create anomaly detection scores from NYC taxi data, writing the results to a Trino table and displaying them in a Superset dashboard.

Closes #142.

Review Checklist

  • Code contains useful comments
  • Documentation added (or not applicable)
  • Changelog updated (or not applicable)
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

@adwk67
Copy link
Member Author

adwk67 commented Oct 31, 2022

Sample output:
image

@adwk67 adwk67 marked this pull request as ready for review November 2, 2022 07:45
@adwk67 adwk67 requested a review from a team November 3, 2022 09:44
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Demo worked fine for me 👍
Iceberg might be a bit of overkill here, but maybe gives us benefits in the future, e.g. compaction during streaming.

@adwk67
Copy link
Member Author

adwk67 commented Nov 7, 2022

bors merge

bors bot pushed a commit that referenced this pull request Nov 7, 2022
## Description

A demo using spark-k8s to create anomaly detection scores from NYC taxi data, writing the results to a Trino table and displaying them in a Superset dashboard.

Closes #142.
@bors
Copy link

bors bot commented Nov 7, 2022

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Spark anomaly detection demo [Merged by Bors] - Spark anomaly detection demo Nov 7, 2022
@bors bors bot closed this Nov 7, 2022
@bors bors bot deleted the 142_spark_anomaly_detection branch November 7, 2022 12:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Anomaly detection demo: Spark-k8s
2 participants