Lineage visualization #3462

jkasiraj · 2022-11-07T19:54:31Z

Issue #, if available:

Description of changes:

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

feature: add __str__ methods to queryLineage output classes

edge.association_type added style changes of graph

edge.association_type added style changes of graph DashVisualizer class added Try except when importing visual modules

feature: query lineage visualizer for general case

…onent handle

Merge pull request #3 from ytlee93/master

feature: query lineage visualizer advanced styling & interactive component handling

Merge pull request #4 from ytlee93/master

sagemaker-bot · 2022-11-07T20:46:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8d3ca3f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-07T20:53:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8d3ca3f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-07T21:00:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8d3ca3f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-07T22:50:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8d3ca3f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-16T23:47:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2ac472a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-16T23:52:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 2ac472a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T00:03:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2ac472a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T00:08:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 2ac472a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T00:24:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 2ac472a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-11-17T00:24:18Z

Codecov Report

Merging #3462 (1d3cdb3) into master (1fa2377) will decrease coverage by 0.54%.
The diff coverage is 79.78%.

@@            Coverage Diff             @@
##           master    #3462      +/-   ##
==========================================
- Coverage   89.17%   88.63%   -0.55%     
==========================================
  Files         204      205       +1     
  Lines       18979    19543     +564     
==========================================
+ Hits        16924    17321     +397     
- Misses       2055     2222     +167

Impacted Files	Coverage Δ
src/sagemaker/automl/automl.py	`91.32% <ø> (ø)`
src/sagemaker/huggingface/model.py	`89.87% <ø> (ø)`
src/sagemaker/pipeline.py	`85.33% <ø> (ø)`
src/sagemaker/sklearn/model.py	`92.98% <ø> (-1.76%)`	⬇️
...sagemaker/workflow/monitor_batch_transform_step.py	`0.00% <0.00%> (ø)`
src/sagemaker/xgboost/model.py	`96.36% <ø> (-1.82%)`	⬇️
...agemaker/model_monitor/clarify_model_monitoring.py	`89.84% <27.58%> (-5.09%)`	⬇️
src/sagemaker/lineage/query.py	`77.36% <46.59%> (-17.61%)`	⬇️
src/sagemaker/session.py	`71.48% <54.54%> (-0.19%)`	⬇️
src/sagemaker/training_compiler/config.py	`80.43% <60.00%> (-2.90%)`	⬇️
... and 39 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2022-11-17T01:08:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1d3cdb3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T01:16:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1d3cdb3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T01:24:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1d3cdb3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T01:26:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1d3cdb3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-11-17T01:38:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1d3cdb3
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navinsoni

Please work with sagemaker-python-sdk regarding this change.

mufaddal-rohawala · 2023-03-22T00:06:12Z

@jkasiraj this PR has been pending for a while, Im closing this, please reopen as needed.

jkasiraj · 2023-05-04T21:11:10Z

Changes merged here: #3825

Yi-Ting Lee and others added 30 commits July 1, 2022 12:59

feature: add __str__ methods to queryLineage output classes

77965db

Merge pull request #1 from ytlee93/master

6d005e2

feature: add __str__ methods to queryLineage output classes

Merge branch 'aws:master' into master

e0c59c2

feature: query lineage visualizer for general case

7359b3d

edge.association_type added style changes of graph

startarn added to lineageQueryResult

d524416

color node by lineage entity

acf64f4

identify startarn node by shape

e07e83c

Add code comments

687d6b6

Double sided arrows handled

e3a4c9d

legend added

1808f87

feature: query lineage visualizer for general case

79b193a

edge.association_type added style changes of graph DashVisualizer class added Try except when importing visual modules

Merge pull request #3 from ytlee93/master

cbe445e

feature: query lineage visualizer for general case

try except raise

60904d5

startarn added

9125eb9

add get element function

3db7fa1

add DashVisualizer class

c135ab9

style check

daa5cc4

Merge branch 'master' into advanced-style

0adc4db

feature: query lineage visualizer advanced styling & interactive comp…

28a9eeb

…onent handle

Merge pull request #1 from Snoyark/master

67346a7

Merge pull request #3 from ytlee93/master

add functions that generate html components and style selectors

7c2b0c3

inject graph data to DashVisualizer task

41d2453

Merge pull request #4 from ytlee93/master

e6078a9

feature: query lineage visualizer advanced styling & interactive component handling

test_lineage_visualize.py created

b74e861

Merge pull request #2 from Snoyark/master

0b35553

Merge pull request #4 from ytlee93/master

PyvisVisualizer added

b6b5ee4

modify options

794fb57

change: change visualization to using pyvis library

baeaace

Merge branch 'master' into large-graph-test

c3d2ff7

pyvis import issue on running lineage test

578acdf

fix: remove no-else-return block in _get_legend_line

2ac472a

Merge branch 'master' into lineageVisualization

1d3cdb3

vikas0203 requested review from a team, navinsoni and knikure and removed request for a team November 23, 2022 20:17

vikas0203 assigned navinsoni Nov 23, 2022

navinsoni suggested changes Nov 23, 2022

View reviewed changes

trajanikant requested review from mufaddal-rohawala and removed request for knikure February 8, 2023 19:57

trajanikant assigned mufaddal-rohawala and unassigned navinsoni Feb 8, 2023

mufaddal-rohawala closed this Mar 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lineage visualization #3462

Lineage visualization #3462

jkasiraj commented Nov 7, 2022

sagemaker-bot commented Nov 7, 2022

sagemaker-bot commented Nov 7, 2022

sagemaker-bot commented Nov 7, 2022

sagemaker-bot commented Nov 7, 2022

sagemaker-bot commented Nov 16, 2022

sagemaker-bot commented Nov 16, 2022

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

codecov-commenter commented Nov 17, 2022 •

edited

Loading

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

sagemaker-bot commented Nov 17, 2022

navinsoni left a comment

mufaddal-rohawala commented Mar 22, 2023

jkasiraj commented May 4, 2023

Lineage visualization #3462

Lineage visualization #3462

Conversation

jkasiraj commented Nov 7, 2022

Merge Checklist

General

Tests

sagemaker-bot commented Nov 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 7, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 16, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 16, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

codecov-commenter commented Nov 17, 2022 • edited Loading

Codecov Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

sagemaker-bot commented Nov 17, 2022

AWS CodeBuild CI Report

navinsoni left a comment

Choose a reason for hiding this comment

mufaddal-rohawala commented Mar 22, 2023

jkasiraj commented May 4, 2023

codecov-commenter commented Nov 17, 2022 •

edited

Loading