-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Implemented write_spmatrix_to_sparse_tensor #28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Egor,
This is great, thanks for your contribution!
src/sagemaker/amazon/common.py
Outdated
@@ -89,6 +108,46 @@ def write_numpy_to_dense_tensor(file, array, labels=None): | |||
_write_recordio(file, record.SerializeToString()) | |||
|
|||
|
|||
def write_numpy_to_sparse_tensor(file, array, labels=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be write_spmatrix_to_sparse_tensor ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, fixed.
|
||
record = Record() | ||
for row_idx in range(n_rows): | ||
record.Clear() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason this isn't done at the end of the for-loop?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Full analogy to the write_numpy_to_dense_tensor
function.
https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/amazon/common.py#L84
src/sagemaker/amazon/common.py
Outdated
@@ -89,6 +108,46 @@ def write_numpy_to_dense_tensor(file, array, labels=None): | |||
_write_recordio(file, record.SerializeToString()) | |||
|
|||
|
|||
def write_numpy_to_sparse_tensor(file, array, labels=None): | |||
"""Writes a numpy array to a dense tensor""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
spmatrix not numpy array
src/sagemaker/amazon/common.py
Outdated
@@ -89,6 +108,46 @@ def write_numpy_to_dense_tensor(file, array, labels=None): | |||
_write_recordio(file, record.SerializeToString()) | |||
|
|||
|
|||
def write_numpy_to_sparse_tensor(file, array, labels=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Thanks again for your submission |
@meownoid Merged in, thanks! |
Since publicly this is being known as "hyperparameter tuning" (or some variation of that), it doesn't make sense that we'd use "hpo" everywhere
BYOM Notebooks - Verbosity and other notes.
Implemented
write_numpy_to_sparse_tensor
which supports anyscipy.sparse
matrix (#27).