Skip to content

Bug with the sagemaker.get_execution_role() #2186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
szamarin opened this issue Mar 3, 2021 · 6 comments
Closed

Bug with the sagemaker.get_execution_role() #2186

szamarin opened this issue Mar 3, 2021 · 6 comments

Comments

@szamarin
Copy link

szamarin commented Mar 3, 2021

Describe the bug
When using get_execution_role() in SageMaker Studio with a non generated IAM role, the method does not return the correct ARN unless the role name contains "AmazonSageMaker-ExecutionRole" and is placed into the "/service-role/" path. These are hard coded into the get_caller_identity_arn() method of sagemaker session. This becomes an issue when deploying the role with CloudFormation. The logical id can not contain dashes, so when I create a role like this:

SageMakerExecutionRole:
      Type: "AWS::IAM::Role"
      Properties:
        AssumeRolePolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - sagemaker.amazonaws.com
              Action:
                - sts:AssumeRole
        Path: /service-role/

The method will fail to include /service-role/ in the arn. I have to explicitly specify the RoleName, or place the role into the root path.

To reproduce
Create a sagemaker execution role without including "AmazonSageMaker-ExecutionRole" in its name and place that role under the "/service-role/" path. Assign the role to a SageMaker Studio domain or user profile. Run sagemaker.get_execution_role() in a Studio notebook. The returned arn will not include "service-role" in the path.

Expected behavior
get_execution_role() should always return the correct arn

@ChoiByungWook
Copy link
Contributor

Similar issue: #2089

@ChoiByungWook
Copy link
Contributor

ChoiByungWook commented Mar 6, 2021

Hey @szamarin,

Is the name of the role that you are using contain SageMakerExecutionRole?

If the name of your role doesn't contain AmazonSageMaker-ExecutionRole, then the method should end up calling IAM's get_role(), which should return the proper Role Arn with the expected path, assuming your role has IAM read permissions.

If the Role Arn is different than what is expected, then it may be due to the role that is being assumed in your SageMaker Studio is different than what you are expecting, as we call STS's get_caller_identity() to retrieve the corresponding role name.

You may need to add IAM read permissions to your role or pass the Role Arn directly, as I believe there isn't a proper way to get the path without it.

When you attempt to call aws iam get-role --role-name <ROLE_NAME> on the expected role, does it return the proper path in your Role Arn?

@ChoiByungWook
Copy link
Contributor

Potentially related PR: #2191

@athewsey
Copy link
Collaborator

I also ran in to a related issue today, trying to use a role which is under a custom prefix (not /service-role/ but something else) and does not include AmazonSageMaker-ExecutionRole:

  • On Studio kernel Python 3 (Data Science) (sagemaker version 2.59.5), it seems to work fine
  • On Studio kernel Python 3 (TensorFlow 1.15 Python 3.7 CPU Optimized) (sagemaker version 2.33.0), the prefix is missing from the returned role ARN

So I think there's a chance this issue as described might now be fixed in the library? But even if so, seems like the fix is not yet propagated to all SageMaker Studio kernel images.

@ronykroy
Copy link

ronykroy commented Mar 29, 2023

Hey
so i ran into the same issue...
The pipeline, which in turn was a part of a SageMaker project, complained that <long prefix>/service-role/<long suffix>AmazonSageMakerServiceCatalogProductsUserRole didn't have

  1. permissions to pass role
  2. get role

I had it fixed by adding

  1. iam:PassRole
  2. iam:GetRole
    to the above issue causing role

justification for adding the above 2 roles..

  1. was hinted at, in the error message
  2. https://github.com/aws/sagemaker-python-sdk/blob/dev/README.rst#aws-permissions

Hope this addresses your issue too

@martinRenou
Copy link
Collaborator

Closing as fixed, please reopen if you think otherwise. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants