Lib conflict with tensorflow-serving-api

[PedroCardoso]

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans):
  Tensorflow
• Framework Version:
  1.6
• Python Version:
  2.7
• CPU or GPU:
  CPU
• Python SDK Version:
  1.2.4
• Are you using a custom image:
  no

Describe the problem

If the tensorflow-serving-api package is installed (with pip) we get errors. To solve this one needs to remove the package, and that is not practical.

Could this be a pip external dependency ?

Minimal repro / logs

----> 2 from sagemaker.tensorflow import TensorFlow
      3 
      4 dnnModel = TensorFlow( entry_point=dnnFile,
      5                                role=role,

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/sagemaker/tensorflow/__init__.py in <module>()
     26     raise AssertionError(message)
     27 
---> 28 from sagemaker.tensorflow.estimator import TensorFlow  # noqa: E402
     29 from sagemaker.tensorflow.model import TensorFlowModel, TensorFlowPredictor  # noqa: E402
     30 

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/sagemaker/tensorflow/estimator.py in <module>()
     23 
     24 from sagemaker.tensorflow.defaults import TF_VERSION
---> 25 from sagemaker.tensorflow.model import TensorFlowModel
     26 
     27 logging.basicConfig()

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/sagemaker/tensorflow/model.py in <module>()
     16 from sagemaker.predictor import RealTimePredictor
     17 from sagemaker.tensorflow.defaults import TF_VERSION
---> 18 from sagemaker.tensorflow.predictor import tf_json_serializer, tf_json_deserializer
     19 from sagemaker.utils import name_from_image
     20 

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/sagemaker/tensorflow/predictor.py in <module>()
     20 from sagemaker.content_types import CONTENT_TYPE_JSON, CONTENT_TYPE_OCTET_STREAM, CONTENT_TYPE_CSV
     21 from sagemaker.predictor import json_serializer, csv_serializer
---> 22 from tensorflow_serving.apis import predict_pb2, classification_pb2, inference_pb2, regression_pb2
     23 
     24 REGRESSION_REQUEST = 'RegressionRequest'

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/sagemaker/tensorflow/tensorflow_serving/apis/inference_pb2.py in <module>()
     14 
     15 
---> 16 from tensorflow_serving.apis import classification_pb2 as tensorflow__serving_dot_apis_dot_classification__pb2
     17 from tensorflow_serving.apis import input_pb2 as tensorflow__serving_dot_apis_dot_input__pb2
     18 from tensorflow_serving.apis import model_pb2 as tensorflow__serving_dot_apis_dot_model__pb2

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/tensorflow_serving/apis/classification_pb2.py in <module>()
     24   serialized_pb=_b('\n,tensorflow_serving/apis/classification.proto\x12\x12tensorflow.serving\x1a#tensorflow_serving/apis/input.proto\x1a#tensorflow_serving/apis/model.proto\"%\n\x05\x43lass\x12\r\n\x05label\x18\x01 \x01(\t\x12\r\n\x05score\x18\x02 \x01(\x02\"=\n\x0f\x43lassifications\x12*\n\x07\x63lasses\x18\x01 \x03(\x0b\x32\x19.tensorflow.serving.Class\"T\n\x14\x43lassificationResult\x12<\n\x0f\x63lassifications\x18\x01 \x03(\x0b\x32#.tensorflow.serving.Classifications\"t\n\x15\x43lassificationRequest\x12\x31\n\nmodel_spec\x18\x01 \x01(\x0b\x32\x1d.tensorflow.serving.ModelSpec\x12(\n\x05input\x18\x02 \x01(\x0b\x32\x19.tensorflow.serving.Input\"\x85\x01\n\x16\x43lassificationResponse\x12\x31\n\nmodel_spec\x18\x02 \x01(\x0b\x32\x1d.tensorflow.serving.ModelSpec\x12\x38\n\x06result\x18\x01 \x01(\x0b\x32(.tensorflow.serving.ClassificationResultB\x03\xf8\x01\x01\x62\x06proto3')
     25   ,
---> 26   dependencies=[tensorflow__serving_dot_apis_dot_input__pb2.DESCRIPTOR,tensorflow__serving_dot_apis_dot_model__pb2.DESCRIPTOR,])
     27 
     28 

.../miniconda3/envs/sagemaker_tf_27/lib/python2.7/site-packages/google/protobuf/descriptor.pyc in __new__(cls, name, package, options, serialized_pb, dependencies, public_dependencies, syntax, pool)
    827         # TODO(amauryfa): use the pool passed as argument. This will work only
    828         # for C++-implemented DescriptorPools.
--> 829         return _message.default_pool.AddSerializedFile(serialized_pb)
    830       else:
    831         return super(FileDescriptor, cls).__new__(cls)

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "tensorflow_serving/apis/classification.proto":
  tensorflow_serving/apis/classification.proto: A file with this name is already in the pool.

• Exact command to reproduce:
  Call TensorFlow Estimator

[PedroCardoso]

This could be related to #139

[mvsusp]

This is duplicated from #139

[mvsusp]

Hi @PedroCardoso ,

I investigated the issue today and got to the following conclusions:

1 - tensorflow-serving-api is only available in python 2 and cannot be installed using python 3. TensorFlow serving is not planning to create a python 3 version (tensorflow/serving#700) although the content of these versions would be the same. That is the main reason for us to maintain a copy of the protobuf messages inside SageMaker Python SDK.

2 - The issue occurs because Python 2 will prioritize loading the system installed module instead of the relative module version. That issue is solved adding future absolute import in any file that loads the reference. I enforced absolute imports in SageMaker to avoid it happening again: #180

3 - I have an additional PR updating the TF serving protobuf messages #181

I will merge these important changes as soon as possible to unblock you.

I appreciate you alerting us the issue and thanks for the patience.

Best,

Marcio

[mvsusp]

Hi @PedroCardoso,

The fix is merged in master. I will release the new version in pypi Monday.

Thanks for the patience.

[mvsusp]

Hi @PedroCardoso,

The release https://github.com/aws/sagemaker-python-sdk/releases/tag/v1.2.5 includes the fixes for the issue.

Thanks again for reporting this issue.