Skip to content

Keras training with Masking layer fails #423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gautiese opened this issue Oct 9, 2018 · 1 comment
Closed

Keras training with Masking layer fails #423

gautiese opened this issue Oct 9, 2018 · 1 comment

Comments

@gautiese
Copy link

gautiese commented Oct 9, 2018

Please fill out the form below.

System Information

  • Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): Tensorflow
  • Framework Version: Latest
  • Python Version: 3
  • CPU or GPU:GPU
  • Python SDK Version:Latest
  • Are you using a custom image:No

Describe the problem

I get the following error :

AlgorithmError: uncaught exception during training: <tensorflow.python.keras.layers.core.Masking object at 0x7f737ca2f710> Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/container_support/training.py", line 36, in start fw.train() File "/usr/local/lib/python2.7/dist-packages/tf_container/train_entry_point.py", line 164, in train train_wrapper.train() File "/usr/local/lib/python2.7/dist-packages/tf_container/trainer.py", line 73, in train tf.estimator.train_and_evaluate(estimator=estimator, train_spec=train_spec, eval_spec=eval_spec) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/training.py", line 451, in train_and_evaluate return executor.run() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/training.py", line 617, in run getattr(self, task_to_run)() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/training.py", line 654, in run_master self._start_distributed_trainin

Code:

def keras_model_fn(hyperparameters):
    model = tf.keras.models.Sequential()
    model.add(Masking(-1))
    model.add(tf.keras.layers.Embedding(input_dim=35, output_dim=5, input_length=5))
    model.add(tf.keras.layers.LSTM(128))
    model.add(tf.keras.layers.Dense(512,activation='relu'))
    model.add(tf.keras.layers.Dropout(0.15))
    model.add(tf.keras.layers.Dense(512,activation='relu'))
    model.add(tf.keras.layers.Dropout(0.30))
    model.add(tf.keras.layers.Dense(512,activation='relu'))
    model.add(tf.keras.layers.Dropout(0.45))
    model.add(tf.keras.layers.Dense(35,activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
@ChoiByungWook
Copy link
Contributor

Can you provide the Python SDK code you used to invoke your training?

apacker pushed a commit to apacker/sagemaker-python-sdk that referenced this issue Nov 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants