File tree 1 file changed +4
-2
lines changed
doc/api/training/smp_versions/latest
1 file changed +4
-2
lines changed Original file line number Diff line number Diff line change @@ -301,8 +301,7 @@ Tensor Parallelism Module APIs
301
301
``post_layernorm `` must be ``True ``.
302
302
- ``post_layernorm ``: If ``True ``, inserts layer normalization at
303
303
the output. At least one of ``pre_layernorm `` and
304
- ``post_layernorm `` must be ``True ``. (Available from
305
- the SageMaker model parallelism library v1.15.0.)
304
+ ``post_layernorm `` must be ``True ``.
306
305
- ``use_alibi `` (bool, default False): Activates Attention with
307
306
Linear Biases (ALiBi) for attention computation.
308
307
ALiBi facilitates efficient extrapolation on input sequences
@@ -314,6 +313,9 @@ Tensor Parallelism Module APIs
314
313
details on the technique.
315
314
(Available from
316
315
the SageMaker model parallelism library v1.15.0.)
316
+ - ``alibi_bias_max `` (int, default 8): Defines the ALiBi base
317
+ value for mask generation. (Available from
318
+ the SageMaker model parallelism library v1.15.0.)
317
319
318
320
- **Methods: **
319
321
You can’t perform that action at this time.
0 commit comments