Skip to content

Commit 4f19de5

Browse files
pintaoz-awspintaoz
and
pintaoz
authored
Add warning about not supporting torch.nn.SyncBatchNorm (aws#5046)
* Add warning about not supporting * update wording --------- Co-authored-by: pintaoz <[email protected]>
1 parent c593687 commit 4f19de5

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

doc/frameworks/pytorch/using_pytorch.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,9 @@ To initialize distributed training in your script, call
375375
`torch.distributed.init_process_group
376376
<https://pytorch.org/docs/master/distributed.html#torch.distributed.init_process_group>`_
377377
with the desired backend and the rank of the current host.
378+
Warning: Some torch features, such as (and likely not limited to) ``torch.nn.SyncBatchNorm``
379+
is not supported and its existence in ``init_process_group`` will cause an exception during
380+
distributed training.
378381
379382
.. code:: python
380383

0 commit comments

Comments
 (0)