Skip to content

Commit 8d9e1b9

Browse files
kausthub-kannantianyizheng02
authored andcommitted
Changed Mish Activation Function to use Softplus (TheAlgorithms#10111)
1 parent 0de09ad commit 8d9e1b9

File tree

1 file changed

+2
-1
lines changed
  • neural_network/activation_functions

1 file changed

+2
-1
lines changed

Diff for: neural_network/activation_functions/mish.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
"""
88

99
import numpy as np
10+
from softplus import softplus
1011

1112

1213
def mish(vector: np.ndarray) -> np.ndarray:
@@ -30,7 +31,7 @@ def mish(vector: np.ndarray) -> np.ndarray:
3031
array([-0.00092952, -0.15113318, 0.33152014, -0.04745745])
3132
3233
"""
33-
return vector * np.tanh(np.log(1 + np.exp(vector)))
34+
return vector * np.tanh(softplus(vector))
3435

3536

3637
if __name__ == "__main__":

0 commit comments

Comments
 (0)