dm.cs.tu-dortmund.de/mlbits/class-nnet-activation-functions/
Activation Functions – Lecture Notes
’(z)} = \smash {\begin{cases} 1 & \text{if }-\! 1\leq z\leq +1 \\ 0 & \text{otherwise}\end{cases}}\]
\[\textcolor[RGB]{246,180,63}{\varphi ’(z)} = \smash {1-\tanh ^2 z} = \smash {1-\varphi (z)^2}\]
Some Activation [...] GB]{132,184,24}{\varphi (z)}:=\smash {\frac{1}{1+e^{-z}}}=\smash {\frac{e^z}{1+e^z}}\]
\[\textcolor[RGB]{246,180,63}{\varphi ’(z)} = \smash {\begin{cases} 1 & \text{if }-\! \tfrac 12\leq z\leq +\tfrac [...] ]{132,184,24}{\varphi (z)}:=\smash {\begin{cases} \phantom{-}1 & \text{if } z \geq +1 \\ -1 & \text{if } z \leq -1 \\ \phantom{-}z & \text{otherwise} \end{cases}}\]
Tangens hyperbolicus:
\[\textcolor[RGB]{132 …