The ReLU function is a very basic activation functions. It is defined as $$ ReLU(x) = max(0,x) $$ That is $x$ if $x \geq 0$, else $0$. The plot of the function is showed in Figure 1.
The following Python3 code has been used for the plot:
def relu(x): return max(0,x) xs = range(-10,11) ys = [relu(x) for x in xs] plt.plot(xs,ys) plt.show()It is quite simple to differentiate, we have that $$ ReLU'(x)= \begin{cases} 0 & x \leq 0 \\ 1 & otherwise \end{cases} $$