# Activation Functions For Deep Learning Using Tensorflow

#### This article implements common activation functions for deep learning using Tensorflow.

Tensorflow is a free and open-source library for machine learning and related taks.

Tensorflow is a symbolic math library, and is also used for machine learning applications such as neural networks & deep learning.

In this article we are going to implement sigmoid, tangent hyperbolic and RELU function using Tensorflow.

## Rectified Linear Unit (RELU) Function

RELU has become the default activation function for different neural networks application because of its simplicity and power.

Mathematical function for RELU is: $$f(x) = max(0,x)$$

Derivative for RELU is: $$f(x) = \begin{cases} \text{1, x>0} \\ \text{0, otherwise} \end{cases}$$

### Souce Code: RELU


import tensorflow as tf
import numpy as np
from matplotlib import pyplot as plt

# Generating data to plot
x_data = np.linspace(-6,6,100)
y_data = tf.nn.relu(x_data)

# Plotting
plt.plot(x_data, y_data)
plt.title('RELU')
plt.grid()
plt.show()


## Sigmoid Function

Mathematical function for sigmoid is: $$f(x) = \sigma(x) = \frac{1}{1+e^{-x}}$$

Derivative of sigmoid function is: $$f'(x) = \sigma(x) ( 1 - \sigma(x) )$$

### Source Code: Sigmoidal Function


import tensorflow as tf
import numpy as np
from matplotlib import pyplot as plt

# Generating data to plot
x_data = np.linspace(-6,6,100)
y_data = tf.math.sigmoid(x_data)

# Plotting
plt.plot(x_data, y_data)
plt.title('Sigmoid Function')
plt.grid()
plt.show()


## Tangent Hyperbolic Function

Another useful activation function is tangent hyperbolic.

Formula for Tangent Hyperbolic: $$f(x) = \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$$

Derivative for Tangent Hyperbolic is: $$f'(x) = ( 1 - g(x)^{2} )$$

### Source Code: Tangent Hyperbolic


import tensorflow as tf
import numpy as np
from matplotlib import pyplot as plt

# Generating data to plot
x_data = np.linspace(-6,6,100)
y_data = tf.math.tanh(x_data)

# Plotting
plt.plot(x_data, y_data)
plt.title('RELU')
plt.grid()
plt.show()