The text introduces the concept of non-linearities in PyTorch for neural networks. It discusses how activation functions can help in solving complex problems and introduces the use of the Heart Failure prediction dataset in PyTorch. It also covers the implementation of neural network architectures and the impact of activation functions on model performance and training. Overall, it emphasizes the importance of activation functions for neural network models.
Is there anything else you would like to know about this text?
PyTorch Introduction — Enter NonLinear Functions
Neural Networks are powerful architectures that can solve complex problems. In this post, we will learn about how non-linearities help solve complex problems in the context of neural networks using PyTorch.
Setting up our Data
In this blog post, we’ll use the Heart Failure prediction dataset available at Kaggle.
Training a Vanilla Linear Neural Network
With our data in place, it’s time to train our first Neural Network. We’ll use a similar architecture to what we’ve done in the last blog post of the series, using a Linear version of our Neural Network with the ability to handle linear patterns.
Enter NonLinearities!
If making our model wider and larger didn’t bring much improvement, there must be something else that we can do with Neural Networks that will be able to improve its performance, right?
That’s where activation functions can be used! In our example, we’ll return to our simpler model, but this time with a twist:
model_non_linear = nn.Sequential(
nn.Linear(in_features=12, out_features=5),
nn.ReLU(),
nn.Linear(in_features=5, out_features=1)
)
What’s the difference between this model and the first one? The difference is that we added a new block to our neural network — nn.ReLU. The rectified linear unit is an activation function that will change the calculation in each of the weights of the Neural Network.
With this small twist in the Neural Network, every value coming from the first layer will have to go through the “ReLU” test.
Now that you know the power of non-linear activation functions, it’s also relevant to know:
- You can add activation functions to every layer of the Neural Network.
- Different activation functions have different effects on your performance and training process.
- torch elegantly gives you the ability to add activation functions in-between layers by leveraging the nn module.
Conclusion
In this blog post, we’ve checked how to incorporate activation functions inside torch Neural Network paradigm. Another important concept that we’ve understood is that larger and wider networks are not a synonym of better performance. Activation functions help us deal with problems that are solved with more complex architectures. They help with generalization power and help us converge our solution faster, being one of the major features of neural network models.
If you want to evolve your company with AI, stay competitive, use for your advantage PyTorch Introduction — Enter NonLinear Functions.
