Neural Networks
Subset of Machine Learning which finds inspiration from the human brain.
Theory
3-Layer NN the scalar looks like this: where and are in the form of
- can be a sigmoid, tanh, relu function
- Hidden Nodes (hidden layer): perform computations and transfer input nodes to output nodes
- Output Nodes (output layer): transferring information from the network to the outside world.
- Connections and weights: each connection transferring the output of a neuron to the input of a neuron. Each connection is assigned a weight.
- Activation function: non-linear function that defines the output of that node given an input or set of inputs
Types of Neural Networks
- Convolutional Neural Network for image processing
- LSTMs for text analysis
- Autoencoders work on compressed data. The encoder can then recreate i.e an image from the compressed “code” that describes the image
- cannot create new images by default, but if you add a normal distribution you can generate similar images
- Generative Adversarial Network is the Generator and the Discriminator playing against each other that check if produced data is similar to real data, over time they get better