Neural Networks

Subset of Machine Learning which finds inspiration from the human brain.

Theory

3-Layer NN the scalar looks like this: $y=f_3(f_2(f_1(x)))$ where $f_1$ and $f_2$ are in the form of $f_1(z)=g_i(W_{iZ}+b_i)$

• $g1$ can be a sigmoid, tanh, relu function

• Hidden Nodes (hidden layer): perform computations and transfer input nodes to output nodes
• Output Nodes (output layer): transferring information from the network to the outside world.
• Connections and weights: each connection transferring the output of a neuron to the input of a neuron. Each connection is assigned a weight.
• Activation function: non-linear function that defines the output of that node given an input or set of inputs

Types of Neural Networks

• Convolutional Neural Network for image processing
• LSTMs for text analysis
• Autoencoders work on compressed data. The encoder can then recreate i.e an image from the compressed “code” that describes the image
  • cannot create new images by default, but if you add a normal distribution you can generate similar images
• Generative Adversarial Network is the Generator and the Discriminator playing against each other that check if produced data is similar to real data, over time they get better