Event Review #3
MIST101 Workshop 3: Convolutional Neural Networks(CNN) was held this Thursday, October 12th. This workshop covered the building blocks, architectures, training methods , variants and applications of CNN.
A simple CNN consists stacks of specialized layers, and every layer acts like a function that transforms one set of activations to another. Three main types of layers are used and they were introduced in detail during the workshop.
Fully connected Layers
As a recap from the last workshop, neurons in a fully connected layer have full connections to all possible activations in the previous layer, without any cycle. However, neural nets with only fully connected layers are computationally expensive to train and are limited in application.
Neurons in convolution layers read the pixel values of a receptive field, and the selected kernels act as filters that process the input data. The convolution between pixel values and kernels makes up the new output of the layer. Stride and padding are two useful quantities to tune the convolution operation.
Pooling progressively reduces the spatial size of the image representations to reduce the amount of parameters and computation in the network.
Techniques such as gradient descent, weight initialization, Dropout method and Batch Normalization were introduced in detail to effectively leverage data learning. Dropout disables random nodes in the training process, and Batch Normalization is able to scale a batch to the center of focus. They both have the advantage of regularization and enable model ensembling. The learnt features from a CNN can be reused, which makes Transfer Learning possible across different computer vision domains.
Variance and Applications
To extend the topic and allow everyone to get a clear sense of real-world applications of CNN, a few state-of-the-art architectures such as GoogleNet, ResNet, were presented in the workshop.
CNN is typically used in areas like object detection and segmentation, image processing and style transfer. It can also be combined with Recurrent Neural Network(RNN), which will be introduced in detail in the next workshop, to handle more complicated tasks such as image captioning and neural translation.
The tutorial shows how to use TensorFlow to train and evaluate a simple neural network for handwritten digit classification using the MNIST data set. Material used in this practical session can be found at: https://github.com/ColinQiyangLi/MIST101/blob/master/Tensorflow_Intro_2.ipynb
Thank you all for coming to our MIST101 workshop #3, we hope you have become familiar with Convolutional Neural Network by now! MIST101 #4, on 7:00-9:00pm October 26th, in GB119, will be talking about Recurrent Neural Network(RNN). RNN is widely applied in language modeling, translation, text classification and robotics.
If you would like to learn more about workshop #3, the slides can be found under this link: https://docs.google.com/presentation/d/13RBeyETvgSJ_2V8zDz1E2fVCNydGyllyMe6WSG4o5bU/edit?usp=sharing
Meanwhile, please take the time to fill out our feedback survey for the event if you haven’t done so!