GSoC application accepted
My application for gsoc was accepted, which means that over the summer I will work on adding multilayer perceptron model and stochastic gradient descent learning algorithms to scikit-learn project.
My plan for this week is to experiment with cython and try to write something that should look like a backpropagation algorithm. The goal is to get familiar with cython and scikit-learn’s coding guidelines. I will have to look at other parts of scikit, which are also written in cython. I have already seen sgd implementation for linear models and I have used some loss functions defined there for my implementation. My goal is to unite these implementations in the future, but for the time being I will have my separate implementation in mlp module.
I think the hardest part will be to implement the core of the sgd algorithm in cython. The linear model’s implementation consists of one huge function, which seems like the easiest solution to avoid compatibility problems between cython and the pure python code.
Everyone who would like to see my progress can watch my github repository, where I have created a new branch gsoc_mlp.
GSoC 2012 — Multilayer Perceptron
Multilayer perceptron with one hidden layer
My GSOC proposal is an implementation of Multilayer Perceptron learning algorithms for scikit-learn library. Multilayer perceptrons are more known as Artificial Neural Networks. Multilayer Perceptrons are composed of interconnecting artificial neurons that are placed in layers.
Each neuron from one layer is fully connected to all neurons from the previous layer. MLP usually contains one input layer (neurons just represent input data), one hidden layer and one output layer (represents output of the network). MLPs are used in machine learning for classification and regression task. They have been used for example for classification of hand-written digits.
Scikit-learn aims to contain every classical learning algorithm and MLP is the one that has been missing until now.
I will implement SGD (stochastic gradient descent) learning algorithms. SGD is a family of heuristic algorithms which are used to approximate the true gradient by a gradient at a single example. The advantage of SGD is that it is awesome for very large datasets and scale very well compared to other often used machine learning techniques (for example Support Vector Machines).
Classification of hand-written digits using MLP classifier (even rows are samples, odd rows are predicted classes)
Learning MLP involve a lot of computing, which isn’t very efficient in Python. That’s why I will use Cython for implementing core parts of learning algorithms. Thanks to Cython the computationaly heavy parts will be translated to C and compiled to run as a native code.
The result of my project will be a new module for creating and learning multilayer perceptrons using stochastic gradient descent algorithms which will be comparable with other machine learning methods. With acompanying documentation this project will be a great source of information for everyone who is interested in machine learning and neural networks.