A Practical Guide to Understanding Stochastic Gradient Descent Methods: Workhorse of Machine Learning

IPython Notebook: Here Introduction: Why Optimization? It is no need to stress that optimization is at the core of machine learning algorithms. In fact this was a big enabler of deep learning; where “pre-training” (i.e. an optimization process) the network was used to find a good initialization for deep models....

A Paper in Thousand Words: Neural Architecture Search with Reinforcement Learning

Paper is found here. One of the key advantages of Deep Models is that they made feature engineering obsolete. With this came a paradim-shift; from engineering robust features to engineering deep architectures, i.e. hyperparameters, for machine learning tasks. This paper uses reinforcement learning (RL) to find the best deep architecture...

RA-DAE: Structurally Adaptive Deep Architecture inspired by Reinforcement Learning

In this post, I’m going to introduce a type of a Stacked Autoencoders (SAE) (Don’t worry if you don’t understand what an SAE is. Will explain later.). And worth a mention, that this is some research work done by me and few colleague from our research lab. So yay for...

Long Short Term Memory (LSTM) Networks: Implementing with Tensorflow (Part 2)

Before proceeding further this assumes an intermediate knowledge about how things work in LSTM networks. If you don’t, please look at Long Short Term Memory (LSTM) Networks: Demystified (Part 1) I’m using the following versions Python: 3.4 Tensorflow: 0.10.0 Let’s get right into it. I’m using code snippets from 6_lstm.ipynb...

Long Short Term Memory (LSTM) Networks: Demystified (Part 1)

Overview Introduction Long Short Term Memory (LSTM) Networks are a popular choice for sequential modelling tasks, due to its ability to store both long and short term. LSTMs are inspired from (Recurrent Neural Networks) RNNs. RNNs function in a similar manner, but does not possess effective ways of storing long-term...

[Part 2] Implementing Deep Learning for WSO2-ML

This section is about the changes made to WSO2-ML during my project. First I will be discussing about the modules and classes that play a major role in the system. This knowledge will be important for understanding changes I’ve done. Key Concepts of WSO2-ML There are 5 concepts you should...