Senior Thesis at CMU (4)
Some transformations, some KL divergence analysis
Posted on January 4, 2017
Happy New Year! I’ve been a bit busy lately with my grad school application and the thesis research experiments, so I haven’t worked on the website update frequently. Sqrt-shift learnable transformation - continue Last time we introduced the sqrt-shift function that is able to adjust itself through backpropagation to simulate...
[Read More]
Tags:
thesis, machine-learning, cmu
Deep Reinforcement Learning on Doom AI
From DQN to A3C+LSTM
Posted on November 30, 2016
Recently I’ve been working on the project of training an AI for Doom, which is a classic FPS game. Luckily, the control side API has been provided by the VizDoom team, so the rest of the task is with how to train it. While I’m not going to get into...
[Read More]
Tags:
project, machine-learning, cmu, game
Senior Thesis at CMU (3) - Mocking BN's bimodal behavior
What happens when we replace BN by some other transformations?
Posted on November 15, 2016
Last time we talked about how our surprising observation of the bimodalization of BN on the distribution led to some hypothesis on its effect. In particular, we observed a (higher) peak on the negative side and another (lower) peak on the positive side. What happens if we manually create such...
[Read More]
Tags:
thesis, machine-learning, cmu
Senior Thesis at CMU (2) - Studying Batchnorm
Mocking the behavior of batch-normalization
Posted on October 31, 2016
In my 10/15 post, I briefly introduced what I was working on in my senior thesis: The Effect ot Pre-ReLU Input Distribution on Deep Neural Network Performance. One interesting aspect that we’ve been looking into is the case of batch-normalization, which works particularly well when combined with ReLU— more specifically,...
[Read More]
Tags:
thesis, machine-learning, cmu
Senior Thesis at CMU (1) - ReLU, DNN and more
What does normalization & ReLU change?
Posted on October 15, 2016
The title of my senior thesis is: The Effect ot Pre-ReLU Input Distribution on Deep Neural Network Performance: Well, this is a not a short title. But really, the gist of this thesis can’t be more straightforward (though challenging): how can we better the performance of neural networks by playing...
[Read More]
Tags:
thesis, machine-learning, cmu
First Post After Migrating My Website!
A welcome message
Posted on October 14, 2016
Well, if you are looking at this post right now, I trust that you have already visited my new personal website. Gradually from today, I will migrate things from the old site to the new one. Meanwhile, I will post some interesting things on the current github page. Topics include:...
[Read More]
Tags:
exciting-stuff