Happy New Year! I’ve been a bit busy lately with my grad school application and the thesis research experiments, so I haven’t worked on the website update frequently. Sqrt-shift learnable transformation - continue Last time we introduced the sqrt-shift function that is able to adjust itself through backpropagation to simulate... [Read More]
Deep Reinforcement Learning on Doom AI
From DQN to A3C+LSTM
Recently I’ve been working on the project of training an AI for Doom, which is a classic FPS game. Luckily, the control side API has been provided by the VizDoom team, so the rest of the task is with how to train it. While I’m not going to get into... [Read More]
Senior Thesis at CMU (3) - Mocking BN's bimodal behavior
What happens when we replace BN by some other transformations?
Last time we talked about how our surprising observation of the bimodalization of BN on the distribution led to some hypothesis on its effect. In particular, we observed a (higher) peak on the negative side and another (lower) peak on the positive side. What happens if we manually create such... [Read More]
Senior Thesis at CMU (2) - Studying Batchnorm
Mocking the behavior of batch-normalization
In my 10/15 post, I briefly introduced what I was working on in my senior thesis: The Effect ot Pre-ReLU Input Distribution on Deep Neural Network Performance. One interesting aspect that we’ve been looking into is the case of batch-normalization, which works particularly well when combined with ReLU— more specifically,... [Read More]
Senior Thesis at CMU (1) - ReLU, DNN and more
What does normalization & ReLU change?
The title of my senior thesis is: The Effect ot Pre-ReLU Input Distribution on Deep Neural Network Performance: Well, this is a not a short title. But really, the gist of this thesis can’t be more straightforward (though challenging): how can we better the performance of neural networks by playing... [Read More]
First Post After Migrating My Website!
A welcome message
Well, if you are looking at this post right now, I trust that you have already visited my new personal website. Gradually from today, I will migrate things from the old site to the new one. Meanwhile, I will post some interesting things on the current github page. Topics include:... [Read More]