Having explored the racecar project for far too long, I wanted to work on something different that could offer newer challenges. I've always wanted to build a game-playing RL agent for complex games like modern boardgames but those still seem too complicated for me right now. I needed something of a stepping stone before I could get there. The classic Connect-4 game has been a…
I've been learning Reinforcement Learning by applying it to a fun problem: Optimizing a race-car's path along a race track. After my previous experiments with a full-state table lookup, in which I was able to get satisfactory results with the Q algorithm, I tried to see if I could achieve better results with Q(λ) or Sarsa(λ) which I suspected would solve some of the problems…
For my first project with Reinforcement Learning (RL), I wanted to pick a problem that could use straightforward approaches yet was not trivial to design. I wanted to use a full-state (or table-lookup) approach instead of a function-approximation (such as neural network) approach. I also wanted to have a deterministic and fully-observable environment (i.e. I wanted actions to have predictable results, and I wanted just…
For my third dataset I picked what looked like a more complex dataset with a classification task. The Internet Ads Dataset has 1558 features and a fair amount of missing data, and the goal was to classify a given online image as either an advertisement or a non-advertisement. It turned out that the vast majority of the 1558 features were simply binary flags for words/phrases that…
I had studied Support Vector Machines lightly in Prof. Ng's ML coursera course, and then with further mathematical rigor in Prof. Paisley's ML ColumbiaX course, but neither involved actually implementing the algorithm. It turns out that these courses never actually taught how to implement the SVM (despite giving off the impression that they had) and so what I assumed would be a straightforward python implementation…
For my second dataset in this series, I picked another classification dataset, the Abalone dataset. However, there are some interesting peculiarities to this dataset compared to other simpler classification datasets: This dataset should ideally be treated as a regression task, since it attempts to predict the age of the Abalone. However, the original investigators attempted a classification task on this dataset, so that is what…
I recently began downloading and working with datasets available online in order to level up on my Machine Learning knowledge. The idea was to: (a) get my hands dirty exploring real world datasets, (b) solidify my theoretical knowledge of ML by implementing the techniques and algorithms, and (c) practice coding in Python and Numpy. This post is about the first dataset I worked with for this purpose.…
Posted on Jan 17, 2018 If you're planning on signing up for the Machine Learning course by Columbia University, or if you've already begun but now find yourself drowning in the deep end, this post is for you. In September 2017, I signed up for CSMM.102x, a Machine Learning course offered by Columbia University on edX and taught by Prof. Paisley. It was the second online…