Over the last 10 weeks I had been following the very popular and successful Stanford Machine Learning course run by Andrew Ng these days. The course covered quite a bit of different material, primarily focusing on the core basics of ML. Namely this was Linear and Logisitc Regression, Neural Networks, PCA (for dimensionality reduction) and k-Means clustering. Recommender systems were also covered but in less details as were anomaly detection (which I really liked). Woven throughout the course was lots of other useful information such as how to handle training data and split it up. Also, very useful information on how to access the performance of your system, and how to look for signs that may suggest where problems lay was given.
The course required that I watch around 2 hours of short (5 – 15 mins) videos which were basically mini “bite-size” presentations which were incredibly well produced and presented in both the actual material covered and the pace at which this was done. Following these videos you had to complete a questionnaire (or two per week) with 5 questions designed to test your general knowledge of the topic. To get a better idea of your understanding, and to provide practical experience you also had to complete programming exercises. Using either MatLab or Octave (which is basic MatLab without the price-tag) you had to complete essential parts of pre-given code that made a particular ML technique work. All in all, each week it was about 8-10 hours work, and to be honest, I probably spend about that much time going on pointless YouTube journeys anyway, so I considered this to be a better use of my time… 🙂
If I had to select three main things that I took from this course, even although it is rather beginner in some respects it would be these:
- Seeing how you can approach a ML problem and implement a system using only a matrix orientated approach (if I had not seen these videos, I would have programmed the system in a VERY different and less efficient way).
- Andrew shows us that setting up a system that works well and being able to assess it’s performance is by no means a dark-art as I had been lead to believe. Using a very clever and useful set of principles, with the cost-function at the core, you can gain wonderful insights into the performance of a system as well as clues as to where your efforts for improvement may be best placed! Really useful stuff there that I haven’t seen in any of the main ML text-books!
- As a more general approach to ML, I see that you need to think of this as a rapid-prototyping task. Don’t spend too much time designing and thinking about this complex system that you’re going to build. Do the opposite! Get a cheap and dirty system up and running (hence the use of MatLab/Octave, they’re high level languages that are good for exactly this) and quickly assess whether this is an avenue worth pursuing. It will be a far better use of your time. Also, attached to this is the need to frequently ask some very critical questions. A common example given is “How much effort or time would it cost to get 10 times the amount of data that we have?”. Generally in ML, you can’t have too much data, but it is certainly possible to have too little…
Anyway, just because I’m a little proud of my minor achievement, here’s a nice PDF of my Certificate of Accomplishment (and I should add I got 100%). 🙂
The course runs again from 16th June 2014, so if you’re interested, get yourself booked in. It only costs time, but it will be time well spent! 😉