December 1, 2018 – Edit this Post on GitHub
How did I get to machine learning?A couple of months ago, I started to listen to the Machine Learning Guide podcast . I found out about it by chance and highly recommend it to get you an introduction for machine learning. Even though I didn’t actively plan about learning ML, it was interesting to hear about all those foreign concepts. There it was again; this excitment when everything is unexplored. I felt like a whole new world in tech opened up for me. It was the same feeling when I finally got the foot into web development .
As I read about a couple of machine learning articles, the course on Machine Learning by Andrew Ng was the by far most recommended to get started in machine learning. I have never taken an online course from start to end before, even though I actively give these online courses myself, but I decided to give it a shot this time. Fortunately, the course had started one week ago. So I enrolled in it a couple of weeks ago and by now finished it. It’s a blast and I recommend everyone who wants to get into ML to take it. But more about it later.
- Math : Matlab, Octave, Julia, R
- Data Analysis : R
- Data Mining : Scala, Java (e.g. Hadoop, Spark, Deeplearning4j)
- Server : [*]
- Performance : C/C++ (e.g. GPU accelarated)
Next, you can see why Python makes so much sense in machine learning. It has a suitable set of libraries for the different tasks assigned to the programming languages from above:
- Math : numpy
- Data Analysis : Pandas
- Data Mining : PySpark
- Server : Flask, Django
- Performance :
- TensorFlow (because it is written with a Python API over a C/C++ engine)
- Keras (sits on top of TensorFlow)
- Math : math.js
- Data Analysis : d3.js
- Data Mining : ?
- Server : node.js (express, koa, hapi)
- Performance :
- deeplearn.js (e.g. GPU accelerated via WebGL API in the browser)
deeplearn.js : The library released by Google is GPU accelerated via WebGL API and used for predicitions by using trained models in inference mode in the browser but also the training mode itself. It mirrors the API of the popular TensorFlow library which is developed by Google as well.
TensorFire and Keras.js : Yet another pair of two GPU accelerated libraries which are used for pre-trained models in inference mode. They allow you to write your models in Keras / TensorFlow with Python and to deploy them to the web.
Machine Learning as an Opportunity for Web Developer
I made my own motivation clear in the beginning of this article. However, that’s not all to the story. THere are plenty of reasons and opportunities to dive into machien learning as web developer.
Second, there are plenty of job opportunities out there in the domain of machine learning. Sure, it is a overly hyped topic in the moment, but not without any reason. Students and researcher in the field are hired straight away from university. There seems to be a huge demand in the general fields of AI, data analysis and machine learning. Bootcamps are popping up or shift their focus to data science . See it as opportunity to take one step beyond web development and maybe to a wider range of job opportunities. Perhaps the market in web development paired with machine learning grows in the next years. But even if it doesn’t, you can learn a programming language suited for machine learnign to apply all your theoretical learnings in it. After all, maybe there comes the time when web developers have to make an important decision to get into a different domain than web development. So why shouldn’t it be machine learning?
Last but not least, there is great effort involved on the side of the machine learners (e.g. deeplearn.js, TensorFire, Keras.js) to enable machine learning in the browser. However, most often the documentation is suited for machine learners entering the web developer domain. It comes with a lot of fundametnal machine learning knowledge which isn’t taught along the way. In return, it makes it difficult for web developers to enter the machine learning domain. Thus there is a great opportunity to pave the way for web developers into the domain of machine learning by making those fundamental topics and ported libraries accessible in an educational way. That’s the point where I try to tie in my knowledge in teaching. I want to give you the guidance if you are keen to enter the field of machine learning as web developer.
Introduction to Machine Learning
If you are familiar with the machine learning topic, feel free to skip this section! Entering the field of machine learning as a newcomer can be a buzzword heavy experience. Where should you start? There is so much terminolgy to clarify in the beginning. Is it AI or machine learning ? What’s all the recent buzz about deep learning ? And from not far apart, data analysis is participating in the topic.
Let’s start our journey with AI (artificial intelligence) . “It is the intelligence of a machine that could successfully perform any intellectual task that a human being can.” There is a great analogy in the Machine Learning Guide podcast to convey the entropy of AI: Whereas the goal of the industrial revolution was it to simulate the physical body through robotics, it is the goal of AI to simulate the brain for mental tasks through algorithms. So how does machine learning relate to AI? Let’s have a look at the subfields of AI:
- searching and planning (e.g. playing a game with possible actions)
- reasoning and knowledge representation (structuring knowledge to come to conclusions)
- perception (vision, touch, hearing)
- ability to move and manipulate objects (goes into robotics)
- natural language processing (NLP)
The last one represents machine learning. As you can see, it is only a subfield of AI. However, it might be the only essential core fragment of AI because it reaches into the other subfields of AI. It reaches into them even more over time. For instance, vision subfield becomes more of an field for applied machine learnign where other techniques, e.g. domain specific algotithms, dominated the domain in the the past. Now often deep neural networks are used for the domain. So what are applicable domains of AI and therefore most often machine learning? A bunch of domains and examples:
- Image Recognition (see referenced linke above)
- Web (e.g. Search Engines, Spam Filters)
- Art (e.g. Painting )
- Autonomous Vehicles (e.g. Tesla Autopilot , awareness comes up for Robots in Warfare )
- Medical Diagnosis
- Playing Games (e.g. Go , StarCraft )
So machine learning is a subfield of AI. Let’s dive into the subject itself. There are a couple of great defintions for machine learning, yet when I started out with the subject, I found the one by Arthur Samuel (1959) most memorable: “The field of study that gives computers the ability to learn without being explicitly programmed.” How does it work in layman’s terms? Basically machine learning can be grouped into three categories: supervised learning, unsupervised learning and reinforcemant learning . It’s quite an evolution from the former to the latter. Whereas the former is more concrete, the latter becomes more abstract (yet exciting and unexplored). The former, supervised learning, gives the best entry point to machine learning and is used therefore in several educational machine learning courses to get into the field. In supervised learning, an algorithm is trained to recognize a pattern in a given data set. The data set is split up into input and outpout data points. The algorithm is trained to map input to output by learning with the given data set ( training phase ). Afterward, when the algorithm is trained eventually, it can be used to make predictions for future input data points to come up with output data points ( inference phase ). During the training phase, a cost function estimates the performance of the current algorithm and adjusts its parameters based on those reflections (penalization). The algorithm itself can be simplified into a simple function to map an input x to an ouput y and is also called hypothesis .
Prediciting housing prices in Portland is one popular machine learning problem for supervised learning. Given a data set of houses whereas each house has a size in square meter (x), the price (y) of the house should be predicted. Thus the data set consists a list of sizes and prices for houses. It is called a training set. Each row in the training set represents a house. The input x, in this case the size of the house, is called a feature of the house. Since there is only one feature for the houses in the trainign set, it is called a univariate training set. If there are more features for a house, such as number of bedrooms and size, it becomes a multivariate training set. Increasing the size of the trainign size (m) and the size of features (n) can lead to an improved prediction of y whereas y is called a label . In a nutshell, a hypothesis is trained with a penalizing cost function to predict labels from features.
Tom Mitchell (1998): “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
The previous use case of prediciting housing prices in Portland is called a regression problem . A linear regression , as explained before, can be used to train the hyptohesis to output continuous values (e.g. housing prices). Another problem in the area of supervised learning to be solved is called classification problem where a logistic regression is used to output categorical values . For instance, imagine you have a trainign set with information about T-shirts. The features, such as width and height, can be used to make predicticions for the categorical sizes X, M and L.
The previous paragraphs were a first glimpse on supervised learning in machine learning. How does unsupervised learning work? Basically there is a given training set with features but no labels y. The algorithm is trained without any given output data in the training set. And last but not least, what about reinforcemtn learning? In reifnrocement learning the algorithm is trained without any given data. It learns from experience by reapting a process. For instance, take this flappy bird which learns to win the game by using neural networks in reifnrcement learning.
Last but not least, there might be another question popping up in your head: What’s the relationship of data science to machine learning? Data science is often associated with machine learning. So one could argue that machine learning bleeds into both domains: data science and artifical intelligence. However, data science has its own subfields such as data mining and data analysis. It can often be used coupled to machine learning, because data mining enables an algorithm to learn from mined data and data analysis enables researchers to study the outcome of the trained algorithm.
How to learn Machine Learning as Web Developer
There are a bunch of resources that I want to recommend for web developers entering the field of machine learning. As for myself, I wanted to stimulate my senses for at least 12 weeks. That’s how long it is said to complete Andrew Ng’s machine learning course. Keep in mind that it’s my personal roadmap and it might not be suited for everyone. But it helped me a lot following a strict routine and having enough learning material along the way. So it might help other web developers too.
If you just want to get a feeling for the topic, start to listen to the Machine Learning Guide up to episode 11. Tyler Renelle has done an amazing job giving an introduction to the topic. Since it is a podcast, just give it a shot while you exercise in a gym. That’s in general a great way to approach the topic to keep your mind sharp in between the learning sessions.
If you start to get excited, the next step would be to enroll in the machine learning course which goes for 12 weeks. It takes you on a long journey from shallow machine learning algorithms to neural networks, from regression problems to clustering problems and from theoretical knowledge in the field to applied implementations in Octave or Matlab. It is intense and challenging, but you can do it by dedicating a couple of hours each week to the course and the exercises.
The machine learning course goes from linear regression to neural networks in 5 weeks. In the end of week 5, I was left with an overwhelming feeling. It was a combination of “Can week 6 become even more complex?” and “Wow, this course taught me all the building blocks to implement a neural network from scratch”. Andrew gives a perfect walkthrough to learn about all these concepts which build up on one another. After all, machine learning has a lot in common with the composition of functions from functional programming. But you will learn about this yourself.
After you have completed week 5 of the machine learning course, you should have a good feeling about what’s machine learning and how to solve problems with it. Afterward, the course continues with shallow algorithms for supervised learning and unsupervised learning. It gives elabroeated guidance of how to improve your implemented machine learning algoriothms and how to scale them for large data sets. When you have completed week 5, you should continue as well wioth the Machine Learning Guide podcast to learn more abotu shallow algorithms and neural networks.
In addition, over the course of those weeks, I read The Master Algorithm by Brilliance Audio to get an overview about the topic, its different perspectives and stakeholders, and its history. After that, I started to read the open source ebook Deep Learning (by Ian Goodfellow and Yoshua Bengio and Aaron Courville). It happened after week 5 of the course and fitted perfectly to all the foundational knowledge I learned so far. So highly recommended books to give you even more guidance along the way. Once I finish the second book, I want to read the free ebooks Neural Networks and Deep Learning by Michael Nielsen and Deep Learning by Adam Gibson, Josh Patterson . Do you have any other book or podcast recommendations? Please leave a comment below.
What else is out there to lean machine learning? After completing the course by Andrew Ng and getting my certificate, I give me a couple of days rest to internalize all those learnings. Perhaps I write more about them too. However, there a bunch of other courses out there which I want to check out.
- Machine Learning Engineer Nanodegree on Udacity
- Deep Learning Specialization on Coursera
- Practical Deep Learning for Coders on Fast.ai
Fast.ai has a course on computational linear algebra as well. In general, machine learning involves lots of math. If you need a refresher on certain topics, I can highly recommend Khan Academy.
- natural (Natural Language Processing)
- Synaptic (Neural Networks)
- Neataptic (Neural Networks)
- WebDNN (Neural Networks; inference mode for Keras, Caffe, Chainer, TensorFlow)
- smv.js (Support Vector Machines) – Demo
- ml.js (shallow machine learning algorithms, math operations)
If you have any other recommendations, please leave a comment below. If you know whether certain libraries are active or not maintained anymore, please reach out as well. I would love to keep this article updated for the future.
More Programming Languages for Machine Learning
Did the article help you? You can share it with your friends on social media , support me on Patreon or take one of my courses
The Road to learn React
Build a Hacker News App along the way. No setup configuration. No tooling. No Redux. Plain React in 190+ pages of learning material. Learn React like 14.500+ readers .