I’m now on my third online Coursera course

Cryptography, Game Theory, and now Machine Learning

When I was talking with Scott Martin about Coursera, he shared his experience. Paraphrased: “I actually felt like I was learning a lot. Sure, it was easier than an equivalent course here at Carnegie Mellon, but that’s also partially because they do a much better job of explaining things in terms that everyone can understand”

Because they don’t require or assume that everyone taking Machine Learning has taken years of advanced Calculus and Probability math classes, they’re force to explain everything in a straightforward manner.

That’s it. Straightforward. Instead of confusing you with all sorts of unnecessary terms, they just tell you what it does.

Oh, sure, all the Greek symbols mean something. But nobody learned how to programming by writing their own operating system from scratch.

No, it’s a step-by-step process. You learn the basics, then you expand on them, learn more advanced concepts. You start by making a dog dance, then a simple video game, and you slowly build up to making something people would play. You don’t dive straight into every little detail of one little item – otherwise, you’ll never actually create something useful.

Maybe it’s just Carnegie Mellon, but that’s something that classes here seem to struggle with. What if I want to learn how to do computer vision (say, tracking an object with a webcamera) without bothering with months of extra “theories” and “theorems”.

It’s true, you need to know all of those theorems if you want to push the field forward and develop new algorithms.

But new algorithms don’t do the world any good. It’s the applications of those algorithms that bring us Google, Netflix, smartphones, etc. The fancier algorithms make them faster, but the more you’re taught about all of the fine-grained details, the less you’re able to apply it practically.

The real-world applicability aside, academia seems to have a grave problem: it’s ok for teachers to make things complicated. The topics are complicated, after all! WRONG. I could explain machine learning to an 8 year old. The fact that most college professors can’t is a damn good indicator that they don’t know how to focus on the most important things. This not only slows the education process and forces people away from thinking about practical applications – it also scares people away from entering the field in the first place. That is why we don’t have enough engineers.

In economics, there’s a theory known as the “Solow Growth Model” – it states that, for a given economy, additional labor and machinery has decreasing returns. Eventually, any economy will reach the point that more workers doesn’t result in more output – they’ll run out of space for factories or cover all the land in farms, or whatever. At that point, the only way for the economy to grow is to increase its “Total Factor of Productivity” – that is, increase the productive output of machinery and workers. Developed economies have all reached this point – America’s GDP only continues to grow because new innovations like faster machines and easier-to-use tools make workers more efficient.

The actual formula for this is Y = zF(K, N). That is, output equals a function of labor and capital, multiplied by the Total Factor of Productivity (z). As we see in the below graph, the function F(K, N) has marginally decreasing returns:

(the blue line is the increase in output caused by an increase in TFP from the red line)

What happens if we apply this to technology innovation?

I propose that technology’s “labor” and “capital” might be “engineering” and “design”. That is to say that engineers make things faster and build new functions, and designers make it faster and easier to use. Overall, adding more engineers and designers creates more technology.

Now, if we’re following the Solow Growth Model, that means that early on, lots of engineers and designers will cause technology to advance quickly. And, if you look at the past 20 years, it’s clear that’s what happened. Initially, there were just a few bleeding edge computer programmers working on a few projects. The technology “economy” has grown quickly, to the point where the world hardly notices if another engineer or designer starts working on a new project.

But, then, what’s the Total Factor of Productivity? How does the technology economy continue to grow?

Hardware. Think about it: every time there’s been a major new device (smartphones, tablets, etc), there’s been a huge boon in the technology economy. Suddenly, individual programmers are having a huge difference, and the pace of innovation soars.

Then, slowly, it fills up. Eventually, almost every possible interesting smartphone app has been made.

The only cause for new growth? New hardware. They add a forward-facing camera, or develop a faster cellular internet chip. They develop new accessories, like the Pebble Bluetooth watch. These hardware advances give devices new functionality that developers are able to exploit to advance the industry.

Sure, it might not be Earth-shattering, but it’s certainly an interesting way to think about technology. And, if you’re a developer, it might be a good reminder that if you want to make money (or make a difference), you should always try to work on and build for the newest, least exploited technologies.

Do you agree?