Sure, robots and AI may destroy a lot of jobs, say the cheerleaders, but they’ll create plenty more jobs. But how vulnerable to automation are these new jobs? Exhibit A: Data Science. MIT News says MIT is on the way to automating a lot of the work Data Scientists now do when using machine learning:
MIT researchers aim to take the human element out of big-data analysis, with a new system that not only searches for patterns but designs the feature set, too. To test the first prototype of their system, they enrolled it in three data science competitions, in which it competed against human teams to find predictive patterns in unfamiliar data sets. Of the 906 teams participating in the three competitions, the researchers’ “Data Science Machine” finished ahead of 615.
In two of the three competitions, the predictions made by the Data Science Machine were 94 percent and 96 percent as accurate as the winning submissions. In the third, the figure was a more modest 87 percent. But where the teams of humans typically labored over their prediction algorithms for months, the Data Science Machine took somewhere between two and 12 hours to produce each of its entries.
Or to put it another way: about 2 years ago I started picking up some machine learning techniques. If I keep working at it, by the time I get good at it odds are the Data Science Machine will be way better at it than I am.
The current version of the Data Science Machine isn’t really a fair comparison to the skills Data Scientists need. With a contest, you know exactly what problem you need to solve from the get-go. In real life, often half the battle is figuring out what problem is possible and/or worthwhile to solve. And there are other issues where it’s unlikely that simple AI will outstrip human creativity any time soon.
Odds are Data Science jobs won’t vanish tomorrow; the demand for Data Scientists is enormous. Then again, that’s why researchers as well as Microsoft, IBM, and many other players are working furiously to automate them out of existence.