Originally published on venturebeat.com.
In 2015, Uber opened a research facility around the corner from Carnegie Mellon University’s National Robotics Engineering Center in a move positioned as a partnership between the two organizations. Within months, dozens of faculty members had left their positions for full-time roles at Uber, draining the center of much of its talent. Other major tech companies have followed a similar path – in 2018, Facebook launched AI labs in Seattle and Pittsburgh headed by former professors.
These stories provide a window into a tug-of-war that’s been playing out between the tech industry and academia. Keen to build products and services that use AI and machine learning, tech firms and other businesses have been hiring away researchers and professors from universities, creating a shortage of academics who can teach the next generation of data scientists. The proportion of computer science PhDs who stay in academia has reached a “historic low,” the Computing Research Association has said.
In waging this battle, however, the tech industry is endangering its own future as well as progress in AI. Besides nurturing tomorrow’s talent, universities host the kind of blue sky research that corporations are often reluctant to take on because the financial returns are unclear. By moving talented researchers out of academia, businesses are disrupting this important basic research.
At the same time, enterprises face a severe shortage of data science and machine learning experts. In an O’Reilly survey of more than 1,600 developers, data scientists, and data analysts released last year, 74 percent of respondents said they consider ML and AI to be a game changer. Yet the lack of qualified talent remains a major hurdle. Universities are trying to address this by offering more courses in data science and machine learning, but this requires qualified professors to teach them – the same people being lured away by industry.
To tackle this conflict, tech companies should change their relationship with the academic community. Instead of hiring academics full time, more companies should allow researchers and professors to split their time between the two worlds. This could mean remaining in academia and working part time for a business, or vice versa. Either way creates a win for both sides. Universities continue to benefit from the brightest minds in AI and data science, while businesses get their in-house expertise and a pipeline for AI talent through internships.
We’ve put this into practice at my own company and it works well. Our chief scientist, Greg Benson, also teaches at the University of San Francisco, where he’s been a professor for more than 20 years. During semesters, Greg spends most of his time teaching and works for my company one or two days a week. During summers and other breaks, he’s full-time at SnapLogic.
We see lots of benefits from this model. USF places around 50 students at tech companies and other businesses each semester, including at our own. These students get valuable experience working on real-world problems and have a path to employment. We’ve hosted about 40 students at our company over the past eight years, and many of those now work for us full time.
These students get to work with real data and develop real products. A few semesters ago, a group of our interns did exploratory work on projects including recommendation engines, natural language processing, and sentiment analysis. The work on recommendation engines proved most fruitful, and we ended up building this into our platform to accelerate app and data integration.
Greg knows the capabilities and limitations of his students, which allows him to place the right interns in the right roles. As they move forward in their careers, we hope the experience will inspire them to stay engaged with the academic community, forge mentorships with students, and even take on teaching roles themselves.
Other tech firms have permitted their AI experts to split their time between industry and academia, such as Facebook’s Yann LeCun. But these tend to be the more prominent researchers who have the clout to call the shots. They are the exception rather than the rule.
With lucrative salaries and inviting perks, it’s not surprising so many researchers pursue business careers; I don’t blame them for their choice. But it’s incumbent on industry to ensure we don’t disrupt the ecosystem that provides important AI research as well as the training ground for the future talent we all need.
There are other ways to help address the AI skills gap, including self-service tools that make it easier for less technical analysts to leverage machine learning without needing to be PhDs themselves. But ultimately we need our skilled professors and researchers to train the next generation of data scientists. Tech companies should encourage and support their data scientists to teach courses at universities and online. It will be a win for all of us in the long run.
