Startups

How To Stem The Global Shortage Of Data Scientists

Comment

Image Credits:

Amy Gershkoff

Contributor
Amy Gershkoff is the chief data officer at Zynga and an adjunct professor at the University of California, Berkeley, where she teaches data science.

Data scientist might be considered the “sexiest job of the 21st century,” with many companies eager to hire data science talent. But 20 years ago the term was barely used, save for a few academic articles.

Even a decade ago, explaining data science to employers was challenging. Few people understood the value of a skill set that combines computer science, statistics, operations research, engineering, business insights and strategy — and the impact it can have on a business.

But things have changed over the last five years. Not only has the term “data science” become commonplace, but data scientists have become highly sought after in the marketplace and lauded in the press.

After all, it was data science that gave consumers personalized shopping recommendations on websites like eBay and Amazon, customized entertainment recommendations on Netflix and Spotify and tailored content and advertising on Facebook, Twitter and Pinterest — all at tens of millions of dollars (if not more) of revenue upside for these businesses.

Over time, organizations started to see the value of data science as the volume of data they had to analyze increased dramatically. For instance, today an estimated 2.5 quintillion bytes of data are created daily. Within this voluminous quantity of data lie insights that could enable companies to gain a competitive edge and provide a better consumer experience, which is why many organizations have now made data science a top priority.

In a KPMG survey of C-suite executives, 99 percent said that analysis of big data was important to their strategy for next year. As the study stated, “Today, the issue is no longer about owning the most data but rather … how to turn data into insights.”

Which is where data scientists come in. The role of a data scientist is to mine enormous volumes of data and pull out key insights. In an era where enterprise-generated data is expected to exceed 240 exabytes daily by 2020, the need for experts trained in extracting insights from data is more important than ever.

So what’s the problem? We have a global talent shortage, and the demand for data scientists continues to grow rapidly, far outpacing the anemic growth in supply. A McKinsey study predicts that by 2018 the number of data science jobs in the United States alone will exceed 490,000, but there will be fewer than 200,000 available data scientists to fill these positions. Globally, demand for data scientists is projected to exceed supply by more than 50 percent by 2018.

The global talent drought may be partially caused by the dearth of universities that offer data science programs dedicated to preparing the next generation of data scientists. Fewer than one-third of U.S. News & World Report’s Top 100 Global Universities offer degrees in data science. Of the 29 universities that offer data science programs, a mere six make them available to undergraduates (the rest are reserved for graduate students).

Moreover, the average cohort size for one of these data science programs is just 23 students. The small cohort size may be partially attributed to the fact that most of these data science programs tend to be offered to graduate students, and sizes of graduate programs tend to be smaller. Still, at 23 students per cohort, we are unlikely to make a meaningful dent in closing the global data science talent gap.

An analysis of the curricula for these data science programs also suggests cause for concern. Most are focused almost exclusively on the computer engineering aspects of data science, with course titles such as Software Design, Parallel Computing and Software Development.

Largely missing from the data science curricula at many of these universities are courses in statistical analysis, insights and strategy. This oversight may have serious consequences for graduating students and their future employers.

Without training in these other areas, data scientists may be capable of designing an algorithm that is mathematically elegant, but doesn’t make strategic sense for the business. They also may not know how to design an experiment to determine whether the algorithm is effective. In other words, computer science skills alone are insufficient for success as a data scientist in today’s marketplace.

Furthermore, many data science programs lack courses that help students apply their technical data science skills to fields such as marketing, operations, product development and supply chain, and industries such as energy, bioinformatics, transportation and healthcare.

While there are some universities that have distinguished themselves for having curricula that focus on the multi-faceted aspects of data science, such as New York University, there are still far too few universities producing enough graduates to meet today’s growing demand for data scientists.

There are three steps we can take to help stem this talent shortage:

  • Increase the number of universities offering data science degrees: To accelerate this, employers can partner with universities to help design and fund the creation of these programs. Accenture’s recent partnership with the University of California, Berkeley to create a new Big Data and Data Science curriculum is one such example. As part of the partnership, Berkeley’s Haas School of Business collaborated with Accenture to create new classes and a lecture series to equip MBA students with the skills needed to analyze big data, including a new course on data analytics.
  • Offer data science programs for undergraduates: Enrollment in data science programs must radically increase — cohort sizes of 23 students simply will not close the talent gap fast enough. Offering data science degree programs to undergraduates as well as graduate students can significantly expand enrollment. Universities also should consider offering courses online and during evenings and weekends in an effort to reach non-traditional students.
  • Launch programs that train analysts to become data scientists: Many organizations are flush with analysts who typically have some data science skills in key areas such as statistics. Zynga, for example, recently started a Data Science Transition Program, which enables high-performing analysts to be apprenticed to a data scientist for twelve months, during which time they learn data science skills on the job while taking online data science courses. At the end of the program, the participant becomes an Associate Data Scientist at the company.

Closing the global data science talent gap won’t be easy, but it’s critical to ensure companies have the ability to make data-driven strategic decisions. If companies and universities work together to address the problem, we can better prepare both students and employers for success.

More TechCrunch

Featured Article

CIOs’ concerns over generative AI echo those of the early days of cloud computing

CIOs trying to govern generative AI have the same concerns they had about cloud computing 15 years ago, but they’ve learned some things along the way.

3 hours ago
CIOs’ concerns over generative AI echo those of the early days of cloud computing

It sounds like the latest dispute between Apple and Fortnite-maker Epic Games isn’t over. Epic has been fighting Apple for years over the company’s revenue-sharing requirements in the App Store.…

Epic Games CEO promises to ‘fight’ Apple over ‘absurd’ changes

As deep-pocketed companies like Amazon, Google and Walmart invest in and experiment with drone delivery, a phenomenon reflective of this modern era has emerged. Drones, carrying snacks and other sundries,…

What happens if you shoot down a delivery drone?

A police officer pulled over a self-driving Waymo vehicle in Phoenix after it ran a red light and pulled into a lane of oncoming traffic, according to dispatch records. The…

Waymo robotaxi pulled over by Phoenix police after driving into the wrong lane

Welcome back to TechCrunch’s Week in Review — TechCrunch’s newsletter recapping the week’s biggest news. Want it in your inbox every Saturday? Sign up here. This week, Figma CEO Dylan…

Figma pauses its new AI feature after Apple controversy

We’ve created this guide to help parents navigate the controls offered by popular social media companies.

How to set up parental controls on Facebook, Snapchat, TikTok and more popular sites

Featured Article

You could learn a lot from a CIO with a $17B IT budget

Lori Beer’s work is a case study for every CIO out there, most of whom will never come close to JP Morgan Chase’s scale, but who can still learn from how it goes about its business.

1 day ago
You could learn a lot from a CIO with a $17B IT budget

For the first time, Chinese government workers will be able to purchase Tesla’s Model Y for official use. Specifically, officials in eastern China’s Jiangsu province included the Model Y in…

Tesla makes it onto Chinese government purchase list

Generative AI models don’t process text the same way humans do. Understanding their “token”-based internal environments may help explain some of their strange behaviors — and stubborn limitations. Most models,…

Tokens are a big reason today’s generative AI falls short

After multiple rejections, Apple has approved Fortnite maker Epic Games’ third-party app marketplace for launch in the EU. As now permitted by the EU’s Digital Markets Act (DMA), Epic announced…

Apple approves Epic Games’ marketplace app after initial rejections

There’s no need to worry that your secret ChatGPT conversations were obtained in a recently reported breach of OpenAI’s systems. The hack itself, while troubling, appears to have been superficial…

OpenAI breach is a reminder that AI companies are treasure troves for hackers

Welcome to Startups Weekly — TechCrunch’s weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday. Most…

Space for newcomers, biotech going mainstream, and more

Elon Musk’s X is exploring more ways to integrate xAI’s Grok into the social networking app. According to a series of recent discoveries, X is developing new features like the…

X plans to more deeply integrate Grok’s AI, app researcher finds

We’re about four months away from TechCrunch Disrupt 2024, taking place October 28 to 30 in San Francisco! We could not bring you this world-class event without our world-class partners…

Meet Brex, Google Cloud, Aerospace and more at Disrupt 2024

In its latest step targeting a major marketplace, the European Commission sent Amazon another request for information (RFI) Friday in relation to its compliance under the bloc’s rulebook for digital…

Amazon faces more EU scrutiny over recommender algorithms and ads transparency

Quantum Rise, a Chicago-based startup that does AI-driven automation for companies like dunnhumby (a retail analytics platform for the grocery industry), has raised a $15 million seed round from Erie…

Quantum Rise grabs $15M seed for its AI-driven ‘Consulting 2.0’ startup

On July 4, YouTube released an updated eraser tool for creators so they can easily remove any copyrighted music from their videos without affecting any other audio such as dialog…

YouTube’s updated eraser tool removes copyrighted music without impacting other audio

Airtel, India’s second-largest telecom operator, on Friday denied any breach of its systems following reports of an alleged security lapse that has caused concern among its customers. The telecom group,…

India’s Airtel dismisses data breach reports amid customer concerns

According to a recent Dealroom report on the Spanish tech ecosystem, the combined enterprise value of Spanish startups surpassed €100 billion in 2023. In the latest confirmation of this upward trend, Madrid-based…

Spain’s exposure to climate change helps Madrid-based VC Seaya close €300M climate tech fund

Forestay, an emerging VC based out of Geneva, Switzerland, has been busy. This week it closed its second fund, Forestay Capital II, at a hard cap of $220 million. The…

Forestay, Europe’s newest $220M growth-stage VC fund, will focus on AI

Threads, Meta’s alternative to Twitter, just celebrated its first birthday. After launching on July 5 last year, the social network has reached 175 million monthly active users — that’s a…

A year later, what Threads could learn from other social networks

J2 Ventures, a firm led mostly by U.S. military veterans, announced on Thursday that it has raised a $150 million second fund. The Boston-based firm invests in startups whose products…

J2 Ventures, focused on military healthcare, grabs $150M for its second fund

HealthEquity said in an 8-K filing with the SEC that it detected “anomalous behavior by a personal use device belonging to a business partner.”

HealthEquity says data breach is an ‘isolated incident’

Roll20 said that on June 29 it had detected that a “bad actor” gained access to an account on the company’s administrative website for one hour.

Roll20, an online tabletop role-playing game platform, discloses data breach

Fisker has a willing buyer for its remaining inventory of all-electric Ocean SUVs, and has asked the Delaware Bankruptcy Court judge overseeing its Chapter 11 case to approve the sale.…

Fisker asks bankruptcy court to sell its EVs at average of $14,000 each

Teddy Solomon just moved to a new house in Palo Alto, so he turned to the Stanford community on Fizz to furnish his room. “Every time I show up to…

Fizz, the anonymous Gen Z social app, adds a marketplace for college students

With increasing competition for what is, essentially, still a small number of hard tech and deep tech deals, Sidney Scott realized it would be a challenge for smaller funds like…

Why deep tech VC Driving Forces is shutting down

A guide to turn off reactions on your iPhone and Mac so you don’t get surprised by effects during work video calls.

How to turn off those silly video call reactions on iPhone and Mac

Amazon has decided to discontinue its Astro for Business device, a security robot for small- and medium-sized businesses, just seven months after launch.  In an email sent to customers and…

Amazon retires its Astro for Business security robot after only 7 months

Hiya, folks, and welcome to TechCrunch’s regular AI newsletter. This week in AI, the U.S. Supreme Court struck down “Chevron deference,” a 40-year-old ruling on federal agencies’ power that required…

This Week in AI: With Chevron’s demise, AI regulation seems dead in the water