<!-- show content if JS disabled --> <style> .delay-enter { opacity: 1 !important; } </style>

How to seize the big data opportunity without burning through your budget

At A Glance

  • With growing data and increasing competition to hire data scientists, teams must cultivate rather than acquire data-science expertise.

  • A culture of data science takes the concept of citizen data scientists a step further, extending their impact across the organization.

  • Companies can meet the challenge by training employees as citizen data scientists, mining insights with access to data and the right tools.

  • Organizations with a data-science culture can reap the benefits of big data, setting the stage for continued success in a data-first world.

By 2020, the digital universe will reach 44 zettabytes, or 44 trillion gigabytes, according to IDC. Translate that into streaming video, and you could watch the World Cup in HD for the next 5.5 billion years  approximately as long as the Earth has been around.

Data Science - Video Streaming

That's an enormous volume of data, which now comes to businesses from browsers, emails, phones, sensors, log files, social media, and much more. Because today’s data is so diverse, it is richer than ever in potential insight but more difficult to sort out effectively.

It’s a challenge no business can ignore. According to the McKinsey Global Institute, how companies use data “will become a key basis of competition and growth.” Instead of attacking this problem individually, or even in small teams, companies need to build a systemic culture of data science. Using the right technology and instilling a scientific approach to data in every employee will help businesses remain competitive, even in the face of increasing competition for data-science talent and growing volumes of complex data.

Sophisticated analytics can substantially improve decision-making, mitigate risks, and unearth valuable insights that would otherwise remain hidden.

McKinsey Global Institute

The difference between data-driven and data science

First, it’s important to draw a distinction between being data-driven and creating a culture of data science. “It is not always easy to discern a correct or accurate conclusion from charts and data,” says Chad W. Jennings, a longtime data scientist with a Ph.D. in aeronautics and astronautics and current Googler. “In a data-science culture, the community helps to check assumptions, vet methods and clear results. There’s a lot more rigor applied to drawing conclusions from data than just throwing a chart into a presentation and declaring, ‘Here’s what this means.’”

This scientific rigor, Jennings notes, is critical to ensuring the right conclusions are drawn from data. Data science is exactly what it sounds like: using the scientific method to obtain insights from data. To lay the foundation for a data-science culture, anyone interpreting data should use basic scientific best practices: starting with a hypothesis, using appropriate methodology, clearly documenting results, expressing conclusions in terms of their statistical significance and margin of error, and undergoing peer review. Like other scientists, data scientists are inherently curious and creative about which questions to ask. But just as a chemist needs a lab or a geologist needs a rock hammer, data scientists need access to the right data and analytics tools in order to be successful.

Rise of the citizen data scientist

The responsibility for drawing statistically accurate conclusions based on data was once the exclusive purview of professional data scientists. But by 2018, according to McKinsey, “the U.S. alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.”

How to seize the big data opportunity without burning through your budget - data science

As the competition for data scientists intensifies, most companies will need to diversify their talent strategies. Citizen data scientists  who, as defined in InformationWeek, are people who leverage data analytics, but whose main job functions aren’t statistics or analytics  can be a powerful complement to in-house data scientists, especially for companies that invest in building a culture of data science.

To be successful, would-be citizen data scientists need:

  • Access to data

  • Curiosity

  • Facility with SQL

  • Domain expertise

  • Collaboration

What does this look like in practice? Consider a technical account manager who supports the technical issues or needs of large customer accounts. Suppose she wants to play a more proactive role in her customers’ success, rather than simply reacting to technical issues when they arise. Giving her access to data about her customers’ usage patterns enables her to ask new questions based on the pain points she’s heard directly from customers  for instance, how might we optimize uptime based on my customer’s usage?  Her first step is to write some queries to analyze the usage.  

Usage might be broken down into questions like these:

  • Does the customer use the service more aggressively at night?

  • Are there certain areas of the world that generate more usage than others?

  • Are certain features of the service getting more traffic than others?

Results from all of these analyses will have statistical variance. If the variance is high enough, deciphering the correct conclusion may be difficult. This technical account manager could send her conclusions and queries to her colleagues with the note, “I ran these queries and drew these conclusions. Can someone run them and check my conclusions?”

The fact that this technical account manager can run these analyses may not be remarkable, but the fact that she can simply share her queries with a cadre of engaged colleagues that also has instant access to the data means that, together, they form a community of data scientists. In effect, you’ve enabled an ad hoc peer-review culture in your company, and that culture will deliver better analyses, more accurate conclusions, and greater value for the company.

Many organizations focus on the need for data scientists, assuming their presence alone will enable an analytics transformation. But another equally vital role is that of the business translator who serves as the link between analytical talent and practical applications to business questions.

McKinsey Global Institute

Building a data-science culture: Step by step

A data-science culture takes the concept of citizen data scientists a step further, extending their impact across the organization.

Step 1: Create a foundation.

Building the right foundation for your data strategy is the first step toward a culture of data science  and without the right tools, it is no small task. In order to enable this culture, companies must first be able to securely store and process huge volumes of critical business data. Next, they must empower their people to access that data anytime, from anywhere. Finally, they must be able to disseminate vetted analyses and conclusions at scale by creating automated dashboards. These capabilities, fundamental to any data strategy, can take years to build in a traditional data warehouse environment. However, modern tools (such as BigQuery) create the foundation for a citizen data science culture immediately.

Step 2: Provide access.

Once a foundation is in place, it’s time to grant access to your citizen data scientists so they can start getting their hands dirty. It also lays the groundwork for collaboration by presenting your employees with a shared challenge. Here, it’s crucial to have technology that empowers rather than overwhelms.

Step 3: Foster experimentation and collaboration.

Not every company has the “fail fast” mentality of a Silicon Valley startup. In some organizations, the notion of experimentation without a defined outcome can engender fear and resistance. In order to encourage your employees to experiment with data, remove expectations of success or failure, and emphasize the importance of simply being curious and asking the right questions. Communication tools like shared documents and chat apps can help foster collaboration and facilitate the sharing of analyses and results. Business leaders should also showcase how data analysis informs their own decision-making process so that employees can see the impact of their work.

You can't expect to create true

citizen data scientists purely by

letting them experiment.

Step 4: Build expertise.

You can’t expect to create true citizen data scientists purely by letting them experiment. You’ll also need to recruit your professional data scientists (or bring in outside expertise) to give a basic crash course in statistical analysis.

However, bear in mind that your citizen data scientists don’t need to become professional data scientists  you need them for their expertise in other areas of the business. If you are using the right technology, much of the lower-level data organization and analysis can be automated, enabling IT and business users to get answers to simple queries with minimal effort.

Step 5: Celebrate success.

It may sound inconsequential, but recognition can go a long way toward establishing a data-science culture. It can also help incentivize the right behaviors by underscoring the importance of scientific methodology and impactful conclusions. Much research has been done on the business impact of data accessibility and usability, so take the opportunity to celebrate successful applications of data in terms of their business impact.

As the volume and potential impact of business data continues to expand, the ability to quickly perform sophisticated analytics will become a key market differentiator. And as competition for data-science talent continues to intensify, companies that can train and empower a community of citizen data scientists will prevail. Building a culture of data science isn’t just about conserving costs or staying competitive. Companies that recruit their in-house data scientists to train and empower other employees will reap the benefits of increased communication, collaboration, and employee retention  ensuring their continued success in a data-first world.