<!-- show content if JS disabled --> <style> .delay-enter { opacity: 1 !important; } </style>
Why I believe the future is open

Why I believe the future is open

Written by Bernard Golden

At a glance

  • Innovation has evolved: Instead of a formal, linear process, companies are embracing a dynamic open model powered with a community.

  • The growth of the internet and its reduced cost of communications means that dispersed collaboration is now possible on a massive scale.

  • Success in the 21st century requires engagement, support, and collaboration with others in open activities. Otherwise teams can fall behind.

  • Four areas of openness are critical to a company’s success: open source, open innovation, open algorithms, and open data.

Twenty-five years ago, if you wanted to to help create the next generation of 
telecommunication services, your path was clear. You would: 

  • Attend a top-notch University like MIT in a handful of developed nations 
  • Continue your education to the Ph.D. level in a specialized field of study 
  • Go to work in the research organization of one of three or four large telecommunication firms — e.g., Bell Labs, the research arm of AT&T 
  • Work your way up in the organization to where you’re finally in a position to contribute to real creative work

Today, none of that is necessary. Instead, if you’re Micheal Ahimbisibwe of Uganda, you can set up a profile on Innocentive, identify research problems submitted by companies interested in innovative solutions, and submit your ideas — and get compensated for them if your submission is accepted.

As Bill Joy, former CTO of Sun Microsystems and later a venture capital investor, put it in what has come to be known as “Joy’s Law”: “No matter who you are, most of the smartest people work for someone else.” 

This means that smart companies seek to broaden their engagement to develop better solutions. Even more important, this means increased opportunity for people across the globe. Men and women with passion and innovative ideas can creation solutions, no longer bound by the tyranny of geography and credentials. Across every domain, the number of participants is growing and the solution set is democratizing.

In a phrase, the future is open.

Why is this occurring now and not ten years ago? Simply stated, the growth of the internet and its reduced cost of communications means that dispersed collaboration is now possible on a massive scale. It’s tempting to think that we’ve already seen the complete impact of the internet, but the truth is we’re just now beginning to experience the full implications of open digital transformation. 

Success in the 21st century will require the ability to deftly engage, support, and collaborate with others in open activities. Failing to do so could mean falling behind. 

Here are four areas of openness that are critical for future success as an open organization:

Open Source

I recently wrote about how open source is a core component in winning the war for talent, but its impact extends far beyond staffing. It’s no exaggeration to say that open source is now the de facto method of software development and distribution. In a “software eating the world” environment, open source is the main dish. 

Here is a figure showing the total number of GitHub repositories by year through 2014. As one can easily see, the trend is clearly turning exponential, indicating that open source is exploding.

Why I believe the future is open - chart

By the end of 2016, GitHub supported 19.4 million repositories, indicating its growth continues on a torrid pace.

Software innovation has clearly shifted to open source. When one looks at any of the new software product categories that have emerged over the past decade, one fact stands out: They are all open source-based.

For example, analytics and big data are technologies in which the past few years have seen innovation and huge growth. A quick look at some of the most significant products in this space confirms that open source is the basis of its innovation: Hadoop, Spark, and Kafka. And they harvest the involvement of large user and contributor bases; for example, Spark’s contributor base has grown from the original academic-based project developers to over 1500 individuals, each of whom helps improve the project for all users.

Key-value and document databases like Cassandra and MongoDB have emerged to address the scale and resiliency requirements of the digital economy — and open source is the basis for the leading products in this category.

The entire field of containers and orchestration is all open source with products like Docker, Mesos, Kubernetes, and CloudFoundry. Proprietary products in this domain are distant also-rans.

This explosion of open source creativity will continue, ultimately resulting in an ecosystem in which open source becomes the default.

Open Innovation

Innocentive is one example of open research, but open approaches to technology commercialization are also emerging. Over the past few years, crowdfunding has made it possible for product creators to obtain financing for their business ideas. The numbers in this chart illustrate the scale of this effort.

More recently, the blockchain ICO phenomenon has emerged. While blockchain is still a nascent concept, many startups and large technology companies perceive the technology as an enormous opportunity. ICOs offer opportunity for innovators to develop new blockchain-based business offerings; they promise to democratize funding and perhaps disrupt the entire venture capital industry.

Open innovation offers opportunity for those previously excluded by reasons of geography, education, or gender to participate and contribute to product creation. Democratizing innovation by growing the pool of contributors not only helps them but also helps innovation sponsors by increasing the diversity of perspectives and experience that help shape solutions with better outcomes as a result.

Open Algorithms

Eight years ago Netflix made a splash with its Netflix Prize — a $1 million reward for anyone who could improve the existing Netflix recommendation system by 10 percent. 

Netflix provided training data to allow contestants to create and validate recommendation algorithms. Over 40,000 teams entered the contest, and one eventually emerged victorious.

Today, sponsoring algorithm contests and providing training data is common. Kaggle (acquired by Google earlier this year) operates contests for companies that need predictive algorithms to address problems in their market. The companies provide training data, and contestants submit proposed algorithms to predict outcomes. All of this is organized and managed by Kaggle.

As companies continue to overlay machine learning onto existing products and services, developing algorithms to analyze the data generated by those richer products/services is critical. And, given the shortage of machine learning talent, implementing methods to increase the number of algorithm developers is critical, so we can expect to see more companies opening up algorithm contests. Here is a current list of some of the existing contests, but chances are that there will be more going forward.

Open Data

If open source is everywhere, open data’s time is at hand. Data owners often find that both they and other parties can benefit when that data is made available.

The benefits of making data available range from the trivial (using New York taxi data to determine if a scene from Die Hard: With a Vengeance is realistic) to the profound (using human genomic data to develop new cancer treatments).  

The great thing about making data available for exploration is serendipity: It’s impossible to predict what outcomes might occur. For example, the person who calculated whether Die Hard was really plausible used the same data to demonstrate the growing gentrification of Brooklyn. It’s unlikely the New York City Taxi & Limousine Commission envisioned either outcome when it released the dataset to the public.

Many states and municipalities have opened up data about local functions to allow residents to understand public safetylocal transportation, and even restaurant inspections.

Releasing data for outside parties to explore and analyze allows organizations to tap into external expertise and develop new offerings and practices. Moreover, it enables these organizations to reach new users and customers, as external parties can use the data in ways the owner cannot envision.

We are in the midst of a transition that is incredibly powerful. Sharing data and code, as well as accepting external contributions of insight and algorithms, promise to profoundly transform how we go through our daily lives. The intertwined growth of the internet and shift from analog to digital processes lowers the barriers to extending the bounds of organizations and incorporate external parties. We appear to be nowhere near the end of this process. No one can predict where we’ll end up, but I believe that the future is open.

Bernard Golden

Written by Bernard Golden

Named one of Wired.com’s 10 most influential persons in cloud computing, Bernard Golden is CEO of Navica and writes an award-winning blog.