I’ve written about this topic previously, and Ridley’s perspective lies fairly close to my own.  We tend to think of ideas as 1) creations of single, brilliant individuals, and 2) unique in history, rather than derivative.  Narratives of great discoveries suggest that ideas are created de novo from individual minds.  However, if we take the time to deeply explore many of the greatest breakthroughs in science, art, etc, we find that the greatest ideas are the result of communities of thinkers and their combination, in unique and creative ways, of pre-existing ideas.  To use Ridley’s metaphor, communities take existing ideas and breed them.  That reproduction results in mutations and recombinations, which create a new species of idea that is superior to existing populations of ideas that we all then benefit from.  Knowledge creation has an evolutionary and communal character to it.

It further reinforces the importance of creating conditions that facilitate the intellectual transmission of diverse ideas and perspectives (what I’ve called “social bumping“).  This can include helping people build diverse social networks, promoting the co-mingling of various subject-matter experts, and encouraging people to share their ideas and perspectives when they might otherwise feel as though it wasn’t their place to speak up.  As Ridley says during his talk, “We all know little bits, but none of us knows the whole.”  Given that is the case, we would be wise to design an environment that maximizes the mating of diverse ideas.

The crowdsourcing of problems could accelerate the time it takes businesses to identify gaps in current product or service offerings and spend more time developing better solutions to those problems for businesses and/or consumers. Of course, the danger is that we don’t just want innovators focusing on immediate problems as it may lead to unexplored paths (which, often times are the most interesting and groundbreaking). However, I would think that certain types of innovators and firms would select themselves into a more direct problem-solving approach to innovation versus a more open, exploratory exercise.

The same could easily be said about academia.  Often times the hardest part of research is determining what it is we need to solve for and which problems are (or, should be) a priority.

Related to this idea, a colleague of mine, Charlie Carpenter, points to a really interesting initiative by the Division of the Social Sciences at Harvard–determining the “hard problems” in the social sciences.  On April 10th, a number of heavy hitters from the social sciences met to discuss what these knotty problems were and how to address them.  The site collects all the discussions by the speakers (be sure to check them out).  It also asks visitors to take a poll, the purpose of which is to help determine what problems are the hardest, the most important, and what problems the speaker may have missed.  You can view and take the poll here.  The group has also set up a Facebook page to allow folks to continue the discussion.

I think this is a great initiative.  Of course, the degree to which it is helpful will be determined largely by three factors:

• Diversity of the respondents: The more diverse the individuals responding, the more reliable the results of the poll will be.  This means not only getting people from multiple disciplines to respond, but also people from outside academia who can bring even greater diversity to the questions.
• Independence of the respondents: The respondents should be independent from one another–that means we need to have people outside of the graduate students of the heavy hitters answering questions, since often times their perspective is heavily influenced by who they have chosen to train under.
• Localization of the respondents: Parts of academia have often been charged with being to Euro- or US-Centric (on the latter point, the study of international relations and political economy are often charged with being too US-Centric).  The poll will work best if they manage to engage respondents from numerous disciplines as well as numerous geographies.

I do not know the extent to which they group is taking seriously the need to manage and engage the responding community in order to get the most out of the crowdsourcing effort, but of course this is also key to such an effort. Also, at present I am not so sure they have done enough to institute and promote the proper incentives for respondents, but I have to investigate more.

Either way, it is a very interesting initiative.

• A prediction market for drug development
• Personal web-based crowdsourcing applications
• IBM’s internal translation project

As always, let me know if I’ve missed anything good out there.

To test this idea, Wired asked writer Evan Ratliff to disappear for one month.  He could tell no one of his plans (not even his family or girlfriend).  They announced the contest online on August 14th, set a bounty for his discovery ($5,000–$3,000 of which was Ratliff’s own money), and invited the general public to find Ratliff through whatever means they liked (preferably, legal).  The contest would be over when someone came face to face with Ratliff, snapped a picture of him, and said the word “fluke”.  All told, it took 17 days to catch Ratliff.  The article is a must read, providing the fascinating details of Ratliff’s moves as well as those of his pursuers.

What occurred to me when reading the article was that, without saying so, Wired essentially crowdsourced the manhunt for Ratliff.  Rather than hire a single investigator or firm to find him, Wired outsourced the task to a large, undefined, diverse group of people (basically, Jeff Howe’s definition of crowdsourcing).  For crowdsourcing to work (or for crowds to be “wise”) the crowd generally has to have three attributes, all of which the hunter group seemed to possess:

• Diversity: The more diverse a crowd is, the more likely it is to be intelligent. Why? Because a diverse crowd will bring many different views to bear on a problem, thereby increasing the likelihood that a solution will be found. From what I can tell, the crowd chasing Ratliff was quite diverse and included people of all ages, professions, and skill sets. And it was an interesting mix of people who eventually nabbed him.
• Independence: The crowd must also be relatively independent, meaning individuals are not reliant on the same source of information. This is important for two reasons. First, when information is independent it’s more likely the ‘errors’ of each individual will cancel each other out. Second, independent individuals are more likely to bring unique information to the table. The article provides evidence for the independence of individuals and clusters of individuals working on the search. While many people were sampling the same information via the Twitter #vanish hashtag, there were tons of individuals bringing their own information to the party.
• Decentralization: The more decentralized, or localized, members of a crowd the smarter it will be, the idea being that the more localized and spread out a crowd is the greater the sources of information that the crowd can pull from. The hunters in this case were highly decentralized, located all over the country and providing local intelligence that the group as a whole could benefit from.

It also occurred to me that the story reveals and illustrates a few other important aspects of crowdsourcing:

• Rewards: Motivating a crowd is a key component when implementing a crowdsourcing strategy to problem solving. If the crowd isn’t properly motivated they are unlikely to put the time and effort into the task at hand. Wired did provide a monetary incentive (the $5K), the crowd seemed just as motivated (if not more so) by the reputational incentive of being known across the Wired-world as the one who found Ratliff. And not to ruin the ending of the article, but readers will see that on balance the reputational motivation won the day in this case. The lesson is that you should take into account multiple types of incentives, not just monetary, when trying to motivate a crowd.
• Self-organization & Collaboration: Even though there are individual incentives to solve the problem, it did not deter hunters from pooling their knowledge and working together in teams. The story mentions that almost immediately folks shared their information and thoughts via the hashtag #vanish, formed groups on Facebook, and even relocated to secure chat rooms to prevent moles from providing Ratliff with counterintelligence. Most of these people had never met each other. Almost instantly hunters self-organized, without the benefit of hierarchical direction. And much like the recent Netflix contest, individual hunters banded together in ad hoc and informal teams to work collaboratively towards finding Ratliff faster than if they simply worked alone.

Be sure to check it out.

Now Helprin does make some valid points; however, I think on balance the conclusions he reaches about new technologies and techniques is overblown and based on his own biases and misunderstandings.  I’d like to speak to a few of them below:

Mark Helprin (MH): Before there was copyright, there was very little incentive for people to actually write things and assemble information. With the development of copyright, all that has increased.

I agree with Helprin here and there is quite a lot of excellent research (particularly by economic historians like Douglass North) to back up the notion that the institution of copyright and associated legal protections such as property rights, patents, etc, led to an explosion in creativity and economic development in the Western world.  So, point for Helprin.

Mary Kuntz (MK): [...]  There’s a lot of activity online right now that’s all about collaboration and crowd sourcing. And some of what gets produced that way people call art. I gather that you would take issue with that?

MH: I would. Crowd sourcing—to me, the words are a nightmare. The great achievement of Western civilization, anyway, has been to end the collective approach to things that marked the early history of man.

In other words, you were defined as part of a group. You were a serf. You were a peasant. You were a slave or whatever. And then in medieval times, you were a member of a guild. And the rights of the individual didn’t really count. What happened with the Greeks and then with Roman law and then over a long, long period culminating in modern times is that we have refined the rights of the individual. Now, obviously collaboration can be very powerful, and it’s important, and we do things in concert with one another. And we advance science that way and many things. But there’s really nothing that can substitute for one mind and one voice.

In a single thought, Helprin does two things: 1) illustrates that he does not truly understand crowdsourcing, but sees it as a threat anyway (and he is surely not alone here); and 2) his entire premise (“nothing can substitute for one mind and one voice) is based on a myth (that of the lone or heroic innovator/creator), one that is contradicted by the two sentences immediately preceding it.

First of all, Helprin’s conceptual connection between crowdsourcing and collectivism is utterly misplaced and flatly wrong. Just because a process or endeavor involves input and cooperation amongst a large number of people it does not mean that one is backsliding against the liberalization of society. I guess Helprin thinks we should disband all those pesky corporations that publish and ship his books, as well as those retail chains that put them in the hands of readers. The brilliance that is Helprin would be the proverbial tree falling in the woods with no one to hear if not for the collective efforts (ideas, labor, etc) of numerous individuals. All crowdsourcing does is provide an alternative (and, in some cases, a better) method for organizing individuals to tackle a collective task or to gather input and insights necessary for the accomplishment of some goal. There is no requirement that by taking part in crowdsourcing one must give themselves over to some collective identity. In fact, many people who participate in crowdsourcing do so during their spare time, and as an alternative to their 9-5 occupation (as Jeff Howe has noted, the process taps into people’s spare cycles). One might argue that crowdsourcing actually provides people with greater outlets for creativity, since those that are electrical engineers by day get a chance to solve complex design problems that have baffled bench chemists at Fortune 500 companies.

Second, Helprin falls into a familiar trap which is basing his argument on the notion that great creativity and discoveries is the result of single, heroic individual. This is hardly the case and there are piles and piles of research illustrating how collaboration is key. Helprin himself notes that advances in, for example, science are dependent on intra- and inter-temporal collaboration. We are better able to tackle complex problems when we have the benefit of diverse input. Additionally, our ability to create and come up with advancements is always dependent on previous work (“standing on the shoulders of giants“) and leveraging or combining creations by others. Scott Berkun, in his wonderful Myths of Innovation, sums it up well:

It wasn’t until the 1500s and the rise of the Renaissance that Western cultures grew comfortable acknowledging people’s creative abilities and individual achievements.

[...] Today, years away form the Renaissance, we’re still attached to the myth of the lone inventors. We do recognize collaboration and partnerships, but we often fall back on tales of lone innovators as heroic figures for reasons of convenience. We insist on isolating credit and dismissing the importance of others. Patent law, by design, credits one or a handful of individuals, assuming not only that ideas are unique and seperable, which is dubious, but that individual names can be given legal ownership of ideas.

Helprin didn’t invent the English language, the novel, the short story, various themes used in his work, and I guarantee that he has on more than one occasion drawn inspiration directly or indirectly from the minds and works of others. To paraphrase Keynes, in the long run everything is derivative.

What cinches Helprin’s lack of perspective for me is this exchange:

MK: Well, let me ask you something. This is just a hypothetical. What if you were working on a new novel and you got stuck on the ending, you just couldn’t find a way to wrap it up and make the plot work. And let’s say you put the problem to a wide following of smart, engaged, intelligent Mark Helprin fans. And one of them or several of them working together came up with the perfect solution. And you used their idea, which, in fact, was better than anything you would come up with. Would the novel that resulted be any less valid?

MH: I don’t know what you mean exactly by valid. But what I can say is I would never do that. If you look at the history of literature, it’s never been done that way. In school, they have what they call “brainstorming,” which I think is a comic-book word. They have what they call “writing webs.” They sit and they criticize each other’s writing, sort of like people in the Soviet on a factory floor.

And this is not the way that it should be. And I guarantee you that the product of this will be far, far less valuable than the product of somebody straining to do his best and taking responsibility for it also.

It’s never been done, so why bother; brainstorming is akin to forced collectivization under the Soviets; and he knows for sure that nothing ‘collectively’ produced could be as good as something produced by an individual.

There’s lots more there so I encourage you to read or listen to the entire interview. As I said, there are moments where Helprin makes solid points (points I agree with), but to say I disagree with his view on crowdsourcing, social technology, and innovation would be a massive understatement.

Over at Advertising Age, Ana Andjelic discuss why digital agencies aren’t ready to lead.  This article was originally posted over at her blog, i [love] marketing.  It is an interesting piece that dovetails nicely with the discussion we recently had around how to organize for innovation.

Digital agencies impress clients with their passion, drive and technology know-how. Clients then say: “You gave us a lot to think about.” Which often means that the account is awarded to someone else. Where digital shops fail is giving confidence to the client that all this momentum will be indeed executed in a well-led marketing campaign. All of this is not new. It is already described in organizational theorist James March’s exploration vs. exploitation dichotomy. The best companies have the optimal balance between the two; those less successful are doing too much of either.

The article focuses on agencies, but there is quite a bit that would apply to other sectors.  Read the whole thing.

The other item is an article over at the Chaordix blog by Shelly Kuipers (President & CEO). Shelly writes about her recent experience at a conference on data governance hosted by IBM. What was the general response to Shelly’s discussion of crowdsourcing’s level of data openness?

“No thanks, at least not yet.” I discovered that among major enterprises in attendance, there is still much work to be done to get social tools well understood and adopted. There seems to be an inherent need to heavily manage user behavior inside these companies via locked down systems and strict data governance. Corporations do not yet have confidence in the tools for fear of compromising confidential and proprietary data.

I am not at all surprised by this response–and I don’t think it is entirely unreasonable. While I am a fan of open-source approaches I also have my risk-averse/risk-mitigation side telling me to be wary. It is a tension that will take some time to work through.

It’s true – knowledge “reshuffling” is key to innovation (or, as you said “when it comes to thinking, we need more disruption, not less.”) This reshuffling, however, creates a lot of “noise” (how do you know what you are looking for before you find it?), and ultimately accounts for less-than-efficient organization, because it creates crazy transaction costs.

And reduction of transaction costs is what made firms show up in the first place (in opposition to markets), as Ronald Coase would have said.

So, the very condition that’s critical for generation of new knowledge and/or recombining old and new knowledge is actually detrimental for efficiency. The question then is, how to combine organizational efficiency with innovation?

I think Ana is right to invoke Coase’s theory and to be concerned about drastically increasing transaction costs and it got me thinking.

My view: while historically firms were able to lessen the search costs associated with valuable information, this no longer applies.  Firms are no longer the most efficient and effective means for collecting and sourcing innovative ideas and insights.  Therefore, as Coase’s theory would predict, firms may need to go beyond their four walls an turn to the market, effectively outsourcing (or co-sourcing) a large portion of idea generation.  Additionally, firms can further reduce information and search costs by leveraging technologies that tag, catalog, and organize knowledge–both within and outside firms.

The basic logic behind traditional, integrated firms is that it was cheaper to bring the various elements of production under one roof.  The idea was that it made sense for firms to enter into long-term contracts with employees because it would eliminate the potentially massive transaction costs associated with, for example, search and information costs one finds in the market.  In theory, when transaction costs are less within a firm than within the broader market, firms will take those items (e.g. labor, technology, distribution, etc.) in-house.  When costs are less in the market, firms will choose to outsource those items.

What many firms are realizing now is that, in many cases, they don’t have the knowledge internally to problem solve and innovate effectively.  The rapid and profuse rise of Open Innovation initiatives by tradional R&D stalwarts provides evidence for this shift.  And even if they do have that knowledge internally, firms are quite inefficient when it comes to quickly locating and leveraging that knowledge.  Firms are typically not structured to efficiently leverage the vast knowledge base they have accumulated internally in the form of their employees, especially as they grow in size and stove-pipe various functions and departments.  It can take weeks, if not months, for firms to locate individuals that have specific knowledge sets that can contribute to a particular problem or project (I’ve seen it firsthand, working with various clients on this very issue).

Given that firms are no longer as efficient at collecting, organizing, and tapping into knowledge under their own roof there are three likely (and logical) outcomes:

1. Firms will continue to outsource this function to minds outside of their four walls.  This could take the form of Open Innovation, crowdsourcing, or partnerships with more specialized firms that have developed platforms for locating and taxonomizing expertise and knowledge holders;
2. Firms will invest more heavily in software and technology that will allow them to catalog the knowledge under their roof, making search and location of that knowledge less costly.  Internal social networks are one way, talent software is another.
3. Firms that fail to adapt to this new reality will fail to innovate as effectively as other firms that due adapt, becoming obsolete.

Ana also brought up a simpler question–how do you know what you are looking for before you find it?  To me, social media makes it quite easy to bump into new perspectives in a quasi-directional or directionaless way.  You can use it to casually challenge your own views and find new and interesting ideas, or you can use it to conduct a focused search for specific experts or knowledge communities.

Ana recently wrote an interesting piece about the impact of organizational structure and systems on performance.  In it, she cited a post I wrote on the role of polymaths in innovation.  The gist of my post was that there is evidence that individuals with a broad, diverse knowledge base contribute more to innovation than highly specialized “experts”.  Ana claimed that what is likely more important than polymaths is the connections between them.  On this point I would also agree.

Weak vs. Strong Ties in a Social Network

The discussion has now turned to how you structure an organization to maximize the likelihood of individuals bumping into others with different sets of knowledge. And away we go…

I wrote the other day that, for me, the big payoff of social media is that it both exposes people to perspectives and knowledge-sets that they normally would not come into contact with. This accomplishes two things: 1) it can help destabilize stale ways of thinking and 2) prevent such stasis from setting in going forward. Social media can also help bring individuals together to collectively problem solve and create. Social media can be as active or passive as one wants it to be. It can be engaged so that people “bump” into new and interesting people (case in point, Ana and I just bumped into each other via our blogs–we were not formally introduced or looking for each other or people with our respective skills, etc.), or it can be used to actively organize a diverse group of minds with the goal of solving a complex problem.

So I would say that to encourage innovative thinking we want to use social technologies to structure collaborations between diverse individuals both within and outside of organizations. For each problem, you would want to bring together differing perspectives from within the organization. There are now platforms that facilitate the crowdsourcing of internal expertise (e.g. Imaginatik), not necessarily crowds outside the organization. This would have the added advantage of preventing the kind of social network ossification (where we end up going back to the same people over and over again for insights) that Ana worries about. Additionally, we can and should add an external module to this as well. Having ready access to external experts further increases the possible diversity of opinion to draw from. This can be done by tapping into an expert network (such as Gerson Lehrman Group, the firm I work for) as well as leveraging external crowdsourcing platforms (e.g. Chaordix).

Another idea is to make social media and social technologies mandatory for all workers. You can set up internal networks and have those networks integrate with external networks to maximize the likelihood of “intellectual bumping”.

So there are some initial thoughts. What do you think?

Rather than take questions from analysts directly, Google has asked Wall Street’s finest to submit their questions through a service called Google Moderator, and is only responding to questions that receive the most votes from other analysts. The selected questions are then read out in a studious monotone by a moderator.

This is an interesting idea.  I don’t know a ton about the evolution of the earnings call convention, but I can imagine such a process (like most conventions) has some built-in inefficiencies and Google Moderator may be one way to ensure that the most relevant questions are asked of executives.  However, one part of the process seems highly inefficient:

There is no chance for analysts to put executives on the spot directly or – most importantly – ask follow-up questions when answers are less than complete.

In contrast, Intel bent over backwards to give analysts more time for thoughtful questions this week, by issuing its CFO’s prepared remarks before the conference call and reducing the usual scripted blather to give more time for questions and answers.

I can’t figure out what the point of trying to hone in on the best questions is if you are not going to provide ample opportunity for follow-up.