Wordy Innovation

My company (EMC) just announced the award winning ideas that emerged from our yearly Innovation Showcase.  Global employees submitted 1,590 ideas in the June to August timeframe. In August, teams of judges from sponsoring organizations reviewed these ideas and selected three finalists (per organization).

One of the things I’d like to explore in this post is the concept of “judging bias”. Before the judging process began my team distributed an article by the MIT Sloan School of Management called ‘Is Your Company Choosing the Best Innovation Ideas”

Our advice to the judging teams came straight from the paper:

“Therefore, it’s essential for senior managers to understand the mechanisms at work when their staff evaluates one another’s ideas, so that executives can hone in on the ideas that will make a real difference to the organization as they move through the innovation funnel.”

One of the “bias” categories that judges need to be aware of is “length of submission”.  If an idea is too short, will judges dismiss it by assuming that the inventor didn’t spend enough time?  Or if an idea is too long, will they dismiss it by categorizing it as too complex, or because it takes too much time to understand?  Consider this quote from the InnovationTools.com blog post:

“The length of an innovation proposal has a major influence on whether managers will select the idea for further consideration. The record shows that proposals that are too long or too short do not get the same level of consideration from managers as those that are the “right” length, given the particular environment. The optimal word length in the company I studied was about 250 words; other organizations will have different norms. The fact is, the lower-level managers who screen proposals are busy and have short attention spans. A proposal that is too long may be eliminated from further consideration irrespective of the potential merit of the idea; a long and turgid proposal frequently suffers from lack of focus. However, a skimpy proposal may also be an indication that the submission is in fact insufficiently thought through and fails to answer the basic questions.”

I’ve written several posts highlighting how running analytics over innovation data (such as idea contests) can lead to broader insights into our corporate innovation culture.

So I asked EMC Distinguished Engineer John Cardente to try and generate a report that correlates “length of proposal” with “idea finalist”. This report would answer the question: ” For the 2011 EMC Innovation Showcase, did longer ideas or shorter ideas fair better?”

According to John, the answer is yes. Below are the estimated probability density functions of the proposal description lengths for finalists and non-finalists.

I asked John to explain density graphs:

“The density graph is more or less a histogram with infinitely small bins.  Like a histogram, the bumps and dips indicate how the distribution of proposal lengths varies with size. The density (y axis) is scaled such that the area under each curve sums to 1.

Both the finalist and non-finalist density curves peak below 500 characters. However, the finalist peak is much smaller with the remaining area spread over the larger character counts. This results in the finalist curve being taller than the non-finalist curve at higher character counts indicating that finalist proposals were generally longer.

While the density plot shows that finalist proposals were generally longer, it doesn’t tell us if the difference is significant or just due to chance. That’s the purpose of the t-test, it tells us that the difference statistically significant and not due to chance alone. “

John used R to perform the t-test to determine if the difference is statistically significant. The table below shows the results.

On average, the length of a finalist’s submission was 827 words (much higher than the average of 506 for a non-finalist).  John puts these results in context:

“Since the 95% confidence ranges don’t overlap, it can be said that the description length is a statistically significant difference between finalists and non-finalists. This doesn’t prove causality but it does suggest that longer descriptions are beneficial.”

I personally judged a large number of ideas, and as I look back on the experience I recall that longer submissions tended to stimulate more possibilities for idea incubation.  Shorter submissions left me wondering, and in those cases I often left a comment for the inventor as a way of gaining more context (which may lead to bias when other judges view the idea!).

What can come from displaying this data?  For inventors, this year’s results may lead to longer and more thoughtful submissions in 2012.  For sponsors, this data may cause them to introduce word count limits on idea submissions. For judges, it may serve as a catalyst to do more digging on shorter-length ideas (to make sure that better ideas are not unduly ignored).

Are there related analytics that may suggest other changes?  I am wondering about the correlation between submission date and finalist status.  Do finalists typically submit ideas later in the contest window? If so would this mean that their ideas are more mature (and perhaps longer in length)?

I look forward to covering this more deeply in future posts.

Don’t miss an article (3,650+) – Subscribe to our RSS feed and join our Innovation Excellence group!

Steve Todd is Director at EMC Innovation Network, and a high-tech inventor and book author “Innovate With Global Influence“. An EMC Intrapreneur with over 180 patent applications and billions in product revenue, he writes about innovation on his personal blog, the Information Playground. Twitter: @SteveTodd