The potential of accidental insights.
The history of innovation is full of examples where seemingly random or unwanted experimental outcomes lead to incalculably valuable discoveries. Seemingly random spikes in innovation often derive from a single unexpected outcome - but one that is observed by a curious researcher. The field of medicine and chemistry are famous for this kind of discovery. Viagra, for example, came from an unexpected side-effect during testing of an anti-hypertension drug. Penicillin famously came from a failed experiment by Alexander Fleming. The pacemaker was conceived after the inventor accidentally installed the wrong part in a heart-monitor he was working on, and created a tiny device that gave off a pulse similar to a heartbeat.
Medicine isn't the only field where these things happen. The microwave oven was developed after an inventor discovered his candy bar was melting faster than it should when he was working near his equipment. Velcro was developed after George de Mestral decided to study the little plant burrs that stuck to his clothes instead of just throwing them away. Corn flakes were first created by an accidental overcooking of a batch of health food. Safety glass was invented after Edouard Benedictus dropped a flask with sticky material inside it and observed it didn't shatter. Dry cleaning, popsicles, Vaseline, Botox - the list could go on and on. Admittedly many inventions and innovations are conceived of on purpose by people seeking what science writer Steven Johnson calls the adjacent next, but a substantial number also come from observing the overlooked or unexpected outcome.
One of the most illustrative of this pattern comes from events that happened in 1850s England. The anti-malarial drug quinine had been isolated from the bark of a Peruvian tree in 1820 and once it was chemically identified, the race to find a way to produce it at industrial scale began. It would take a while - the first formally announced successful synthesis wouldn't happen until 1944. But the search led to many discoveries, one of which had surprising repercussions.
William Henry Perkins was a clever young man who made his way into the relatively new Royal College of Chemistry in 1853. Chemistry was still very new as a formal science. It's worth remembering that for all the amazing mathematical and scientific insights discovered a century before by Sir Isaac Newton, he probably spent more time studying alchemy than physics.
Perkins began his study under the guidance of a German professor, August Wilhelm von Hofmann. Von Hofmann believed that organic chemistry had the potential to synthesize in the laboratory things that normally only came from plants. This concept of laboratory synthesis of natural compounds had only recently been proven true with the synthesis of the nitrogen-rich fertilizer compound urea in the late 1820s.
Synthesizing quinine would have obvious therapeutic medicinal value, and in the expanding British empire its accomplishment would be a reliable path to fame and probable financial success. Von Hofmann's quest became of great interest to young William Henry Perkins.
In 1856, Perkins was only 18 and back at home on Easter vacation when he tried some experiments in a rudimentary home laboratory towards the goal of synthesizing quinine. The experiment did not work and just left a sticky brown mess of organic compounds.
It was during the process of cleaning up this messy failed experiment that Perkins made an important observation and discovery. When he added alcohol to the preparation to clean up, he saw something that was relatively unusual at the time: the color purple.
The reason purple was an uncommon color - aside from flowers and such - is that the most effective source of the color as a pigment for dyes was the gland of a particular sea-snail (bolinus brandaris). It took thousands of these snails to produce a gram of the dye pigment known as Tyrian purple. The dye creation process was smelly, labor intensive, and therefore expensive. The result was a scarcity-driven market that made the price of the dye so expensive that only the very wealthy and powerful could afford the color. The association of purple to royalty is directly tied to this economic reality. This had been the situation for more than a thousand years.
This brings us back to Perkins. He could have ignored the colored gunk on his lab equipment, but he knew chemistry and he knew the importance of industrial fabric dyes. He was living in a time when the economic power of the industrialized world was a dominant concern in England and especially within his own school of chemistry.
He hadn't synthesized quinine, but he had discovered the first synthetic dye capable of industrial-scale use. It's a fascinating story, but not one I can detail in this article. It does need to be said that this finding and its efficacy in dying cloth gives us the color known as mauve, and led to many other discoveries both in England and in Germany with an explosion of inexpensive new colors. These colors changed the hue of the European fashion world, and would affect painting and art as well.
Recall that Perkins was working on finding a medical breakthrough. His work would eventually lead to a very, very important one but it would take place in Germany decades later. Let's hopscotch through the steps to this next bit of serendipity:
- In the early 1870s, Italian chemist Camillo Golgi dyes brain tissue with the same chemical used in photographic plates, silver-nitrate. The nerve tissue was notoriously difficult to study before this, because it looked too similar to surrounding tissue under the microscope. Histological staining is born.
- In the late 1870s, German chemist Karl Weigert begins using industrial produced dyes to stain biological samples.
- At the same time, Weigert's cousin, Paul Ehrlich, was also studying the process of dying biological samples. He notes that in some cases, bacteria will take up the dye while the surrounding tissue does not.
- In 1883 Ehrlich marries Hedwig Pinkus. At some point in the next few years, one cold day, she lights up a wood burning stove to make her husband more comfortable. His staining samples are affected by this and he notices that both the speed and efficacy of the process is improved by the heat. Serendipity again.
- Ehrlich's work was foundational to the development of chemotherapy. He spent much of the rest of his life working on a search for dyes and drugs that would target specific diseases and ignore the rest of the host body. His work would produce the first widely used medicine for the treatment of syphilis before the discovery of the antibiotic penicillin in the 1940s. (Note: The discovery of penicillin is also a story of serendipity.)
- In 1908 Ehrlich would receive the Nobel prize in medicine for his work on immunology.
Today, we still have very few examples of what Ehrlich truly wanted - a "magic bullet" for treating disease. But the search goes on.
While I find researching the history of medical breakthroughs and their connections to one another a fascinating subject, there are some emergent lessons that are applicable to broader applications outside that specific field.
Observation
These breakthroughs began when the results of the experiments was not what was expected. It so easy and so common to find results one doesn't expect and to discard them as useless. We can't know how many important discoveries were delayed by not noticing the value of an unexpected outcome, but statistically it must be a vast number.
Often when we're pressed for time we can't ponder the significance of such outcomes, but some curious portion of us have taken note and explored such findings. This has been a critical vector for discovery.
Lesson: If you get weird outcomes, even if you don't have time at the moment to figure out why, it may be useful to at least take note of the strangeness and discuss it with colleagues.
Insight
When you discover something new or unexpected you still have to determine if it is useful or not. Sometimes the new insight may not be of use to you directly, but may be of use to other individuals or groups. In the case of the synthetic dye mauve, Perkins knew the value of dyes and saw an opportunity to develop a new business (and a new field of chemistry) through this insight. His work was not in fabric but he had developed cross-domain awareness. It is because of the value of unexpected cross-domain insights that I always try and emphasize the utility of learning from fields outside of my own interest and expertise. This has been one of the functions of the popular TED Talks symposiums. I don't think every talk necessarily provides astonishing insight you'll find applicable, but because of the breadth of their scope it is highly likely that consuming them may inform you of potential new insights you weren't expecting.
The filtering process of separating useless outcomes from insights is difficult to teach. It requires a body of experience and discernment, but don't think that there aren't vast domains of ignorance waiting to be conquered. New insights in computing, physics, medicine and many other "mature" fields do remain.
Your own opportunities for such discovery may not be world-changing, but even small process changes can have massive impact on productivity if one discovers them and also capitalizes on them.
Exploitation
It is not enough to discover and understand a new phenomenon. You have to be able to make use of it. This is at least one reason why innovation is thwarted. The pathway to successfully making use of a new discovery looks like this:
Observation > Insight > Exploitation
But there are barriers to exploiting a new discovery.
Inertia is one of the most common barriers to change in established business. Such changes might threaten how work is done and the status quo often has entrenched guards. People usually don't want to have their jobs or ways of doing business disrupted even if the outcome is to their benefit financially.
Funding is almost always an obstacle to change. Unless you've got control of the budget, you're likely going to have to convince some people that your new insight is fiscally justified. Sometimes people work for years to get their good idea implemented, and then see it called "an overnight success."
Competition will also be paying attention and trying to implement your best ideas for themselves. While patents and copyright laws might protect some kinds of innovation, others are more challenging and require a constant vigilant search for the next big insight to nudge your own business to better and more optimal outcomes.
Sales can also be an obstacle. Sometimes people have to be shown how you're meeting a need that they may not even know they had. Even having the best technology may not guarantee your success in the marketplace. A good sales and marketing team will understand your innovation and help make it out into the marketplace.
Conclusion
I really like the word stochastic. It describes a process with repeating but random and unpredictable patterns. If one keeps on the lookout for new insights, the potential for stochastic innovation is always going to emerge. Be vigilant and ready to make the most of it.
It is also worth noting that the so-called "serendipity" at play in these historical narratives all depended on many people doing a lot of work. We have names here for "the hero inventors" but they all worked together as a community, sharing their work, arguing over implementation, and slowly and collectively pushing science's domains of expertise farther down the line towards a better understanding of the natural world. Praising, rewarding, and remembering pioneers is a very human thing to do, but each of these "heroes" are merely avatars of an ecosystem of innovation built on the work of thousands of contributors.
Addendum: Another sort of knock-on effect of Ehrlich's research came when people started making "elixirs" from the compound his work had produced. A product called "Elixir Sulfanilamide" was created as a sort of multi-purpose cure-all by American S. E. Massengill through the work of his head chemist, Harold Watkins. The Massengill formulation included diethylene glycol to turn the bitter powder into a drinkable solution. Unfortunately, they did not perform adequate testing and it turned out to be a lethal drug for more than 100 victims across the country, many of them children. The resulting scandal led to the creation of the United States' Food & Drug Administration.
