The Slow Hunch by Nick Grossman

I've been reading Steven Johnson's new book, Where Good Ideas Come From: The Natural History of Innovation. It's an enjoyable read, in large part because of Steven's ability to draw connections across seemingly unrelated subject areas to tell a unified story. I'm only about halfway through, but in a nutshell: innovation is a natural phenomenon; the basic patterns are the same whether we're talking about coral reef ecosystems, the human brain, urban systems, or technology development. Each chapter draws from examples across biology, psychology, sociology and technology to tell the story. (This approach isn't new for Johnson - one of his prior books, Emergence: The Connected Lives of Ants, Brains, Cities, and Software, tells the story of bottom-up organizing by looking at examples from biology, neurology, urbanism and technology.) There is a chapter in Good Ideas about the "slow hunch": an idea that builds over time, in pieces, often without the explicit awareness of the builder. Part of what makes a slow hunch possible is the cataloging of notes and observations over time, and the ability to look back and draw associations. Many prominent thinkers (Locke, Darwin), accomplished this by keeping a "commonplace book" -- a journal of quotes, observations, sketches, and half-baked ideas -- that they would review regularly. According to Johnson, as important as writing down the notes is re-reading them, in order to make connections that you otherwise might miss:

"Each rereading of the commonplace book becomes a new kind of revelation. You see the evolutionary paths of all your past hunches: the ones that turned out to be red herrings; the ones that turned out to be too obvious to write; even the ones that turned into entire books. But each encounter holds the promise that some long-forgotten hunch will connect in a new way with some emerging obsession."

Later, Johnson talks about his own digital commonplace book, a tool called DEVONthink:

"I keep all these quotes in a database using a program called DEVONthink, where I also store my own writing: chapters, essays, blog posts, notes. By combining my own words with passages from other sources, the collection becomes something more than just a files storage system. It becomes a digital extension of my imperfect memory, an archive of all my old ideas, and the ideas that have influenced me. There are now more than five thousand distinct entries in that database, and more than 3 million words -- sixty books' worth of quotes, fragments, and hunches, all individually captured by me, stored in a single database. Having all that information available at my fingertips is not just a quantitative matter of finding my notes faster. Yes, when I'm trying to track down an article I wrote many years ago, it's now much easier to retrieve. But the qualitative change lies elsewhere: in finding documents that I've forgotten about altogether, finding documents that I didn't know I was looking for. What makes the system truly powerful is the way it fosters private serendipity."

Steven's writing is an example of this in action -- it seems clear now that this system helps him make connections he might have otherwise missed. For example:

"This can create almost lyrical connections between ideas. Several years ago, I was working on a book about cholera in London and queried DEVONthink for information about Victorian sewage systems. Because the software had detected that the word "waste" is often used alongside "sewage," it directed me to a quote that explained the way bones evolved in vertebrate bodies: namely, by repurposing the calcum waste products created by the metabolism of cells. At first glance that might seems like an errant result, but it sent me off on a long and fruitful tangent into the way complex systems -- whether cities or bodies -- find productive uses for the waste they create. That idea became a central organizing theme for one of the chapters in the cholera book."

So: it appears that re-reading your old notes, and ideally doing so with some amount of fuzz and randomization, can lead to the development of new ideas. I like it. I certainly spend a lot of time capturing notes, but my process for reviewing them is scattershot: I'll read back through old tweets & tumbls, occasionally search through my delicious bookmarks (though not that often), and sometimes scan an old notebook, but I don't have a good way of tying it all together. But, unlike Steven, I don't keep all of my notes in a single database. I use the web, and my notes are everywhere -- in blog posts I've written, tweets I've sent, bookmarks I've tagged, comments I've left, and (increasingly) in my Tumblog. I do it this way because there is value in building these ideas together with others -- I want people to read my tweets, tumbls, blog posts, and bookmarks, and connect them with their own. I could just as easily keep all of these tidbits locked up on my computer, but I believe that there's more value in opening them up. So, tools like DEVONthink (and Evernote), that assume you are using their system in isolation, aren't much help for me. I do keep private notes, but those aren't in a single system either -- they're scattered across the internet and my computer. I have hundreds of notes in Notational Velocity (a SimpleNote client for Mac -- before that I used Yojimbo for the same purpose), and I keep a journal in the form of a private wordpress blog (which serves the same purpose as OhLife would), and I have other notes in google docs and etherpads. So, what I want is a tool that helps me review across all of these sources, brings them together in a nice way, makes searching easy, and allows for some amount of fuzziness and random discovery. Basically an "open" commonplace book that understands my distributed landscape of knowledge bits. A commonplace book that works like the web. My ideal version of this tool would run locally on my computer, mostly so that it could access local files, and also for speed. The idea is something like this:

I could imagine an advanced searching interface, but for day-to-day, I'd really just want something simple and fast that integrates into my workflow (the way that Quicksilver does) without feeling like a big heavy app. There are certainly neighboring precedents in this general area of tools that organize my web-based presence. Some of my favorites are TripIt (on the left, below) which takes my travel info and repackages it in a way that's useful to me, and Momento (on the right), which takes a a bag of feeds and turns it into a nice timeline view:

Momento is maybe the closest to what I'm talking about, in that it understands that my experience is distributed, and that it's value is tying it together for me. It's serving a different purpose than the one I'm talking about, but it is in the right family. I could easily see Evernote moving in this direction, and I would be psyched if they did, but I think it would be a lighter lift to just start with simple aggregating and searching, and not worry about all of the content capture that Evernote handles. So, that's the idea. After all this, I would be pleasantly surprised to see that this already exists, but if I does, I haven't seen it yet.

"Each rereading of the commonplace book becomes a new kind of revelation. You see the evolutionary paths of all your past hunches: the ones that turned out to be red herrings; the ones that turned out to be too obvious to write; even the ones that turned into entire books. But each encounter holds the promise that some long-forgotten hunch will connect in a new way with some emerging obsession."

Later, Johnson talks about his own digital commonplace book, a tool called DEVONthink:

"I keep all these quotes in a database using a program called DEVONthink, where I also store my own writing: chapters, essays, blog posts, notes. By combining my own words with passages from other sources, the collection becomes something more than just a files storage system. It becomes a digital extension of my imperfect memory, an archive of all my old ideas, and the ideas that have influenced me. There are now more than five thousand distinct entries in that database, and more than 3 million words -- sixty books' worth of quotes, fragments, and hunches, all individually captured by me, stored in a single database. Having all that information available at my fingertips is not just a quantitative matter of finding my notes faster. Yes, when I'm trying to track down an article I wrote many years ago, it's now much easier to retrieve. But the qualitative change lies elsewhere: in finding documents that I've forgotten about altogether, finding documents that I didn't know I was looking for. What makes the system truly powerful is the way it fosters private serendipity."

Steven's writing is an example of this in action -- it seems clear now that this system helps him make connections he might have otherwise missed. For example:

"This can create almost lyrical connections between ideas. Several years ago, I was working on a book about cholera in London and queried DEVONthink for information about Victorian sewage systems. Because the software had detected that the word "waste" is often used alongside "sewage," it directed me to a quote that explained the way bones evolved in vertebrate bodies: namely, by repurposing the calcum waste products created by the metabolism of cells. At first glance that might seems like an errant result, but it sent me off on a long and fruitful tangent into the way complex systems -- whether cities or bodies -- find productive uses for the waste they create. That idea became a central organizing theme for one of the chapters in the cholera book."

The Slow Hunch by Nick Grossman

The Slow Hunch by Nick Grossman

The Slow Hunch by Nick Grossman

The Slow Hunch by Nick Grossman

CityScience

CityScience

Wanted: An Open Commonplace Book

Wanted: Strategic Networking Tool

Wanted: An Open Commonplace Book

Wanted: Strategic Networking Tool