Open data is a huge driver of innovation.  Traveling around NYC is better because the MTA opens up route, schedule and real-time data for people to build apps with.  Responding to natural disasters is easier when data is open and interoperable.  As we continue to collect more data about ourselves and our environments, from how we learn to how we sleep, to how much energy we use, the possibilities for building on top of it are literally endless.

Most of the work I’ve done in open data has had to do with the government — getting the government to “think like a platform” and open up data.  There is always resistance — opening data means relinquishing control and handing over power.  But, generally speaking, “government” has become more and more open to these ideas (for example, see the US recently issued a new open data executive order [PDF]).

But of course, there is a whole other side of open data.  The data that we produce, and companies collect, as we traverse the web and the world.  The times ran an article this weekend on the paradox of personal data: companies collect data about you, use it internally and re-sell it to marketers, but most times they won’t give it back to you:

“OUR mobile carriers know our locations: where our phones travel during working hours and leisure time, where they reside overnight when we sleep. Verizon Wireless even sells demographic profiles of customer groups — including ZIP codes for where they “live, work, shop and more” — to marketers. But when I called my wireless providers, Verizon and T-Mobile, last week in search of data on my comings and goings, call-center agents told me that their companies didn’t share customers’ own location logs with them without a subpoena.”

I fully expect old, stodgy companies like Verizon and National Grid not to get this.  And it makes sense that big internet companies like Google and Facebook are afraid of giving users access to “what they have on them”  

It’s a fair question who “owns” data that users create (directly and indirectly) as they use web and mobile services.  On the one hand, there would be no data if the users didn’t do the things that produce it!  On the other hand, platforms and companies invest huge sums to architect the experiences that make these activities (and by extension, the act of collecting the data) possible.  That’s true whether you’re Verizon, Google or Foursquare — there would similarly be no data if the platforms didn’t exist and didn’t build systems to collect it.  So there is a shared interest here.

But it seems like a huge, huge opportunity for emerging companies that are gathering user data and haven’t yet built sketchy complex business models monetizing that data.  Giving users open access to that data — in the form of APIs — would not only benefit users by letting them do more with their data, it would also build trust.  I am way more likely to entrust my data with a company when I know they are going to share it back with me openly.

Of course, opening up this data carries risks — security (privacy breaches, identity theft) and the potential for users to leave your service more easily among them.  But my gut is that the benefits will outweigh the risks and platforms that embrace user access to data become more valuable and more beloved for it.

At Albert’s suggestion, I’ve started reading Nicholas Taleb’s Antifragile.  

It really is a powerful idea to think of things that get better with stress, not worse.  And it’s really interesting to think about how to build antifragility into your life in various ways.

Here’s one (tiny) example: email.  I hate email and I am bad at it. I am constantly in email debt.

It gets worse when I get busy and when I travel a lot.  Things like conferences give me an excuse to ignore email for a day or two, so I do.

But what if there was a way to get better at email, the busier you are?

One way of achieving that is to do email by phone only.  High turnover, limited depth.  The limitations on typing speed on the phone actually make it easier to do more email, not harder (in many cases, not all).

Anyway, that’s just one tiny example.  Will keep thinking about more.

I’ve been working on an internal web app for the past few weeks.

I’m not a “real” programmer, so as usual I just figure things out as I go along and make lots of mistakes, but I always learn new stuff.  This is how I learned programming in the first place, got my first gigs building websites, then ended up at OpenPlans, Code for America and now USV.  My favorite thing about the web is how it makes it possible to just pick a direction and learn as you go.  It’s amazing, really.

My background is that I’m good with front-end development (html/css/js) and decent with PHP (have built a ton of wordpress websites, plugins, themes), but I quickly trail off from there.   So I’m very comfortable in some places, and not so much in others (linux admin, deploying python apps, etc).

But for this project, I decided to try something new.  I’m using

Open data is a huge driver of innovation.  Traveling around NYC is better because the MTA opens up route, schedule and real-time data for people to build apps with.  Responding to natural disasters is easier when data is open and interoperable.  As we continue to collect more data about ourselves and our environments, from how we learn to how we sleep, to how much energy we use, the possibilities for building on top of it are literally endless.

Most of the work I’ve done in open data has had to do with the government — getting the government to “think like a platform” and open up data.  There is always resistance — opening data means relinquishing control and handing over power.  But, generally speaking, “government” has become more and more open to these ideas (for example, see the US recently issued a new open data executive order [PDF]).

But of course, there is a whole other side of open data.  The data that we produce, and companies collect, as we traverse the web and the world.  The times ran an article this weekend on the paradox of personal data: companies collect data about you, use it internally and re-sell it to marketers, but most times they won’t give it back to you:

“OUR mobile carriers know our locations: where our phones travel during working hours and leisure time, where they reside overnight when we sleep. Verizon Wireless even sells demographic profiles of customer groups — including ZIP codes for where they “live, work, shop and more” — to marketers. But when I called my wireless providers, Verizon and T-Mobile, last week in search of data on my comings and goings, call-center agents told me that their companies didn’t share customers’ own location logs with them without a subpoena.”

I fully expect old, stodgy companies like Verizon and National Grid not to get this.  And it makes sense that big internet companies like Google and Facebook are afraid of giving users access to “what they have on them”  

It’s a fair question who “owns” data that users create (directly and indirectly) as they use web and mobile services.  On the one hand, there would be no data if the users didn’t do the things that produce it!  On the other hand, platforms and companies invest huge sums to architect the experiences that make these activities (and by extension, the act of collecting the data) possible.  That’s true whether you’re Verizon, Google or Foursquare — there would similarly be no data if the platforms didn’t exist and didn’t build systems to collect it.  So there is a shared interest here.

But it seems like a huge, huge opportunity for emerging companies that are gathering user data and haven’t yet built sketchy complex business models monetizing that data.  Giving users open access to that data — in the form of APIs — would not only benefit users by letting them do more with their data, it would also build trust.  I am way more likely to entrust my data with a company when I know they are going to share it back with me openly.

Of course, opening up this data carries risks — security (privacy breaches, identity theft) and the potential for users to leave your service more easily among them.  But my gut is that the benefits will outweigh the risks and platforms that embrace user access to data become more valuable and more beloved for it.

At Albert’s suggestion, I’ve started reading Nicholas Taleb’s Antifragile.  

It really is a powerful idea to think of things that get better with stress, not worse.  And it’s really interesting to think about how to build antifragility into your life in various ways.

Here’s one (tiny) example: email.  I hate email and I am bad at it. I am constantly in email debt.

It gets worse when I get busy and when I travel a lot.  Things like conferences give me an excuse to ignore email for a day or two, so I do.

But what if there was a way to get better at email, the busier you are?

One way of achieving that is to do email by phone only.  High turnover, limited depth.  The limitations on typing speed on the phone actually make it easier to do more email, not harder (in many cases, not all).

Anyway, that’s just one tiny example.  Will keep thinking about more.

I’ve been working on an internal web app for the past few weeks.

I’m not a “real” programmer, so as usual I just figure things out as I go along and make lots of mistakes, but I always learn new stuff.  This is how I learned programming in the first place, got my first gigs building websites, then ended up at OpenPlans, Code for America and now USV.  My favorite thing about the web is how it makes it possible to just pick a direction and learn as you go.  It’s amazing, really.

My background is that I’m good with front-end development (html/css/js) and decent with PHP (have built a ton of wordpress websites, plugins, themes), but I quickly trail off from there.   So I’m very comfortable in some places, and not so much in others (linux admin, deploying python apps, etc).

But for this project, I decided to try something new.  I’m using