Saturday, January 25, 2014

Transit homebrew busts out in 2013

There was a terrific article in The Atlantic Cities by Emily Badger that described the phenomenon of how technology changes our perception of waiting for the bus. Emily outlines the evidence that not only does the psychology of waiting change for riders, but simply having the real-time data for arrivals is more important than the frequency of the bus. Simply knowing when to get to the stop to minimize your wait time and having the confidence you haven’t missed your bus has a dramatic impact on ridership and rider satisfaction.

I’m thrilled to have played a big part in bringing this to Madison with the Madison transit API. What started out as a weekend hack three years ago has become a core Madison service.

Hitting the two million mark

2013 was the year the API entrenched itself as a pillar in the Madison transit system and I’m blown away by it. With very little overhead or attention from me, here’s what happened in the last twelve months.
  • 11 new developers for a total of 48
  • API traffic went from 100,000 requests/week to 2,000,000 requests/week

Apps released by developers this year


Watch the Transit API visualized in real time.

Saturday, January 11, 2014

Communication and your identity

photo courtesy of Google

Google ruffled a lot of feathers in the privileged tech world this week with their new Gmail integration in Google+. It seems like an obvious integration to me. I just wish they’d market the integration differently.

The email address is a pretty arbitrary endpoint for communication. If I want to share my profile publicly on a social network, why wouldn’t I want someone to be able to reach out to me? How and when or if I chose to respond is up me. That’s a feature that belongs in my inbox - whether it is gmail, Yahoo or US Postal.

My email address is not my identity.

It’s like tying your phone number to a phone carrier. Why should a phone number be forced to route itself to one physical phone? I should be able to route a phone call to any device or service that I want.

The Facebook team has executed this brilliantly with their messaging interface. One identity, multiple interfaces - email, chat and SMS.

So stop beating up on Google and get ready for more integration. They’re doing the obvious. If you don’t like it, perhaps you should reevaluate your public profile.

Sunday, June 2, 2013

Celebrating National Day of Civic Hacking

This weekend was National Day of Civic Hacking. An incredibly well organized, distributed and disconnected network of events across the country where people came together to build tools and policies to make their communities better.

I love this type of work. It's the inspiration behind SMSMyBus, the underlying API that powers 40+ transit applications, and our local civic hacking organization, HackingMadison.

Although we didn't get an event organized for Madison, I wanted to contribute something. I had a few hours each morning this weekend to mash my keyboard on a project so I picked something small. There has always been a lot of interest around the data in the transit API so I decided to expose some of it by visualizing the API requests in real-time.

The browser app displays a Google Map and drops red circles on the bus stop locations as API requests come in for those stops. Fun. Simple. Transparent.

http://api.smsmybus.com/map


Sunday, March 24, 2013

Life should be a bowl of ice cream


A public service announcement for my peers... Enjoy your parents. Spend time with them like it's your last.

My dad is visiting this weekend and we're not doing anything important. We watch basketball. Drink beer. Compare career notes. Turn society's debates into math problems. Tour the city like tourists. Run errands. Sit in silence. And eat ice cream.

It's wonderful.

The truth is, these visits are numbered. His mobility is decreasing and his mental acumen is dulling. Enough to make everyone quite sad if we stopped to focus on it. If we stopped to stress on any of the messy, strained bits of families that weave into the fabric of life, we would miss opportunities like this weekend.

But we don't. And that is the secret sauce of the wonderful.

These moments have been unfortunately aided by tragic events. The loss of a sister and daughter and premature mortality that Parkinson's brings changes your lens.

That's the tragedy behind the tragedy. What if you didn't need to have your lens prescribed?

That's your challenge. Look passed the messy stuff. Scoop your dad ice cream. Sit back. And enjoy the simple part of life.


Sunday, November 18, 2012

Goodbye Posterous. Hello Blogger.

I've been hosting my blog on Posterous since I started writing in 2008. Earlier in the year, they sold the business to Twitter with no guarantees that the service would be around long term.

I kept waiting for them to offer a tool to extract/port content and it hasn't arrived. So I decided to scratch an itch and do it myself. I ported my blog to Blogger this weekend and here's how I did it.

Problem scope

The blog content is the easy(ish) part, but if you want to preserve the links to the original content, it can be tricky. Depending on your search engine and social media traffic, those thinks can be valuable.

First, every blogging platform uses its own slug path structure. Second, if your blog is in a subdomain, you have to find a way to preserve that subdomain with the addresses on the new blogging platform. For example, the parking API post has two different URIs.

Posterous :

www.gregtracy.com/adding-parking-to-the-madison-api-homebrew

Blogger :

blog.gregtracy.com/2012/01/adding-parking-to-madison-api-homebrew.html

Migrating content

Content migration is a tedious process. But the current option for getting Posterous content to Blogger is well documented on the web.
  1. Import Posterous content into Wordpress 
  2. Export Wordpress to an XML file
  3. Use wordpress2blogger to transform the XML file for Blogger import
  4. Import XML file into Blogger
This flow is documented in detail here. Lifehacker has also documented a process for migrating content using the auto-post feature inside Posterous, but it is tedious.

It doesn't necessarily end there, however. There are two glaring problems that make this process linger on forever. First, the content is poorly formatted so you'll find yourself editing posts in Blogger to clean up formatting. Second, multimedia sometimes doesn't import correctly or references files on Posterous or Wordpress so you need to upload that content to Blogger.

Someone will eventually build a migrator by mashing up the Posterous API with the Blogger API to make this all go away. In a different life, I might have taken the time to do it. But until someone does, the content migration is a little messy.

Building the redirector

This was the piece I was most interested in because it's simple and fun to build. 
  1. Use the Posterous API to get a list of all post URI slugs and their post dates
  2. Construct a slug map for Posterous URIs to Blogger URIs utilizing the dates and slugs
  3. Build an app that takes any Posterous URI slug and redirects the page to the Blogger URI
  4. Setup DNS so your old blog points to your app

Posterous listing

I wrote the following nodejs app to grab the full list of public posts from Posterous.


Blogger listing

If you can get access to the Blogger API, which requires a special request to the Blogger team, you can do the same thing I did with Posterous. The challenge is finding the ways Blogger changes slugs. For example, they remove "the" and "a" from most post slugs. 

Finding these differences is tedious so I actually recommend making sure you get the mapping correct for your top ten blog posts and then just crowd source the rest. Keep track of the page misses with logging (see below for my solution) and then update the map as you find mistakes. 

In the end, the map looks something like this:

{ 'awesome-post-title-wins : { date : '/2012/11/' , blogger_slug : 'awesome-post-title' }

With those details, you can get from any legacy slug - /awesome-post-title-wins - to any Blogger URI - /2012/11/awesome-post-title

Redirector application

I chose to use Google App Engine to host my redirector, but you could use anything. The logic is very simple. Just lookup the inbound request in your slug map and redirect to the Blogger URI. It looks a lot like the following.


My full implementation, including the slug map can be found on github. My implementation also includes a miss tracker which is what I've used for understanding which redirects are failing. I'm persisting the routes and a counter for the number of occurrences to understand what's missing. I can find all of the misses using the datastore viewer in the App Engine dashboard.

The incidentals

There are a couple of miscellaneous resources that you'll need to add to your map file as well. 
  • RSS feed
  • favicon.ico
  • robots.txt
  • Apple touch icons
  • Posterous "tags" (Blogger "labels")
All of those can be redirected, but creating static files for your favicon and apple touch icons allow you to personalize that content better on your new domain.

Domain mapping

Now that you have an application that redirects content to Blogger, you need to map that application to your old domain. In my case, I updated the DNS for www.gregtracy.com to point to my app. The combination of App Engine and Google Apps makes this very easy.

Inside your App Engine dashboard, select "Application Settings" and use "Add Domain" to map the application to your subdomain. These steps walk you all the way through DNS setup.

That's it. What details did I miss?  http://blog.gregtracy.com

Monday, July 16, 2012

Cancer stole one of the good ones today

Around 1,500 people die from cancer every day.
 
But today that number feels different. Today, my own sister became a statistic.


Marla Tracy Pak was diagnosed with glioblastoma multiforme last August. Fought an ugly and losing battle for 10 months, and left us today, leaving a big void in the TracyPak Tribe.

I was asked to do a PechKucha talk the day I got the news last August. I called an audible and instead of talking about civic hacking, I talked about human qualities that make families, tribes and communities better. The qualities that my sister had.

I neglected to share that presentation last year, but here are the highlights...

  1. Be kind to every person you interact with and talk to strangers like you’ve met them before.

  2. Have empathy.

  3. Support the things your friends are passionate about regardless of your opinion.

  4. Celebrate nostalgia within your tribe.

  5. Spoil your nieces, nephews and neighbors’ kids.

  6. Converse with others with infinite optimism in your voice.

  7. Send surprise packages to people.

  8. Call an out of town friend or sibling and setup a long-weekend visit just because you can.

  9. Make a choice to relate to people rather than drawing contrasts.

Success in your personal relationships defines who you are. We should all be a little bit more like Marla.


Saturday, February 18, 2012

Spam blockers and colors

The Astronomy Picture of the Day email service I created several years ago has been getting pummeled by spam critters. I was getting up to five signups a day with bogus email addresses. Most of them came from Hotmail, AOL and Yahoo. With the new pricing structure of Google App Engine, it was beginning to cost me in the pocketbook so I had to address it.

The solution I came up with was part spam filter and part fundraiser. I decided to draw more attention to the PayPal donate button by requiring new signups to type the color of the button into one of the form fields. A couple of interesting things happened.

First, the rate of donations didn't change. I've had one modest donation in two weeks. 

Second, the spam ended. Overnight. It was fantastic.

Most interestingly, the little test showed fun insight into how we all see colors. When I originally wrote the test, the answer was "yellow". At the very last minute I decided that I should throw in "orange" as a possible answer as well and it was a good thing I did. Since deploying that change 15 days ago, there have been 51 signups - 26 orange and 25 yellow. I don't know why, but his fascinates me. It seems obvious to me that it is yellow. :)

What color do you see?