Posts Tagged ‘open data’



In Hamilton,Ruby on December 4, 2013 by Matt Grande Tagged: , , , ,

Over the past couple days, in the evenings, I’ve been working on a small project.

Frequent readers of my blog (ha!) know that I enjoy map making and things of that nature. I decided to do a more local project, this time, using Hamilton’s transit data.

All the routes

The script I’ve created takes a transit schedule (in GTFS format) and either places all the routes on one image, or creates thousands of images and turns them into frames in a video.

Here’s a quick overview of how it works:

  1. Determine “significant” trips (in this case, I’m only looking at weekday trips).
  2. Generate a list of stops with their times and latitudinal / longitudinal co-odrdinates.
  3. Convert the lat/long into x/y.
  4. Plot to an image.

If you want to take a deeper look into the code, here it is on github. If you want to see the image again, feel free to click here. If you’re all about the videos, maybe give this link a click?



All the roads in Canada

In .Net on May 6, 2013 by Matt Grande Tagged: , , ,

If you’re just here for the image, click here (10638×9101, 1.17MB). If you want more information, please read on!

I recently saw a map of Oregon containing only the roads, and nothing more.  I thought it was really interesting how you could see where the population centres and natural features (lakes, mountains, etc) were.  I decided to try something with Canada.

The first challenge was finding a dataset containing all the roads in Canada. Luckily, it was on StatsCanada’s website, but it took longer than expected to find.

I opted for the GML format, since it’s the one I’m most familiar with.  The full file is a fairly large 1.9GB.  Here’s a sample of one road, Wilcox Street in Hamilton, Ontario:

<fme:lrnf000r12g_e gml:id="id00987e75-8fb2-4224-8e82-f2b97dd86307">
<gml:LineString srsName="_FME_0" srsDimension="2">
<gml:posList>7197110.1 878145.377099998 7197136.3143 878156.274300002</gml:posList>

The most important piece of information in there, for my purposes, is the <gml:posList> tag.  In this case, the data is stored as pairs of Eastings & Northings.  This means that for the above road, it travels from 7197,110 metres East and 878,145 metres North of the origin to 7,197,136 metres East and 878,156 metres North of the origin.

I decided to write a script to extract only the data I needed.  Essentially:

if (line.StartsWith("<gml:posList>"))

This brought the file size from 1.9GB to 448MB, and the number of lines from 49,352,141 to 1,983,127.

With this smaller file, I could parse each data point to find the boundries.  This helped me determine the file size to use.  I used a ratio of 500 metres per pixel which, I felt, gave a good balance between detail and file size (although it is quite large, still).

Now that I had all the boundaries and all the points, I was able to plot them on an image using .Net’s graphics library.  A few dozen lines of code, and a few seconds of processing, and I had my map.

Code will be relased on my BitBucket shortly. Stay tuned!