Grant Bourque

My face

Read, write, delete, repeat.

Feed GitHub Profile LinkedIn Profile

23 March 2020

Personal project: crime map

Each time the weekly crime blotter popped up in my news feed, I found it difficult to interpret.

Without knowing every street very well, how do I know what’s happening near me and where clusters of incidents are occurring? It seemed like a map would help visualize the data and that would be an interesting project. For a college class I developed a desktop GUI application to plot real-time IP addresses on a world map, but I hadn’t worked with maps in the browser before and I wanted that in my repertoire.

I started on processing the article to make the data easier to work with. While the article’s text is structured and uses a predictable format, it is not exactly machine-readable in the sense of automatically obtaining a list of objects in a programming language. I used Ruby and RSpec to do test-driven development, starting with a small sample of the article as my input and deciding what I wanted the resulting data structure to look like. I figured eventually this data would be used in JavaScript, so an array of hashes that can be converted to JSON was a safe bet. With that goal now backed by a test, I wrote more tests and methods to do smaller sub-tasks such as dividing the input up by categories and extracting attributes from the incident lines. Regular expression capture groups get the job done!

With a few web searches, I decided to use Leaflet for the map frontend. They have an excellent quick start guide and once I obtained a free API key from Mapbox I had a working map on a page! From there I tweaked my viewbox and experimented with adding markers.

I realized that to add my data to the map, I would need to convert the addresses into geographic coordinates. For this task, I used the Nominatim API. I created a new Ruby class responsible for adding the geocode data to my data set. Since my data set was relatively small (~200 addresses) and a one-time conversion is all I needed, I felt that using the publicly hosted API with a self-imposed request throttle would meet their usage guidelines. In my Ruby script I directed the output of the parsed incidents into the geocoder class and wrote the resulting data set to a JSON file. Only two addresses in my data set failed to geocode and I manually added coordinates for those using Google Maps, which gives you the latitude and longitude when you click a spot on the map.

Now I had the map and machine-readable data to add to the map! For security reasons, browsers do not allow direct access to a local file in JavaScript. I could have hosted the file on a local server or implemented a file input on the page, but I did something much simpler. I pasted the JSON file contents into the JS code and assigned it to a variable. The freedom of personal project prototypes is great.

Iterating over the data to add markers with associated pop-ups code gave me the desired results: Screenshot of my crime map I found semi-transparent circles to be better than typical map pin markers because the data uses blocks instead of specific addresses and the compounding opacity indicates that there are multiple reports on the same block.

While it is not a fully automated setup or a polished frontend experience, I was satisfied with the development and how it turned out. I later found out that Open Data BR has their own crime map with more features and a full year’s worth of data anyway!