Horse with a Pointy Hat

Musings of a data scientist and recovering astrophysicist

How to run a half-marathon in 10 lines of python

I got into running a number of years ago and one of the ways I've always found productive to get me out and make sure I get a bit of exercise is to enter races. That way I know I better make an effort to train or I'm going to look like an idiot on race day. A couple of months ago I was registered to run in the ABP Southampton Half Marathon. A couple of years ago there was a feature that allowed you to register your social media accounts and then they would tweet/post your progress out as you ran. This wasn't available in 2016 and didn't seem to be available this either so I decided to engineer my own solution.

Read more…

Being a data scientist in a “post-truth”, “alternative fact” world

This is a reposting of a guest piece I did for the Pivigo S2DS blog

Regardless of where on the political spectrum you fall I think everyone would agree that there has been something of a shake-up in the political landscape in the past twelve months with both the Brexit referendum result and the election of Donald Trump to the US Presidency. Many pundits have remarked that these were revolts against the "establishment" and there are potentially more quakes to come in 2017. In the course of recent political events we have borne witness to the rise of the "post-truth" era with its "alternative facts"; the expression that people have "had enough of experts". In a time when data is considered the "oil of the 21st century" and that technology and data science is changing the way we work and live, I find it concerning that some parts of society are developing an aversion to facts and a distrust of scientists and experts.

A brilliant (if scary) example of this was a CNN interview by Alisyn Camerota with former US politician and former presidential candidate Newt Gingrich: the full interview is on YouTube, Feelings trump FBI Stats!. In the interview, the topic of crime in the US was brought up (I'm aware that I'm cutting down the interview for brevity but I promise I'm not taking anything out of context):

CAMEROTA: ... violent crime across the country is down. We're not under siege in the way that we were in say, the 80s.

GINGRICH: ...The Average American, I will bet you this morning, does not think crime is down, does not think they are safer.

CAMEROTA: But we are safer, and it is down.

GINGRICH: No, that's your view.

CAMEROTA: It's a fact.

GINGRICH: I just -- no. But what I said is also a fact ... people don't think that their government is protecting them ... I understand your view. The current view is that liberals have a whole set of statistics which theoretically may be right, but it's not where human beings are. People are frightened.

CAMEROTA: ... but hold on, Mr. Speaker, because you're saying liberals use these numbers, they use this sort of magic math. This is the FBI statistics. They're not a liberal organization.

GINGRICH: No, but what I said is equally true. People feel it.

CAMEROTA: They feel it, yes, but the facts don't support it.

GINGRICH: As a political candidate, I'll go with how people feel and I'll let you go with the theoreticians.

From that exchange it sounds like feelings trump facts; pun fully intended. And Gingrich is right that there has been an uptick in certain cities, he could even have claimed using the FBI statistics that 2015 violent crime was up on 2014 and 2013. But, and I've plotted the FBI violent crime stats below, it's clear that violent crime has been on a downward trend since 1996: people may not feel safe but the data shows us they are safer than they were when Barack Obama came into office! And just because you "feel" something doesn't make it true.

Read more…

Neo4j Graph Hack 2016: Where to avoid cycling accidents

It's a little late but it's taken me a while to finally get this blog up and running; Now that it is live I'm catching up publishing interesting projects from the past. I've been intrigued by graph databases since I discovered them and curious about what they are capable of (another example can be seen in this blog post). So having won a ticket to the Graph Connect Europe 2016 conference I thought I'd take advantage of attending the graph hack hosted by Neo Technologies the night before. I also invited a friend, Amy, along. Together we formed Crash Dodgers (follow the link to get the actual Python notebook) and decided to see if we could find the most dangerous spots to hire bicycles in London, in the 2-3 hours we had available to us!

tldr: We won the "best app" award

Read more…