Civil partnerships: the ONS and the data journalist

The ONS release of data on civil partnerships highlights some of the the challenges of managing an open data approach when “all the data” becomes an expectation and lines get blurred.

Yesterday, David Ottewell head of UK newspaper group Trinity Mirror’s (excellent) data journalism unit, tweeted his frustration at the way the Office of National Statistics (ONS) published a recent data set.

Earlier, the ONS had published a statistical bulletin on Civil Partnerships in the UK. Something they do a lot of. Indeed the ONS have published a bulletin on this topic for the last five years or so. But there was a change this year, as the bulletin author, Elizabeth McLaren noted

Following the introduction of marriages of same sex couples, the number of civil partnerships formed has declined dramatically. Due to the relatively small number of civil partnerships formed in 2015, datasets have not been published alongside this bulletin — data have been included in the bulletin instead.

The the issue of small numbers — a concern that the data in combination with location information, could inadvertently identify individuals is always a concern with data releases — wasn’t one that washed.

Ottewell voiced a concern that this was a huge “policy change’ for the ONS.

In terms of policy, the ONS has always maintained a ‘publish when we can’ approach to open data. But open data publishing is, and should be driven by demand and Darren Waters, managing editor of publishing at the ONS noted, that in this case the demand wasn’t there.

It was an argument that, again, didn’t carry much weight with Ottewell who questioned the consultation process noting it was a problem with “the consultation not the data”

As it stands the ONS have gone back to look at the data and the possibility of delivering a regional breakdown of the data. That’s something that’s key for Ottewell and his unit who create content that services hundreds of local newspapers — without the abstraction of data they can’t make it work in their journalism.

Points of interest

The whole debate struck some, watching from the sidelines as odd, noting that the changeable nature of data availability was nothing new. But beyond the policy and practice I think there are a couple of issues that are worth consideration:

The expectation of consistency there is clearly a expectation here that because this dataset has been made available before in a certain format that it will be again. Whether that’s right or wrong (or rendered moot by consultation) the expectation that once you let the data out into the open it’s always open, is one that is a really big challenge for publishers of data where a misstep means a step back with related communities.

The role of the ONS as publisher – it’s clear that with the appearance ofvisual.ons in the last year the presentation of stats at the ONS has taken a ‘qualitative turn’. Its perhaps unsurprising given Darren Waters background. But is that really supporting the stated public aim that the ONS “helps users understand our responsibilities and makes clear what topics of statistics are available for use and re-use.” Or is it taking the ONS going beyond simply explaining its role to taking an editorial position. Is the ONS doing data journalism? Is that a problem?

Do rich interactives produced by the ONS blur the line between communicating what it does and an editorial response and is any of it open data?

Ottewell is less concerned about the “ONS doing journalism” than he his the approach to opening up data stopping journalist doing their jobs. It’s that concern – the slippery slope of what’s next on the list of data to be selectively used – that highlights the balancing act between effective internal use of the data (always to be encouraged) and effectively enabling others to use it. In this case its using it for storytelling and insights, but as has long been argued more generally for open data (and earlier incarnations of PSI reuse) it shouldn’t stifle legitimate activity outside of government.

Open data means less data – The consultation on civil partnership data has an interesting statement

In future, the number of tables published may decrease and be replaced by more interactive open data options, whereby the user can effectively create their own tables according to their needs.

The key word there is ‘interactive’ (and I’m hoping that the concept of‘interactive open data’ doesn’t become a thing) It’s a commonly held belief that visualisation is a good way to get people engaged with data and the effectiveness of making it interactive is built on the assumption of more personal insights. But it’s important to reflect, especially with government data, on the extent to which a data set can be said to be really open if you can’t access the underlying data. Better engagement and accountability shouldn’t be mutually exclusive with open data.

What next…

* I’ll update this post when and if there’s any change.*

The data behind the release is now available