mapgubbins

Prior to yesterday’s release of the Cabinet Office’s Open Data White Paper there was speculation, mainly in an article first published Wednesday evening on the Guardian website, that the UK Government would also use this occasion to announce a review of charging for information requested under the Freedom of Information Act.

It’s unclear whether the Guardian’s original draft was just based on a misunderstanding, or whether the Cabinet Office picked up on the initial negative responses and decided it would really rather not face the wrath of both open data campaigners and freedom of information activists on the same day.

Either way the Guardian article was substantially revised with comments from a Cabinet Office spokeswoman, and the original headline ‘Public services data to be published - but at a price’ was replaced by the much more congenial ‘Mass of government data on public services to be published’.

Although the Open Data White Paper was rather a damp squib for those of us hoping for some commitments on release of core reference data, it was apparently well-received by those concerned mainly with public sector transparency. The success of the launch event was also helped by a colourful revamp of the Government’s public data portal Data.gov.uk.

Had the Government decided to float a FOI charging proposal yesterday, the social media reaction would have eclipsed any positive coverage of the open data paper.

FOI charging therefore remains an outstanding and highly contentious issue for another day. Probably another day soon; we know that the Ministry of Justice, which for some reason has lead responsibility for the freedom of information brief, is keen on charging. 

It should be said however that if charging for FOI responses and charging for re-use of public sector data are two ideas that are easy to confuse, the Government has only itself to blame. Recent legislative changes seem to have been designed to deliberately conflate requests for access to information with licensing to re-use that information, when the subject of the information request is a publicly-owned dataset.

Following changes to the Freedom of Information Act brought in last month (via Section 102 of the Protection of Freedoms Act), the Government has effectively combined the procedures for requesting FOI access to a dataset and requesting a licence to re-use that dataset.

Annex B in yesterday’s Open Data White Paper establishes clearly that the standard route for requesting re-use of any public dataset that is not already available is now to submit a FOIA request.

The FOIA changes include a new power to charge fees in relation to release of datasets for re-use. Although the original intention may not have been to allow charging for access alone, the actual language of the FOIA changes is suspiciously ambiguous. The provisions seem to require public bodies to prepare the data for re-use anyway, i.e. to do the work for which they could charge a fee, even if the requestor only wants to access the data.

The practical effect is that public bodies will have the option to charge for responding to a FOIA request whenever that request is for a public dataset, unless or until they receive clearer guidance to the contrary from Government.

Public bodies have not yet had formal guidance from Government on interpretation of these FOIA changes or on the appropriate shape of any charging policies. That may fall within the scope of new responsibilities given to the Information Commissioner’s Office. The ICO is expected to introduce a revised model publication scheme in April 2013.

Open data campaigners will hope that any such guidance re-establishes the presumption that public bodies should charge at most the marginal cost of preparing the data for re-use. Under the previous Government public bodies were (in principle at least) required to justify any charging policy that was not based on a marginal cost pricing approach.

P.S. I wrote a previous analysis of these problems back in February, when the Protection of Freedoms Bill was under scrutiny in Parliament.

When the Cabinet Office announced back in November, as one of the Open Data measures in the Autumn Statement, that

from 1 March 2012 Land Registry will make available “Price Paid information” showing all residential property sales in England and Wales at address level

what do you suppose it meant by that?

It seemed simple enough. By law all residential property sales are lodged with Land Registry. Land Registry maintains a Price Paid information dataset with the full address, date of transfer, property type, whether the property is new and whether it is freehold or leasehold. 

The full dataset contains records of more than 15 million house transfers, going back to 1 January 1995.

At the moment licensing costs for the full Price Paid dataset effectively put it beyond the means of non-profit organisations, SMEs, app developers and members of the public.

Based on Land Registry’s charging policy you need pretty deep pockets to license the full dataset or even the most recent few years of data. The licensing fee for Price Paid data from 2005 to present, for example, is more than £50,000 plus an annual subscription to maintain re-use rights. The full dataset is mainly used only by large firms ancillary to the property and conveyancing sectors.

However Price Paid information is potentially very useful for any number of other applications, both commercial and non-commercial.

When the Government said in the Autumn Statement that it would “make available for free a range of core reference datasets … to support the development of high-value data businesses” the Price Paid information seemed like an excellent choice.

Which brings us to March 2012

Based on the Autumn Statement announcement Land Registry was expected to release the Price Paid information yesterday, as open data under the Open Government Licence.

That deadline has slipped, and the latest word on the open data grapevine is that the Price Paid information for residential property sales in January 2012 will now be released at the end of this month.

You will then be able to download that data from the Land Registry website, to use as you like.

But that’s all: one month of data. The following month the data for February 2012 will be available, and so forth.

Monthly updates will be open data — but the bulk of the Price Paid dataset will remain closed, still available only on commercial terms to Land Registry’s big business customers.

If you’re a researcher who wants to use the historic data for socio-economic analysis, or a small developer with a cool idea for a new app, you’re out of luck. Come back in a couple of years, when the monthly updates have built up into a sufficiently large body of open data to be useful.

What’s happened here is called “descoping”

Descoping is the practice of abandoning or weakening the objectives of a project, usually to save money. In a normal business context, when that project is work on behalf of a client, the client can minimise the potential for descoping by compelling performance to an agreed specification.

When it comes to open data releases, descoping is usually a sign that the public body involved isn’t really committed to opening up their data.

My guess is the chain of events went something like this. The Cabinet Office was keen to launch the Public Data Group with a package of new open data. The Price Paid information fit the criteria because it’s a reference dataset and likely to be popular with open data developers and SMEs once the cost barrier is removed.

Land Registry derives most of its income from registrations and copies of land records, so could afford to offer the Price Paid information as open data — if only to show willingness. (Many of the other datasets held by Land Registry contain personal data and are therefore not eligible for open data release.)

But the Cabinet Office only cares about the big picture. Once the Autumn Statement announcement was agreed it left Land Registry to handle the practicalities of delivering the data. Land Registry decided that by taking a narrow interpretation of the announcement it could get away with releasing updates only, and safeguard some of its revenue.

Or could this just be a misunderstanding?

Perhaps the Cabinet Office only ever expected Land Registry to release the monthly updates, and the Autumn Statement was just short on detail.

We can test that by looking at the Public Data Group business case prepared by the Cabinet Office to support the open data package announced in the Autumn Statement. The business case was released recently in response to a Freedom of Information request from Harry Metcalfe via the WhatDoTheyKnow? website.

In the business case Land Registry figures estimate an annual loss of revenue of £600,000 from open data release of the Price Paid information. We are told that Land Registry has 37 existing subscribers for this data, and additionally receives about ten ad hoc requests per month for specific subsets of the data.

Combine that with the figures in Land Registry’s charging policy for the Price Paid information (annual subscription £2,200 + VAT and £0.008775 + VAT per address update supplied), along with Land Registry’s statistics on the number of houses sold each year in England and Wales.

It’s clear that, even as a ballpark estimate, a £600,000 annual loss of revenue is plausible only if it includes Land Registry’s income from licensing of the historic “archived” data.

There go the economic benefits

The PDG business case as a whole has a major weakness: its benefit analysis is based mainly on a generic application of the economic arguments for open data.

If the Cabinet Office had made some attempt to attach benefits to each of the specific datasets in the Autumn Statement package, with reference to their actual characteristics and potential uses, that would have discouraged any descoping of the release.

In respect of the Price Paid information, the business case counts the benefit of savings to existing licensees of the data (based on the full loss of revenue estimated by Land Registry). The business case also notes that “this move will enable more SMEs to establish added value services and products based on the core reference data to some of the key public information organisations in the UK”.

Now that the release has been descoped, that analysis no longer holds. If Land Registry has successfully protected most of its licensing revenue, that will correspondingly reduce the input cost savings to existing licensees. And if most of the dataset remains closed, there is much less scope for innovation by SMEs. Existing big business licensees will retain their market advantages.

So are the monthly updates good for anything?

Open data release of monthly updates to the Price Paid information is better than nothing, of course. As suggested above this data will become more useful as the months go by and it builds up into a larger body of open data.

Developers may find innovative uses that only require data for new residential property transactions. I can’t think of that many myself, but innovations are by definition non-obvious.

The most immediate application for this data is direct marketing. Prior to this release, the terms under which the Price Paid dataset was available from Land Registry precluded use of the data for direct marketing purposes.

As the monthly data is available under the Open Government Licence, that direct marketing restriction cannot apply.

As a near-term effect of this open data release, new homeowners can look forward to targetted advertising from small and medium sized businesses that could not previously afford the Price Paid data.

In fact, I understand that (in the interests of fairness no doubt) Land Registry plans to remove the direct marketing restriction from licensing terms for its commercial customers. They will now also be able to use the closed historical Price Paid data for direct marketing purposes.

Result!

Seriously though.

Land Registry’s Price Paid information is a core national reference dataset that documents residential property transactions in England and Wales at address level over a period of more than 15 years.

The monthly updates are only an incremental addition to that body of data and, by themselves, have nowhere near the same potential for economic use as open data.

This release is a damp squib for the Cabinet Office’s open data agenda, and just one more disappointing example of the gap between rhetoric and follow-through in the Government’s approach to innovation and support for SMEs.

If the Cabinet Office is serious about unlocking public sector data (and the jury is still out on that) it needs to do a lot better.

The Protection of Freedoms Bill is a wide-ranging piece of UK legislation currently at the report stage in the House of Lords.

This post is about Clause 102, a small and unassuming part of the Bill that tinkers with the Freedom of Information Act in order to make it easier for the public to access and re-use datasets held by government bodies. Or at least that’s the idea.

I start from the premise that the Cabinet Office is probably genuine in supporting broad re-use of public information. However the Cabinet Office doesn’t always have a complete grasp of detail or the influence to drive its transparency agenda forward. Other departments (particularly the Ministry of Justice) are intriguing against greater public access to information.

It looks to me as if Clause 102, as drafted, has potential to reduce rather than increase re-use of public datasets, and that as a knock-on effect it may actually reduce FOI access to public data as well.

Clause 102 in the Protection of Freedoms Bill covers “release and publication of datasets held by public authorities”. Ostensibly it supports a ‘right to data’ that will enable the public to request and use government-held datasets and ensure they are updated on a regular basis. It does this by amending and expanding Section 11 of the Freedom of Information Act 2000 (FOIA).

As noted in a Home Office factsheet, the FOIA currently “provides for access to information held by public authorities” but it “makes no express provision in respect of datasets, their availability for re-use or their publication in a re-useable format.” The implication is that this is a flaw in the FOIA.

As policy background, the Cabinet Office has included in its Business Plan an action to create a new ‘right to data’. An impact statement signed off by Francis Maude last year indicated the Cabinet Office was committed to encouraging re-use of public data sets. An enhanced ‘right to data’ also featured prominently in the Cabinet Office’s recent Open Data consultation.

However the Cabinet Office intends to create this new right to data “in conjunction with” the Ministry of Justice. Responsibility for FOIA along with a large swathe of information policy fall within the remit of the MoJ, for rather arcane reasons that I’ll come to a bit further along.

There are strong reasons to mistrust the MoJ’s commitment to the ‘right to data’. Based on its written submission to the current Select Committee inquiry into pre-legislative scrutiny of the FOIA, the MoJ is leading a charge to roll back the FOIA and restrict public access to government information.

It may also be significant that when the Cabinet Office published the responses to its Open Data consultation a couple of weeks ago, the MoJ response was the only one redacted in its entirety.

At first glance the wording of Clause 102 in the Protection of Freedoms Bill seems to support the ‘right to data’ as advertised, by requiring public bodies to “so far as reasonably practicable, provide the information to the applicant in an electronic form which is capable of re-use” in response to a valid FOI request.

The problem is the subsequent language that enables public bodies to “charge a fee in connection with making the relevant copyright work available for re-use”.

That “in connection with” locution is rather odd. Intentionally or not it serves to blur the distinction between access to information under FOI (which does not require a licence) and additional re-use rights (which do require a licence).

As currently drafted, Clause 102 would seem to oblige the public body to prepare data for re-use, and to enable it to charge the FOI requestor for the cost of that work, even if the requestor only wants to access the information and has no interest in re-using it. FOIA as it stands allows public bodies to charge for the cost of “processing” the request, and the Clause 102 amendments will necessarily increase that cost.

That’s effectively equivalent to enabling the public body to require an increased fee simply for providing access to the data, which will have a chilling effect on requests for data and is therefore dangerous to the core democratic purpose of the FOIA.

It’s quite true that the FOIA does not currently provide for re-use of data. Copyright and licensing considerations are actually irrelevant to requests under both the existing FOIA and the Environmental Information Regulations (EIR). The copyright status of the data may preclude re-use but that by itself will not preclude access, i.e. it does not give rise to an exemption under either FOIA or EIR.

However that is no flaw in the FOIA, because the Government already has a framework to encourage re-use of public sector information: the Re-use of Public Sector Information Regulations 2005 (RPSIR).

Responsibility for enforcing RPSIR sits with the Office for Public Sector Information (OPSI), which is also responsible for the Open Government Licence. OPSI was originally a Cabinet Office body, but was merged with The National Archives in 2006. National Archives is an agency of the Ministry of Justice.

In principle OPSI is supposed to promote the re-use of public sector information. It operates the Information Fair Trader Scheme, which among other things accredits and regulates public bodies that want to charge fees for re-use of their information.

In practice, under the current Coalition Government, the OPSI has done very little to promote the re-use of public information.

On the contrary last year National Archives acted to undermine take-up of the Open Government Licence by introducing a new government licence for non-commercial use of information. That option sent a signal to public bodies encouraging them to release their data holdings but retain any revenue opportunities from re-use for themselves or selected business partners, rather than making them available more generally as open data and potentially driving innovation.

It looks to me as if the real effect of Clause 102, as drafted, will be to give public bodies carte blanche to charge for re-use of (and potentially access to) data. It eliminates the motivation to seek accreditation under the Information Fair Trader Scheme, without providing any alternative mechanism to make public bodies justify their charging policies.

In other words, less a ‘right to data’ than a ‘right to buy data’.