a blog by Owen Boswarva

10 Oct

Post: 10 October 2014

The Department for Communities and Local Government has released Green Belt boundaries for England as a polygon dataset, for re-use under an open data licence.

The dataset may be downloaded in shapefile format from the OpenDataCommunities website.

I’ve written previously about the arguments for open data release of this dataset, as has Alasdair Rae of the University of Sheffield.

Another version of DCLG’s Green Belt dataset has been in circulation since 2012 when it was published on the Telegraph website. However yesterday’s release is the first time DCLG has officially made a Green Belt dataset available on open data terms.*

* Update: Jeni Tennison points out that this dataset is also available via a link from a catalogue record. The link seems to have been added in May or June.


The national Green Belt dataset is compiled from spatial data provided to DCLG by local planning authorities. (Earlier this week DCLG published new planning guidance on green belt boundaries.)

I’ve had a quick look at the data. Following are some notes:

  • This Green Belt dataset is not the same as the version released in 2012 by the Telegraph. However, like the Telegraph version, the dataset appears to represent green belt coverage for the 2011-12 year.
  • DCLG publishes annual National Statistics on green belt coverage. Statistics for the 2012-13 year were published in March 2014, so I would have thought DCLG could also release a 2012-13 version of the boundary data. One of the main purposes of the statistics is to track year-on-year changes in green belt coverage. These changes will be easier to visualise and understand if we have a time series of open data releases.*
  • It is difficult to reconcile the polygons in the spatial dataset closely with the hectare figures in the statistics, for some geographic areas. The spatial dataset is serviceable, and I appreciate there are variations in the quality of the underlying data provided by local planning authorities. However I think there is an argument for regularising the open spatial dataset so that it is interoperable with the National Statistics table, i.e. with one object for each local planning authority and matching hectare calculations.

* Update 16 October 2014: Statistics for 2013-14 have now also been published.

Image credit: Produced from the dataset discussed above. © Department for Communities and Local Government. Contains Ordnance Survey data © Crown copyright and database right 2012.

3 Oct

Post: 3 October 2014

National Archives is consulting on UK implementation of the revised EU Directive on the reuse of public sector information (the ‘PSI Directive’). This is a process called “transposition”, and will take the form of new PSI regulations to replace the existing Re-use of Public Sector Information Regulations 2005.

The PSI regulations are of great relevance to the future availability of open data in the UK, because they set out the rules under which some public bodies are allowed to charge for re-use of public data assets.

The Open Data Institute has published a draft submission to the consultation, in which it raises a number of concerns (discussed below).

The NA consultation is open until 7 October if you want to submit your own response.

Although the consultation is being run by National Archives, I first want to place it within the wider context of Government policy on information rights.

Where’s OPSI?

When the original PSI regulations were introduced the UK body responsible for information policy was the Office of Public Sector Information. OPSI was then part of the Cabinet Office reporting structure.

In 2006 OPSI was folded into National Archives, which is an executive agency of the Ministry of Justice. At the end of 2010 the current Government shut down the OPSI website.

Search on GOV.UK today and you will be hard-pressed to find much evidence that OPSI still exists. OPSI is also difficult to recognise on the National Archives site itself. As ODI says in its submission, ”the identity and visibility of OPSI has been obscured.”

This matters because under the PSI regulations OPSI is responsible for handling public complaints about barriers to re-use of public sector information. If potential complainants can’t figure out who they should complain to, that undermines the credibility of the regulatory process.


MoJ and information rights

The wider point is that, although Cabinet Office leads on the Government’s transparency agenda (which includes open data), practical authority over information policy and information rights now sits with MoJ.

In addition to its oversight of National Archives and OPSI, MoJ funds the Information Commissioner’s Office (ICO), which handles complaints about freedom of information and data protection.

As a consequence of these separate policy streams, there is a noticeable disconnect between Cabinet Office’s rhetoric in favour of open data and MoJ’s legislative approach to public sector information.

UK negotiating position on the revised PSI Directive

In December 2011 the European Commission launched an Open Data Strategy for Europe, along with the original proposals for a revised PSI Directive. Those proposals included new measures intended to strongly advance the open data agenda.

Of primary interest: the Commission proposed to explicitly limit charges for re-use of PSI to recovery of marginal costs incurred for reproduction and dissemination (other than in exceptional cases).

However those proposals were subsequently eviscerated in committee, based on amendments put forward by the UK.

The UK amendments to the Commission proposals created a broad exception to marginal cost pricing, for libraries, museums and archives, and for “public sector bodies that are required to generate revenue to cover a substantial part of their costs relating to the performance of their public tasks”. (See in particular Amendments 35, 78, 84 and 92. Sajjad Karim is a British MEP.)

According to the final text of the revised PSI Directive, those public bodies will be allowed to charge fees to recover not only the marginal costs of reproduction and dissemination, but also the cost of collection and production “together with a reasonable return on investment”.

In other words those public bodies will be allowed to recover the costs of creating information for its core purpose (the delivery of a public task), and not just additional costs related to re-use or production of added value.

The UK amendments are similar to the wording in the existing PSI regulations, i.e. they operate to reverse proposals from the Commission that would have put tighter constraints on charging and bolstered the open data agenda.

The UK’s negotiating position was in direct opposition to the basic principle underlying open public data, i.e. the argument that if government already needs to fund creation of the information then re-use for other purposes should be free (or charged at the marginal cost of reproduction).

There is a more detailed explanation of the UK’s negotiating position, in a March 2013 letter to the European Scrutiny Committee from MoJ minister Helen Grant MP.

It is clear from that letter that the UK Government was keen to protect the commercial interests of its data-rich trading funds (Ordnance Survey, Met Office, et al). However in my view the exceptions to marginal cost pricing in the final PSI Directive are sufficiently broad that other public bodies could take advantage of them quite easily.

Meanwhile …

MoJ has until July 2015 to enact regulations that transpose the revised PSI Directive into UK law, and this process is the subject of the current consultation.

However, in the meantime, MoJ has introduced amendments to the Freedom of Information Act that also deal with licensing and charging for re-use of public information. I’ve written previously about those amendments and why I think they are hostile to open data.

Of particular interest are the accompanying Fees Regulations, which contain some familiar language. Public authorities may charge fees for re-use, to recover:

(a) the cost of collection, production, reproduction and dissemination of the relevant copyright work, and

(b) a reasonable return on investment.

The Fees Regulations seem to go further than the revised PSI Directive, because they apply to all public authorities rather than just those that are actually required to generate revenue. There could even be potential for a legal challenge here: arguably the Fees Regulations constitute “gold-plating" to UK transposition of the revised PSI Directive. (The MoJ consultation notes that there "may be a need" for further amendments to FOI.)

Does the current National Archives consultation matter?

Given that the consultation is about transposition of amendments already agreed during negotiations in Europe, the scope to influence changes to the proposals is quite narrow. It would have been more meaningful if MoJ had consulted on formulation of the UK negotiating position, back in 2013.

There are five questions in the consultation, and only one of those relates to charging. National Archives has not released a draft of the new PSI regulations, so there is a lack of detail. Some of the background work in the impact assessment is interesting. However the consultation seems to be only a token effort to seek public views on the general approach to selected parts of the process.

Comments on the Open Data Institute’s draft submission: Redress

I agree with many of the concerns raised in the ODI draft submission. However on the question of the redress mechanism I tend to side with the Government proposals. 

The Government proposes to retain OPSI as the investigative body for PSI complaints, with referral to a First-tier Tribunal (likely the Information Rights Tribunal) for a legally binding decision.

ODI would prefer that, rather than providing for referral to a First-tier Tribunal, the Government should empower ICO to both investigate complaints and to make legally binding decisions.

I cannot see how the ODI’s approach would serve the interests of complainants. Not only does it take away the recourse to the courts offered by the Government proposals, it removes the appeal stage (fulfilled by APPSI in the current process) entirely.

ODI seems to think it is proposing that PSI complaints should be handled in the same way as FOI complaints. But ICO’s decisions on FOI (and DPA etc.) can be appealed to the FTT (and upwards). ICO’s decisions are binding but not necessarily final, and it is not unusual for the FTT to rule against ICO. There is no reason at all to suppose ICO would be less fallible in making decisions on PSI complaints.

Removing the recourse to appeal is only in the interests of complainants if we expect ICO to uphold most complaints. We don’t have any reason to think that will be the case; ICO’s record on FOI and other complaints is irrelevant because it is based on different legislation, and OPSI has handled too few PSI complaints to provide credible data.

Could ICO substitute for OPSI, but with a right of appeal to the FTT? Perhaps — but where is the advantage? There is little practical difference between an OPSI “recommendation” and an ICO “binding decision” if parties to the complaint have recourse to the FTT either way.

Notwithstanding the recent introduction of re-use into the Freedom of Information Act, ICO has no track record on handling PSI complaints. It is already under-resourced to manage its casework on FOI/EIR and data protection. Even if OPSI staff and resources were transferred to ICO, PSI could well be seen as a low priority relative to FOI and DPA. I think it makes more sense to retain OPSI as a specialised unit within the information policy environment of National Archives.

The one advantage ICO has over OPSI is a recognisable public profile. But that is a function of the Government’s decision to conceal OPSI. Rather than transfer OPSI’s functions to ICO, the Government should give OPSI back it’s own public-facing identity. The Government should also require public bodies to include information about the OPSI complaints process in any responses they make to requests for re-use of PSI.

Comments on the Open Data Institute’s draft submission: Charging

I concur with ODI’s response to Question 4, as far as it goes. As I think ODI has recognised, it is difficult to comment in detail on the proposed approach to charging because National Archives hasn’t actually set out any new criteria, either for deciding when public bodies are entitled to charge above marginal cost or for how those charges should be calculated.

I think ODI is quite right to be concerned that the charging regime under the new PSI regulations could be implemented in a manner contrary to the stated aim of the PSI Directive.

My recommendations in this regard are as follows:

  • National Archives should retain APPSI, as a source of advice and expertise even if it is no longer required as a formal part of the complaints process.
  • National Archives should also retain the Information Fair Trader Scheme, and the Government should explicitly require all public bodies that want to charge above marginal cost to join that scheme. This requirement should take precedence over the Fees Regulations in FOI.
  • Or: the Government should just jettison the language that allows recovery of costs of collection and production and a return on investment, and restore the more limited scope for charging set out in the original Commission proposals for the revised PSI Directive.

Yes, the last point is an unrealistic expectation. But that is the approach Government should take if it wants to show full support for the open data agenda, and maximise re-use of public sector information. The only barrier is political will.

Update 7 October 2014:

I have now submitted my own views to the National Archives consultation.

Image credit: Ministry of Justice by Charles Hoffman (CC BY-SA 2.0)

1 Oct

Post: 1 October 2014

Earlier this week Heather Savory, chair of the Open Data User Group (ODUG), argued in a post that Ordnance Survey’s model for licensing of geographic data is a “fundamental barrier” to maximising the beneficial use of publicly owned data in Great Britain.

Ordnance Survey does publish some very useful open data products, but its most detailed and useful geographic data is still available only on commercial terms (with special arrangements for public sector organisations).

ODUG is calling on the Government to push OS to open up most of (or all) of its data, in order to promote competition and develop the wider information market. I very much support this call.

In fairness, Ordnance Survey has done a pretty good job promoting reuse of its existing open data since the launch in 2010. A preliminary economic study prepared in 2012 estimated the OS OpenData programme would generate net GDP growth of between £13.0m and £28.5m per year by 2016. A recent project to add open data to the popular Minecraft game has had favourable publicity and, more importantly, helped introduce open geographic data to a wider, younger audience.

OS OpenData is a success story — and it is time to build on that success. But Ordnance Survey is constrained by the trading fund model, which requires it to put revenue generation ahead of the wider interests of Britain’s information economy. As in 2010, moving forward will require political support at ministerial level.

National Information Infrastructure

ODUG and the Cabinet Office are currently drawing up a list of datasets that form the National Information Infrastructure. The first draft, released in October last year, was rather uneven — but the decision to include all of Ordnance Survey’s geographic data was both sensible and obvious. OS data is not only essential to the NII — it is in many respects the foundation.

One of the key concepts behind the NII is the idea that the utility of public data is amplified by interactions between datasets. The more datasets are released as open data, the more potential there is to link those datasets in new and innovative ways.

In her post Heather Savory uses the example of the flood data Environment Agency is planning to release next year (perhaps sooner). She suggests analysis of that flood data would be more productive if we also had more detailed open data from Ordnance Survey.

At the moment OS OpenData gives us postcode centroids (in Code-Point Open) that we could use with EA flood data. But what if we also had coordinates of individual addresses, or even building outlines?

Maximising Reuse of Open Data: An Illustration

It’s always difficult to anticipate all the ways in which open data might be reused. However I would like to expand on Heather’s example, drawing on my own experience using EA flood data and OS data together in a business context. Below is a (simplified) illustration of the difference that better OS data can make to the public understanding of flood risk.

The main flood dataset Environment Agency intends to release as open data is called Risk of Flooding from Rivers and Sea (RoFRS), although it was until recently better known as the National Flood Risk Assessment (NaFRA). This dataset is an output from modelling the EA uses to prioritise investment in flood defences. RoFRS is also the source of flood risk information used mostly widely in the UK insurance industry.

RoFRS describes the likelihood of flooding at any given location, using four categories of risk (High, Medium, Low and Very Low) averaged across a grid of 50m x 50m cells. At high level, on a map, RoFRS looks like this:


Drill down to the level of an individual postcode and the data looks like this:


As you can see the data is kind of blocky at this resolution. Like any flood risk model RoFRS is only “indicative”, particularly when laid over small areas of geography. (It’s generally difficult to produce a definitive view of flood risk at address level without a site survey.) But indicative is still useful for many purposes, and better than nothing even when thinking about the risk to individual properties.

So how do we do that? The EA flood data gives us an idea of the relative likelihood of flooding for areas of land, but we need to geolocate our addresses (the flood risk “receptors”) as well. This is where the Ordnance Survey data comes in. 

At the moment, if we rely only on OS OpenData, all we have is the postcode centroids available in Code-Point Open. In the example above that centroid is on the edge of a High likelihood grid cell. We know the addresses themselves are in the vicinity of that point, but not their precise coordinates.

So, to be prudent, we might assume all of the addresses in that postcode are at a High risk of flooding. (There are techniques we can use, such as buffering, to get a more nuanced view even without address points. But that takes additional expertise.)

Here’s a better option:


If we had a polygon drawn around all of the address points in the postcode, as above, we could use GIS techniques to average the likelihood of flooding within that polygon. That would give us a better representation of the risk within the postcode, even without geolocating the individual addresses.

To do this we would need OS’s Code-Point with polygons product, which is not currently open data.

This isn’t a great approach anyway, because much of the space within the polygon is road rather than properties — and in a rural postcode properties might be few and far between. But it’s a better approach than relying only on the postcode centroid.

Or here’s an even better option:


Here we have the address points themselves. Ordnance Survey produces a number of geocoded address products, the most comprehensive of which is OS AddressBase Premium. None of those products are currently available as open data.

As you can see above, using the address points allows us to differentiate the indicative level of flood risk by individual address, rather than lumping the properties all together within the postcode. Of course we wouldn’t be confident of these results if buying a house, but for mechanical analysis of a large portfolio of properties this is a respectable approach.

But … there’s an even better option:


Here we have building outlines, in addition to address points, for the individual properties. As you can see, in some instances the address point is outside the flood cell but part of the property is within the flood cell. That gives us a more detailed understand of the likelihood of flooding at each address. (Though bear in mind we are really pushing the resolution of the flood model here.)

We don’t have the building outlines as open data. They are included within the Topography Layer of OS MasterMap, Ordnance Survey most detailed and comprehensive mapping product.

Beyond Addresses - Other Contexts

In the above I’ve mentioned various Ordnance Survey data products that might be useful for analysing flood risk in a residential postcode. However OS publishes many other datasets, such as the OS MasterMap ITN Layer and the OS MasterMap Sites Layer, which would be similarly useful if analysing flood risk to transport infrastructure, commercial properties, etc. And risk to property is only one context in which OS data can amplify the utility of flood data; there are applications in reducing risk to life and promoting community resilience as well.

Disclaimer: The above is an illustration only. Proper analysis of flood risk, even in the jejune environment of the insurance industry, is a bit more complicated. If you’re a hydrologist please don’t make fun of me.

Image credits: The images in this post may contain Ordnance Survey data and/or Environment Agency data or derivations thereof. Or maybe I just drew all the lines with a ruler. Who knows? Derived data is confusing.

5 Sep

Post: 5 September 2014

This post is also published on the Royal Statistical Society’s StatsLife site.

I’ve written before about the UK National Information Infrastructure, a Cabinet Office project to identify the more important and useful public data assets and perhaps nudge some of them towards open data release. I’ve been critical of progress so far, but the basic idea is sound. This is potentially an important initiative for open data and public sector information in general.

Recently the Open Data User Group published a position paper on the UKNII, as an attempt to engage wider interest and reinvigorate discussion. The UKNII was also covered in yesterday’s all-star open data briefing at techUK in London.

There is one major barrier to this project that Cabinet Office seems to have shied away from in its initial draft of the UKNII. Of course there is still plenty of work to do in persuading Government departments to unlock the data they hold. But some important datasets, of relevance to the daily life of the nation, are not held or maintained by Government departments or public bodies — they are in the private sector.

I’m not talking here about commercially sensitive data that big businesses use to compete in the marketplace. Neither is this a broadside against the evils of privatisation; we are where we are. But the fact is that, in the UK at the moment, companies and business organisations have primary responsibility for maintaining key reference datasets on transport, utilities, banking, the food supply and in many other areas.

The open data community could make good use of some of those datasets, in apps and as a source of analytic insights. The arguments for release are not quite the same as for government data, because these datasets may not be directly funded from taxation. But selectively, as part of the UKNII conversation, I think we should be on the look out for particular datasets that are (a) authoritative and irreplacable in their domain, and (b) important to public transparency or the operation of services across that domain. In other words, datasets that we would normally expect Government to make available if that area of the economy was state-run rather than market-led.

Last summer HM Treasury announced that banks and building societies would publish mortgage lending data based on postcode sectors. The outputs are not perfect — the banking industry hasn’t grasped all the nuances of open data release — but in principle this is a useful contribution to the body of national information.

Banking is to some extent a special case, because following the financial crisis of 2007-08 there is a broad consensus that lending practices need to be more transparent. But it is not difficult to identify other datasets held by the private sector that, arguably, should be more widely available for reuse.

Below are five examples of datasets that I personally would like to see released, either by the relevant business organisations on their own initiative or, even better, with coordination from Government. These particular examples are datasets with a geographic theme.

1. Supermarket locations


All of the major supermarket chains maintain datasets with the names, locations and opening times of their stores, as the basis of “locator” or “finder” search functions on their websites. (E.g. Tesco, Morrisons, Lidl.)

There is a clear public interest in availability of a single combined source of data on these locations, so that people can more easily find their nearest sources for groceries (irrespective of brand). There is also considerable potential for analytic use of this data, as a basis for study of food supply chains, public health and accessibility of services.

At the moment bulk data on supermarket locations is included in Ordnance Survey’s Points of Interest product, but there is no open source. To see the difference this makes to the UKNII, have a look at DfT’s excellent accessibility statistics release: of the eight key services measured by DfT, food stores is the only one for which the underlying point data is not available to the public (“commercial data not available for release”).

[ Update 10 September 2014: Tony Hirst points out that an open list of supermarket addresses can be extracted from food hygiene ratings data published by the Food Standards Agency. Some of those records are a few years old but, in the absence of open data from the supermarket chains themselves, this is certainly a useful alternative. ]

2. ATM locations


In a similar vein there is no national open data available for ATMs, i.e. cash machine locations. At least one local authority (Sunderland) publishes a local list. However the national dataset is maintained by LINK, an industry-backed scheme. The public can search the dataset from the LINK website, but bulk data is available only on commercial terms.

As with supermarkets, access to free cash machines — particularly in rural and disadvantaged areas — is a matter suitable for public scrutiny. Policy on that issue should be supported by public availability of the data.

3. Post box locations


Release of the Postcode Address File (PAF) has long been a cause célèbre for the UK’s open data community; never more so than in the run-up to last year’s privatisation of Royal Mail. However PAF was not the only piece of information infrastructure lost to the public when Royal Mail moved into the private sector.

Royal Mail holds data on post box locations (and collection times) in two databases, the Central Collections Management Database and the Final Plate Database. Although this location data has never been explicitly released as open data, Royal Mail was compelled to provide access to it (in response to Freedom of Information requests) following an ICO decision in 2011. Third-party developers have made good use of the data: see here, here and here.

Post-privatisation, Royal Mail is no longer subject to FOI. Official updates to post box locations are no longer available as free bulk data.

In my view the Government should fix that problem by prevailing on Royal Mail to release bulk data on post box locations, under an open licence, as a function of the company’s special status as designated provider of the Universal Service.

4. Water company boundaries


In England and Wales at least, pretty much every level and type of administrative geography is now available in GIS formats for reuse as open data, via either ONS or Ordnance Survey.

Water company boundaries are a glaring exception. There are a few high-level raster maps around — on the Water UK and Ofwat websites, for example — but the boundary data itself is not readily available. That makes it difficult to analyse water company services and activities in conjunction with other location data, or to build apps that, for example, enable consumers to identify their likely water company by entering a postcode.

Individual water companies maintain their operating boundaries as polygons called Water Resource Zones. The boundaries are provided to the Environment Agency as part of the companies’ Water Resource Management Plans. For internal use the EA collates those polygons into a combined layer called Water Company Boundaries. However due to “complex rights issues”, i.e. water company ownership of the underlying data, the EA cannot release that dataset for external reuse.

It is unlikely that this boundary data has any great commercial value to the individual water companies. In my view the Government should negotiate with Water UK, as representative of the water companies, for release of these boundaries as open data.

5. Mobile phone base stations


Information rights aficionados will be familiar with the extremely protracted battle to secure release of Ofcom’s Sitefinder database under the Environmental Information Regulations. This battle raged up and down the court system, from the original information request in 2005 until a final ruling in 2012 that required Ofcom to release the data.

The Sitefinder dataset is now available in bulk as well as via Ofcom’s search interface. I have released an enhanced and tidied-up version of the bulk data via ShareGeo Open.

Unfortunately the EIR outcome was rather a Pyrrhic victory. Ofcom has not updated Sitefinder since May 2012 and some of the mobile network operators had withdrawn their participation well before then.

However there is still substantial public interest in the locations of mobile phone base stations. (That’s why they’re so often built into church towers and steeples, or otherwise disguised within the built environment.) In my view the Government should require network operators to make base station locations public, as a regulatory requirement rather than on the previous voluntary basis.

Image credits: Supermarket check out by Velela (CC BY-SA 3.0); Worn ATM by Jason Cupp (CC BY-SA 2.0); PostBox1010037 by Bashereyre (released into the public domain); Water supply map nicked from Ofwat site (“complex rights issues”); Mobile phone base station, Heol-y-Broom, by eswales (CC BY-SA 2.0)

1 Aug

Post: 1 August 2014

Yesterday the Environment Agency added some additional open data releases to its Datashare pageThese datasets are reusable under the Open Government Licence.

River and Coastal Maintenance Programme

This download contains two spreadsheets, one describing the EA’s “Frequent” maintenance programme and the other its “Intermittent” maintenance programme. These programmes are about managing flood and coastal erosion risk.

Several comments:

1. Although the download page and spreadsheet names indicate this data relates to the 2013/14 programmes, I think it is actually forward-looking and relates to activities planned for the 2014/15 year. The Intermittent spreadsheet is titled “Intermittent Maintenance 2014/15”. I have asked EA to clarify.

2. EA has only released spreadsheet data, and not the spatial data for the Flood Risk Management System areas to which the data applies. I have obtained a copy of the spatial data (see my post from February) but the EA released that under its more restrictive "EA OpenData" terms, which are not fully compliant with the Open Definition. Ideally I would like to see the FRMS polygons released under the OGL (or the OS OpenData Licence, if they contain OS derived data).

3. EA has published the spreadsheets in the closed XLSX format. In line with the Government’s push to adopt open standards it would be better to use a format that is directly supported by free software, such as XLS or ODS. That said, XLSX is an improvement on releasing the data only in a 33MB XLSM file.

4. I note the spreadsheets omit the Flood Defence Grant in Aid amounts allocated for each watercourse/asset. I think EA should try to be transparent about those figures; both Defra and to a lesser extent EA itself have been criticised in the past for obfuscating the amounts spend on flood protection.

River Habitat Surveys - Survey Details and Summary Results

This is a large spreadsheet. The dataset is described on as follows:

River Habitat Survey (RHS) is the Environment Agency standard for collecting data on the physical character and quality of river habitats across the UK. It is a substantial dataset of significant research interest. RHS is a standard field survey of a 500m stretch of river where data is collected in a replicable manner. At 50m intervals a ‘spot-check’ is conducted to record specific details about bank and channel physical attributes, man-made modifications, land uses and vegetation structure. Since 1994 approximately 24,000 surveys have been carried out. The bulk of surveys were carried out between 1994 to 1997 and 2006 to 2008. Surveys are still carried out for specific drivers, for example assessing habitat availability and Water Framework Directive.

I haven’t explored this dataset in any detail but it looks interesting.

See the AfA286 metadata in EA’s Information for Re-Use Register for more technical details.

Water Framework Directive datasets

EA has also released three new WFD spatial datasets.

WFD - Management Catchments Cycle 2 (Draft) is described as follows:

Management Catchments are the geographical units for which action plans are drafted in implementing the Water Framework Directive (WFD). Catchments have an action plan published that relates to all waterbodies that fall within its boundaries.

WFD - Operational Catchments Cycle 2 (Draft) is described as follows:

Operational Catchments (Cycle 2) show how Water Framework Directive (WFD) work is grouped geographically for practical management purposes.

WFD - River Basin Districts Cycle 2 (Draft) is described as follows:

River Basin Districts are the geographical units showing the area of land and sea, made up of one or more neighbouring river basins together with their associated groundwaters and coastal waters for assessment and action under the Water Framework Directive (WFD).


I confess I have difficulty keeping track of the differences between all the WFD spatial data. Of course it’s better to have too many datasets than too few; only a few years ago it was difficult to find any open spatial data that provided a hydrologically significant segmentation of the country.

For purposes of practical reuse, however, its worth noting that the three datasets above are at a more general level of geography than the WFD - River Waterbody Catchments Cycle 2 (Draft) dataset, previously released by the EA as open data.

Recorded Flood Outlines (not released)

There have been some hints online that EA is preparing to release its Recorded Flood Outlines dataset as open data.

This spatial dataset, previously known as Historic Flood Outlines, contains the individual location outlines and approved attributes for records of historic flooding. See the AfA008 metadata in EA’s Information for Re-Use Register for more technical details.

Recorded Flood Outlines has substantially more potential for reuse than the more “niche” datasets mentioned above. I have worked with this dataset previously in a commercial context, and am quite looking forward to its release as open data. However it’s understandable that there might be some protracted discussions within the EA prior to making the data available to the general public.

Image produced from the WFD datasets described above. Attribution: “Contains Environment Agency information © Environment Agency and database right”.

Page 1 of 17