G2. BUILDING FOOTPRINTS SOURCES - colouring-cities/manual GitHub Wiki

Introduction

This section provides information on:

Building footprint use in CCRP platforms

Building footprints form the basic building blocks for Colouring Cities platforms. Footprints allow CCRP open data to be captured, collated, verified and visualised. They can also be used to infer other building characteristics such as typology, height and land use, on their own or in conjunction with other attributes. To produce the highest quality open data possible, Colouring Cities platforms require access to the highest quality, regularly updated, most comprehensive, geometrically precise footprints. These are most commonly held by national mapping agencies. Where access to national mapping data is restricted, charged for, or not available, other open datasets will need to be used. In recent years open footprints have come more widely available owing to: the release, in a number of countries (mainly since 2015), of open property tax datasets; increased interest from OpenStreetMap in collating building footprints datasets, and open datasets now produced by Microsoft and Google using satellite imagery and AI, producing block level footprints at global scale. This has radically changed the level of interest in granular building attribute data and significantly increased opportunities to understand and analyse stocks. It has also raised a number of security and privacy issues which now need to be addressed.

Privacy and security issues

Building level footprints are only used in CCRP platforms to capture, collate, verify and visualise non-personal open attribute data at building level. CCRP platforms do not aim to release open footprint datasets at building level. This is due to security and privacy concerns regarding the use of footprints to collect and visualise of personal data relating to building occupants. e.g relating to income or health. Colouring Cities national platforms are asked to control building level footprints behind firewalls regardless of whether these are released by public bodies open data. Data relating the interior of buildings is also considered by the CCRP to be private. Visualisation of, for example, energy ratings, energy use and heat loss, at building are is seen grey area requiring further discussion with visualisation currently recommended at block not building level, again even where these are released publicly released at building/property level. Wider discussion regarding regulation in relation to spatial data collection, is promoted by the CCRP, especially owing to growing commercial interest in this area. (Concern regarding microscale visualisations also extend to georeferenced point data at building level, though not used in CCRP platforms).

Building footprint dataset generation

Papers and links providing information on building footprint generation, comparison and quality are collected below:

VIDA release of google-microsoft-open-buildings
[Building footprint data for countries in Africa: to what extent are existing data products comparable?] Looks at similarities, differences and coverage of Google v2, OSM, Ecopia and Microsoft building footprints as of January 2023.https://www.researchsquare.com/article/rs-3334423/v1

CCRP platforms - Footprint sources

Below is a brief summary of sources of footprint data used Colouring Cities academic hosts in partner countries.

COLOURING AUSTRALIA

Current building footprints are an amalgamated set of polygons obtained from OpenStreetMap and Microsoft Bing Maps. Work and negotiations are currently underway to obtain authoritative building datasets from governmental and private custodians including Geoscape and Vexcel. For Colouring Australia's paper explaining issues with use of Microsoft/Bing footprints see here

COLOURING BAHRAIN

Currently being discussed with Bahrain's national mapping agency Survey and Land Registration Bureau and Ministry of Works, Municipality Affairs and Urban Planning. -Esri representative office in GCC.

COLOURING BRITAIN

The Colouring London prototype has, since 2016, used Ordnance Survey MasterMap polygons as its sole source of footprint data. OSMM offers the highest quality, most comprehensive and updated footprints available for the UK. Updates are ongoing and available for integration. OSMM data are accessed from Ordnance Survey (OS), the Uk's national mapping agency, under an OS Public Sector Geospatial Agreement signed by the Greater London Authority. Additional authorisation to use OSMM was also required from OS.

The agreement with OS requires that vectorised footprints, to which Crown copyright restrictions apply, must be protected from download. Footprints are therefore transformed into raster tiles which facilitate data capture and visualisation but are nor released. Coordinates are not permitted to be derived from footprints/polygons, though OS release since 2020 of Unique Property Reference Numbers (UPRNs) now enables coordinates to be freely accessed.

Each OS building feature is taken as a "building", so building_id <-> toid is 1 to 1 Commonsense buildings could span multiple features, or multiple buildings could be represented by a single building. Buildings may have Unique Property Reference Numbers (UPRNs) (multiple properties with multiple UPRNs within one building). UPRNs may span several buildings (on a more complex site, covered by one parent UPRN)

The Colouring London prototype is now expanding to become Colouring Britain- with CCRP platforms engineered to be easily scaled. Unactivated OSMM footprints can be viewed on the Colouring London site by zooming out of London. It was originally envisaged that Colouring Cities code would be tested by individual cities and towns in Britain/UK, and that separate OSMM licences would be negotiated with. OS has said that no further access can be permitted to OSMM other than for possible two other test sites.

Microsoft Bing footprints have been identified as too aggregated for use in data capture for most CCRP classes. Experiments are currently being made using the UK INSPIRE open land parcel dataset and OS Open Map Local which is a more accurate version of Microsoft Bing (i.e. at block level. Building level footprints will continue to be kept behind a fire wall for security purposes until the wider implications of releasing building level footprint data are better understood. (Last updated 7.10.2023)

OS historical maps also offer access to historical footprints at c20 year increments from c1870. Vectorisation of footprints within historical maps is further under 'Historical footprints' below.

For further information on footprint use in Colouring London please see here. (Last updated 7.10.2023)

COLOURING CANADA/MONTREAL

The geospatial data collected for building footprints comes from various sources published by Natural Resources Canada, City of Montreal. The primary semantic information was extracted from a city-run cadastral data portal, which provides georeferenced building lot shapefiles and building footprints for six boroughs in Montreal. To expand coverage, we incorporated satellite-based image segmentation data from Natural Resources Canada, covering the entire province. We generated individual lot-level buildings for higher accuracy by segmenting the satellite-derived footprints using the city's cadastral parcel data. Building heights were then added using LiDAR data. We pre-processed the data after a spatial join operation, eliminating duplicate entries and polygon artifacts through area-based filters.

Data Sources

COLOURING COLOMBIA

The data source used for initial testing will be provided by the IDECA (Spatial data infrastructure of Bogota). This local authority integrates more than 98 local departments and spatial layers in an open a reusable license. Base Map of Bogota: https://www.ideca.gov.co/recursos/mapas/mapa-de-referencia-para-bogota-dc Buildings: https://www.ideca.gov.co/recursos/mapas/predios-bogota-dc. Other datasets considered include OpenStreetMap (OSM) and local datasources. The local team will implement a GIS system to help the validation and integration required to access all layers.

COLOURING GERMANY

To be completed https://opendata.dresden.de/ https://www.geodaten.sachsen.de/

COLOURING GREECE

A hybrid model is currently being tested for Athens combining footprint from the 2001 Census and held by the Hellenic Statistical Authority with OSM data. The advantage of Hellenic Statistical Authority data is that it is comprehensive government data. the disadvantage is that is out of date and the next data will not be released until the next in Census in XXX. The advantage of OSM data is that it is relatively up-to-date for the semi-central municipalities in Athens central Athens and allows platform users to download Colouring Athens footprints as open shapefiles. The disadvantage is that footprints only cover parts of the city, it is not known when more will be added or when existing footprints will be updated.

Hybrid

athens footprints 2

COLOURING LEBANON

Building footprint attributes used in the construction of Colouring Beirut were obtained from the National Center for Scientific Research CNRS Lebanon. Footprints were manually digitized over the entire city of Beirut by CNRS-L using aerial photos at 15cm resolution and VHR pan-chromatic satellite images from Pleaides-1A at 70cm resolution.

COLOURING INDONESIA

Building footprints are available from the Indonesian government, free of charge, available on Jakarta Satu website. Link and image to be added

COLOURING SWEDEN

Building footprints are available free from the Swedish government at national level. However, as with Colouring Britain, though footprints can be used to capture and visualise building attribute data, they are not currently permitted to be downloaded for public use.

Historical building footprint data - integration within CCRP platforms

Historical maps and other types of historical information are integrated into CCRP platforms wherever possible to support public understanding of the evolution of urban areas, to improve predictive models, to increase understanding of the way in which typologies change and adapt over time, and to better understand stocks as complex dynamic systems. However availability, granularity, precision and openness of digitised historical maps varies considerably across countries with, for example, some countries such as the UK lucky enough to have access to building level national mapping agency maps at national scale at c20 year increments from c1870 to the late 20th century, to other countries where no historical maps may be available. A key role of CCRP platforms is to also demonstrate the value of existing paper map archives, the importance of archiving of building footprint datasets year on year to allow change to be tracked.

Digital copies of historical paper maps (geoferenced and in raster format) can be easily integrated as layers within CCRP platform interfaces. These can be overlaid with either current or historical vectorised building footprints, at building footprints. An example of overlaying of current vectorised footprints onto historical maps and collecting data on survivals and demolitions can be seen at https://colouringlondon.org/view/age/.

A key challenge has been to access_historical vectorised footprints_ at building level for specific timepoints, necessary to collate and collect historical information, as well as to track change. The CCRP works with the Alan Turing Institute's Computer Vision for Digital Heritage Special Interest group to support research into AI and machine learning approaches to footprint vectorisation, and offer use of the platform prototype for experimentation. It currently collaborates on historical footprint research with a group of research institutions, based in Switzerland, Germany and the UK; the Digital Humanities Laboratory at the École Polytechnique Fédérale de Lausanne/EPFL, The Leibniz Institute of Ecological Urban and Regional Development/IOER, The University of Bristol's MAPHIS initiative (part of the Mapping History international project), and The University of Cambridge. As part of the collaboration, vectorised building footprints will be generated from Ordnance Survey historical maps, integrated within the Colouring London prototype. They aim is to provide:

Open historical footprints able to be downloaded and experimented with, within diverse areas of research including analysis of typology adaptation, mutation, persistence and resilience, and identification of underlying rules of operation within stocks, and locked-in patterns and cycles;
Mini filing cabinets into which historians, and others, can upload historical data (e.g. historical land use or attributes of the building if demolished);
Links to historical sources and archive collections providing information on the history of individual sites.

Building footprint extraction from historical maps - academic paper links

Relevant publication links may be found below

Petitpierre, R., & Guhennec, P. (2023). Effective annotation for the automatic vectorization of cadastral maps. Digital Scholarship in the Humanities. https://doi.org/10.1093/llc/fqad006