A. ABOUT - colouring-cities/manual GitHub Wiki

Overview

The Colouring Cities Research Programme (CCRP) is an open data initiative, run by The Alan Turing Institute, set up to help improve the quality, efficiency, sustainability and resilience of national building stocks. The CCRP develops open code for open data platforms that release high quality data on the composition, operation and dynamic behaviour of stocks, for everyone to use. The project was initiated at University College London in 2016 with the international research programme launched at the Turing in 2020.

The CCRP drives capture and open release of building attribute data though an international network of open databases/data visualisation platforms managed by CCRP academic partners. It works to support the United Nations' Sustainable Development Goals and New Urban Agenda, to effect a step-change in the scale and quality of spatial building attribute data available for use in research and practice within and across countries, and to accelerate insights into the stock as a dynamic system using AI and machine learning.

The CCRP is a branded academic programme that enables researchers, from across continents, involved and/or interested in open data and building stock research, to pool knowledge, and co-develop the Colouring Cities network. Each CCRP platform is managed and funded independently by CCRP academic partners, at country level, and each follows an agreed set of protocols drafted by the Alan Turing Institute. The CCRP releases all core code, and additional country specific code, via GitHub. Its standardised open building attribute datasets are accessible via its international platform network under an ODbL licence. The CCRP also publishes all its methods within its Open Manual and promotes The Turing Way which looks to advance ethical, reproducible, and collaborative data science.

Alan Turing argued that "the isolated individual does not develop any intellectual power …The search for new techniques must be regarded as carried out by the human community as a whole, rather than by individuals". The CCRP programme brings together collaborators from across sectors, communities and countries, and from science & technology, the humanities and the arts to help solve complex problems relating to cities and to building stocks. It looks to capture information from - and provide data for - as many individuals and organisations as possible: from government, industry, and the third sector involved in the practical design, construction, management, monitoring, conservation, retrofitting of buildings; from academic bodies and others involved in building/city related research, and from citizens themselves who hold substantial knowledge of how well buildings and local areas 'work'. The project brings together an international group of researchers with expertise in multiple areas of research including: data science, computer science and software engineering, artificial intelligence & machine learning, urban science, industrial ecology, urban morphology, physics, environmental science, material science, building construction, civil engineering, geomatic engineering, GIS and cartography, conservation, housing, planning and regeneration, architecture, history, architectural history, graphic design, data ethics, computer vision, procedural modelling, the mathematics of cities, open data and colour theory.

Kinds of data collected

Colouring Cities platforms use digitised building footprints (of the highest quality, openness and geographic coverage available) as their basic building blocks. These, when combined with Colouring Cities open code, act as mini filing cabinets, allowing information on individual buildings to be easily captured, collated, verified, visualised and released. Live co-creation of maps through the colouring-in of footprints forms a key feature of platform design. (Data collection at building level is critical to understand how the physical 'cells' of a city work, and how to make them work better and last longer). Four data capture methods are experimented with: bulk upload of open public datasets, crowdsourcing at building level, live streaming of official data, and computational generation, as well as feedback loops between these. Through this process issues relating to data fragmentation, incompleteness, inaccessibility, aggregation, inconsistency, security and accuracy are gradually addressed.

The CCRP's core open code repository on GitHub currently allows for the capture and collation of around 100 spatial building attribute datasets shown below. These are grouped into 12 subject areas and are added to over time. Selection of datasets is based on analysis of academic papers relating to sustainability science, urban science, resilience analysis and urban complexity, UK stakeholder consultation on the Colouring London prototype (2015-19), international partner consultation, risk assessment, and live platform testing. New types of dataset are included in consultation with CCRP academic partners.

Mission Statement

The Colouring Cities Research Programme

• advocates for open access to building level data on building stocks to help produce more equitable, resilient and sustainable cities in line with UN New Urban Agenda (NUA) goals

• addresses fragmentation, omission, inaccuracy and incompleteness of data on building stocks, necessary to meet NUA goals, by combining open knowledge methods with computational approaches to large-scale data capture and verification

• supports the development of a network of open data platforms that help identify common problems occurring within stocks and drive collaborative problem solving whilst being inclusive, beautiful and fun;

• demonstrates the value and efficiency of an academic governance model and trusted data management framework and shows how CCRP platforms can be set-up and run sustainably, at low cost;

• tests diverse methods of data capture and feedback loops between these, to maximise data accuracy, geographic coverage, richness and quality, and to engage all stakeholders including citizens in the process of developing sustainable urban areas;

• develops open tools that advance collaborative research on building stocks across science, humanities and the arts;

•. promotes a model for informal, non-competitive, inclusive, international research collaborations that allow knowledge on stocks to be easily pooled from different disciplines and sectors, and shared across cities and countries;

• provides longitudinal data to support use of AI and machine learning able to identify underlying patterns and cycles occurring within cities and across geographies and temporal periods

• raises awareness of the need to prioritise platform users and building occupiers well-being when releasing spatial building attribute data; works to increase debate on data ethics within this field.

International participants

The CCRP is made up of international public research institutions, involved in building stock research, interested in developing and managing Colouring Cities platforms to support common research goals.

Colouring Cities Research Programme platforms are currently being built across nine countries, representing four continents (Asia, Oceania, Europe and South America). Participating countries are: Australia, Bahrain, Britain, Canada, Colombia, Greece, Germany, Lebanon,Indonesia and Sweden.

Academic institutions currently testing Colouring Cities platform code are as follows:

  • Colouring Australia: The University of New South Wales (City Futures Research Centre)
  • Colouring Bahrain: The University of Bahrain (Urban & Housing Lab, Dept of Architecture)
  • Colouring Britain/Colouring London prototype and CCRP management. (The Alan Turing Institute (Urban Analytics Programme)
  • Colouring Canada:Concordia University. Canada Excellence Research Chair in Smart, Sustainable and Resilient Communities and Cities
  • Colouring Colombia: The District University of Bogotá - Universidad Distrital Francisco Jose de Caldas (The Department of Cadastral and Geodesy Engineering and the NIDE research group)
  • Colouring Germany: The Leibniz Institute for Ecological Urban and Regional Development
  • Colouring Greece: The National Technical University of Athens (Urban Planning Research Lab, School of Architecture, Geochoros Geospatial Analysis and GIS Research Group, School of Rural, Surveying and Geoinformatics Engineering)
  • Colouring Indonesia: King’s College London (Department of Geography) & Institut Teknologi Bandung (Department of Urban and Regional Planning, Department of Geodesy and Geomatic Engineering, Green Infrastructure Initiative GIZ, Climate Change Center)
  • Colouring Lebanon: The American University of Beirut (Urban Lab and Department of Physics)
  • Colouring Sweden: Mälardalen University (School of Business Society and Engineering and Department of Energy, Building and Environment).

Colouring Sydney

Operational model & strategy

The CCRP uses a low maintenance academic governance model to maximise platform efficiency, quality and longevity, and to help develop trust amongst users. Data ethics, user well being, and research integrity are prioritised. The diagram below illustrates the current relationship between the Alan Turing Institute, CCRP international partners, and country level stakeholders/data contributors. The Turing currently provides oversight, prototype code and technical support for set-up; CCRP international academic partners set up and manage demonstration models and national Colouring Cities platforms. Stakeholders from academia, government, industry, the third sector, and citizens then use platforms to add and access attribute data for their specific needs whether these be the development of a digital twin, observation of change to the management of a development proposal or a school project. The collaborative maintenance model and diversity of data capture methods allows data to volunteered by individuals and organisations, at national, regional and local level, of a kind, at a time, at a scale, and in a way, that suits their needs. As collaborative work increases, responsibility for CCRP oversight and chairing is likely to be shared between academic partners.

governance model

Creating effective platforms relies on initial engagement, at country level, with through detailed consultation. (In the UK, detailed discussions were undertaken with over 60 national, regional and local stakeholders during the initial development of the Colouring Cities prototype, Colouring London. In Greece, over 500 consultation responses and 30 in-depth sector-specific interviews were carried out prior to [Colouring Athens] being launched.

Each Colouring Cities platform is currently at a different stage of development. We anticipate demonstration platforms to go live for all the above countries this year. Discussions are also being held with academic colleagues in Switzerland, with potential collaboration also being explored in Bangladesh and Vietnam. See here for further information on CCRP international partners and CCRP platform links.

Funding

Since 2016 over £1,000,000 of research funding has been secured by participating countries for Colouring Cities research. Help-in-kind of a similar value has also been provided during the same period by academic partners and stakeholders. (In the UK for example Ordnance Survey has given access to resources valued c£50,000 a year since 2016 for Colouring London prototype research). Individual grants secured by countries are recorded and updated on dedicated CCRP partner pages within the Open Manual. At the start of 2023 over 45 academics and software engineers over nine countries were contributing directly to the project.

Highlights October 2022-June 2023:

  • Launch of Colouring Australia in Sydney at the 7th Smart Data Smart Cities & 17th 3D GeoInfo Conference (Oct 2022)

  • Colouring Dresden awarded 50K euro prize funded by The Federal Ministry of Education and Research (Dec 2022)

  • Colouring Sweden receives seed funding to begin construction of demonstration platform (Jan 2023)

  • Release of open source code by The Alan Turing Institute to test the first streaming of UK planning application data into footprints and colour-coding by progress status (Jan 2023)

  • Reference to the CCRP in the Global Infrastructure Hub's (G20) Annual report (Jan 2023)

Examples of applications for CCRP data currently being explored by The Alan Turing Institute and CCRP partners

CCRP data are being collected across countries ready for use in a wide range of applications. These include: Housing stock auditing ; building lifespan calculation; building adaptability and resilience assessment; energy performance visualisation; building morphology and health and health; public engagement in planning; demolition tracking; developer performance tracking; material waste flow simulation across time; building typology classification and geolocation for retrofit; typology rules for procedural simulation models; heritage protection; engagement of communities in assessing building quality; disaster management/reconstruction crowdsourcing; risk assessment; using age diversity in resilience prediction; dynamic tissue classification; location data ethics, and in the development of digital twins.

Examples of applications in which Colouring London data have been tested as part of research collaborations include: Procedural London, visualisation of Greater London Authority Planning data, and commercial 3D city model development(VuCity mobile app). In Australia, Colouring Australia development is being funded as part of the Australian Housing Data Analytics Platform. In Germany,Colouring Dresdenis being experimented with initially as an open citizen science platform supporting climate-friendly architecture.

impact diagram 2

Key Research questions

The main questions the CCRP asks is: Can open-source code platforms providing open data on the composition, performance and dynamics of building stocks, that use diverse methods of data capture and actively engage built environment stakeholders in their development and maintenance, be successfully set up and managed by research institutions? Can this be done at low cost to effect a step-change in the amount and quality of data on stocks available for integration, synthesis, dissemination, and analysis across countries, necessary to support their sustainable and resilient development?

To answer these the CCRP explores a distributed management model for low maintenance open building attribute platforms, overseen by academic hosts and designed as live, experimental projects, able to be adapted and improved over time in collaboration with stakeholders. Platforms are also used as test beds for experimentation with diverse approaches to data capture, synthesis, analysis and application, and as free tools that support open, visual auditing of the stock; performance and risk monitoring; public education and school curricula; and data capture in emergency situations. Colour and data visualisation required to support stakeholder engagement is central to platform design.

As part of the process of platform design, development and testing a number of sub questions are also asked. These include:

  • What are the main datasets needed in open data platforms to support the development of sustainable and resilient building stocks?
  • Of these which are currently missing or restricted? How does this differ across countries?
  • What methods can be used to address these data gaps?
  • How can we make data as accurate and reliable as possible?
  • How do crowdsourcing data capture methods compare to automated methods and can feedback loops be created to improve data quality and coverage?
  • How can datasets best be maintained and updated, at low cost?
  • Why are spatial data so important?
  • At what scale are data needed and why?
  • How can data on stocks as dynamic systems best be collected, and why are these so important and so difficult to capture?
  • How can we best track change to, loss and survival of typologies over long periods of time and how can typology geocation impact on the efficiency of retrofitting of buildings?
  • What is the most efficient way to build, manage and fund platforms?
  • How can platforms be designed to maximise engagement from built environment stakeholders/experts to create collaboratively maintained systems? * To what extent is this critical for project sustainability and who should be consulted?
  • What other functions could platforms have to increase their cost effectiveness, efficiency and sustainability?
  • How can we ensure the highest standards of data ethics and the security and privacy of both platform users and building occupiers/owners
  • What is the best way to test reproduction the platform model at international scale? How can we best assess CCRP success?
  • How can we best illustrate applications of the data?
  • What are the main challenges and limitations faced?

CCRP platforms and the CCRP Wiki record findings in relation to these questions.

Examples of questions the CCRP aims to answer as data are captured include:

  • What kinds of buildings, and how many, exist within and across countries and where - e.g typology, age, use, size etc?
  • Which perform best i.e. are the most energy efficient? Which do communities think work best?
  • Is there a relationship between typology form and socio-cutural, economic and environmental performance? and does this compare across countries?
  • How long do buildings with similar attributes tend to last and why? Which should we be saving/adapting? Default position should be all?
  • How many buildings have been retrofitted and where? Can knowledge of typology location accelerate the retrofit process?
  • How can granular building attribute be used by academia, government, industry, the third sector and communities to improve our understanding of the relationship of the physical environment to the socio-cultural, economic and environmental performance of stocks cities as a whole?
  • What locked-in patterns and cycles can we see relating to stocks across countries using historical/time series data? What learning can be shared on the success of existing sustainability and resilience strategies?
  • How can we rapidly expand the volume of data and increase the scale and speed of analysis using AI and machine learning?

Context

Buhler et al. (Buhler et al. 2021), note that infrastructure remains one of the least innovative and digitalised sectors, largely as a result of a high degree of fragmentation of data and knowledge. New methods of sharing and reusing infrastructure data within connected open and trusted data ecosystems is viewed as essential to promote responsible digital governance, foster innovation and to exploit advances in advanced and emerging technologies (e.g. artificial intelligence and digital twins) and to enable the sector to become more resilient and efficient and able to meet ambitious global decarbonisation and environmental protection goals. Agile, resilient learning platforms are called for, supporting continuous improvement, innovation and adaptation, and allowing for multiple stakeholders and experts to co-work to meet common goals. The CCRP tests a model for open infrastructure data platforms that meet these requirements, curated by academic departments but maintained by diverse stakeholders at national level.

Since the 1990s, there has been widespread acceptance that cities need, as a matter of urgency, to become more ‘sustainable’ (Chávez et al., 2018; United Nations Framework Convention on Climate Change, 2021). Sustainable development has been defined by the United Nations as a process ensuring humanity 'has the ability to make development sustainable to ensure that it meets the needs of the present without compromising the ability of future generations to meet their own needs.' (World Commission on Environment and Development, Para 27, 1987). In 1994, the United Nations Framework Convention on Climate Change (UNFCCC) was set up to support the stabilisation of greenhouse gas concentrations in the atmosphere (United Nations Framework Convention on Climate Change, 2021). In 2015, the UN’s 2030 Agenda for Sustainable Development was adopted by all UNFCCC member states (United Nations Department of Economic and Social Affairs, 2021). In it, 169 targets were set out to provide a shared vision for, and pathway to, sustainable development at a global scale and to the reduction of global greenhouse gas emissions. The UN’s 17 Sustainable Development Goals (SDGs) as laid out in its 2030 Agenda for Sustainable Development (United Nations, 2015) are shown below.

image

In 2016, The United Nation's New Urban Agenda (NUA) (United Nations, 2016) was adopted by 193 UNFCCC members to deliver the 11th SDG– that of ‘Sustainable Cities and Communities’ (ibid.). The NUA vision is for nations to work together to create ‘just, safe, healthy, accessible, affordable, resilient and sustainable cities and human settlements to foster prosperity and quality of life for all’ (ibid, para. 11). In the NUA, the form, quality and condition of the physical fabric is seen as intimately linked with issues of social equity and citizen well-being, and as being ‘grounded in the integrated and indivisible dimensions of sustainable development: social, economic and environmental’ (ibid., para. 24). Cities are also described as the ‘source of solutions, rather than the cause of the challenges that our world is facing today. If well-planned and well-managed, urbanization can be a powerful tool for sustainable development for both developing and developed countries’ (ibid, p. iv).

The NUA states that to move towards sustainable urban models a ‘paradigm shift, based on the science of cities’ is needed (ibid.). Such a science, though not specifically defined, includes a range of scientific disciplines and fields of research.

A city's largest and most important socio-economic resource is its building stock. The most relevant scientific research into the sustainability, quality, efficiency and resilience of building stocks is currently being undertaken within urban science, in which the mathematics of cities, and underlying rules governing their behaviour as complex systems are investigated (Batty, 2007); in urban morphology, in which the urban fabric is rigorously classified and analysed, and patterns of long-term dynamic behaviour observed (Kropf, 2017); in the science of form, in which the logic of urban form, and the relationship between form and the performance of buildings are studied (Steadman, 2014); and in sustainability science, in which diverse disciplines are brought together to create ‘a new approach to deal with complex, long-term global issues, such as human-induced climate and ecosystem change from broad perspectives’(UNESCO, 2016). Here questions are asked about the adaptability, vulnerability, and the capacity of human-environment systems, including stocks, to be resilient - i.e. to respond to stresses over time (Kates, 2011., p.19450), about the ‘metabolism’ of cities and stocks and their energy and material flows, and how low carbon economies and sustainable cycles of adaptation and reuse, retrofit and repair can be achieved (Aksözen et al., 2017; Chavez et al, 2018; Tanikawa and Hashimoto, 2009). Knowledge and data held within these and other relevant disciplines is however poorly connected, as is the vast body of expertise on buildings held across government, industry, the third sector and within communities themselves.

Buildings and built infrastructure comprise 70-80% of total capital in industrialised countries (Hassler, 2009). Buildings are a city’s most significant physical objects (Shlosser et al., 2020), representing ’the largest physical, economical and cultural capital of a society’ (Bradley and Kohler, 2007, p. 530). Data on buildings are also needed by academia, government, industry, the third sector, and local communities in multiple applications relating to housing, planning, property construction and management, energy, conservation, heritage, tourism, urban economics, health and education. Buildings are also where citizens spend most of their money and most of their time (Building Performance Institute of Europe, 2011)as well as being where the greatest potential for energy reduction in cities lies, with the building stock contributing to nearly half global energy consumption and greenhouse gases (ref). Owing to the scale of natural, social and financial capital held in stocks these resources also function as essential physical, economic and socio-cultural reserves needing to be drawn from in future (Thomsen et al., 2011a).

Stocks are, in addition, complex, dynamic systems, though up until recently acknowledgement of temporal dynamics in the scientific study of cities has been 'almost entirely absent’ (Batty, 2007, p. 3). Construction, demolition and adaptation are however core processes which operate concurrently and continuously within stocks at a range of rates and scales - from rapid, large-scale development and destruction across multiple blocks to minor removals and additions within plots and buildings themselves. Each generates a constantly changing amount of form and space, of varying quality, available for society to use for different types of activity. The rate of change, and number, quality, physical form, use and location of buildings built, adapted and demolished over a specific period of time will have profound implications in terms of a city’s ability to both to flourish in socio-cultural and economic terms, and its ability to moderate its ‘metabolism’ in terms of flows of materials and energy, and related greenhouse gas emissions over time (Chávez et al., 2018; Reyna and Chester, 2015; Tanikawa and Hashimoto, 2009). Comprehensive data required to support analysis and modelling of stocks as dynamic systems are therefore now urgently needed, at a scale and volume never required before (Steadman et al. 2020). Historical data allowing tracking of long-term change and evolution in stocks i.e. spatial building age data, lifespan data and data on typology survival rates, though difficult to collect in numerical form in increasingly sought after. To address this issue new multidisciplinary approaches across humanities and science are required (Aksözen et al., 2017), and are also tested by the CCRP.

Problems with incompleteness, fragmentation, aggregation, inconsistency, inaccuracy and inaccessibility of many types of building attribute data necessary to support sustainability research have in fact now been raised by many scientific studies (Aksözen et al., 2017; Evans et al. 2017; Huuhka and Lahdensivu, 2014; Miatto et al., 2017; Tooke et al., 2013). Difficulties with access at international level were already being noted by Kohler and Hassler in 2002; ‘Most studies are seriously limited by the absence of reliable statistical data, international research confirm this lack world-wide’ (Kohler and Hassler, 2002, pp. 231–232). Scientific models on stock behaviour continue, in many countries to rely on assumed rather actual data, with information on form, age (and lifespan) ‘very seldom available reliably’ (Huuhka and Lahdensivu, 2014, p. 3). The ‘size, structure and the dynamics of its [the stock’s] change are not well known. In order to estimate effects of changing conditions, it is necessary to ascertain what constitutes a specific building stock and understand its dynamic change by modelling the process, based on actual data’. (Bradley and Kohler, 2007, p. 530). Reasons include longstanding fragmentation of knowledge owing to: individual sectors’ interest in/focus on specific types of building (for example the state’s in social housing owing to the scale of investment); the prioritisation of new build and technological innovation by the construction industry and architectural profession, and planning, over adaptation and reuse (ibid.) supported by tax incentives. Lack of drivers to encourage governments to invest in public auditing and monitoring of stocks as a whole; to publicly release spatial building attribute data collected for taxation purposes; to address limitations of using small non-spatial samples for housing analysis, and aggregated and assumed data; and to investigate the relationship of its physical form with socio-economic and environmental performance, and building longevity, and negative cyclical patterns over long time periods, have also been contributing factors.

the Alan Turing Institute- Promoting collaborative, trustworthy research in the field of data science and AI

The CCRP is managed by the Alan Turing Institute, which was set up in 2015 by the UK government as the National Institute of Data Science. In 2017 Artificial Intelligence was added to its remit. The Turing collaborates with 13 UK universities an there are over 500 Turing research fellows working in diverse areas of research.

Research excellence is the foundation of the Institute. Turing researchers collaborate across disciplines to generate impact, both through theoretical development and application to real-world problems. The Institute is fuelled by the desire to innovate and add value. Its goals are to:

  • Advance world-class research and apply it to real-world problems: innovate and develop world-class research in data science and artificial intelligence that supports next generation theoretical developments and is applied to real-world problems, generating the creation of new businesses, services, and jobs.
  • Train the leaders of the future: train new generations of data science and AI leaders with the necessary breadth and depth of technical and ethical skills to match the UK’s growing industrial and societal needs.
  • Lead the public conversation: through agenda-setting research, public engagement, and expert technical advice, drive new and innovative ideas which have a significant influence on industry, government, regulation, or societal views, or which have an impact on how data science and artificial intelligence research is undertaken.

The Turing Way, was set up by The Alan Turing Institute to involve and support a diverse community of contributors in making data science more accessible, comprehensible and effective for everyone. Its principles are promoted by the CCRP, and CCRP partners encouraged to participate in the development of the open handbook.

turing way

The CCRP provides a friendly, inclusive, considerate, informal, creative, non-competitive and experimental space for researchers interested in multidisciplinary problem solving relating to the building stock and its sustainability, and in data ethics and open database design. Our programme is intended to be as welcoming as possible and to encourage and celebrate diversity of skills, gender and cultural background within CCRP teams, contributor groups and Colouring Cities' audiences. The graphic below shows Turing core values advanced through the CCRP.

turing values