Data modelling - ONSdigital/DRAFT_DE_learning_roadmap GitHub Wiki
Data modelling is the process of creating a visual representation or abstraction of an organisation's data and its relationships. It involves defining the structure of the data to be stored in a database or other data storage system. The primary purpose of data modelling is to facilitate a clear understanding of the data, its relationships, and how it supports business processes.
Although data modelling should be a part of data engineers remit in DGO we are in the privileged position of working alongside data architects who often know a lot more on the subject. This page was put together with the help of resources provided by the DAB team and their learning pathway. We mostly use Erwin data modeler at the ONS so some resources focus on this specific software.
Awareness: show an awareness of different data models and tools, and understand when they could be used. Show an awareness of industry-recognised data modelling patterns and standards.
Working: explain the concepts and principles of data modelling. Produce, maintain and update relevant data models for an organisation’s specific needs. Reverse-engineer data models from a live system.
Practitioner: produce relevant data models across multiple subject areas. Explain which models to use for which purpose. Understand industry-recognised data modelling patterns and standards, and when to apply them. Compare and align different data models
Expert: understand the concepts and principles of data modelling and can produce relevant data models. Work across government and industry, recognising opportunities for the reuse and alignment of data models in different organisations. Design the method to categorise data models within an organisation.
Awareness
At awareness level you should:
- be familiar with different types of entity-relationship data model (ERD - conceptual, logical physical)
- be able to read and understand a data model
- be able to explain the benefits of using a data model for the purposes of a particular project
YouTube: Introduction to Data Modelling (2:20)
Working
At working level you should:
- be able to identify which type of data model would Be appropriate for a given situation and/or audience
- be able to seek information for the creation of data models via research and stakeholder engagement
- be familiar with different types of data structure: relational, dimensional, tree, object oriented
- be able to create conceptual and logical data model based on relevant business information
- be able to explain the benefits of using a data model for on going maintenance and use of a data asset
- be familiar with different types of data model notation, e.g. ERD and UML
- understand the principles of database normalisation. At this level of skill you should be the stage where you are starting to build or maintain very simple data models yourself.
Percipio: (Book) Hoberman - Data Modelling Made Simple: this may have to be acquired from somewhere other than Percipio.
YouTube Playlist: Erwin Data Modeler Tutorial: this resource is a little old but should still be useful for the basics.
Practitioner
At the practitioner level you should:
- be able to validate a data model
- be able to design a physical data model from a logical data model, for particular technologies
- be able to apply a data model to aid data migration, integration, extraction and acquisition
- be able to check that a model adheres to the standards for data modelling which are expected
The BCS certificate in systems modelling is not free and it is not going to be required anywhere in DGO but we have included the website as a resource here because you can find some useful free resources there. See also chapter 5 of the DAMA-DMBOK2 and this group of Erwin specific resources put together by DAB.
Expert
At the expert level you should be able to lead on development and management of complicated data models and be able to reverse engineer a physical data model from a variety of databases e.g. SQL Server, Oracle, Hive, Excel etc.