Home - USG-SCOPE/data-dictionary GitHub Wiki

SCOPE is a cooperative effort of the U.S. government's statistical agencies to coordinate and collaborate. SCOPE = Statistical Community of Practice and Engagement.

This data-dictionary project is offering proposals for the data dictionaries to characterize the data sets made available by the statistical agencies.

Our first, 1.0 list of metadata (schema) variables to characterize fields in data sets is uploaded as a Word document: SCOPE - Metadata Scheme for Data Dictionaries - final.docx or a similar version can be read in the browser here.

Federal staff who would like more information, please contact Peter Meyer (BLS) or log into Statipedia and search for SCOPE and Metadata.

Next steps:

• adapt pyjstat to bring in "units" and "extensions" -- consider where to store that info based on issues discussed at https://github.com/pydata/pandas/issues/2485 (done)
• speed up pyjstat's downloads for large data (per jmvg) (jmvg did it; not yet integrated into all versions)
• adapt pyjstat to meet 2.0 JSON-stat spec
• choose license here, following up the issue and https://blog.creativecommons.org/2014/05/09/white-house-supports-cc0-for-federal-government-datasets/
• integrate with Project Open Data's 1.1 schema for catalogs: https://project-open-data.cio.gov/v1.1/schema/, https://project-open-data.cio.gov/v1.1/metadata-resources/
• Gather comments on this draft
• Continue to make example metadata for data sets
• Define the datatypes in this document
• Learn to create the JSON files that represent data sets, eventually with programs doing the work
• Learn How to use github for SCOPE

Gray Brooks has asked us to review the Project Open Data specs, harmonize them where possible, and then ask for consideration of our proposals about data dictionary in the Project Open Data issues database: https://github.com/project-open-data/project-open-data.github.io/issues

This group may work on the format of data returned by Web API requests: dates (which are known to come back in different formats per Mark Elbert), location data which can be in latitude and longitude, category systems aka code sets.

Related materials: JSON analyzer and supporting library links

Regarding datatypes 'interval' and 'ratio' -- these language comes from the research of S.S. Stevens (1959). It is known in the field of metrology. A reference to Stevens' work would clarify this document. Re Stevens see: On the theory of scales of measurement, the main 1946 paper laying these terms out, and his bio on Wikipedia.