Web Conference 2022.05.17 Curb - openmobilityfoundation/curb-data-specification GitHub Wiki

Web Conference - Curb Working Group

  • Every other week Tuesday call at 9am PT, 12pm ET, 5/6pm CET

Conference Call Info

Meeting ID: 898 5980 7668 - Passcode 320307
https://us02web.zoom.us/meeting/register/tZ0lcuCgrjwsHNyZRagmc86b12iCmWGBHfjq

One tap mobile: +13126266799,,89859807668#,,,,*320307# US (New York)

Dial by phone: +1 929 436 2866 (US) (Find your local number)

Agenda

Main Topics

  1. Welcome (5 mins) - Jacob Larson, Omaha
  2. Overview of OpenAPI (10 mins) - Michael Schnuerle, OMF
  3. Passport and Developer Tools (15 mins) - Michael Danko, Passport
  4. Lacuna and validating with YAML (15 mins) - Eric Mai, Lacuna
  5. Feedback and Q&A (10 mins) - Michael Schnuerle, OMF

Organizers

  • Hosts: Jacob Larson, Omaha
  • Note Taker: John Good, Ford AV
  • Facilitator: Michael Schnuerle, OMF
  • Outreach: Angela Giacchetti, OMF

Recap

Notes

Action Items

  1. Issue #87
    1. Working Group participants provide feedback on how IT departments are consuming data specifications, which are easiest to use, and any challenges they face
    2. Working Group participants provide feedback on potential usefulness of OpenAPI for CDS
  2. Determine willingness and level of support to CDS OpenAPI documentation over the coming months

Minutes

  • Introduction, Jacob Larson: OpenAPI for documenting the Curb Data Specification
  • OMF's objective is to get the Curb Data Specification out into the cities for use
  • OMF's goal is to build CDS in the open
  • Recap of development process: all of the work is done on Github, every other Tuesday we have a public working group meeting
  • Next meeting: May 31, 2022, on privacy

Michael – overview of OpenAPI

  • Real-world examples of companies using it or something similar

  • Why you should care about this technology

  • The intent today is to get feedback/Q&A, see if you think it is a fit for our curb work

  • APIs: a digital interface to expose functionality or receive data between software applications

  • These can be documented to establish interoperability on a standard basis between user groups

  • People using the curb or monitoring the curb – allows data to flow in a secure way between parties in a standardized format

  • OpenAPI: an openly defined industry standard way to describe HTTP/web-based APIs

  • Instead of just using text on Github, you can describe them as a digital structure.

  • Open does not mean “public” here, just created in the open and free to use

  • The OpenAPI specification is managed by the OpenAPI Initiative (previously Swagger spec), much like CDS is managed by OMF

  • Benefits of OpenAPIs: simplifes collaboration, common understanding to explain what functionality is available within an API – automated code generation & testing of API endpoints, automatic generation of documentation and visualizations to assist with API adoption

  • Instead of it just being a document on Github with static documentation that you have to read, there is more interactivity

  • There is sample code to test things – which inputs work and what does the output look like

  • Ability to have online/offline tools to validate and document to better understand CDS and its various API components

Complementary benefits:

  • OpenAPI specification – industry-standard spec that outlines how they are structured
  • Mechanism to display how the documentation file is displayed / ensuring that it is readable and easy to understand
  • Key Question for OMF and the Curb WG: Why and how?
  • If we collectively put CDS into the OpenAPI format for the next release, why would we and do this?

Next steps:

  • Passport/Lacuna presentations
  • Michael follow-up discussion

Michael Danko, Passport and OpenAPI

  • Key objective of this presentation: How we are using these specifications
  • How is Passport currently using OpenAPI?

Passport has published multiple APIs – can demonstrate how other partners can interact with our APIs:

  • Access token API

  • Payment tokenization API

  • We still integrate specific functionality at the request of our clients

Why did we do this? What are the benefits?

  • OAS files help people understand the different functionality and how it integrates with another application

  • A common standard language

  • OAS – a language to convey the functionality and technical limitations of the spec

  • Source of truth, and change management

  • APIs go through different versions and functionality changes over time

  • Adopting OpenAPI has allowed us to improve our change management, and understand when we are making a “breaking change” and changes that allow us to preserve backwards compatibility

  • The raw code file is intended to be read by software

  • In the OpenAPI version, it is human readable, and easier to understand how data is structured

  • As we accelerate adoption, this automated documentation becomes more important

  • Example: We are seeing an uptick of smart/connected ALPR devices

  • Easy way for our customers to connect those devices to Passport APIs, to determine who has paid for parking

  • Provided a technical example of what we are providing

  • Stoplight.io is a piece of web-based software to automate the creation of the OpenAPI standard documentation, and to help API creation

  • Public-facing

Eric Mai, Lacuna

  • Technical info: To write an OpenAPI spec: it’s just a .json (JavaScript Object Notation) or .yaml file
  • YAML: a more concise way to define JavaScript objects
  • OpenAPI provides for a rich and complete spec

Options for OpenAPI implementation:

  • Specify the format of the request

  • Examples where people can look at the types of API implementation

  • Metadata for any object in the spec – why things are the way they are

  • Can define administrative processes, including contacts and tech support

  • License that is used

  • It can also help optimize the validation process

  • Having this level of documentation can help ensure that we have not unintentionally changed the shape of requests or format

Question: how do you handle version control?

  • Lacuna: we don’t have conflicting versions, we update all at once

  • The other part that helps – each back-end application has its own OpenAPI spec, e.g. a geography

  • Would only define payload shapes around those

  • Segmentation based on division of responsibilities

  • Lacuna: we are not currently validating request payloads in MDS payloads right now

Michael – an open question is how we would incorporate this into CDS?

  • CDS is a bit different than Lacuna’s APIs – it will also live in Github to some extent (to ensure collaboration) – but we would have to make sure that both versions are up-to-date
  • Having a technical contract is super useful for both producers and users of the API

Michael – we expect to receive data in a specific format

  • An OpenAPI spec can be used on both sides, for the creators and users of the spec

Brian, SDOT – just so I’m understanding correctly, it is not written into OpenAPI?

  • Eric, Lacuna – they have a separate MDS implementation that they have created
  • Michael, OMF – we have put MDS in a different validation/documentation system

Michael Schnuerle, OMF

  • Essentially, using OpenAPI would be digitally describing CDS

Key Questions:

  • Would our members/cities find this useful?

  • Do we have enough critical mass to move forward?

  • If you are interested in this, let us know. Email us.

  • Background: This request originated in a Github feature request.

Kenya, SFMTA: building on Brian’s question, beyond the current validation method. I am curious why MDS hasn’t used OpenAPI?

  • Michael: We are currently working towards an OMF best practice

  • Background: Currently with OMF, we’ve used a python validation schema, embedded in the spec itself

  • Michael: this screenshot is what has been used to validate MDS for the last three years

  • The text file on the right is very similar OpenAPI

  • In March 2019, this issue came up for MDS

  • There has been a lot of questions and discussions on this over the years

Important Note: Michael: if we go through with this for CDS, we would likely go through with this on an MDS perspective

  • We would discuss with MDS committee to see how it could be implemented

  • We would determine what type of tool that we would use, and then we would do that for both

  • Michael: when MDS was created in 2018, OpenAPI was not as well defined and widely used

  • As it has changed from Swagger to OpenAPI, it has become more widely accepted

  • Michael: JSON schema is what we used to define the MDS, but it doesn’t have the same adoption and ecosystem that OpenAPI has

  • There is an ecosystem of libraries, a python script that validates/matches your JSON splot

  • Reference: Json-schema.org/implementations.html

  • However, this spec format is not as nice to work with from a human readable perspective

  • The Python validator option is written into the MDS spec

Key Questions for the Working Group

  1. How do your IT departments like to consume data specifications?
  2. What challenges have you seen with consuming data specifications?
  3. Are there any data specifications that were particularly easy for your IT departments to consume?
  4. If we published in this format (OpenAPI), does that look useful?
  5. What kind of information would you like to see there?
  6. How would you be willing to support this work (volunteer or financial) over the coming months?

Key Questions to help us decide?

  • Does this help implement CDS?
  • What other information would you like to see around this?
  • How can you support (if you find this useful)?

Implementation Parameters

  • Michael – some members of the steering committee could help lead that effort
  • We don’t currently have a mechanism to fund one-off projects at OMF
  • We might need a funding mechanism for a one-time purpose
  • It might be a bigger task than OMF’s staff capacity
  • We have paid third-parties previously for major version updates for schema updates

Important Clarifications, from James Wilson:

  • OpenAPI is an extension of JSON schema – all three can interoperate
  • OpenAPI is a superset of JSON schema
  • Michael – therefore, it might work out of the box on the MDS…
  • OpenAPI is a more web-oriented representation of the data
  • Michael – essentially, we would already have a very good starting point
  • More support of the author compared to the JSON schema

Matt Davis, Populus

  • Populus uses OpenAPI to document our APIs that we provide
  • It is a widely adopted tool
  • We are using a web development framework that includes generating OpenAPI documentation from the code itself – not a separate project
    • e.g. the API tooling itself can generate it – a tool called FastAPI, but similar things exist for many web frameworks

Do any of the tools that OMF use have the option to automatically create it?

  • Matt – for some of the APIs, it is not the auto-generated one
  • Developers of populus.ai
  • At Developers.populus.ai, there is a Manually maintained file
  • Matt – I will put it into the chat

API Examples from Populus:

  • List MDS Geographies / List MDS Policies

  • Policies/geographies/agencies API for MDS

  • Michael – once you define it in OpenAPI spec, you get a well-developed web-based interactive documentation that you can manipulate in different ways

Next Steps

  • Michael – we will be working towards the next release of CDS, either 1.1 or 2.0
  • Feel free to leave your thoughts and comments in the Github issues page

  Chat From Allie Moise, Passport to Everyone 11:02 AM Will the recording be shared after? shared* From OMF Zoom to Everyone 11:03 AM Yes the recording will be shared afterwards on the meeting page, like we have with our other CDS meetings. https://github.com/openmobilityfoundation/curb-data-specification/wiki/Web-Conference-2022.05.17-Curb#agenda From Allie Moise, Passport to Everyone 11:03 AM Thank you! From Kristin Tufte (UW - TCAT) to Everyone 11:21 AM No questions at the moment, but super useful, thanks! From Michael Danko (Passport Labs) to Everyone 11:28 AM A "technical contract" is super useful for both the creators and consumers of an API to be on the same page. From Matt Davis (Populus, he/him) to Everyone 11:39 AM It's a spec called jsonschema. https://json-schema.org/ From Matt Davis (Populus, he/him) to Everyone 11:49 AM https://api.populus.ai/v2/api-docs https://api.populus.ai/v2/dev-docs

Participants • Michael Schnuerle, OMF • Jacob Larson, City of Omaha • Eric Mai, Lacuna • Dominika Kowalska Bernhart, Flowbird • Alex Demisch, SFMTA • Brian Hamlin, Seattle DOT • Deepanshu Girdhar • Jacob Malleau, CurbIQ • James D. Wilson, VADE.AI • Jean Kao, Populus • Joe Bryan • John Good, Ford AV LLC • Kenya Wheeler, San Francisco MTA • Kristin Tufte, UW – TCAT • Marisa Mangan, SANDAG • Mark Mockett, USDOT Volpe Center • Michael Danko, Passport Labs • Tom Carranza, Los Angeles DOT • Trevor Thomas, City of Santa Monica • Tristan Charreton