OTST meeting minutes - UnionInternationalCheminsdeFer/OSDM GitHub Wiki

Link to OTST Team Meeting

2024-04-23 Meeting

  • **OTST scenarios development:

P.Heuguet presented to the OSDM EC meeting the statuts with the difficulty to developp all scenarios required only based on the free time of OTST members participating on volontary basis. A lot was already done by the team delivering the scenario development framework, and on Sqills side the validation test tool.

An proposition was done to use an UIC budget to issue a tender requesting an external independant company to developp the scenarios. The OTST team is still responsible to define the scenarios to be developped (producing a Scenario Functional Raquirement document) and controling anthe developpment done by the external supplier, and validating the scenarios delivered.

OTST team disucssed the principals of the tender

  • Deliverables : The tender does not define a number of scenarios to be delivered, but more a process to request a scenario developpment and a pricing per scenario based on tee-shirt sizing principals. The rdelveloppemnt request will be managed on a "go with the flow" principal".
  • The team suggested to add in the Tee-shirt sizing elements not only added validation tests but also mofified "validation test" to evaluate the size of the developpment.
  • The team suggested also to add in the process for the scenario price evaluation. A first step would be for the supplier to provide an estimation of the tee-shirt size after analysing the SFR document. but the final sizing will be reviewed after the developpemnt as potentially some new modification could have been introduced during the development phase and must be taken in account.
  • The team suggested to provide more details on the documentation the external supplier must provide : Scenario Solution Document (SSD). The SDD should allow a company willing to use the scenario to understand what parameters they must provide to launch the scenarios, what use case is tested and what elements are tested with which expected results. The SSD should also be the entry point for a developper willing to maintain the scenario to understand the scenario architecture. More detail could be found then after in the scenario code with remarks .
  • Sqill Test Tool : Tim booked a slot this week to test the last changes done on the 2 first scenarios and finalise theri intehration in the OSDM official environment. He will also during this slot validate the last version of the test tool, and officialised its delivery.

As a reminder, in a first stage, the Validation Tool will run on a Sqills operational environment, and later, Sqills will provide the tool inas an open source. At this stage the code will be availble on the official OSDM environment. Details on how this open source will be delivered and hosted will be discussed later.

The team discussed about the namine to be given to the tool, and agreed on Osdm Validation Tool (OVT)

2024-03-15 Meeting

OTST UIC PATRIC team meeting ( P.Heuguet, Tim Van Den Berg, Josef Petrak, Clemens Gantert, Pierre Watier

  • OTST scenarios and Sqill Test Tool : The URL dedicated for OTST was provided " testing.osdm.io ". Josef and Tim will work together to migrate OTST environment on this official one.
  • Update the OTST solution to potentially run the same scenario multiple time with different data setting.
  • Authentification, enrich the scenario to support more authetification processecus and versions
  • Authentification : Need to define a minimum set of authentification to be certified (for security reasons) and even if normally out of scope OSDM
  • In all scenario, it is necessary to add a schema checking assertion
  • In the scenario configuration, need to add a parameter to specify the OSDM version used
  • OTST scenario error report: The current test version of the OTST Test Tool includes HTML report, could integrate later a PDF version. For each scenario run, there will be a dedicated detailed report page§ with all information
  • Can we have access in the detailed report on the message content tested (referencing the link where to find the message content
  • If an error message is return by the called application, can this error code can be dosplayed
  • As a certification could integrate several sceanrios, could we have a summary report providing for each scenario run its reference, the status (number of failed assertoons) and clicking on the number of failed assertion accessingt o the detailed report ?.Of course having on the other way the capability to retur to the Report summary from detailed report.
  • PATRIC certification content: PATRIC team will define the minimum list of sceanarios allowing to provide a first certification level. It should integrate
  1. Positive test
    • Running the functional with a succesful results
  2. Negative test
    • Requesting a funtionality not supported by the application and check error message provided
    • like Bicycle, Meal, etc...
  3. Versioning test
    • Send a request with a version higher than supported one..and check the error message sent back
    • Send a request witha lower version and see behavior
    • ....
  • As all the scenarios required for negative and versioning tests are not yet available and OTST or PATRIC team does not have development capacity, we will check with teh first company asking for certification if they are willing to contruibute providing test scenarios (based on OTST framework) that will reviewed and validated by teh OTST team as compliant to be integrated in OTST catalog.

2024-02-27 Meeting

  • OTST environment inOSDM.IO : The URL dedicated for OTST was provided " testing.osdm.io ". Josef and Tim will work together to migrate OTST environment on this official one.

  • OTST scenario for certification: The proposal to use a scenario ref inbedded inside the customer data file was discussed. Confirmed as feasable allowing to have in one file under customer control the list of scenarios he is willing to run and associated customer dat to be used for this scenario. Based on it, Postman and Sqills tool can run automaticcally all listed scenario and provide results. It is highlighted that only the scenario reference is not enough, today each scenario includes also scenario type parameters (like both when runiing admission and resevation) Those parameters are still rrquired to fine tune the setting of the scenario behavior.

For the results, Sqills will look at the possibility to have a report including a first chapter listing the overall scope to targeted certification using the scenario reference and scenario types informations. And then a second chapter providing a detailed report on the certificaation outcome

  • OTST certification scope : Some important use case from a distributor popint of view were discussed (like a technical ticket cancelation when aone of the ticket of a multiprovider solution is failing). Technically speaking this is not currently a mandatory OSDM functionnality, which does not prevent to have bilateral agreements to support it and request tobe cerrtified on this optional function.After discussion, this specific case can be discussed at the technical forum in the scope of error handling if someone is willing to make it mandatory.

2024-02-20 Meeting

  • OTST environment inOSDM.IO : Still waiting for the domain name to finalise the setting to store the library in a dedicated domain name. Josef will check with Andreas the status of this domaine name creation. Odile tested the "how to" video to run the tests, found some issues that was fixed. Once this new environment will be ready, Odile and Pierre will use the content of this new OSTST repository to test how to retrieve and run on postman sceanrios.

  • How to video : A new set of video was created by Tim. This new set will be reviewed to be run on final OTST environment. In the mean time, the temporary link to this video will be shared with Pierre to have his feed back (in 2 weeks) on potential issues he faced running the tests.

  • Scenario validation reporting: Discussion about what granularity msut be used to define the scope of a certification. Odile proposed to look at NDC catalogue https://retailing.iata.org/armi/docs/ as a reference. Odile will try to find someone in Amadeus knowing this process to present it. Patrick will do the same with IATA with Marie. One decision taken is to use as IATA a sceanrio identifier, allowing to use this identifier in the data file the company who wants to be validated will create to run his validation. The main advantage is to avoid impact on the validation tool each time a new scenario is created.

  • Certification scope : Question about what type of companies are in the scope Railways, 3rd party distributors and / or IT providers ? Currently, OTST team is focussing on Railways certification. TRhere is a grey area between a Railway OSDM interface and a 3rd party distributor as a Railway can provide thrught its interface combine offers like a distributor. What can be certified will depend on what OTST team is capable to deliver. The target is to focuss on what is generated 80 % of the transaction volumes.

  • Scenario user's feed back: SBB used the stand alone seat reservation sceanrion to check its NOVA OSDM integration. A feedback presentation will be done next week

2024-01-30 Meeting

  • Validation scenario delivery : In 2023 the team delivered the 2 first scenarios and the development framework. To deliver all necessary scenario, OTST team is proposing to work with the test teams of each carriers implementing OSDM to work together and having them helping in developping the scenrios required for their internal validation using OTST framework. The first team that will be contacted is SNCF. A dedicated kick off meeting will be organised to define the way of workig together. P.Heuguet will organgise the pre kick off meeting with Tim, Josef and Pierre Watier from SNCF next week.

  • Scenario validation reporting: Before increasing the number of scenarios the team propose to have a review on the way test are reporting. It will be at the agenda of the next session.

  • Certification process : A first pilot will be run with Iryo based on the scenarios delivered in 2023, the kick off meeting planned fr tomorrow is postponed due to calendar constrains. OTST team is in support to provide the tools and scenarios. UIC PATRIC audit team will lead to define the certification process. OTST is invited on the 21st of March in Bern during the next PATRIC team meeting to present all tools developped including the Sqills apps.

  • Official OTST repository framework : Josef worked with Andreas to setup the environment. It should be finalisied next week, and then Josef will migrate the OTST dedicated pages on this new environment.

2023-12-19 Meeting

  • Validation scenario delivery : 2 scenrios were delivered. One allowing to tes admissiçon without or with reservation and another one to test Stand Alone seat reservation. Both were tested on a "stand alone" postmant PC (Patrick). Working perfectly. The one for Adission plus seat reservation was tested by Martin from Turnit succesfully for an admission with manadtory reservation
    Both scenario will be updated to support request by tripserach or trip seclection. For the trip search, filtering is necessary to garante that only offers linked with the tyoe of scenarios to be validated will be returned. Filtering used will be the train numbers. The user can put multiple trains numbers (if a train has multiple one's or if he wants to test a trip with multiple legs).
    All scenarios delivered by OTST does not allows to test a "ramdom" response.

  • Scenario validation reporting: Before increasing the number of scenarios the team propose to have a review on the way test are reporting.

2023-12-05 Meeting

  • Iryo validtation / cerification : Juanjo presentation on the scope of their certification request. Scope is IRT sale and refund all. Already the first scenario under finalisation cover the sale process. Need to developp the refund all afterwards. The 24 Feb 24 target is achievable. Patrick reminded that scenario developped inside OTST are guaranted as being fair and impratila despite however member of OTST is delivering it, as all members can review its content and check its behaviour on their own OSDM implementation. OST members comfirmed the work is done in an open and fair manner.

  • Validation tool demonstration: Tim did a demonstration of the solution, running locally the scenario in its postman environment (he downloaed the OTST postman libraries in its own environment). He also demonstrated the same scenario run from the validation tool. Aside the user firendly interface facilitating the tester work, the advantage of the Validation tool is its capability to always check the scenario version he is running and secure it is always using the latest version. While locally a user could forget to updated the scenarios he is using in the long term.

  • Postamn libary mgmt:To use the scenario in a local environment, we should provide a link to the library repositry. We have 2 choices

  1. We use the GITHUB library link (Patrick had in lind there were drawback discarding this solution. Need to be cross checked)
  2. Have in UIC a repository URL with a sync mechanism with github. In this case a mecanism should be put in place to secure this repository is fully in sync with github content.
  • Postman OSDM github environment : Patrick to check with clemens to have it available.

2023-11-28 Meeting

  • First OTST scenario delivery : Tim will finalise its modification and upload them on Bileto github for a final review with Josef Josef will work on the scenario documentation to finalise it. Tim is planning to do a dmo to the next meeting of the validation tool with this fist scenario.

  • Validation scenario catalog: Iryo asked to be cerified and agreed to pilot this activity. They will comme to the next week ot present what OSDM capabilities they want to certify and discuss with us on how to organise the delivery of required scenarios.

SBB on is side would like to use the first sceanrio about Seat reservation to check SBB OSDM implementation for this feature. Patrick will exchange directly with Josef on this point.

2023-11-14 Meeting

  • First OTST scenario delivery : Tim provided its proposed update, Josef and Tim will test it deiver a user guide to use it for someone to use it in its environment. Potentially it ciuld be fine tune organising the code with ore stuff in the library, but already useable as such. This scenario must be enriched for the validation library to meet the scenario objective. Those validation steps should be implemented in libraries to be reusabe. It should not impact the scenario skeleton. Tim propose to work this week on the offer validation allowing to have a first scenario skeleton with offer validation (still pre booking, booking, ticketing validation to be deliver).

  • Validation scenario catalog: The proposal to be focussed on first certification (SAMTRAFIKEN, SJ, Iryo, etc...) the best is to focus on what they are willing to certify as MVP. The OTST team propose to deliver the scenario in an agile mode, along the way, alowing to certifying step by step as soon as a scenario is ready rather to wait 3 month to deliver all at the same time. Odile proposed to look at IATA NDC and the way they organise the NDC capapility catalog 5airline Retailing Index) as a source of inspiration. Link to IATA web page : https://retailing.iata.org/armi/

  • Next steps: Tim/Josef : Finalise the 1st scenario adding the user guide allowing someone to use it in ots own environment. Tim: Propose to enrich the 1st scenario Offer validation based on what you have developped internally Patrick: Contact Iryo, SJ and Samtrafiken and others implementing OSDM to have their owon lwished list of scenarios and priority for delivery plan Odile: to prepare a presentation on NDC ARMI providing a list of NDC capabilities certification list

2023-11-06 Meeting

  • First OTST scenario delivery : It modifuied the scenario to use the postman environment viariables in which he provided a data_config parameter with a list of config info in rest/json format...and a backup with a link to a cloud based file. After discussion the team agreed it is a good solution allowing to run on postamn in stand alone or on the Sqills validation tool. Tim will load the updated scenario on the Bileto github for final review with Josef.

  • Validation scenario catalog: After a long discussion, the team agreed to based the catalog on OSDM capabilities (like sales of adissions, sales of offers including admission and reservati or offer with stand alone reservation only.... Participants will propose some list of scenarios for next meeting. A second item was adressed, how to manage the different manner to build a request (like for example for the offer based on O&D, Train ID or trip ID. The team agreed it should be triggered buy the parameters configures in teh config file, the scenario adapting its self the request depending on the config file.

2023-10-31 Meeting

  • Demo environement : Sqills provided a vodeo based on the demo he did on the 24rd of October. This was presented to the PES UIC members on the 27th October during the PES.

  • First OTST scenario delivery : Tim sent its last updates to Josef. They will organise a meeting to progress this week. Tim also worked to see how the config file can be setup localy by the postman user for him to easly configure it. No obvious ways were found, He will send a mail to Sayantan from AMadeus to croos check with him. Otherwise, the configuration setup will be more complewx but still acjievable. with a detailed documentation to explain it.

2023-10-24 Meeting

  • Demo environement : Sqills shown a demo of the tool configuring it for the Sqills sandbox running the scenarios and showing the reports, and them the same for Bileto environment showing we have an environment agnostic solution. A recorded demo will be done for the PES

  • First OTST scenario delivery : Patrick tested the Bileto scenario locally on its PC and was working fine. He couuld not find a way to change the CSV config file, and asked if it will be possible in the future, as tester would have teh choice to run it using the Sqills tool, or locally in standoanle on its postman. This functionalility was not planned to be deliver in the version Josef did, and Tim and Josef will look at the different way to achieve this goal, but do not see an obvious way.

  • Catalog of validation scenarios : As the first scenario is almost finished, The team proposed to start the work listing all the required scenarios that must be delivered for the first validation set based on mandatory fields. Patrick proposed to start this activity on the 7th of November.

2023-10-10 Meeting

  • Demo environement : Sqills sandbox migration finalised, new credential, new end points. Communication was sent too users. Bileto delivered its sandbox. Can request access on osdm.cz for the moment. Josef will provide credentials this week.

  • First OTST scenario delivery : Tim run the first scenarion on Sqills Sandbox. Failiong on seat reservation request as supporting the selection with preference while Bileto support by seat number request. As a result to test both business logique, better to have a dedicated scenario per business function. Also Sqills identified that OSDM "collection" parameters were set in the Authentication" collection (not part of OSDM). Tim will propose changes to well split those 2 collections. Tim and Josef will work to fine tune the scenario in the next 2 weeks.

  • OSDM Validation Tool : Tim did a demo of the validation tool. It shown how to configre, run the scenario and what could be reported. Work in progress.

  • Next meeting : Due to member availability, next meeting is on the 24th of October

2023-09-26 Meeting

  • Demo environement : Sqill is migrating its demo environment to use its last aoolication version. The end poinbt to be used is changing. Tim will send communication to all Demo environement users. BILETO is finalising its Biletto demo environment delivery. As soon as ready, they will communicate.

  • First OTST scenario delivery : Josef delivered to Tim the first "workable" version of the scenario. Tim and Josef plan a commun test session next Monday. Status will be done next week.

  • OSDM Validation Tool : Tim is planning to do a demo potentially next week.

2023-09-19 Meeting

  • Scenario input variables setting : Josef upadted the OSDM collection to use a CSV input file pour setting the variable. He could not use the "legacy" POSTMAN way of integrate it, but got a way to do it. He could not finalise to do a demo during the meeting but should achieve it in next hours / day. As soons as finalised Josef will share it with Tim who will integrate it in Sqills validation tool to plan a demo as soon as possible.

  • OSDM starter collection : Josef created a dedicated collection in which he manage the authorisation (out side OSDM scope but required) and the OSDM collection setup up (like loading the necessary libraries). DIsucssion on the best way to structure the "starter" collection is not a structuring prioirity and is postponed

2023-09-12 Meeting

  • Postman scenario coding : Josef finalise its coding, and has now a separet "non OSDM" collection to request the Bileto token. This collection could be different for another environment (like Sqills sand box). This "Non OSDM" TOken collection must be run prior starting the OSDM collection.

  • Scenario input variables setting : Discussion on how to mange the input variables required to run the collection. For offer request using as input a trip description, the nummber pof legs, stops can be different from on provider to another. To ease the config set up Josef will have an external CSV file with generic variables defined that can be extended for example depending on the number of legs, etc (ref to postman doc https://learning.postman.com/docs/collections/running-collections/working-with-data-files/ ). His objective is to deliver it next week.

  • Next steps : Setup of the "OSDM standard" environment is not urgent, ^Subject to be tacklle when the fistscenario will be finalised. Once Josef will have deliver its final version of the scenario (using the trip as search criteria), the team will focuss on this one to enrich it with validation tests. Teh goal is to finalise this first scenario with all necessary validations tests before starting duplicating it for another type of Offer request scenario (based of O&D search) or based a a trip id.

2023-09-05 Meeting

  • Postman scenario coding : Josef continue to improve his code. For the dev environnent, the proposal is to have it on Josef private environnent for the moment (Josef can give access to anyboduy willing to contribute)until we have a stabilised version which will be then loaded on the "OSDM stanadr OTST github" environnent that we need to define.

For the static javascript files, for the moment Josef use Digital Ocean and its automation to publish those JS files continuously upon each commit. We will have to define later one how to store them in an OSDM "standard" environement.

2023-08-29 Meeting

  • Postman scenario coding : Josef from Bileto with the support of Martion from Turnit implemented the Postman scenario using Javascript libraries based on Sayantan from Amadeus recommendations. They have running scenarios, but need more time to finalised. They expect to deliver a final version mid september. Hosting the library in github is not the most performant solution. They recommend to have a dedicated library hosting solution. Action : Check in which environment the OTST libraries should be hosted.

  • Postman scenarion enhancement : Once the code framework is ready, we could enhance the scenario content adding more validation tests in the scenarios. Action: Identify volonteers willing to work on validation tests integration.

2023-07-31 Meeting

  • Postman scenario coding : Josef from Bileto started to work on coding based on Sayantan recommendation. He will work during the summer break to deliver a first version in september. Sayantan propose his support on request, by mail exchange or meeting if required.

2023-06-27 Meeting

  • Postman scenario coding : Sayantan from Amadeus did a presentation on how he reorganised the Postman scenario using Javascript libraries and data definition at collection / scenario level. He could not finish coding the offer part but will continue for the next meeting. On their side OTST members will look at Sayantan proposal to discuss next week how it could be reused.
  • Number of scenarios: Restart the discussion on the necessity to deplicate or not scenarios as all complexity is mainly in the offer response to be handled. Discussion postponed as priority done on the Scenario coding for the moment.

2023-06-20 Meeting

  • All of the features require starting with obtaining an offer. It makes sense to create a collection just to issue offer based on different parameters. If doesn't fail, the offer id may be reused in use case-specific scenarios or re-run from the offer collection without further validation.
  • Mapping different kinds of offers we actually need for the use cases (in env. variables)
  • Josef also presented how railML certification is performed - the consultant provides "unknown data set" and then is able to query the system. This zeroes feature differences based only on products that different vendors/carriers offer and helps test same set of function features. For future discussion if OSDM APIs could also be tested this way,

2023-05-23 Meeting

Meeting Minutes:

  1. Direct booking scenario: Tim presented the scenario, and announced that mid next week, Sqills will have its sand box supporting OSDM V3. The scanario will be loaded on GITHIB during the week.
  2. Architecture of scenarios: Based on this first scenario, it is agreed it will be easier to managed a list of scenarios per function to validate rather than a unique scenario polymorphique.

2023-05-16 Meeting

Meeting Minutes:

  1. Direct Booking test scenario : Next steps (Tim / Josef) Due to Murphy law, Tim could not deliver the final version of the forst scenario. Hopefully next week.
  2. Admission Resa scenario : Updated by Martin on the script loaded on Github Martin did not progress. No updates.
  3. Exchange with Postman experts : Sayantan Roy from Amadeus) : Sayantan explained few solution he is using to managed Amadeus test scenario collection in a configurable / maintenable manner. Here are his top recommendations a) Handling Strict Variables via environment variables

b) For all test related variables utilize only global variables

c) For framework one can create it and utilize it via git repo and eval the code in each test

d) Using htmlextra for reporting while running through newman.

e) Using base tests to run different data sets.

To progress on the subject, P.Heuguet will produce a document summarizing OTST team objective allowing Sayantan to better understand what we are willing to achieve, and Tim will try to send befor next session his first scenario. Tim scenario is built on V3 version, and cannot run as such on the Sqills sandbox which is on V2 version. Potentially he will try to deliver a V2 version of the scenario

  1. Meeting hour change: To facilitate the access to the session it was decided to move it a 1:30 PM.

2023-05-09 Meeting

Agenda proposal:

  1. Direct Booking test scenario : Next steps (Tim / Josef) Tim is back from vacation. He did a demonstration on how it is developped to support different type of test scenarios like shopping with reduction card or without. and will try to deliver a first clean version of the scenario end of the week
  2. Admission Resa scenario : Updated by Martin on the script loaded on Github MArtin is working in paralell on its set of scenario, he will continue to invest on them and progress. The team agreed on the value at this stage to work in parallel on different way of developping , we will converge at a later stage.
  3. Insomia test collection : Presentation from Mattias Matthias tested Imsomia, but found it less mature, less stable than Postam Also he looked at the Postman JSON export capability allowing to manage the dev of OST scenarios in github. He confirmed after some test it is working well and it is stable.
  4. Exchange with Postman experts: Amadeus who is using extensively POstman propose to ahave an exchange with one of its expert next week. To better prepare the exchange OST team will share its first scenario in advance.

2023-05-02 Meeting

Turnit created a new scenario based on the initial scenario. The poll request is created Some issues to be solved . Some on the Turnit test environement and some other on the scenario improvment to be done with Josef / Tim. In Turnit scenario refund was changed adding the release seat step first. For the trip, should we use trip specification or trip search

How to use/manage variable naming convention and inventiory to be sure someone will not duplicate an existing variable.

  • You can create variable at different level, Matthias Rainer had the issue as postman manage variable or at global level or at collection level. One way to solve is to have one scenario per collection.
  • In this case, could we run the same collection multiple time in parallele ?
  • Need to check with Sqills who is developping the Test apps using those scenarios.
  • P.Heuguet asked if a postman expert could be helpfully fastracking the definition of the postman scenario architecture ?
  • P.Ludwig propose to provide SBB dedicated apps developped in JAVA to developp the scenarios avoiding postman limitation on variable handling.
  • Bileto proposed HTTPPIE must not at the same level than postman running scenarios in parallel. Missing features.
  • Postman is widly use , thus securing support
  • Could it be better in the paid version of Postman, from Mathias perspective, not really
  • Mathias found a postman function 'random' taht could be used to have consistant testing. We can also import mibrary. Mathais will investigate

2023-04-25 Meeting

The first test case finalisation is pending ressource availability that are on cvacation. In standby for the moment.

Turnit could not deliver their test case this week due to workload. Will replan their delivery target

ÖBB will provide additional test scearios in a few weeks, exact target not yet finalised.

Due to ressources availability, there are delays in the targeted deliveries.

2023-04-18 Meeting

The first test case identified a small bug in the specification.

The first test case of Turnit will be ready next week.

ÖBB will provide additional test scearios in a few weeks.

2023-03-28 Meeting

Before teh start of the meeting, welcoming Patrick Ludwing representing SBB who is starting its OSDM implementation. Patrick coordinating SBB teams working on the subject.

1) Scenarios catalogue : Not addressed this week

2) Scenario configuration : Tim provided feed back on testing the scenario in his test environment. Main issue is about "request" in which potentially a railway could ask an optional field (as if it was mandatory). in the scenario tested the example was : productCategory in the offer request:

"service": { "productCategory": { "name": "{{service_type_name}}", "shortName": "{{service_type_name}}", "productCategoryRef": "urn:{{namespace_stations}}:sbc:{{service_type_brand_code}}" }, Discussions on different solution:

  1. Force anyone implementing to be in a positon to send response with request using only mandatory fields : Easiest way to solve the PB, not not sure it is posisble to impose this rule. To be discussed with the OSDM expert team on Friday morning. Patrick will raise the point to Andreas and Clemens.
  2. Analyse the error message to check that an optional field is missing to have the carrier managing the request: Not sure there qre OSDM rules defined to have such error message response. More chance to have a response "no offer available" rather than Mandatory field missing...
  3. Analyse all potential optional fields that should be used, and include them in the configuration capabilities to adapt the scenario depending on the Railway behaviour: Possible but adding more complexity in the scenario architecture.

Next steps : Tim will continue to test to check the scenario behaviour will propose some evolution to solve issues found and share its results with Josef to push on github e new version of the scenario that could be then enhanced by OSTST members to enrich its validation capability.

3) Scenario validation : Schema : Tim integreted it in the scenario prototype and will finalise its tests to validate and push the enhancement to Josef

4) Scenario validation : Content :

In standby until point 2 and 3 above will be finalised.

2023-03-21 Meeting

1) Scenarios catalogue :

No update done by the team this week, participants will try to find spare time to propose updates for next week.

The scenarios will try to be generic and flexible enough to check differents features in one scenarion. For example a sles scenarion includes offer/booking / fullfilment verbs and will be in capcity to validate different type of booking NRT with or without mandatory reservation / IRT

But some dedicated scenario will have to be done if in the middle of the process a new verb is used like a sale scenarion including a seat graphical reservation.

Same about after sale, as we have 2 different type of aftersales, full or partial refund.

Clemens will enhanced the document adding an excel file allowing that all OSDM verbs / features are covered by defined scenarios

2) Scenario configuration :

Josef updtaded github with the folowing updates:

  1. Support OSDM V3

  2. Decoupling the authentication process scenario which is out of OSDM scope from the OSDM scenario which only need the certification token

  3. Removing all steps dedicated to BILETO specific elements

  4. Environment include the setup of station code type, station code, carrier code

The scenario is a direct reservation, booking only the seat reservation.

Tim will test this week this version of the prototype and privide his feed back

3) Scenario validation : Schema :

TIM did a scenario testing the schema based on the GITHUB OSDM JSON schema reference file. WOrking fine. Before integrating it in the proto, Tim is willing to validate an "invalid" schema and check how the rport is done.

Once done the code will be shared with Josef to be integrated in the proto scenario.

4) Scenario validation : Content :

In standby until point 2 and 3 above will be finalised.

2023-03-14 Meeting

1) Scenarios catalogue :

Clemens created a new page listing the OSDM features and their type Mandatory, Conditional or Optional. Offer is not included as anyway a prerequite for a booking. Hidden feature in booking. Next step is to have the OTST team reveiwing the list and updating it if necessary.

2 questions addressed:

1- Should the first scenario should include conditional features ?

2- Should we integrate offer search as forst scenario is a direct booking use case ?

Those 2 will be further discussed in the way the prototype is developped.

Clemens also cretaed a JSON reference file on github for the V3.0.0. This file can be used to test JSON schema in OTST scenario. Clemens will start a draft proposal to define the content of the first scenario. This content will be updated based on the evolution of the prototype under development.

Action : OTST team to review the propose feature list and update it if necessary to have a final version Action : Clemens to propose a first draft version of the first scenario with its features.

2) Scenario configuration :

Josef started to test but had issues due to version differences. Decision : OTST focus on V3.0.0. No investment to be done to retrofit the V3 scenario for V2 version. As Skills has a local V3 version platform Josef will upload its last version of the scenario, and Sqills will test it on its local environment providing feedback on issues, and proosing also enhancements.

Autentification process. Difficult to make it configurable. **Decision :**The Autentification process is out of scope of OTST scenarios. The tester will have to provide as an input a valid token to run its OTST scenarios. It is recommended to have a token valid at least for more than 10mm to avoid issue.

Josef will focus on "domain" configuration URL end point, Station Code / Operator code type etc...)

3) Scenario validation : Schema :

AJV is a potential solution to test the schema. Sqills propose to integrate schema validation on its scenario, and to propose this enhancement for the prototype.

4) Scenario validation : Content :

For the content validation, we should take in account also process type of OSDM tech team recommandation (like the recommendation on NRT with mandatory reservation with price only in acco price or reservation price). OTST will implement a test to check the implementation is compliant with the OSDM expert team reco, and will report a warning with proposed alternate solution compliante with OSDM. Once the first prototype scenario will be ready, it is propose to share it with all team mebers allowing them to run it on their own environment and check if with the test they run on their side they can propose enhancements

Next OTST weekly meeting will then focuss on finalising this first proto scenarios.

2023-03-07 Meeting

1) 1st test scenario prototype delivery :

Some adaptation must be done on the scenario to manage fir API parameter, the Oauth version, and also in the request parameters like for exemple the station type parameter and try to make this configurable. Sqills and Bileto will work to continue the valiodation of the prototype on the Sqills environment. Decision : Sqills / Bileto to plan a dedicated meeting to review the scenario.

2) Sqills demonstration about a scenario running in postman console :

Based on the Sqill Sandbox demo scenarios, Sqills developp a proof of concept allowing to run a postman scenario chaining different request in sequence, testing their resuls and deliverying a report. Very promosing demo.

3) Catalogue of scenarios :

Discussion about the number of scenarios to be developped. Two ideas to ficus on the most important one's

  1. Focus on mandatory elements
  2. Focus on Pareto law (20% of the functions representing 80% of the volume)

Clemens proposed to start a doc on the scenario scope on the WIKI.

4) OSDM schema validation :

A first discussion on possible way to integrate in the scenario the validation of the OSDM schema. At least schema validation has 2 sides

  1. The format
  2. The content

A link to a postamn page explaining how to do a schema validation with postman was shared https://postman-quick-reference-guide.readthedocs.io/en/latest/schema-validation.html#

Few ideas shared like Converting the YML file in JSON and use it in postman, Using a Java script library. To be continued

4) Scenario solution architecture :

Return of experience from Turnit (Martin) who used in the past a library (https://unpkg.com/[email protected]/dist/forge.min.js) as an environment variable. it dropped the performance significantly. Sadly no comparison available right now.

Discussion on the subject to be continued

2023-02-28 Meeting

1) 1st test scenario prototype delivery :

Josef Petrak from Bileto delivered the first version of the Direct Booking test scenario prototype. It is defined based on 2.0 OSDM specs. Will be migrated soon to V3.0 version.

Decision : The test scenarios will be based on version V3.0 of OSDM. Bileto, Sqills and Turnit will provide V3.0 test environment to test them

2) Test scenario configuration :

Some mandatory parameters should be configurable like

  • End Point (the URN to target)
  • Origin and Destination stations
  • Hour of departure
  • PAX info potentially with birth date or age depending on Railway behavior.

2 ways of entering the parameters

  1. Updating the parameters in the Postman environment ( current way)
  2. A script asking the parameters to the end user and updating postman config. Need some devs, maybe for a later stage

3) Scenario test report :

Action: Tim Van Berg from Sqills will test the 1st prototype on the Sqills Sandbox He will see what type of reporting can be produced based on the Postman result checks done.

4) Scenario solution architecture:

Postman cannot allow to have code done to support shared functions or libraries to avoid some copy/paste between the different scenarios.

Decision: the team agreed to keep Postman and live with its coding limitation and find solutions (like embedding a function to be stored in the environment). The focus is today on delivering the first test prototype. Once done this discussion will be reopen.

2023-02-21 Kick off Meeting

Discussion:

On top of the Functional (offer / Booking / Fullfillment, etc…) and Feautures ( Nbr of PAX, Reduction cards, Seat resa, Ancilaries…), there are other dimensions:

  1. Mandatory / Optional parameters
  2. Providers / Distributors

The focus is to do a validation of the OSDM Railways implemenation to re-inforce the message about the willingness of the sector to have a standardisation of their distribution. The priority is givent o this validation.

On the distribution side; the validation will be more diffcult as anyway it should be beyond only the exhange of messae bt more on the way the provider responses are used and displayed. It is out of scope of OTST

Catalogue matrix:

Two options are envisaged

  1. A full catalogue matrix listing all possible functionnalities / featured to be tested, and the Railways selecting the the cases he wants to validate.
  2. A minimum list of functions the provider wants to test (like Trips, Offers, Booking, Fullfilment, Refund all, Excnaheg All; Partial refund, Partial Exchange,….) that he is supporting.

Features are discovered by the scenario based on the offers responses, and tested automatically by the scenario. For example if the offers answer propose ancilaries, the scenario will try to book, fullfil those ancilaries.

The team decided to go for option 2 with the provider defining at the beginning the level on function coverage it supports and proposing in its test environnement the necessary data allwong to test the full ranges of features he wants to validate. It is the responsibility of the provider to secure its test environment is providing the level of data necessary to validate all the features he is willing o use on production.

The test scenarios will provide a report listing all features tested succesfully.

On some specific use cases like passenger type, the scenario, will test the 2 possible options (age or PAX type), in the report at least one of them should be succesfull. No need to define in advance wich one is used. The scenario will check automatically both options.

As a future enhancement, we could have as an input a matric of features the provider is willing to support and check that the test report is compliant with this wish list. This enhancement is out of scope of the OTST 2023 delivery scope

Conclusions: Catalogue matrix is limited to the list of functions the provider is supporting Test scenarios will detect which features are offered in the offer response and will validate them automatically, producing a report listing the features tested succesfully, and the one failing.