Objective:

This documentation aims to portray precisely the cbuildgen-specific tests, providing a roadmap for their seamless translation or migration into Golang integration tests designed for cbuild2cmake to avoid any regressions.

Observations:

Upon analyzing the test cases, a central observation emerges:

• The majority of tests focus on verifying the successful generation of CMakeLists.txt and validating its content using reference comparison.
• The distinctions between various test cases stem from the utilization of diverse input projects, each configured with distinct configurations, resulting in the generation of diverse nodes within the CMakeLists.txt file.
• Presently, the reference comparison excludes lines containing paths (i.e., "/") and comments.

Notes:

• No preliminary setup or pre-configuration is required exclusively for running tests related to cbuildgen.
• Some of the sample projects utilize local packs
• CMSIS_PACK_ROOT & CMSIS_COMPILER_ROOT env variables are utilized by cbuildgen (if not set, default is used)
• Installation setup is only required when we want to validate if the generated CMakelists.txt file is buildable.

Test Analysis

Test Objective	Scenario	Expectations	comments	Migration to cbuild2cmake
CMakelists.txt Generation/validation	AccessSequence	• Successful operation • Correct generation of the CMakeLists.txt file • AccSeq is resolved	Acc Seq like $BPack, $DPack, $BName, $DName etc needed to be implemented in csolution	$packroot Access Sequence should be impelemted in cbuild2cmake and respective integration test is needed
	AccessSequence Project with missing BoardName	• Successful operation • Correct generation of the CMakeLists.txt file • AccSeq is resolved		NA
	Invalid AccessSequence	• Error • No CMakeLists.txt file generation		NA
	AccessSequence Project with unknown BoardName	• Error • No CMakeLists.txt file generation		NA
	--update (Fixed versions)	• Successful operation • Correct generation of the CMakeLists.txt file		NA
	Branch protection	• Successful operation • Correct generation of the CMakeLists.txt file with branch protection		Integration test
	GPDSC Bundle	• Successful operation • Correct generation of the CMakeLists.txt file		NA
	GPDSC Bundle with components	• Successful operation • Correct generation of the CMakeLists.txt file		NA
	projects with duplicated source filenames under different directories	• Successful operation • Correct generation of the CMakeLists.txt file with correct file mapping		Integration test
Arguments validation	Invalid arguments	• Error		Unit test
	Projects having different Translation controls	• Successful operation • Correct generation of the CMakeLists.txt file with translation controls correctly mapped		Integration test
	--outdir	• Generation of clog file under the defined path		NA
	Multilevel --outdir, --intdir	• Generation of clog file under the defined outdir path • Generation of CMakeLists file under the defined intdir path		NA
	Absolute path support --outdir, --intdir	• Generation of clog file under the defined outdir path • Generation of CMakeLists file under the defined intdir path		NA
	Relative path support --outdir, --intdir	• Generation of clog file under the defined outdir path • Generation of CMakeLists file under the defined intdir path		NA
	No arg test, -V, --version, -h, --help, Multiple commands, Missing target arg, Invalid argument	• Error		Unit test
Commands validation	packlist	• Successful operation • Correct generation of the .cpinstall and .cpinstall.json		NA
	packlist without CMSIS_PACK_ROOT	• Successful operation • Correct generation of the .cpinstall and .cpinstall.json		NA
	packlist command with invalid pack repo ( no CMSIS_PACK_ROOT)	• Successful operation • Correct generation of the .cpinstall and .cpinstall.json		NA
	packlist with --intdir	• Successful operation • Correct generation of the .cpinstall and .cpinstall.json under specified path		NA
	layer "extract"	• Successful operation • Correct generation of the .clayer files and validate for completeness		NA
	layer "compose"	• Successful operation • Correct generation of the project under the expected file tree		NA
	layer "add"	• Successful operation • Correct generation of the project with an added layer under the expected file tree		NA
	layer "remove"	• Successful operation • Correct generation of the project with layer removed under the expected file tree		NA
	Aux commands: mkdir, rmdir, touch	• Successful operation		NA (update cbuild to use cmake to do these oprations)
	multiple commands	• Error		Unit test
	Each supported command should have an individual help	• Successful operation		Unit test
Misc	Emaples project: • AC6 • GCC • IAR • MultiTargetAC6 • MultiTargetGCC • MultiTargetIAR	• Successful CMakelist generation and build		• An integration test (Not toolchain specific) for the generation/validation of the CMakelist • Integration test would be good for all Multitarget test (Since different target bring different translation controls). • cmsis-toolbox would be a better place to validate the successful build.
	cpackget tests			• cmsis-toolbox would be a better place to validate cpackget
	debian package			NA
	Installer tests			NA