Try tutorial for building tools using LibTooling and LibASTMatchers - shijunjing/edk2 GitHub Wiki
- https://clang.llvm.org/docs/LibASTMatchersTutorial.html
- https://github.com/shijunjing/llvm-project/tree/tutorial
- CppCon 2018: James Bennett “Refactoring Legacy Codebases with LibTooling”
- Fred Tingaud “How to Refactor Millions of Line of Code Without Alienating your Colleagues”
- CppCon 2019: Paul Licameli “Dynamite the Monoliths! Taming dependency cycles in a 20-year old...”
jshi19@ub2-uefi-b01:~/llvm/llvm-project$ mkdir LibASTMatchersTutorial/ && cd LibASTMatchersTutorial/
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ cmake -G Ninja ../llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DCMAKE_BUILD_TYPE="MinSizeRel" -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_INSTALL_PREFIX=../minsizerelinstall/ -DLLVM_ENABLE_Z3_SOLVER=OFF -DLLVM_CCACHE_BUILD=ON
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ninja
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ninja check
... ...
... ...
********************
********************
Failed Tests (3):
Clang :: Tooling/clang-check-offload.cpp
lit :: googletest-timeout.py
lit :: shtest-timeout.py
Testing Time: 569.85s
Unsupported : 17095
Passed : 41058
Expectedly Failed: 86
Failed : 3
FAILED: CMakeFiles/check-all
cd /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial && /usr/bin/python3.6 /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/tools/extra/test /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/tools/extra/clangd/test/../unittests /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/tools/extra/clangd/test /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/test /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/utils/lit /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/test
ninja: build stopped: subcommand failed.
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ninja clang-test
... ...
... ...
********************
********************
Failed Tests (1):
Clang :: Tooling/clang-check-offload.cpp
Testing Time: 324.44s
Unsupported : 489
Passed : 18172
Expectedly Failed: 22
Failed : 1
FAILED: tools/clang/test/CMakeFiles/check-clang
cd /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/test && /usr/bin/python3.6 /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/tools/clang/test
ninja: build stopped: subcommand failed.
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ninja install
... ...
-- Up-to-date: /home/jshi19/llvm/llvm-project/releaseinstall/lib/cmake/llvm/./UseLibtool.cmake
-- Up-to-date: /home/jshi19/llvm/llvm-project/releaseinstall/lib/cmake/llvm/./FindGRPC.cmake
-- Up-to-date: /home/jshi19/llvm/llvm-project/releaseinstall/lib/cmake/llvm/./HandleLLVMOptions.cmake
-- Up-to-date: /home/jshi19/llvm/llvm-project/releaseinstall/lib/cmake/llvm/./CheckCompilerVersion.cmake
Meet the ld link cannot find the libstdc++ issue after configure CMAKE_CXX_COMPILER (optional, even waste time...)
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ccmake ../llvm/
Change Dir: /home/jshi19/llvm/llvm-project/LibASTMatchersTutorial/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/ninja cmTC_13757 && [1/2] Building CXX object
CMakeFiles/cmTC_13757.dir/testCXXCompiler.cxx.o
[2/2] Linking CXX executable cmTC_13757
FAILED: cmTC_13757
: && /home/jshi19/llvm/llvm-project/releaseinstall/bin/clang++
CMakeFiles/cmTC_13757.dir/testCXXCompiler.cxx.o -o cmTC_13757 && :
/usr/bin/ld: cannot find -lstdc++
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
Solve above issue by creating the soft link for libstdc++.so as below. Details see: https://ubuntuforums.org/archive/index.php/t-15120.html
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ sudo ln -s /usr/lib32/libstdc++.so.6 /usr/lib32/libstdc++.so
jshi19@ub2-uefi-b01:~/llvm/llvm-project$ mkdir clang-tools-extra/loop-convert
jshi19@ub2-uefi-b01:~/llvm/llvm-project$ echo 'add_subdirectory(loop-convert)' >> clang-tools-extra/CMakeLists.txt
jshi19@ub2-uefi-b01:~/llvm/llvm-project$ vim clang-tools-extra/loop-convert/CMakeLists.txt
... ...
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ cmake -G Ninja ../llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DCMAKE_BUILD_TYPE="MinSizeRel" -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_INSTALL_PREFIX=../minsizerelinstall/ -DLLVM_ENABLE_Z3_SOLVER=OFF -DLLVM_CCACHE_BUILD=ON
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ ninja loop-convert
Test the loop-convert tool on a simple cc file:
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ cat ~/llvm/wrongcode/loop-convert/simple-loops.cc
#include <stdio.h>
#define Log(str) printf(#str)
#define IO_LEN_ADDR 0x7ffe10015710;
void DMA_Read(void * buffer, int size){
//assert(addr_check(buffer, size)==TRUE);
int datalen = *(volatile int*)IO_LEN_ADDR;
printf("datalen=%d \n", datalen);
if(size >= datalen){
printf("Start reading DMA\n");
//DMAStart(buffer, datalen);
printf("DMA transfer is triggered\n");
}else{
printf("Buffer is too small.\n");
//... //exception handling
}
}
int main()
{
int buffer[0x100];
for (int i = 0; i < 5; i++){
printf("buffer address=%p \n", buffer);
DMA_Read(buffer, 0x100);
}
return 0;
}
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ bin/loop-convert ~/llvm/wrongcode/loop-convert/simple-loops.cc
Could not auto-detect compilation database for file "/home/jshi19/llvm/wrongcode/loop-convert/simple-loops.cc"
No compilation database found in /home/jshi19/llvm/wrongcode/loop-convert or any parent directory
fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Running without flags.
ForStmt 0x55cd8b712e58
|-DeclStmt 0x55cd8b712aa8
| `-VarDecl 0x55cd8b712a20 used i 'int' cinit
| `-IntegerLiteral 0x55cd8b712a88 'int' 0
|-<<<NULL>>>
|-BinaryOperator 0x55cd8b712b18 '_Bool' '<'
| |-ImplicitCastExpr 0x55cd8b712b00 'int' <LValueToRValue>
| | `-DeclRefExpr 0x55cd8b712ac0 'int' lvalue Var 0x55cd8b712a20 'i' 'int'
| `-IntegerLiteral 0x55cd8b712ae0 'int' 5
|-UnaryOperator 0x55cd8b712b58 'int' postfix '++'
| `-DeclRefExpr 0x55cd8b712b38 'int' lvalue Var 0x55cd8b712a20 'i' 'int'
`-CompoundStmt 0x55cd8b712e38
|-CallExpr 0x55cd8b712c88 'int'
| |-ImplicitCastExpr 0x55cd8b712c70 'int (*)(const char *__restrict, ...)' <FunctionToPointerDecay>
| | `-DeclRefExpr 0x55cd8b712c50 'int (const char *__restrict, ...)' lvalue Function 0x55cd8b6fec80 'printf' 'int (const char *__restrict, ...)'
| |-ImplicitCastExpr 0x55cd8b712cb8 'const char *' <ArrayToPointerDecay>
| | `-StringLiteral 0x55cd8b712c00 'const char [20]' lvalue "buffer address=%p \n"
| `-ImplicitCastExpr 0x55cd8b712cd0 'int *' <ArrayToPointerDecay>
| `-DeclRefExpr 0x55cd8b712c30 'int [256]' lvalue Var 0x55cd8b712988 'buffer' 'int [256]'
`-CallExpr 0x55cd8b712dd8 'void'
|-ImplicitCastExpr 0x55cd8b712dc0 'void (*)(void *, int)' <FunctionToPointerDecay>
| `-DeclRefExpr 0x55cd8b712d70 'void (void *, int)' lvalue Function 0x55cd8b710e40 'DMA_Read' 'void (void *, int)'
|-ImplicitCastExpr 0x55cd8b712e20 'void *' <BitCast>
| `-ImplicitCastExpr 0x55cd8b712e08 'int *' <ArrayToPointerDecay>
| `-DeclRefExpr 0x55cd8b712d30 'int [256]' lvalue Var 0x55cd8b712988 'buffer' 'int [256]'
`-IntegerLiteral 0x55cd8b712d50 'int' 256
- Also build and test in windows VM w10-uefi-b01
Generate VS2017 meta files:
C:\steven\LLVM\llvm-project\_build_debug>cmake ..\llvm -G "Visual Studio 15 2017" -A x64 -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE="Debug" -DLLVM_ENABLE_ASSERTIONS=ON -Thost=x64 -DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra -DCMAKE_INSTALL_PREFIX=C:\steven\LLVM\llvm-project\_install_debug -DLLVM_ENABLE_Z3_SOLVER=OFF -DLLVM_TARGETS_TO_BUILD="X86"
click C:\steven\LLVM\llvm-project_build_debug\LLVM.sln and open the VS2017 to build the loop-convert target(Solution'LLVM'-->Clang executables-->loop-convert):
Test the loop-convert tool on above same cc file::
C:\steven\LLVM\llvm-project\_build_debug>Debug\bin\loop-convert.exe C:\steven\LLVM\wrongcode\loop-convert\simple-loops.cc
Error while trying to load a compilation database:
Could not auto-detect compilation database for file "C:\steven\LLVM\wrongcode\loop-convert\simple-loops.cc"
No compilation database found in C:\steven\LLVM\wrongcode\loop-convert or any parent directory
fixed-compilation-database: Error while opening fixed database: no such file or directory
json-compilation-database: Error while opening JSON database: no such file or directory
Running without flags.
ForStmt 0x1a51816f898
|-DeclStmt 0x1a51816f490
| `-VarDecl 0x1a51816f400 used i 'int' cinit
| `-IntegerLiteral 0x1a51816f468 'int' 0
|-<<<NULL>>>
|-BinaryOperator 0x1a51816f508 '_Bool' '<'
| |-ImplicitCastExpr 0x1a51816f4f0 'int' <LValueToRValue>
| | `-DeclRefExpr 0x1a51816f4a8 'int' lvalue Var 0x1a51816f400 'i' 'int'
| `-IntegerLiteral 0x1a51816f4c8 'int' 5
|-UnaryOperator 0x1a51816f548 'int' postfix '++'
| `-DeclRefExpr 0x1a51816f528 'int' lvalue Var 0x1a51816f400 'i' 'int'
`-CompoundStmt 0x1a51816f878
|-CallExpr 0x1a51816f6b8 'int'
| |-ImplicitCastExpr 0x1a51816f6a0 'int (*)(const char *const, ...) __attribute__((cdecl))' <FunctionToPointerDecay>
| | `-DeclRefExpr 0x1a51816f680 'int (const char *const, ...) __attribute__((cdecl))':'int (const char *const, ...)' lvalue Function 0x1a5181d7000 'printf' 'int (const char *const, ...) __attribute__((cdecl))':'int (const char *const, ...)'
| |-ImplicitCastExpr 0x1a51816f6e8 'const char *' <ArrayToPointerDecay>
| | `-StringLiteral 0x1a51816f630 'const char [20]' lvalue "buffer address=%p \n"
| `-ImplicitCastExpr 0x1a51816f700 'int *' <ArrayToPointerDecay>
| `-DeclRefExpr 0x1a51816f660 'int [256]' lvalue Var 0x1a51816f368 'buffer' 'int [256]'
`-CallExpr 0x1a51816f818 'void'
|-ImplicitCastExpr 0x1a51816f800 'void (*)(void *, int)' <FunctionToPointerDecay>
| `-DeclRefExpr 0x1a51816f7a8 'void (void *, int)' lvalue Function 0x1a518171a30 'DMA_Read' 'void (void *, int)'
|-ImplicitCastExpr 0x1a51816f860 'void *' <BitCast>
| `-ImplicitCastExpr 0x1a51816f848 'int *' <ArrayToPointerDecay>
| `-DeclRefExpr 0x1a51816f760 'int [256]' lvalue Var 0x1a51816f368 'buffer' 'int [256]'
`-IntegerLiteral 0x1a51816f780 'int' 256
Continue add code as https://github.com/shijunjing/llvm-project/commit/5bd5f0f915424c3822f8f9613403b7826689dd6f . Then run again:
jshi19@ub2-uefi-b01:~/llvm/llvm-project/LibASTMatchersTutorial$ bin/loop-convert ~/llvm/wrongcode/loop-convert/simple-loops.cc
Error while trying to load a compilation database:
Could not auto-detect compilation database for file "/home/jshi19/llvm/wrongcode/loop-convert/simple-loops.cc"
No compilation database found in /home/jshi19/llvm/wrongcode/loop-convert or any parent directory
fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Running without flags.
Potential array-based loop discovered.