GPUPluginStructure - akuporos/openvino GitHub Wiki
GPU plugin structure
Historically GPU plugin was built on top of standalone clDNN library for DNNs inference on Intelยฎ GPUs, but at some point clDNN became a part of OpenVINO, so now it's a part of overall GPU plugin code.
OpenVINO GPU plugin is responsible for:
- IE Plugin API implementation.
- Translation of model from common IE semantic (ov::Function) into plugin specific one (cldnn::topology) which is then compiled into gpu graph representation (cldnn::network).
- Implementation of OpenVINO operation set for Intelยฎ GPU.
- Device specific graph transformations.
- Memory allocation and management logic.
- Processing of incoming InferRequests using clDNN objects.
- Actual execution on GPU device.
As Intel GPU Plugin source code structure is shown below:
src/plugins/intel_gpu - root GPU plugin folder
โโโ include
โ โโโ intel_gpu - library internal headers
โ โ โโโ graph - headers for internal graph representations
โ โ โโโ plugin - definition of classes required for OpenVINO plugin API implementation
โ โ โโโ primitives - primitive definitions for all supported operations
โ โ โโโ runtime - abstraction for execution runtime entities (memory, device, engine, etc)
โ โโโ va
โโโ src
โ โโโ graph - all sources related to internal graph representation
โ โ โโโ graph_optimizer - passes for graph transformations
โ โ โโโ impls - definition of primitive implementations
โ โ โโโ include - headers with graph nodes
โ โ
โ โโโ kernel_selector - OpenCLโข kernels (host+device parts) + utils for optimal kernels selection
โ โ โโโ common - definition of some generic classes/structures used in kernel_selector
โ โ โโโ core - kernels, kernel selectors, and kernel parameters definitions
โ โ โโโ actual_kernels - host side part of OpenCLโข kernels including applicability checks, performance heuristics and Local/Global work-groups description
โ โ โโโ cache - cache.json - tuning cache of the kernels which is redistributed with the plugin to improve kernels and kernel parameters selection for better performance
โ โ โโโ cl_kernels - templates of GPU kernels (device part) written on OpenCLโข
โ โ โโโ common - utils for code generation and kernels selection
โ โโโ plugin - implementation of OpenVINO plugin API
โ โ โโโ ops - factories for conversion of OpenVINO operations to internal primitives
โ โโโ runtime
โ โโโ ocl/ - implementation for OpenCLโข based runtime
โโโ tests
โ โโโ test_cases
โ โโโ test_utils
โโโ thirdparty
โโโ onednn_gpu - [oneDNN](https://github.com/oneapi-src/oneDNN) submodule which may be used to accelerate some primitives
โโโ rapidjson - thirdparty [RapidJSON](https://github.com/Tencent/rapidjson) lib for reading json files (cache.json)
One last thing that is worth mentioning is functional tests which is located in the following location:
src/tests/functional/plugin/gpu
Most of the tests are reused across plugins, and each plugin only need to add test instances with some specific parameters.
Shared tests are located here:
src/tests/functional/plugin/shared <--- test definitions
src/tests/functional/plugin/gpu/shared_tests_instances <--- instances for GPU plugin