② C++ elements#
Intel® DL Streamer C++ interface allows to develop new functional blocks (source, transform and sink elements) with minimal effort. The interface does not use any of GStreamer types, so elements/blocks written in this interface can be used in both GStreamer-based and non-GStreamer applications.
The C++ elements implementation typically inherit one of base classes which partially implement some functions in corresponding abstract interfaces:
BaseSource partially implements Source
BaseTransform partially implements Transform
BaseTransformInplace partially implements Transform
so that C++ element implements only remaining virtual functions (mostly read
, process
and write
functions) as shown in diagram below
![digraph {
node[shape=record,style=filled,fillcolor=lightskyblue1]
edge[dir=back, arrowtail=empty]
Desc[label = "{ElementDesc|+name\l+description\l+author\l+params\l+input_info\l+output_info\l+flags\l+create\l}"]
Element[label = "{«interface»\nElement|+ init()\l+ get_context(...)\l}"]
Source[label = "{«interface»\Source|+ set_output_info(...)\l+ get_output_info(...)\l+ read()\l}"]
Transform[label = "{«interface»\nTransform|+ set_input_info(...)\l+ set_output_info(...)\l+ get_input_info(...)\l+ get_output_info(...)\l+ process(TensorPtr, TensorPtr)\l+ process(FramePtr, FramePtr)\l}"]
TransformInplace[label = "{«interface»\nTransformInplace|+ set_info(const FrameInfo&)\l+ process(TensorPtr)\l+ process(FramePtr)\l}"]
Sink[label = "{«interface»\Sink|+ set_input_info(...)\l+ get_input_info(...)\l+ write(FramePtr)\l}"]
BaseESource[label = "{BaseElement\<Source\>|+ init_once()\l+ init()\l+ get_context(...)\l}"]
BaseT1[label = "{BaseElement\<Transform\>|+ init_once()\l+ init()\l+ get_context(...)\l}"]
BaseT2[label = "{BaseElement\<TransformInplace\>|+ init_once()\l+ init()\l+ get_context(...)\l}"]
BaseESink[label = "{BaseElement\<Sink\>|+ init_once()\l+ init()\l+ get_context(...)\l}"]
BaseSource[label = "BaseSource"]
BaseTransform[label = "BaseTransform"]
BaseInplace[label = "BaseTransformInplace"]
BaseSink[label = "BaseSink"]
Element->{Source Transform TransformInplace Sink}
Source->BaseESource
Transform->BaseT1
TransformInplace->BaseT2
Sink->BaseESink
BaseESource->BaseSource
BaseT1->BaseTransform
BaseT2->BaseInplace
BaseESink->BaseSink
}](../_images/graphviz-15e3f6e01bf35acd6f8530399b7a97d7bfeadb7a.png)
C++ interfaces and base classes#
Many examples how to create C++ elements can be found on github repository in folder src and sub-folders.
Element description#
The ElementDesc structure is used to describe an element input/output capabilities and supported parameters, and provide instance creation function. The most important are the following fields:
name
– Name of element. Same name is used for both GStreamer and direct programming applications.params
– Array of parameters supported by element. In case of GStreamer, those will be translated to GStreamer properties.input_info
– Types of input Frames that element can consume. In case of GStreamer interop, it will be represented as sink capabilities.output_info
– Types of output Frames that element can produce. In case of GStreamer interop, it will be represented as source (src) capabilities.create
– Pointer to a function that creates an instance of an element.
Here’s example of ElementDesc structure for simple post-processing element:
// Element parameters
ParamDescVector params_desc = {
{"method", "Method used to post-process tensor data", "max", {"max", "softmax", "compound", "index"}},
{"labels_file", "Path to .txt file containing object classes (one per line)", std::string()},
{"layer_name", "Name of output layer to process (in case of multiple output tensors)", std::string()},
{"threshold", "Threshold for confidence values", dflt::threshold, 0.0, 1.0},
};
// Element description
ElementDesc tensor_postproc_label = {
.name = "tensor_postproc_label",
.description = "Post-processing of classification inference to extract object classes",
.author = "Intel Corporation",
.params = ¶ms_desc,
.input_info = {MediaType::Tensors},
.output_info = {MediaType::Tensors},
.create = create_element<PostProcLabel>,
.flags = 0};
How to create instance of C++ element#
Instance of C++ element can be created using functions create_source
, create_transform
, create_sink
.
These functions take pointer to ElementDesc
, initialization parameters (as std::map
) and optional context
pointer as parameters:
auto ffmpeg_source = create_source(ffmpeg_multi_source, {{"inputs", inputs}}, ffmpeg_ctx);
See direct programming samples ffmpeg_openvino and ffmpeg_dpcpp for examples.