GVA::AudioEvent Class Reference

This class represents audio event - object describing audio event detection result (segment) and containing multiple Tensor objects (inference results) attached by multiple models. For example, it can be audio event with detected speech and converts speech to text. It can be produced by a pipeline with gvaaudiodetect with detection model and gvaspeechtotext element with speechtotext model. Such AudioEvent will have start and end timestamps filled and will have 2 Tensor objects attached - 1 Tensor object with detection result and other with speech to text tensor objectresult. More...

#include <audio_event.h>

Collaboration diagram for GVA::AudioEvent:

Public Member Functions
Segment< gulong >	segment () const
	Get Segment of AudioEvent as start and end timestamps, timestamps are presentation time. More...
std::string	label () const
	Get AudioEvent label. More...
double	confidence () const
	Get AudioEvent detection confidence (set by gvaaudiodetect) More...
std::vector< Tensor >	tensors () const
	Get all Tensor instances added to this AudioEvent. More...
Tensor	add_tensor (const std::string &name)
	Add new tensor (inference result) to this AudioEvent with name set. To add detection tensor, set name to "detection". More...
Tensor	detection ()
	Returns detection Tensor, last added to this AudioEvent. As any other Tensor, returned detection Tensor can contain arbitrary information. If you use AudioEvent based on GstGVAAudioEventMeta attached by gvaaudiodetect by default, then this Tensor will contain "label_id", "confidence", "start_timestamp", "end_timestamp" fields. If AudioEvent doesn't have detection Tensor, it will be created in-place. More...
int	label_id () const
	Get label_id from detection Tensor, last added to this AudioEvent. More...
	AudioEvent (GstGVAAudioEventMeta *meta)
	Construct AudioEvent instance from GstGVAAudioEventMeta. After this, AudioEvent will obtain all tensors (detection & inference results) from GstGVAAudioEventMeta. More...
void	set_label (std::string label)
	Set AudioEvent label. More...
GstGVAAudioEventMeta *	_meta () const
	Internal function, don't use or use with caution. More...

Protected Attributes
GstGVAAudioEventMeta *	_gst_meta
	GstGVAAudioEventMeta containing fields filled with detection result (produced by gvaaudiodetect element in Gstreamer pipeline) and all the additional tensors, describing detection.
std::vector< Tensor >	_tensors
	vector of Tensor objects added to this AudioEvent (describing detection & inference results), obtained from GstGVAAudioEventMeta
Tensor *	_detection
	last added detection Tensor instance, defined as Tensor with name set to "detection"

Detailed Description

This class represents audio event - object describing audio event detection result (segment) and containing multiple Tensor objects (inference results) attached by multiple models. For example, it can be audio event with detected speech and converts speech to text. It can be produced by a pipeline with gvaaudiodetect with detection model and gvaspeechtotext element with speechtotext model. Such AudioEvent will have start and end timestamps filled and will have 2 Tensor objects attached - 1 Tensor object with detection result and other with speech to text tensor objectresult.

Constructor & Destructor Documentation

AudioEvent()

GVA::AudioEvent::AudioEvent ( GstGVAAudioEventMeta *  meta )

inline

Construct AudioEvent instance from GstGVAAudioEventMeta. After this, AudioEvent will obtain all tensors (detection & inference results) from GstGVAAudioEventMeta.

Parameters

meta	GstGVAAudioEventMeta containing AudioEVent information and tensors

Member Function Documentation

_meta()

GstGVAAudioEventMeta* GVA::AudioEvent::_meta ( ) const

inline

Internal function, don't use or use with caution.

Returns

pointer to underlying GstGVAAudioEventMeta

add_tensor()

Tensor GVA::AudioEvent::add_tensor ( const std::string &  name )

inline

Add new tensor (inference result) to this AudioEvent with name set. To add detection tensor, set name to "detection".

Parameters

name	name for the tensor. If name is set to "detection", detection Tensor will be created and set for this AudioEvent

Returns

just created Tensor object, which can be filled with tensor information further

confidence()

double GVA::AudioEvent::confidence ( ) const

inline

Get AudioEvent detection confidence (set by gvaaudiodetect)

Returns

last added detection Tensor confidence if exists, otherwise 0.0

detection()

Tensor GVA::AudioEvent::detection ( )

inline

Returns detection Tensor, last added to this AudioEvent. As any other Tensor, returned detection Tensor can contain arbitrary information. If you use AudioEvent based on GstGVAAudioEventMeta attached by gvaaudiodetect by default, then this Tensor will contain "label_id", "confidence", "start_timestamp", "end_timestamp" fields. If AudioEvent doesn't have detection Tensor, it will be created in-place.

Returns

detection Tensor, empty if there were no detection Tensor objects added to this AudioEvent when this method was called

label()

std::string GVA::AudioEvent::label ( ) const

inline

Get AudioEvent label.

Returns

AudioEvent label

label_id()

int GVA::AudioEvent::label_id ( ) const

inline

Get label_id from detection Tensor, last added to this AudioEvent.

Returns

last added detection Tensor label_id if exists, otherwise 0

segment()

Segment<gulong> GVA::AudioEvent::segment ( ) const

inline

Get Segment of AudioEvent as start and end timestamps, timestamps are presentation time.

Returns

Start and end time of AudioEvent

set_label()

void GVA::AudioEvent::set_label ( std::string  label )

inline

Set AudioEvent label.

Parameters

label

Label to set

tensors()

std::vector<Tensor> GVA::AudioEvent::tensors ( ) const

inline

Get all Tensor instances added to this AudioEvent.

Returns

vector of Tensor instances added to this AudioEvent

---

The documentation for this class was generated from the following file:

• /home/gst-libs/gst/videoanalytics/audio/audio_event.h