class GVA::AudioEvent

Overview

This class represents audio event - object describing audio event detection result (segment) and containing multiple Tensor objects (inference results) attached by multiple models. For example, it can be audio event with detected speech and converts speech to text. It can be produced by a pipeline with gvaaudiodetect with detection model and gvaspeechtotext element with speechtotext model. Such AudioEvent will have start and end timestamps filled and will have 2 Tensor objects attached - 1 Tensor object with detection result and other with speech to text tensor objectresult. More…

#include <audio_event.h>

class AudioEvent {
public:
    // construction

    AudioEvent(GstGVAAudioEventMeta* meta);

    // methods

    Segment<gulong> segment() const;
    std::string label() const;
    double confidence() const;
    std::vector<Tensor> tensors() const;
    Tensor add_tensor(const std::string& name);
    Tensor detection();
    int label_id() const;
    void set_label(std::string label);
    GstGVAAudioEventMeta* _meta() const;
};

Detailed Documentation

This class represents audio event - object describing audio event detection result (segment) and containing multiple Tensor objects (inference results) attached by multiple models. For example, it can be audio event with detected speech and converts speech to text. It can be produced by a pipeline with gvaaudiodetect with detection model and gvaspeechtotext element with speechtotext model. Such AudioEvent will have start and end timestamps filled and will have 2 Tensor objects attached - 1 Tensor object with detection result and other with speech to text tensor objectresult.

Construction

AudioEvent(GstGVAAudioEventMeta* meta)

Construct AudioEvent instance from GstGVAAudioEventMeta. After this, AudioEvent will obtain all tensors (detection & inference results) from GstGVAAudioEventMeta.

Parameters:

meta	GstGVAAudioEventMeta containing AudioEVent information and tensors

Methods

Segment<gulong> segment() const

Get Segment of AudioEvent as start and end timestamps, timestamps are presentation time.

Returns:

Start and end time of AudioEvent

std::string label() const

Get AudioEvent label.

Returns:

AudioEvent label

double confidence() const

Get AudioEvent detection confidence (set by gvaaudiodetect)

Returns:

last added detection Tensor confidence if exists, otherwise 0.0

std::vector<Tensor> tensors() const

Get all Tensor instances added to this AudioEvent.

Returns:

vector of Tensor instances added to this AudioEvent

Tensor add_tensor(const std::string& name)

Add new tensor (inference result) to this AudioEvent with name set. To add detection tensor, set name to “detection”.

Parameters:

name	name for the tensor. If name is set to “detection”, detection Tensor will be created and set for this AudioEvent

Returns:

just created Tensor object, which can be filled with tensor information further

Tensor detection()

Returns detection Tensor, last added to this AudioEvent. As any other Tensor, returned detection Tensor can contain arbitrary information. If you use AudioEvent based on GstGVAAudioEventMeta attached by gvaaudiodetect by default, then this Tensor will contain “label_id”, “confidence”, “start_timestamp”, “end_timestamp” fields. If AudioEvent doesn’t have detection Tensor, it will be created in-place.

Returns:

detection Tensor, empty if there were no detection Tensor objects added to this AudioEvent when this method was called

int label_id() const

Get label_id from detection Tensor, last added to this AudioEvent.

Returns:

last added detection Tensor label_id if exists, otherwise 0

void set_label(std::string label)

Set AudioEvent label.

Parameters:

label

Label to set

GstGVAAudioEventMeta* _meta() const

Internal function, don’t use or use with caution.

Returns:

pointer to underlying GstGVAAudioEventMeta