class GVA::AudioFrame

Overview

This class represents audio frame - object for working with AudioEvent and Tensor objects which belong to this audio frame . AudioEvent describes detected object (segments) and its Tensor objects (inference results on AudioEvent level). Tensor describes inference results on AudioFrame level. AudioFrame also provides access to underlying GstBuffer and GstAudioInfo describing frame’s audio information (such as format, channels, etc.). More…

#include <audio_frame.h>

class AudioFrame {
public:
    // construction

    AudioFrame(GstBuffer* buffer, GstAudioInfo* info);
    AudioFrame(GstBuffer* buffer, const GstCaps* caps);
    AudioFrame(GstBuffer* buffer);

    // methods

    GstAudioMeta* audio_meta();
    GstAudioInfo* audio_info();
    std::vector<AudioEvent> events();
    const std::vector<AudioEvent> events() const;
    std::vector<Tensor> tensors();
    const std::vector<Tensor> tensors() const;
    std::vector<std::string> messages();

    AudioEvent add_event(
        long start_time,
        long end_time,
        std::string label = std::string(),
        double confidence = 0.0
    );

    Tensor add_tensor();
    void add_message(const std::string& message);
    void remove_event(const AudioEvent& event);
    void remove_tensor(const Tensor& tensor);
};

Detailed Documentation

This class represents audio frame - object for working with AudioEvent and Tensor objects which belong to this audio frame . AudioEvent describes detected object (segments) and its Tensor objects (inference results on AudioEvent level). Tensor describes inference results on AudioFrame level. AudioFrame also provides access to underlying GstBuffer and GstAudioInfo describing frame’s audio information (such as format, channels, etc.).

Construction

AudioFrame(GstBuffer* buffer, GstAudioInfo* info)

Construct AudioFrame instance from GstBuffer and GstAudioInfo. This is preferred way of creating AudioFrame.

Parameters:

buffer	GstBuffer* to which metadata is attached and retrieved
info	GstAudioInfo* containing audio information

AudioFrame(GstBuffer* buffer, const GstCaps* caps)

Construct AudioFrame instance from GstBuffer and GstCaps.

Parameters:

buffer	GstBuffer* to which metadata is attached and retrieved
caps	GstCaps* from which audio information is obtained

AudioFrame(GstBuffer* buffer)

Construct AudioFrame instance from GstBuffer. Audio information will be obtained from buffer. This is not recommended way of creating AudioFrame, because it relies on GstAudioMeta which can be absent for the buffer.

Parameters:

buffer

GstBuffer* to which metadata is attached and retrieved

Methods

GstAudioMeta* audio_meta()

Get audio metadata of buffer.

Returns:

GstAudioMeta of buffer, nullptr if no GstAudioMeta available

GstAudioInfo* audio_info()

Get GstAudioInfo of this AudioFrame. This is preferrable way of getting audio information.

Returns:

GstAudioInfo of this AudioFrame

std::vector<AudioEvent> events()

Get AudioEvent objects attached to AudioFrame.

Returns:

vector of AudioEvent objects attached to AudioFrame

const std::vector<AudioEvent> events() const

Get AudioEvent objects attached to AudioFrame.

Returns:

vector of AudioEvent objects attached to AudioFrame

std::vector<Tensor> tensors()

Get Tensor objects attached to AudioFrame.

Returns:

vector of Tensor objects attached to AudioFrame

const std::vector<Tensor> tensors() const

Get Tensor objects attached to AudioFrame.

Returns:

vector of Tensor objects attached to AudioFrame

std::vector<std::string> messages()

Get messages attached to this AudioFrame.

Returns:

messages attached to this AudioFrame

AudioEvent add_event(
    long start_time,
    long end_time,
    std::string label = std::string(),
    double confidence = 0.0
)

Attach AudioEvent to this AudioFrame. This function takes ownership of event_tensor, if passed @start_time: start time stamp of the segment @end_time: end time stamp of the segment.

Parameters:

label	object label
confidence	detection confidence

Returns:

new AudioEvent instance

Tensor add_tensor()

Attach empty Tensor to this AudioFrame.

Returns:

new Tensor instance

void add_message(const std::string& message)

Attach message to this AudioFrame.

Parameters:

message

message to attach to this AudioFrame

void remove_event(const AudioEvent& event)

Remove AudioEvent.

Parameters:

event

the AudioEvent to remove

void remove_tensor(const Tensor& tensor)

Remove Tensor.

Parameters:

tensor

the Tensor to remove