audiomate.corpus¶
This module contains all parts needed for using a corpus. Aside the main corpus class
audiomate.Corpus
, there are different loaders in the audiomate.corpus.io
.
CorpusView¶
-
class
audiomate.corpus.
CorpusView
[source]¶ This class defines the basic interface of a corpus. It is not meant to be instantiated directly. It only describes the methods for accessing data of the corpus.
Notes
All paths to files should be held as absolute paths in memory.
-
all_label_values
(label_list_ids=None)[source]¶ Return a set of all label-values occurring in this corpus.
Parameters: label_list_ids (list) – If not None, only labels from label-lists with an id contained in this list are considered. Returns: A set of distinct label-values. Return type: set
-
all_tokens
(delimiter=' ', label_list_ids=None)[source]¶ Return a list of all tokens occurring in one of the labels in the corpus.
Parameters: - delimiter (str) – The delimiter used to split labels into tokens
(see
audiomate.annotations.Label.tokenized()
). - label_list_ids (list) – If not None, only labels from label-lists with an idx contained in this list are considered.
Returns: A set of distinct tokens.
Return type: set
- delimiter (str) – The delimiter used to split labels into tokens
(see
-
contains_issuer
(issuer)[source]¶ Return
True
if the given issuer is in the corpus already,False
otherwise.
-
contains_track
(track)[source]¶ Return
True
if the given track is in the corpus already,False
otherwise.
-
feature_containers
¶ Return the feature-containers in the corpus.
Returns: A dictionary containing audiomate.container.FeatureContainer
objects with the feature-idx as key.Return type: dict
-
issuers
¶ Return the issuers in the corpus.
Returns: A dictionary containing audiomate.issuers.Issuer
objects with the issuer-idx as key.Return type: dict
-
label_count
(label_list_ids=None)[source]¶ Return a dictionary containing the number of times, every label-value in this corpus is occurring.
Parameters: label_list_ids (list) – If not None, only labels from label-lists with an id contained in this list are considered. Returns: A dictionary containing the number of occurrences with the label-value as key. Return type: dict
-
label_durations
(label_list_ids=None)[source]¶ Return a dictionary containing the total duration, every label-value in this corpus is occurring.
Parameters: label_list_ids (list) – If not None, only labels from label-lists with an id contained in this list are considered. Returns: A dictionary containing the total duration with the label-value as key. Return type: dict
-
name
¶ Return the name of the dataset (Equals basename of the path, if not None).
-
num_feature_containers
¶ Return the number of feature-containers in the corpus.
-
num_issuers
¶ Return the number of issuers in the corpus.
-
num_subviews
¶ Return the number of subviews in the corpus.
-
num_tracks
¶ Return number of tracks.
-
num_utterances
¶ Return number of utterances.
-
split_utterances_to_max_time
(max_time=60.0, overlap=0.0)[source]¶ Create a new corpus, where all the utterances are of given maximal duration. Utterance longer than
max_time
are split up into multiple utterances.Warning
Subviews and FeatureContainers are not added to the newly create corpus.
Parameters: - max_time (float) – Maximal duration for target utterances in seconds.
- overlap (float) – Amount of overlap in seconds. The overlap is measured from the center of the splitting. (The actual overlap of two segments is 2 * overlap)
Returns: A new corpus instance.
Return type:
-
stats
()[source]¶ Return statistics calculated overall samples of all utterances in the corpus.
Returns: A DataStats object containing statistics overall samples in the corpus. Return type: DataStats
-
stats_per_utterance
()[source]¶ Return statistics calculated for all samples of each utterance in the corpus.
Returns: A dictionary containing a DataStats object for each utt. Return type: dict
-
subviews
¶ Return the subviews of the corpus.
Returns: A dictionary containing audiomate.corpus.Subview
objects with the subview-idx as key.Return type: dict
-
total_duration
¶ Return the total amount of audio summed over all utterances in the corpus in seconds.
-
tracks
¶ Return the tracks in the corpus.
Returns: A dictionary containing audiomate.track.Track
objects with the track-idx as key.Return type: dict
-
utterances
¶ Return the utterances in the corpus.
Returns: A dictionary containing audiomate.corpus.assets.Utterance
objects with the utterance-idx as key.Return type: dict
-
Corpus¶
-
class
audiomate.corpus.
Corpus
(path=None)[source]¶ The Corpus class represents a single corpus. It extends
audiomate.corpus.CorpusView
with the functionality for loading and saving. Furthermore it provides the functionality for adding/modifying assets of the corpus like tracks and utterances.Parameters: path (str) – Path where the corpus is stored. (Optional) -
feature_containers
¶ Return the feature-containers in the corpus.
Returns: A dictionary containing audiomate.container.FeatureContainer
objects with the feature-idx as key.Return type: dict
-
classmethod
from_corpus
(corpus)[source]¶ Create a new modifiable corpus from any other CorpusView. This for example can be used to create a independent modifiable corpus from a subview.
Parameters: corpus (CorpusView) – The corpus to create a copy from. Returns: A new corpus with the same data as the given one. Return type: Corpus
-
import_issuers
(new_issuers)[source]¶ Add the given issuers/issuer to the corpus. If any of the given issuer-ids already exists, a suffix is appended so it is unique.
Parameters: issuers (list) – Either a list of or a single audiomate.issuers.Issuer
.Returns: - A dictionary containing idx mappings (old-issuer-idx/issuer-instance).
- If a issuer is imported, whose id already exists this mapping can be used to check the new id.
Return type: dict
-
import_subview
(idx, subview)[source]¶ Add the given subview to the corpus.
Parameters: - idx (str) – An idx that is unique in the corpus for identifying the subview. If already a subview exists with the given id it will be overridden.
- subview (Subview) – The subview to add.
-
import_tracks
(import_tracks)[source]¶ Add the given tracks/track to the corpus. If any of the given track-ids already exists, a suffix is appended so it is unique.
Parameters: import_tracks (list) – Either a list of or a single audiomate.tracks.Track
.Returns: - A dictionary containing track-idx mappings (old-track-idx/track-instance).
- If a track is imported, whose idx already exists this mapping can be used to check the new id.
Return type: dict
-
import_utterances
(utterances)[source]¶ Add the given utterances/utterance to the corpus. If any of the given utterance-ids already exists, a suffix is appended so it is unique.
Parameters: utterances (list) – Either a list of or a single audiomate.tracks.Utterance
.Returns: - A dictionary containing idx mappings (old-utterance-idx/utterance-instance).
- If a utterance is imported, whose id already exists this mapping can be used to check the new id.
Return type: dict
-
issuers
¶ Return the issuers in the corpus.
Returns: A dictionary containing audiomate.issuers.Issuer
objects with the issuer-idx as key.Return type: dict
-
classmethod
load
(path, reader=None)[source]¶ Loads the corpus from the given path, using the given reader. If no reader is given the
audiomate.corpus.io.DefaultReader
is used.Parameters: - path (str) – Path to load the corpus from.
- reader (str, CorpusReader) – The reader or the name of the reader to use.
Returns: The loaded corpus.
Return type:
-
classmethod
merge_corpora
(corpora)[source]¶ Merge a list of corpora into one.
Parameters: corpora (Iterable) – An iterable of audiomate.corpus.CorpusView
.Returns: A corpus with the data from all given corpora merged into one. Return type: Corpus
-
merge_corpus
(corpus)[source]¶ Merge the given corpus into this corpus. All assets (tracks, utterances, issuers, …) are copied into this corpus. If any ids (utt-idx, track-idx, issuer-idx, subview-idx, …) are occurring in both corpora, the ids from the merging corpus are suffixed by a number (starting from 1 until no other is matching).
Parameters: corpus (CorpusView) – The corpus to merge.
-
name
¶ Return the name of the dataset (Equals basename of the path, if not None).
-
new_feature_container
(idx, path=None)[source]¶ Add a new feature container with the given data.
Parameters: - idx (str) – An unique identifier within the dataset.
- path (str) – The path to store the feature file. If None a default path is used.
Returns: The newly added feature-container.
Return type:
-
new_file
(path, track_idx, copy_file=False)[source]¶ Adds a new audio file to the corpus with the given data.
Parameters: - path (str) – Path of the file to add.
- track_idx (str) – The id to associate the file-track with.
- copy_file (bool) – If True the file is copied to the data set folder, otherwise the given path is used directly.
Returns: The newly added file.
Return type:
-
new_issuer
(issuer_idx, info=None)[source]¶ Add a new issuer to the dataset with the given data.
Parameters: - issuer_idx (str) – The id to associate the issuer with. If None or already exists, one is generated.
- info (dict, list) – Additional info of the issuer.
Returns: The newly added issuer.
Return type:
-
new_utterance
(utterance_idx, track_idx, issuer_idx=None, start=0, end=inf)[source]¶ Add a new utterance to the corpus with the given data.
Parameters: - track_idx (str) – The track id the utterance is in.
- utterance_idx (str) – The id to associate with the utterance. If None or already exists, one is generated.
- issuer_idx (str) – The issuer id to associate with the utterance.
- start (float) – Start of the utterance within the track [seconds].
- end (float) – End of the utterance within the track [seconds].
inf
equals the end of the track.
Returns: The newly added utterance.
Return type:
-
relocate_audio_to_single_container
(target_path)[source]¶ Copies every track to a single container. Afterwards all tracks in the container are linked against this single container.
-
relocate_audio_to_wav_files
(target_path)[source]¶ Copies every track to its own wav file in the given folder. Every track will be stored at
target_path/track_id.wav
.
-
save_at
(path, writer=None)[source]¶ Save this corpus at the given path. If the path differs from the current path set, the path gets updated.
Parameters: - path (str) – Path to save the data set to.
- writer (str, CorpusWriter) – The writer or the name of the reader to use.
-
subviews
¶ Return the subviews of the corpus.
Returns: A dictionary containing audiomate.corpus.Subview
objects with the subview-idx as key.Return type: dict
-
tracks
¶ Return the tracks in the corpus.
Returns: A dictionary containing audiomate.track.Track
objects with the track-idx as key.Return type: dict
-
utterances
¶ Return the utterances in the corpus.
Returns: A dictionary containing audiomate.corpus.assets.Utterance
objects with the utterance-idx as key.Return type: dict
-