Aleksandr Diment

Publications

2019

• A. Diment, E. Fagerlund, A. Benfield and T. Virtanen. Detection of Typical Pronunciation Errors in Non-native English Speech Using Convolutional Recurrent Neural Networks. In proc. of International Joint Conference on Neural Networks (IJCNN), 2019.
  • View abstract
  • Export BibTeX
  • DownloadPDF

  A machine learning method for the automatic detection of pronunciation errors made by non-native speakers of English is proposed. It consists of training word-specific binary classifiers on a collected dataset of isolated words with possible pronunciation errors, typical for Finnish native speakers. The classifiers predict whether the typical error is present in the given word utterance. They operate on sequences of acoustic features, extracted from consecutive frames of an audio recording of a word utterance. The proposed architecture includes a convolutional neural network, a recurrent neural network, or a combination of the two. The optimal topology and hyperparameters are obtained in a Bayesian optimisation setting using a tree-structured Parzen estimator. A dataset of 80 words uttered naturally by 120 speakers is collected. The performance of the proposed system, evaluated on a well-represented subset of the dataset, shows that it is capable of detecting pronunciation errors in most of the words (46/49) with high accuracy (mean accuracy gain over the zero rule 12.21 percent points).

  @INPROCEEDINGS{Diment19_PL,
    author={Diment, Aleksandr and Fagerlund, Eemi and Benfield, Adrian  and Virtanen, Tuomas},
    booktitle={Proceedings of the International Joint Conference on Neural Networks (IJCNN)},
    title={Detection of Typical Pronunciation Errors in Non-native English Speech Using Convolutional Recurrent Neural Networks},
    year={2019}
  }
  

• A. Mesaros, A. Diment, B. Elizalde, T. Heittola, E. Vincent, B. Raj, and T. Virtanen. Sound Event Detection in the DCASE 2017 Challenge. In IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, in press.
  • View abstract
  • Export BibTeX
  • DownloadPDF

  Each edition of the challenge on Detection and Classification of Acoustic Scenes and Events (DCASE) contained several tasks involving sound event detection in different setups. DCASE 2017 presented participants with three such tasks, each having specific datasets and detection requirements: Task 2, in which target sound events were very rare in both training and testing data, Task 3 having overlapping events annotated in real-life audio, and Task 4, in which only weakly-labeled data was available for training. In this paper, we present the three tasks, including the datasets and baseline systems, and analyze the challenge entries for each task. We observe the popularity of methods using deep neural networks, and the still widely used mel frequency based representations, with only few approaches standing out as radically different. Analysis of the systems behavior reveals that task-specific optimization has a big role in producing good performance; however, often this optimization closely follows the ranking metric, and its maximization/minimization does not result in universally good performance. We also introduce the calculation of confidence intervals based on a jackknife resampling procedure, to perform statistical analysis of the challenge results. The analysis indicates that while the 95% confidence intervals for many systems overlap, there are significant differences in performance between the top systems and the baseline for all tasks.

  @article{Mesaros2019_TASLP,
      author = "Mesaros, Annamaria and Diment, Aleksandr and Elizalde, Benjamin and Heittola, Toni and Vincent, Emmanuel and Raj, Bhiksha and Virtanen, Tuomas",
      doi = "10.1109/TASLP.2019.2907016",
      title = "Sound event detection in the {DCASE} 2017 {C}hallenge",
      journal = "IEEE/ACM Transactions on Audio, Speech, and Language Processing",
      note = "in press",
      year = "2019",
      keywords = "Sound event detection, weak labels, pattern recognition, jackknife estimates, confidence intervals",
  }

2018

• J. Nikunen, A. Diment, T. Virtanen. Separation of Moving Sound Sources Using Multichannel NMF and Acoustic Tracking. In IEEE/ACM Transactions on Audio, Speech and Language Processing, 2018.
  • View abstract
  • Export BibTeX
  • DownloadPDF

  In this paper we propose a method for separation of moving sound sources. The method is based on first tracking the sources and then estimation of source spectrograms using multichannel non-negative matrix factorization (NMF) and extracting the sources from the mixture by single-channel Wiener filtering. We propose a novel multichannel NMF model with time-varying mixing of the sources denoted by spatial covariance matrices (SCM) and provide update equations for optimizing model parameters minimizing squared Frobenius norm. The SCMs of the model are obtained based on estimated directions of arrival of tracked sources at each time frame. The evaluation is based on established objective separation criteria and using real recordings of two and three simultaneous moving sound sources. The compared methods include conventional beamforming and ideal ratio mask separation. The proposed method is shown to exceed the separation quality of other evaluated blind approaches according to all measured quantities. Additionally, we evaluate the method's susceptibility towards tracking errors by comparing the separation quality achieved using annotated ground truth source trajectories.

  @ARTICLE{Nikunen18,
    author={J. {Nikunen} and A. {Diment} and T. {Virtanen}},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title={Separation of Moving Sound Sources Using Multichannel NMF and Acoustic Tracking},
    year={2018},
    volume={26},
    number={2},
    pages={281-295},
    keywords={acoustic signal processing;blind source separation;covariance matrices;direction-of-arrival estimation;matrix decomposition;optimisation;source separation;time-varying filters;Wiener filters;acoustic tracking;single-channel Wiener filtering;spatial covariance matrices;ideal ratio mask separation;annotated ground truth source trajectories;source spectrogram estimation;multichannel nonnegative matrix factorization model;multichannel NMF model;source tracking;moving sound source separation;time-varying source mixing;SCM;directions of arrival estimation;objective separation criteria;beamforming;optimisation model;Acoustics;Direction-of-arrival estimation;Microphones;Mathematical model;Estimation;Spectrogram;Array signal processing;Sound source separation;moving sound sources;time-varying mixing model;microphone arrays;acoustic source tracking},
    doi={10.1109/TASLP.2017.2774925},
    ISSN={2329-9290},
    month={Feb},
  }
  

2017

• A. Diment and T. Virtanen. Transfer Learning of Weakly Labelled Audio. In proc. of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2017.
  • View abstract
  • Export BibTeX
  • Data synth code
  • DownloadPDF

  Many machine learning tasks have been shown solvable with impressive levels of success given large amounts of training data and computational power. For the tasks which lack data sufficient to achieve high performance, methods for transfer learning can be applied. These refer to performing the new task while having some prior knowledge of the nature of the data, gained by first performing a different task, for which training data is abundant. Previously shown successful for machine vision and natural language processing, transfer learning is investigated in this work for audio analysis. We propose to solve the problem of weak label classification (tagging) with small amounts of training data by transferring the abstract knowledge about the nature of audio data from another tagging task. Three neural network architectures are proposed and evaluated, showing impressive classification accuracy gains.

  @INPROCEEDINGS{Diment17_TL,
    author={Diment, Aleksandr and Virtanen, Tuomas},
    booktitle={Applications of Signal Processing to Audio and Acoustics (WASPAA), 2017 IEEE Workshop on},
    title={Transfer Learning of Weakly Labelled Audio},
    year={2017},
    month={Oct},
    keywords={transfer learning, tagging, weak labels, audio}
  }
  

• T. Virtanen, A. Mesaros, T. Heittola, A. Diment, E. Vincent, E. Benetos, B. M. Elizalde. (eds.) Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017). Tampere University of Technology. Laboratory of Signal Processing, 2017. 142 p.
  • Export BibTeX
  • Check the website
  • DownloadPDF

  @book{DCASE17_book_bib,
  	title = "Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017)",
  	author = "Tuomas Virtanen and Annamaria Mesaros and Toni Heittola and Aleksandr Diment and Emmanuel Vincent and Emmanouil Benetos and Elizalde, {Benjamin Martinez}",
  	year = "2017",
  	month = "11",
  	publisher = "Tampere University of Technology. Laboratory of Signal Processing"
  }
  

• A. Mesaros, T. Heittola, A. Diment, B. Elizalde, A. Shah, R. Badlani, E. Vincent, B. Raj, and T. Virtanen. DCASE 2017 challenge setup: tasks, datasets and baseline system. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017). November 2017.
  • View abstract
  • Export BibTeX
  • Listen to demos
  • DownloadPDF

  DCASE 2017 Challenge consists of four tasks: acoustic scene classification, detection of rare sound events, sound event detection in real-life audio, and large-scale weakly supervised sound event detection for smart cars. This paper presents the setup of these tasks: task definition, dataset, experimental setup, and baseline system results on the development dataset. The baseline systems for all tasks rely on the same implementation using multilayer perceptron and log mel-energies, but differ in the structure of the output layer and the decision making process, as well as the evaluation of system output using task specific metrics.

  @inproceedings{DCASE2017challenge,
      Author = "Mesaros, A. and Heittola, T. and Diment, A. and Elizalde, B. and Shah, A. and Badlani, R. and Vincent, E. and Raj, B. and Virtanen, T.",
      title = "{DCASE} 2017 Challenge Setup: Tasks, Datasets and Baseline System",
      booktitle = "Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017)",
      month = "November",
      year = "2017",
      keywords = "Sound scene analysis, Acoustic scene classification, Sound event detection, Audio tagging, Rare sound events"
  }
  

• M. Valenti, A. Diment, G. Parascandolo, S. Squartini, T. Virtanen. DCASE 2016 Acoustic Scene Classification Using Convolutional Neural Networks. In proceedings of the 2017 International Joint Conference on Neural Networks, Anchorage, Alaska, USA.
  • View abstract
  • Export BibTeX

  This paper presents a novel application of convo- lutional neural networks (CNNs) for the task of acoustic scene classification (ASC). We here propose the use of a CNN trained to classify short sequences of audio, represented by their log-mel spectrogram. We also introduce a training method that can be used under particular circumstances in order to make full use of small datasets. The proposed system is tested and evaluated on three different ASC datasets and compared to other state-of-the- art systems which competed in the “Detection and Classification of Acoustic Scenes and Events” (DCASE) challenges held in 2016 and 2013. The best accuracy scores obtained by our system on the DCASE 2016 datasets are 79.0% (development) and 86.2% (evaluation), which constitute a 6.4% and 9% improvements with respect to the baseline system. Finally, when tested on the DCASE 2013 evaluation dataset, the proposed system manages to reach a 77.0% accuracy, improving by 1% the challenge winner’s score.

  @INPROCEEDINGS{Valenti17,
    AUTHOR="Michele Valenti and Aleksandr Diment and Giambattista Parascandolo and Stefano Squartini and Tuomas Virtanen",
    TITLE="{DCASE} 2016 Acoustic Scene Classification Using Convolutional Neural Networks",
    BOOKTITLE="Proc. of the 2017 International Joint Conference on Neural Networks",
    ADDRESS="Anchorage, Alaska, USA",
    DAYS=14-19,
    MONTH=may,
    YEAR=2017,
  }
  

Thesis supervision

• E. Nieminen. Building and Visualizing a Percussion Dataset Using Deep Audio Embeddings and Dimensionality Reduction, BSc thesis, Tampere University, 2020.
• View abstract
• Export BibTeX
• Source code
• Download PDF

In this bachelor's thesis, a dataset consisting of samples of various drums and percussion is built and a method for visualizing it is described. An insight to the process of planning, recording and labeling an audio dataset is provided, and further on, a network embedding based visualization technique is applied to the dataset. The dataset visualization method consists of two parts. First, high-dimensional embeddings produced by OpenL3, a pre-trained deep convolutional neural network model, are obtained. Then, the embeddings are projected to a 2-dimensional plane with the t-Stochastic Neighborhood Embedding (t-SNE) algorithm. Promising results are presented, as the method is able to separate different drum types and articulations into distinct clusters in the 2D plane, and moreover, it is able to recognize dynamic variations inside the classes themselves and organize the samples respectively. Additionally, Python programs are written to assist in the dataset labeling and visualization tasks, and the code and the dataset alongside with pre-computed embeddings are published online.

@thesis{Nieminen20_BSc,
    author = {Elias Nieminen},
    title = {Building and Visualizing a Percussion Dataset Using Deep Audio Embeddings and Dimensionality Reduction},
    school = {Tampere University},
    address = {Finland},
    year = {2020},
    type={Bachelor's Thesis}
    }

• V.-V. Eklund. Data Augmentation Techniques for Robust Audio Analysis, MSc thesis, Tampere University, 2019.
• View abstract
• Export BibTeX
• Download PDF

Having large amounts of training data is necessary for the ever more popular neural networks to perform reliably. Data augmentation, i.e. the act of creating additional training data by performing label-preserving transformations for existing training data, is an efficient solution for this problem. While increasing the amount of data, introducing variations to the data via the transformations also has the power to make machine learning models more robust in real life conditions with noisy environments and mismatches between the training and test data.

In this thesis, data augmentation techniques in audio analysis are reviewed, and a tool for audio data augmentation (TADA) is presented. TADA is capable of performing three audio data augmentation techniques, which are convolution with mobile device microphone impulse responses, convolution with room impulse responses, and addition of background noises. TADA is evaluated by using it in a pronunciation error classification task, where typical pronunciation errors of Finnish people uttering English words are classified. All the techniques are tested first individually and then also in combination.

The experiments are executed with both original and augmented data. In all experiments, using TADA improves the performance of the classifier when compared to training with only original data. Robustness against unseen devices and rooms also improves. Additional gain from performing combined augmentation starts to saturate only after augmenting the training data to 30 times the original amount. Based on the positive impact of TADA for the classification task, it is found that data augmentation with convolutional and additive noises is an effective combination for increasing robustness against environmental distortions and channel effects.

@thesis{Eklund19_MSc,
  author = {Ville-Veikko Eklund},
  title = {Data Augmentation Techniques for Robust Audio Analysis},
  school = {Tampere University},
  address = {Finland},
  year = {2019},
  type={Master's Thesis}
  }

• T. Honka. One-shot Learning with Siamese Networks for Environmental Audio, BSc thesis, Tampere University, 2019.
• View abstract
• Export BibTeX
• Source code
• Download PDF

In the recent years deep learning based approaches have dominated different types of classification problems. Usually these approaches require large amounts of training data to train a model capable of generalizing to any unseen data of the same type. However, in some applications it might be difficult to gather training data efficiently and it would be beneficial to classify new samples using only a few or even a single training example.

For us humans the knowledge from previously learned concepts is relatively easy to transfer to unfamiliar concepts, therefore many researchers have experimented with this idea in machine learning classification tasks. The idea of only using a single labelled example to classify unseen data is known as one-shot learning and has been successful especially in the field of computer vision. Many of the modern approaches for one-shot learning utilize a special neural network architecture named siamese network. This architecture can be trained to predict similarities between inputs, and can be used for a metric-based approach to one-shot learning. Siamese networks have been used for different audio related tasks before, however their usage in one-shot learning for audio classification has received less attention compared to computer vision.

The purpose of this thesis is to extend the idea of one-shot learning to environmental audio classification and see if this approach is feasible. The proposed system was trained and evaluated on the ESC dataset, consisting of 50 different environmental audio categories. The final one-shot evaluation was done to 5 completely unseen classes, using only a single example of each class when performing the classification. The results show that convolutional siamese networks are indeed a valid approach to the difficult one-shot classification task for environmental audio.

@thesis{Honka19_BSc,
    author = {Tapio Honka},
    title = {One-shot Learning with Siamese Networks for Environmental Audio},
    school = {Tampere University},
    address = {Finland},
    year = {2019},
    type={Bachelor's Thesis}
    }

• V.-V. Eklund. Audio dataset creation, BSc thesis, Tampere University of Technology, 2018.
• View abstract
• Export BibTeX
• Source code
• Download PDF

The goal of this thesis was to build an easily accessible and approachable environmental audio dataset for benchmarking machine learning algorithms. The samples in the created dataset were collected from Freesound, which is an online database consisting of sounds uploaded by its users. The dataset consists of three classes, each containing 50 instances of original recordings with annotations for 5-second segments. The classes are 'dog', 'bird' and 'rain', and the dataset was therefore given the name DBR dataset. Additionally, a script for evaluating support vector, random forest and k-NN classifers with the dataset was written. The building and evaluation stages were documented step-by-step and a Jupyter notebook tutorial for using the dataset was created. The dataset, the scripts and the notebook were published online.

@thesis{Eklund18_BSc,
  author = {Ville-Veikko Eklund},
  title = {Audio dataset creation},
  school = {Tampere University of Technology},
  address = {Finland},
  year = {2018},
  type={Bachelor's Thesis}
  }

• M. Valenti. Convolutional neural networks for acoustic scene classification, MSc thesis, Tampere University of Technology, 2016.
  • View abstract
  • Export BibTeX
  • Download PDF

  In this thesis we investigate the use of deep neural networks applied to the field of computational audio scene analysis, in particular to acoustic scene classification. This task concerns the recognition of an acoustic scene, like a park or a home, performed by an artificial system. In our work we examine the use of deep models aiming to give a contribution in one of their use cases which is, in our opinion, one of the most poorly explored. The neural architecture we propose in this work is a convolutional neural network specifically designed to work on a time-frequency audio representation known as log-mel spectrogram. The network output is an array of prediction scores, each of which is associated with one class of a set of 15 predefined classes. In addition, the architecture features batch normalization, a recently proposed regularization technique used to enhance the network performance and to speed up its training.

  We also investigate the use of different audio sequence lengths as classification unit for our network. Thanks to these experiments we observe that, for our artificial system, the recognition of long sequences is not easier than of medium-length sequences, hence highlighting a counterintuitive behaviour. Moreover, we introduce a training procedure which aims to make the best of small datasets by using all the labeled data available for the network training. This procedure, possible under particular circumstances, constitutes a trade-off between an accurate training stop and an increased data representation available to the network. Finally, we compare our model to other systems, proving that its recognition ability can outperform either other neural architectures as well as other state-of-the-art statistical classifiers, like support vector machines and Gaussian mixture models. The proposed system reaches good accuracy scores on two different databases collected in 2013 and 2016. The best accuracy scores, obtained according to two cross-validation setups, are 77% and 79% respectively. These scores constitute a 22% and 6.1% accuracy increment with respect to the correspondent baselines published together with datasets.

  @MastersThesis{Valenti16_MSc_bib,
    author = {Michele Valenti},
    title = {Convolutional Neural Networks for Acoustic Scene Classification},
    school = {Tampere University of Technology},
    address = {Finland},
    year = {2016},
    }

Teaching

• 2016-2017:
  • Advanced Signal Processing Laboratory.
• 2014-2015:
  • Audio and Speech Processing.
• 2013-2014:
  • Audio and Speech Processing.
  • Analysis of Audio, Speech and Music Signals.
• 2012-2013:
  • Speech recognition.

Page design based on theme by orderedlist, CC BY-SA 3.0.