Global Patent Index - EP 3113180 A1

EP 3113180 A1 20170104 - METHOD FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL AND APPARATUS FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL

Title (en)

METHOD FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL AND APPARATUS FOR PERFORMING AUDIO INPAINTING ON A SPEECH SIGNAL

Title (de)

VERFAHREN ZUR DURCHFÜHRUNG EINER AUDIO-EINBLENDUNG IN EIN SPRACHSIGNAL UND VORRICHTUNG ZUR DURCHFÜHRUNG EINER AUDIO-EINBLENDUNG IN EIN SPRACHSIGNAL

Title (fr)

PROCÉDÉ ET APPAREIL PERMETTANT D'EFFECTUER DES RETOUCHES AUDIO SUR UN SIGNAL VOCAL

Publication

EP 3113180 A1 20170104 (EN)

Application

EP 15306085 A 20150702

Priority

EP 15306085 A 20150702

Abstract (en)

Audio inpainting is a technique for recovering missing audio samples. Known inpainting methods are not suitable if the missing speech samples are very different from the remaining available speech signal, or if a gap covers an entire word or a sequence of words. An improved method for speech inpainting comprises synthesizing speech or using natural speech recordings for a gap that occurs in a speech signal by using a transcript of the speech signal, converting the synthesized or natural speech by voice conversion according to the original speech signal, and blending the synthesized or natural converted speech into the original speech signal to fill the gap. The disclosed speech audio inpainting technique plausibly recovers lost speech parts with the help of synthetic speech generated from the text transcript or natural speech recordings of the missing part. The synthesized speech is modified by voice conversion to fit with the original speaker's voice.

IPC 8 full level

G10L 19/005 (2013.01)

CPC (source: EP)

G10L 19/005 (2013.01)

Citation (applicant)

  • AMIR ADLER; VALENTIN EMIYA; MARIA JAFARI; MICHAEL ELAD; REMI GRIBONVAL; MARK D. PLUMBLEY: "Audio inpainting", IEEE TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, vol. 20, no. 3, 2012, pages 922 - 932, XP011397627, DOI: doi:10.1109/TASL.2011.2168211
  • P. SMARAGDIS ET AL.: "Missing data imputation for spectral audio signal", PROC. IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP, 2009
  • J. LE ROUX ET AL.: "Computational auditory induction as a missing data model-fitting problem with Bregman divergence", SPEECH COMMUNICATION, vol. 53, no. 5, 2011, pages 658 - 676
  • DRORI ET AL.: "Spectral sound gap filling", PROC. ICPR, 2004, pages 871 - 874, XP010724530, DOI: doi:10.1109/ICPR.2004.1334397
  • JANI NURMINEN; HANNA SITEN; VICTOR POPA; ELINA HELANDER; MONCEF GABBOUJ: "Voice Conversion, Speech Enhancement, Modeling and Recognition- Algorithms and Applications", 2012, INTECH
  • HIDEKI KAWAHARA: "Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited", 1997 IEEE INTER- NATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, ICASSP'97, 21 April 1997 (1997-04-21), pages 1303 - 1306
  • Y. BAHAT; Y. Y. SCHECHNER; M. ELAD: "Self-content-based audio inpainting", SIGNAL PROCESSING, vol. 111, 2015, pages 61 - 72
  • D. ELLIS, DYNAMIC TIME WARP (DTW) IN MATLAB, 2003, Retrieved from the Internet <URL:http://www.ee.columbia.edu/-dpwe/resources/matlab/dtw>
  • TODA, T.; BLACK, A.W.; TOKUDA, K.: "Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory", AUDIO, SPEECH, AND LANGUAGE PROCESSING, IEEE TRANSACTIONS ON, vol. 15, no. 8, November 2007 (2007-11-01), pages 2222,2235, XP011192987, DOI: doi:10.1109/TASL.2007.907344
  • AIHARA, R.; NAKASHIKA, T.; TAKIGUCHI, T.; ARIKI, Y.: "Voice conversion based on Non-negative matrix factorization using phoneme-categorized dictionary", ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2014 IEEE INTERNATIONAL CONFERENCE ON, 4 May 2014 (2014-05-04), pages 7894,7898

Citation (search report)

  • [A] US 2015023345 A1 20150122 - SCHECHNER YOAV [IL], et al
  • [A] US 2011165912 A1 20110707 - WANG QINGFANG [CN], et al
  • [A] AMIR ADLER ET AL: "Audio Inpainting", IEEE TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, IEEE SERVICE CENTER, NEW YORK, NY, USA, vol. 20, no. 3, 1 March 2012 (2012-03-01), pages 922 - 932, XP011397627, ISSN: 1558-7916, DOI: 10.1109/TASL.2011.2168211

Designated contracting state (EPC)

AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

Designated extension state (EPC)

BA ME

DOCDB simple family (publication)

EP 3113180 A1 20170104; EP 3113180 B1 20200122; PL 3113180 T3 20200601

DOCDB simple family (application)

EP 15306085 A 20150702; PL 15306085 T 20150702