(19)
(11)EP 3 459 078 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
04.11.2020 Bulletin 2020/45

(21)Application number: 17726861.2

(22)Date of filing:  17.05.2017
(51)International Patent Classification (IPC): 
G11B 27/28(2006.01)
G11B 27/00(2006.01)
G10L 19/02(2013.01)
G10L 21/04(2013.01)
(86)International application number:
PCT/EP2017/061895
(87)International publication number:
WO 2017/198737 (23.11.2017 Gazette  2017/47)

(54)

APPARATUS AND METHOD FOR PROCESSING A MULTICHANNEL AUDIO SIGNAL

VORRICHTUNG UND VERFAHREN ZUR VERARBEITUNG EINES MULTIKANAL-AUDIOSIGNALS

APPAREIL ET PROCÉDÉ DE TRAITEMENT D'UN SIGNAL AUDIO MULTICANAL


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 20.05.2016 EP 16170723
14.07.2016 EP 16179531

(43)Date of publication of application:
27.03.2019 Bulletin 2019/13

(73)Proprietor: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
80686 München (DE)

(72)Inventors:
  • UHLE, Christian
    92289 Ursensollen (DE)
  • KRATZ, Michael
    91054 Erlangen (DE)
  • KLOSE, Paul
    91058 Erlangen (DE)
  • LEONARD, Timothy
    91058 Erlangen (DE)
  • LUVIZOTTO, André
    90403 Nürnberg (DE)
  • SCHARRER, Sebastian
    90518 Altdorf (DE)

(74)Representative: Zinkler, Franz et al
Schoppe, Zimmermann, Stöckeler Zinkler, Schenk & Partner mbB Patentanwälte Radlkoferstrasse 2
81373 München
81373 München (DE)


(56)References cited: : 
WO-A1-2008/046967
US-A1- 2005 137 729
WO-A1-2012/167479
US-A1- 2007 094 031
  
  • DOLSON M: "THE PHASE VOCODER: A TUTORIAL", COMPUTER MUSIC JOUR, CAMBRIDGE, MA, US, vol. 10, no. 4, 21 December 1986 (1986-12-21), pages 14-27, XP009029676, ISSN: 0148-9267 cited in the application
  • JEAN LAROCHE ET AL: "Improved Phase Vocoder Time-Scale Modification of Audio", IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 7, no. 3, 1 May 1999 (1999-05-01), pages 323-332, XP011054370, ISSN: 1063-6676 cited in the application
  • LEE E ET AL: "PhaVoRIT: A Phase Vocoder for Real-Time Interactive Time-Stretching", ICMC. INTERNATIONAL COMPUTER MUSIC CONFERENCE. PROCEEDINGS, XX, XX, 6 November 2006 (2006-11-06), pages 1-8, XP002424910, cited in the application
  • FLANAGAN J L ET AL: "Phase vocoder", BELL SYSTEM TECHNICAL JOURNAL, AT AND T, SHORT HILLS, NY, US, vol. 45, no. 9, 1 November 1966 (1966-11-01), pages 1493-1509, XP011629282, ISSN: 0005-8580, DOI: 10.1002/J.1538-7305.1966.TB01706.X [retrieved on 2014-03-15]
  • LAROCHE J ET AL: "Phase-vocoder: about this phasiness business", APPLICATIONS OF SIGNAL PROCESSING TO AUDIO AND ACOUSTICS, 1997. 1997 I EEE ASSP WORKSHOP ON NEW PALTZ, NY, USA 19-22 OCT. 1997, NEW YORK, NY, USA,IEEE, US, 19 October 1997 (1997-10-19), page 4pp, XP010248209, ISBN: 978-0-7803-3908-8
  • PUCKETTE M: "Phase-locked vocoder", APPLICATIONS OF SIGNAL PROCESSING TO AUDIO AND ACOUSTICS, 1995., IEEE ASSP WORKSHOP ON NEW PALTZ, NY, USA 15-18 OCT. 1, NEW YORK, NY, USA,IEEE, US, 15 October 1995 (1995-10-15), pages 222-225, XP010154670, DOI: 10.1109/ASPAA.1995.482995 ISBN: 978-0-7803-3064-1
  • M. PORTNOFF: "Time-scale modification of speech based on short-time Fourier analysis", IEEE TRANSACTIONS ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING., vol. 29, no. 3, 1 June 1981 (1981-06-01), pages 374-390, XP055405797, USA ISSN: 0096-3518, DOI: 10.1109/TASSP.1981.1163581
  • PORTNOFF M R: "TIME-FREQUENCY REPRESENTATION OF DIGITAL SIGNALS AND SYSTEMS BASED ON SHORT-TIME FOURIER ANALYSIS", IEEE TRANSACTIONS ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, IEEE INC. NEW YORK, USA, vol. ASSP-28, no. 1, 1 February 1980 (1980-02-01), pages 55-69, XP000984575, ISSN: 0096-3518, DOI: 10.1109/TASSP.1980.1163359
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description


[0001] The invention refers to an apparatus for processing a multichannel audio signal. The multichannel audio signal comprises a plurality of - i.e. at least two - channel signals. The apparatus performs a time scale modulation of the multichannel audio signal. The invention also refers to a corresponding method and a computer program.

[0002] Time scale modification (TSM) refers to the processing to slow down or speed up the playback of an audio signal without affecting its pitch. TSM in combination with sample rate conversion also enables to change the pitch without modifying the tempo. The challenge for TSM is to maintain all other characteristics of the audio signal (except either tempo or pitch) and in particular the sound quality. The processing should not produce audible artefacts.

[0003] For single-channel input signals, the main important characteristic is the timbre. For signals having more than one channel, also spatial characteristics need to be maintained. The spatial characteristics comprise the position and width of the direct sound source and the diffuseness of ambient sound. They can be quantified by inter-channel level differences (ICLD), inter-channel time differences (ICTD), inter-channel phase differences (ICPD) or and inter-channel coherence (ICC).

[0004] Two fundamentally different approaches to time scale modification exist. One is applied in the time domain and the other is applied in the frequency domain.

[0005] Processing in the time domain uses a synchronised overlap-add (SOLA) scheme. The signal is cut into overlapping frames and these frames are shifted and combined to stretch or shrink the signal. The shifting position is typically computed by maximising a measure of similarity, e.g. correlation between the signal frame and its shifted copy.

[0006] This method in the time domain is of low computational complexity. It yields good results for monophonic (in contrast to polyphonic) signals, e.g. speech or flute tones, because the shifting offset can be determined as an integer multiple of the period of the fundamental frequency in order to avoid discontinuities and destructive interference in the output signal. In other words, the shifted signal frames are added in a phase coherent way. For polyphonic inputs with multiple tones having different fundamental frequencies, the shifting cannot be determined such that wave form similarity is fulfilled for all tones (pitches).

[0007] For many musical signals, better results in terms of sound quality are obtained by applying the processing in the frequency domain. This method uses e.g. the phase vocoder scheme [1] as shown in the block diagram shown in Fig. 1 and briefly explained in the following.

[0008] The input audio signal x(n) is transformed into the frequency domain using a short-time Fourier transform (STFT). Equivalently, another type of filterbank or transform can be used for which the inverse processing can be applied with sufficiently small reconstruction error.

[0009] In the mentioned embodiment, the input signal x(n) is cut into overlapping frames and a Discrete Fourier transform (DFT) is computed for each frame according to the following Equation (1), yielding a short-time Fourier transform (STFT) representation of the signal, also referred to as STFT coefficients (or spectral coefficients), according to



[0010] A time frame index is denoted by m, k is a discrete frequency index with 0 ≤ k ≤ N - 1, and wa is a window function. The normalized angular frequency Ωk is given by Ωk = 2 π k / N. The DFT has size N and Ra is an analysis hop size.

[0011] The indices for time and frequency are omitted in the description when possible for brevity.

[0012] The output time-domain signal y(n) is computed from output spectral coefficients Y(m, k) in the synthesis stage by means of the inverse of the STFT which is performed in two steps:
Firstly, an inverse Discrete Fourier transform is computed for each of the M frames according to



[0013] Secondly, an overlap-add procedure is applied according to

with optional synthesis window ws(n) and synthesis hop size Rs.

[0014] A time scale modification is achieved by setting the synthesis hop size Rs and the analysis hop size Ra to different values: the signal is stretched in time if Ra < Rs and is shrinked if Ra > Rs.

[0015] The analysis window wa and synthesis window ws are chosen such that if Ra = Rs and Y(m, k) = X(m, k), then the input and output signal are identical.

[0016] Short-time Fourier transform coefficients X(m, k) for real-valued input signals (which is the case for audio signals considered here) are complex-valued numbers that can be expressed in polar coordinates by their magnitude |X| and phase Φx as



where j = √-1.

[0017] If the two hop sizes Ra and Rs differ, i.e. if Ra ≠ Rs, then the phases of Y(m, k) need to be modified such that "horizontal phase coherence" is achieved. This means that for a sinusoid of constant frequency, successive frames overlap coherently without discontinuities or phase cancellation (destructive interference).

[0018] The phase vocoder approach is appropriate for polyphonic inputs, e.g. musical recordings. Its drawback is that the modification of the phase can produce an artefact known as "transient smearing", i.e. the temporal envelope of the signal is modified such that note onsets are perceived as having less attack and sound less percussive. Additional processing can be applied to the output phase to mitigate the transient smearing, e.g. by applying a method called "phase locking" [2] or by resetting the phase during periods of silence [3].

[0019] Following a suitable procedure of modifying the phase, an output is obtained. The spectral coefficients of the output can be written with polar coordinates as Y = |Y| exp(jΦy) where Φy denotes the modified phase. The process of computing the phase Φy is in the following referred to as phase adaptation (PA).

[0020] Various ways to process two-channel audio input signals are known:
One option is to downmix the multi-channel signal to a single-channel signal, i.e. adding scaled versions of all channels, and to process the single-channel (mono) signal. Processing a mono mixdown of the input signal has the disadvantage that the stereophonic information is lost and thereby the sound quality is reduced.

[0021] A different option is to process the separated input channel signals independently. The main disadvantage of separately processing each channel signal is that arbitrary decorrelation between the channels is introduced which distorts the stereo image. Because the phase adaptation of time scale modification is a signal dependent processing, the relations between the phases of the individual channels are not preserved when the corresponding channel signals are different. The distortions of the spatial information can be perceived as blurring or widening of the stereo image of direct sound sources (singers or soloist, for example).

[0022] The patent application WO 2008/046967 A1, NOKIA CORP [FI], OJALA PASI [FI], LAKANIEMI ARI [FI], VIROLAINEN JUSSI, 24 April 2008, discloses time scaling of multichannel audio signals, comprising buffering an encoded audio input signal comprising at least one combined "sum" signal of a plurality of audio channels, and one or more corresponding sets of "side information" parameters describing a multi-channel sound image. i.e. salient inter-channel spatial cues compactly coded as "BCC side information". The length of at least one audio frame is changed by adding or removing a segment of the combined signal, and the one or more sets of side information parameters are modified corresponding to the change in length of the at least one audio frame of the combined signal.

[0023] An object of the invention is to improve existing time scale modification methods with respect to the quality of the spatial characteristics - e.g. the stereo image - of the processed signal.

[0024] The object is achieved by an apparatus as well as by a method.

[0025] The object is achieved by an apparatus for processing a multichannel audio signal. The multichannel audio signal is comprising a plurality of - or at least two - channel signals. The apparatus is configured for performing a time scale modulation of the multichannel audio signal, i.e. the multichannel audio signal is slowed down or sped up without affecting its pitch. The apparatus comprises a phase adaptor and a separator. The phase adaptor is configured for providing at least one processed signal by modifying a phase of a signal based on a combination of the channel signals. The separator is configured for providing separated signals based on the at least one processed signal.

[0026] The invention improves processing signals and especially audio or speech signals. Especially addressed is the problem of processing stereophonic input signals having two or more channels.

[0027] An advantage of the invention is that the spatial characteristics of the input audio signal are preserved such that the perceived stereo image is not distorted. In particular, the positions of the sound sources and the diffuseness are not changed due to the TSM processing.

[0028] The invention thus addresses the problem that the spatial characteristics of the output signal are severely distorted in the state of art, which is most noticable when listening to sound sources panned to the center of the stereo image.

[0029] The apparatus performs the time scale modification of the multichannel audio signal by performing a phase adaptation. In the state of art, different procedures of modifying the phase (including phase locking and other means) are known.

[0030] One embodiment for modifying the phase comprises to modify the phase such that the phase propagation between adjacent frames with the synthesis hop size Rs is identical to the phase propagation of the input signal for the analysis hop size Ra. This ensures that the horizontal phase coherence (i.e. the temporal evolution of the phases in each frequency bin) is maintained. This is achieved by computing the instantaneous frequency at time frame m given the input phase of the current and the preceding time frame and the analysis hop size Ra and the STFT (Short Time Fourier Transform) parameters. The desired phase propagation is computed using the instantaneous frequency and the synthesis hop size Rs.

[0031] In an additional embodiment, the foregoing method is enhanced by a method comprising "phase locking". Phase locking aims to improve vertical phase coherence, i.e. to maintain the relation of the phase between adjacent frequency bins in each frame. This improves the sound quality as is for example noticeable when processing music signals containing transients or percussive notes.

[0032] The phase adaptor is configured to adapt the phase of at least one combination of the channel signals comprised by the multichannel audio signal. For this, the signals to be processed by the phase adaptor are given with polar coordinates by a magnitude and a phase.

[0033] The separator following the phase adaptor provides separated signals based on the processed signal, i.e. based on the signal with a modified phase. The separator reverses the combination of signals and extracts or generates separated signals.

[0034] Hence, the apparatus modifies the phase of at least one combination of channel signals and provides individual modified signals by separating the phase adapted combination of channel signals into separated signals.

[0035] The inventive phase adaptation is most suitable as an extension to time scale modification using the phase vocoder, i.e. processing in the frequency domain. However, it can also be applied as an extension to TSM in the time domain. To this end, the time-domain TSM is configured to process the at least one signal that is obtained by combining the channel signals comprised by the multichannel audio signal. The separator following the time-domain TSM provides separated signals based on the processed signal.

[0036] In an embodiment, the phase adaptor is configured for providing N processed signals by modifying phases of N signals based on combinations of N channel signals. The separator is configured for providing N separated signals based on the processed signals. N is a number of channel signals comprised by the multichannel audio signal and is a integer greater than or equal to two. In an embodiment, N is the number of all channel signals comprised by the multichannel audio signal. Hence, N channel signals are converted into N combined signals that are - after the phase adaptation - separated into N signals.

[0037] According to an embodiment, the N combinations of the channel signals are linear combinations of the channel signals. In an embodiment, the phase adaptor handles various combinations of channel signals, whereas the combinations are linear combinations, e.g. the sum or the difference of channel signals.

[0038] In an embodiment, N equals two (N = 2) and the two combinations of the two channel signals are a sum and a difference signal. In a different embodiment, N is greater than two.

[0039] In a further embodiment, the apparatus comprises a transformer. The transformer is configured for providing transformed signals by transforming signals from the time domain into the frequency domain. The transformer allows to transform signals from the time into the frequency domain and, subsequently, to process these signals in the frequency domain. The signals to be transformed are in one embodiment the channel signals comprised by the multichannel audio signal.

[0040] According to an embodiment, the transformer is configured for providing transformed signals by applying a Short Time Fourier Transformation. The Short Time Fourier Transform or alternatively Short Term Fourier Transform (STFT), is a Fourier-related transform of local sections of a signal as it changes over time. In practice, a longer signal is divided into shorter segments of equal length and the Fourier transform is computed separately on each segment.

[0041] In a further embodiment, the apparatus comprises a combiner, wherein the combiner is configured for providing combined signals based on the channel signals. The transformer is configured for providing transformed signals based on the combined signals by applying a transformation. Here, the channel signals are combined and the combinations are transformed into the frequency domain.

[0042] In a different embodiment, the sequence of combiner and transformer is reversed. In this embodiment, the combiner comprised by the apparatus is configured for providing combined signals based on transformed signals provided by the transformer. In this embodiment, the combiner receives transformed signals from the transformer and combines them in order to provide combined signals. Thus, the channel signals are individually transformed into the frequency domain and are combined in this domain by the combiner.

[0043] In a further embodiment, the combiner is configured for providing a sum signal as a combined signal by calculating a sum of two transformed signals. In this embodiment, a linear combination of the channel signals via their transformed signals in the form of a summation is used to provide at least one combined signal. If the multichannel audio signal comprises, for example, two channel signals as stereo signals, then both channel signals are added to each other for obtaining one combined signal being a sum signal. In a different embodiment, the combiner calculates a sum of channel signals of at least two channel signals.

[0044] According to an embodiment, the combiner is configured for providing a difference signal as combined signal by calculating a difference between two transformed signals. In this embodiment, the linear combination of signals is a difference between two signals. The signals to be combined are here transformed signals. In a different embodiment, the combiner calculates a difference between two channel signals.

[0045] According to an embodiment, the combiner is configured for providing combined signals by applying a mixing matrix - called g - having dimensions N times N to a multichannel signal comprising the N transformed signals, according to = gX

[0046] N is a number of channel signals comprised by the multichannel audio signal and is the matriced multichannel signal based on the transformed signals and represents, thus, the combined signals in a general form.

[0047] With other words: the combiner is configured for providing the combined signals by applying a mixing matrix having dimensions N times N to the N transformed signals - preferable in a matriced format - wherein the transformed signals are based on the N channel signals belonging to the multichannel audio signal.

[0048] As the mixing matrix has the same number of columns and rows, the number of signals to be combined equals the number of combined signals, as shown in the following equation:



[0049] Applying the mixing matrix is equivalent to computing the k-th channel signal of the combined signal by summing all channel signals of the multichannel audio signal multiplied by the corresponding elements of the k-th row of the matrix, as shown in the following equation:



[0050] In a different embodiment, the defined mixing matrix is applied to a signal based on the N channel signals. The mixing matrix allows to combine any given number of signals, being either transformed or channel signals.

[0051] In a further embodiment, the phase adaptor is configured for modifying the phases by applying a phase vocoder method. A phase vocoder is a vocoder scaling both the frequency and time domains of audio signals by using phase information.

[0052] According to an embodiment, the phase adaptor is configured for providing processed signals by their polar coordinates having amplitudes and modified phases. Thus, in one embodiment with N = 2 channel signals Ys, Yd, the processed signals are given by: Y s/d = |Ys/d|exp(jΦs/d) with Φs/d = arg Ys/d and j = √-1.

[0053] The separator provides single signals based on the processed signals that are based on the combined signals. The signals provided by the separator are to be used as time scale modified channel signals. Hence, according to an embodiment if the multichannel audio signals comprises N channel signals, then the separator provides N separated signals.

[0054] According to an embodiment, the separator is configured for providing a separated signal based on a difference between two of the processed signals.

[0055] In a further embodiment, the separator is configured for providing a separated signal based on a sum of two of the processed signals.

[0056] In an embodiment, the separator applies a factor - for example 0,5 - to the amplitude of the respective combination of the processed signals.

[0057] According to an embodiment, the separator is configured for providing N separated signals by applying an inverse mixing matrix having dimensions N times N to a matriced signal based on the N processed signals. N is a number of channel signals comprised by the multichannel audio signal. This embodiment allows to process any given number of processed signals and to provide the corresponding number of separated signals. The inverse mixing matrix is the inverse of the mixing matrix used for obtaining the combined signals.

[0058] According to an embodiment, the apparatus comprises a corrector, wherein the corrector is configured to modify the separated signals by replacing amplitudes of the separated signals with amplitudes based on amplitudes of the channel signals comprised by the multichannel audio signal.

[0059] This embodiment comprises to process a linear combination of the channel signals and to apply a magnitude correction procedure after the signals have been modified in the frequency domain.

[0060] The novelty of the presented method is, thus, in this embodiment two-fold:
  1. 1) to apply the TSM processing to the preferably linear combinations, e.g. to the sum signal and the difference signal and
  2. 2) to apply a processing for restoring the magnitude of the output signal in the time-frequency domain in order to restore the inter-channel level differences (ICLD).


[0061] In a further embodiment, the corrector is configured to replace the amplitudes of the separated signals with amplitudes of the corresponding transformed signals provided by a transformer, i.e. by the amplitudes of the channel signals in the frequency domain.

[0062] The corrector, thus, replaces the amplitudes of the separated signals having the adapted phase by the amplitudes of the corresponding signals before the adaptation. Hence, the original amplitudes are restored.

[0063] The following embodiments take care of signals in the frequency domain and allow to process them in the time domain.

[0064] In an embodiment, the apparatus comprises an inverse transformer. The inverse transformer is configured for providing modified channel signals based on the separated signals by applying an inverse transformation. This inverse transformation transforms in an embodiment signals from the frequency into the time domain.

[0065] According to another embodiment, the apparatus comprises an inverse transformer. The inverse transformer is configured for providing modified and corrected channel signals based on corrected signals provided by the corrector by applying an inverse transformation.

[0066] According to a different embodiment, the inverse transformer is configured for applying an inverse Short Time Fourier Transformation.

[0067] The inverse transformer is, thus, configured to reverse the kind of transformation performed in a step preceding the phase adaptation.

[0068] According to an embodiment, the apparatus comprises an extractor, wherein the extractor is configured for providing channel signals comprised by the multichannel audio signal. In this embodiment, the apparatus e.g. receives the multichannel audio signal and the extractor provides the individual channel signals. In a different embodiment, the channel signals are separately submitted to the apparatus.

[0069] The following embodiment allows to facilitate the computational steps and the requirements for the units.

[0070] According to an embodiment, the apparatus is configured to perform steps on combinations of channel signals based on a difference with less precision than on different combinations of the channel signals.

[0071] The object is also achieved by a method for processing a multichannel audio signal.

[0072] The method comprises at least the following steps:

providing at least one combined signal based on channel signals comprised by the multichannel audio signal,

providing a processed signal by performing a time scale modulation of the combined signal, and

providing modified channel signals based on a separation of the processed signals.



[0073] The time scale modulation is in one embodiment performed by a phase adaptation.

[0074] According to an embodiment, the method further comprises modifying amplitudes of modified channel signals by replacing the amplitudes with amplitudes based on amplitudes of the corresponding channel signals.

[0075] In this embodiment, the following steps happen: The channel signals are combined into combined signals. The combined signals or signals based on the combined signals undergo a phase adaptation in order to perform the time scale modification. The phase adapted signals are separated in separate signals. The signals comprise phases and amplitudes. The amplitudes/magnitudes of these signals are replaced by amplitudes based on the channel signals.

[0076] The embodiments of the apparatus can also be performed by steps of the method and corresponding embodiments of the method. Therefore, the explanations given for the embodiments of the apparatus also hold for the method.

[0077] The object is also achieved by a computer program for performing, when running on a computer or a processor, the method of any of the preceding embodiments.

[0078] The invention will be explained in the following with regard to the accompanying drawings and the embodiments depicted in the accompanying drawings, in which:
Fig. 1
shows a block diagram of a time scale modification method in the frequency domain according to the state of art,
Fig. 2
illustrates a block diagram of the inventive time scale modification method for input audio signals with two channels,
Fig. 3
provides schematically an embodiment of the apparatus,
Fig. 4
provides a different embodiment of the apparatus and
Fig. 5
illustrates a general version of the embodiment of Fig. 4.


[0079] A block diagram of an embodiment of the inventive method is depicted in Fig. 2 for the example of input signals having two channels (e.g. the left and right channel of stereo sound).

[0080] The input audio signal is x = [x1 x2]T, where x1 denotes the first channel signal and x2 denotes the second channel signal. A short-time Fourier transform (STFT) representation is computed for x1 and x2, yielding X1 and X2, respectively, in the step STFT.

[0081] According to the invention, a sum signal Xs and a difference signal Xd are computed in the step named MS from the channel signals - here: the transformed channel signals X1 and X2 - of the multichannel audio signal according to





[0082] In a different embodiment, the order of STFT and the combination of the signals in step MS are switched. This takes into consideration, that computing the sum and difference signals can also be performed in the time domain and that the STFT can be computed from the time domain sum and difference signals. It is however advantageous to apply the order as shown in Fig. 2 to reduce the computational load.

[0083] The sum signal Xs and the difference signal Xd are then processed by means of a phase adaptation (PA) method, e.g. by using the phase vocoder method described in [2], or any other time scale modification method. The processed sum signal and difference signal are denoted by Ys and Yd, respectively.

[0084] This is followed by an inverse transformation in step inverseMS, e.g. by transforming the signals Ys and Yd by an inverse short-time Fourier transform.

[0085] The processed signals for the first and the second channel are obtained in the shown embodiment by applying the following Equations (8) and (9), i.e. the inverse processing of Equations (6) and (7).

[0086] Hence, the two separated signals Y1 and Y2 are given in the shown embodiment by:





[0087] In a general form, the computation of the sum signal and the difference signal (as examples of the combined signals) are expressed in matrix notation as

with STFT coefficients of the input signal X = [X1 ... XN]T, a number of channels N, STFT coefficients of the matriced signal = [1... N]T, and a mixing matrix g of size N times N.

[0088] For example, the sum signal Xs and the difference signal Xd - as given by Equations (6) and (7) - are obtained for N = 2 by setting

and = [XsXd]T.

[0089] The inverse processing of the matrix operation, i.e. computing the channel signals Y = [Y1 .. YN]T from the matriced signal = [1 ... N]T is obtained from

where is the result of the PA processing applied to and g-1 is the inverse of matrix g. With this generalization, the proposed method can also be applied to signals having more than two channels.

[0090] A very advantageous step of the shown embodiment is to correct the magnitude of the spectral coefficients such that the resulting complex-valued spectral coefficients have the phase of the result of the separation step using Equations (8) and (9) and the magnitude of X1 and X2 of the transformed channel signals.

[0091] The separated signals are given in polar coordinates as:

with Φy,1/2 = arg Y1/2 and j = √-1.

[0092] Hence, for the shown two-channel input signals, the corrected signals Z1 and Z2 following the step MC are given by:





[0093] This step ensures that the inter-channel level differences (ICLD) of the audio signal is preserved.

[0094] Hence, in this step the magnitudes of the processed signals are substituted by the original amplitudes of the transformed channel signals.

[0095] For the general case of input signals having more than two channels, each output signal after the inverse matrix operation - i.e. after the separation in separated signals following the individual phase adaptation - is modified in an embodiment such that its magnitude is replaced by the magnitude of the corresponding - preferably transformed - channel signal (before matricing, i.e. before computing the combinations of different channel signals).

[0096] The PA processing has a considerable computational complexity. It involves various processing steps that can be implemented with reduced precision in order to reduce the computational load.

[0097] For example, the computation of the polar coordinates (magnitude and phase) of a complex number given by its Cartesian coordinates (real and imaginary component) can be implemented with lower or higher precision. Often, computations at lower precision have lower computational costs but introduce an error.

[0098] The computational load can be reduced by taking advantage of the fact that for typical audio signals (e.g. musical recordings or broadcast signals) the sum signal has higher energy then the difference signal. Errors that result from approximations in the computation have a smaller effect when they are introduced in the difference signal and greater effect when they occur in the sum signal. This can be exploited by applying methods with smaller precision for computing the difference signal Yd and methods with larger precision when computing the sum signal Ys.

[0099] Another means for reducing the computational load is to skip certain processing steps. For example, the phase locking can be skipped when computing the difference signal Yd. Phase locking refers to an additional processing step for improving the sound quality. For reducing the computational load, the phases for the difference signal Yd are computed such that the phase propagation is maintained without applying the phase locking processing.

[0100] Another means for reducing the computational load is to apply the processing with high-quality (large precision including all processing steps like phase locking) only up to a maximum frequency value. For a digital signal sampled at 48 kHz, for example, the high quality processing is only applied to frequency bands up to a maximum value of 10 kHz. The maximum frequency up to which the processing is applied with the best possible quality can be further reduced for the computation of the difference signal.

[0101] Fig. 3 shows an embodiment of the apparatus 1 which performs a time-scale modification of a multichannel audio signal 100. The multichannel audio signal 100 may comprise more than two channel signals. The time-scale modification is achieved by a phase adaption.

[0102] An extractor 2 retrieves the channel signals x1, x2 comprised by the multichannel audio signal 100. In the shown embodiment, there are just two channel signals. Nevertheless, the invention is not limited to two channel signals.

[0103] In a different - and not shown - embodiment, the channel signals x1, x2 are separately provided to the apparatus 1. Hence, for such an embodiment no extractor is required.

[0104] The channel signals x1, x2 are signals in the time domain and are, in the shown embodiment, submitted to a transformer 3. The transformer 3 transforms the channel signals x1, x2 into the frequency domain and, thus, provides transformed signals X1, X2.

[0105] The transformed signals X1, X2 are submitted to a combiner 4. The combiner 4 combines the transformed signals X1, X2 - in the given example - by applying linear combinations of them, e.g. by calculating a sum Xs of the transformed signals and a difference Xd between them. The number of combined signals is in one embodiment identical to the number of channel signals.

[0106] In a different - not shown - embodiment, the sequence of the transformer 3 and the combiner 4 is reversed. This implies that the combiner 4 combines the channel signals and the transformer 3 transforms the combined signals in this different and not shown embodiment.

[0107] In the depicted embodiment, the combined signals - being combined transformed channel signals - Xs and Xd are submitted to the phase adaptor 5.

[0108] The phase adaptor 5 modifies the phases of the combined signals Xs and Xd and provides processed signals Ys and Yd. The processed signals Ys and Yd have adapted phases reflecting the appropriate time scale modification of the combined signals Xs and Xd. Hence, the combined signals are either slowed down or sped up.

[0109] In order to obtain time scale modified channel signals, the processed signals Ys and Yd are separated by the separator 6 providing separated signals Y1 and Y2 that undergo an inverse transformation by an inverse transformer 7. The resulting modified channel signals y1 and y2 are time signals and have the desired time scale.

[0110] Fig. 4 shows a different embodiment of the apparatus 1.

[0111] The structure of the apparatus 1 shown in Fig. 4 is similar to the embodiment shown in Fig. 3. The difference between both embodiments is given by the units following the separator 6. For the explanation of the other elements see the description of Fig. 3.

[0112] The separator 6 in the embodiment shown in Fig. 4 also provides the separated signals Y1 and Y2. These frequency domain signals Y1 and Y2 are submitted in this embodiment to a corrector 8 that is upstream to the inverse transformer 7.

[0113] The corrector 8 replaces the amplitudes of the separated signals Y1 and Y2 by the amplitudes of the corresponding transformed signals X1 and X2, i.e. with the amplitudes or magnitudes before the phase adaptation and especially before the combination of the channel signals.

[0114] The resulting corrected or amplitude modified signals Z1 and Z2 (compare equations (12) and (13)) are submitted to the inverse transformer 7 and are transformed into the time domain as modified and corrected channel signals z1, z2.

[0115] In order to enable the correction, the transformer 3 is connected with the corrector 8.

[0116] In a further, not shown embodiment, the sequence of the transformer 3 and the combiner 4 are switched and the transformer 3, thus, transforms the combined signals. For the correction of the separated signals Y1, Y2 the corrector 8 refers therefore to additional transformations of the corresponding channel signals x1, x2.

[0117] The embodiment of Fig. 5 is a generalized version of the one shown in Fig. 4.

[0118] Here, the multichannel audio signal 100 comprises N channel signals x1, x2, ... xN where N is a integer greater than two.

[0119] The channel signals x1, x2, ... xN being time signals retrieved by the extractor 2 are submitted to the transformer 3 providing the transformed signals in the frequency domain X1, X2,... XN which are here given by a vector X (alternatively given by X).

[0120] The following combiner 4 provides a vector of linear combinations = [1 ... N]T by applying equation (10).

[0121] It follows the phase adaptor 5 providing processed signals (here given by a vector: ) that are separated by the separator 6. The separated signals Y1, Y2, ... YN (here given by a vector Y) are corrected with regard to their amplitudes by the corrector 8. The corrected signals Z1, Z2, ... ZN (given here by a vector Z) are submitted to the inverse transformer 7 yielding N modified and corrected channel signals z1, z2, ... zN. The switch from upper case to lower case indicated the transformation from the frequency to the time domain.

[0122] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

[0123] The inventive transmitted or encoded signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0124] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0125] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0126] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0127] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0128] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0129] A further embodiment of the inventive method is, therefore, a data carrier (or a non-transitory storage medium such as a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.

[0130] A further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

[0131] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

[0132] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0133] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0134] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0135] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

References



[0136] 
  1. [1] M. Dolson, "The Phase Vocoder: A Tutorial", Computer Music Journal, vol. 10, pp. 14 - 27, 1986.
  2. [2] J. Laroche and M. Dolson, "Improved Phase Vocoder Time-Scale Modification of Audio", IEEE Transaction on Speech and Audio Processing, vol. 7, no. 3, pp. 323 - 332, 1999.
  3. [3] T. Karrer, E. Lee, and J. Borchers, "PhaVoRIT: A Phase Vocoder for Real-Time Interactive Time-Stretching," in Proc. of ICMC, 2006.



Claims

1. Apparatus (1) for processing a multichannel audio signal (100) comprising a plurality of channel signals (x1, x2), wherein the apparatus (1) is configured for performing a time scale modification of the multichannel audio signal (100), and wherein the apparatus (1) comprises:

a combiner (4) for providing combined signals (Xs, Xd) based on the channel signals (x1, x2), wherein the combiner (4) is configured for providing a sum signal (Xs) by calculating a sum of two channel signals (x1, x2) and for providing a difference signal (Xd) by calculating a difference of said two channel signals (x1, x2);

a phase adaptor (5), wherein the phase adaptor (5) is configured for providing at least one processed signal (Ys) to perform time scale modification by modifying a phase of the sum signal (Xs), and optionally a processed difference signal (Yd) by modifying a phase of the difference signal, wherein the phase adaptor (5) is configured for modifying the phase by applying a phase vocoder method, wherein the phase adaptor (5) is configured for modifying the phase by applying a phase locking, or wherein the phase adaptor (5) is configured for modifying the phase so that a horizontal phase coherence is maintained, and

a separator (6), wherein the separator (6) is configured for providing separated signals (Y1, Y2) based on the at least one processed signal (Ys) and the difference signal (Xd) or the optionally processed difference signal (Yd), wherein the separator (6) is configured for reversing the combination of the channel signals, applied by the combiner (4).


 
2. Apparatus (1) of claim 1, wherein the phase adaptor (5) is configured for providing N processed signals (Ys, Yd) by modifying phases of N signals (Xs, Xd) based on combinations of N channel signals (x1, x2),
wherein the separator (6) is configured for providing N separated signals (Y1, Y2) based on the processed signal (Ys, Yd), and
wherein N is a number of channel signals (x1, x2) comprised by the multichannel audio signal (100).
 
3. Apparatus (1) of claim 2, wherein the N combinations of the channel signals (x1, x2) are linear combinations of the channel signals (x1, x2).
 
4. Apparatus (1) of any one of claims 1 to 3, wherein the apparatus (1) comprises a transformer (3), and
wherein the transformer (3) is configured for providing transformed signals (X1, X2) by transforming signals from the time domain into the frequency domain.
 
5. Apparatus (1) of claim 4, wherein the transformer (3) is configured for applying a Short Time Fourier Transformation.
 
6. Apparatus (1) of any one of claims 4 or 5,
wherein the combiner (4) is configured for providing combined signals (Xs, Xd) based on the channel signals (x1, x2), and
wherein the transformer (3) is configured for providing transformed signals based on the combined signals by applying a transformation.
 
7. Apparatus (1) of claim 4 or 5,
wherein the combiner (4) is configured for providing combined signals (Xs, Xd) based on transformed signals (X1, X2) provided by the transformer (3).
 
8. Apparatus (1) of claim 7, wherein the combiner (4) is configured for providing a sum signal (Xs) by calculating a sum of two transformed signals (X1, X2).
 
9. Apparatus (1) of claim 7 or 8, wherein the combiner (4) is configured for providing a difference signal (Xd) by calculating a difference between two transformed signals (X1, X2).
 
10. Apparatus (1) of claim 7, wherein the combiner (4) is configured for providing the combined signals (Xs, Xd) by applying a mixing matrix (g) having dimensions N times N to the N transformed signals (X1, X2) based on the N channel signals (x1, x2) belonging to the multichannel audio signal (100), and
wherein N is a number of channel signals (x1, x2) comprised by the multichannel audio signal (100).
 
11. Apparatus (1) of any one of claims 4 or 5, wherein the transformer (3) has an analysis hop size Ra,
wherein the apparatus further comprises an inverse transformer (7), wherein the inverse transformer (7) has a synthesis hop size Rs, and
wherein the synthesis hop size Rs and the analysis hop size Ra are set to different values, so that the time scale modification is a signal stretching in time, when Ra < Rs, or so that the time scale modification and is a signal shrinking in time, when Ra > Rs.
 
12. Apparatus (1) of any one of claims 2 to 11, wherein the separator (6) is configured for providing a separated signal (Y1, Y2) based on a difference between two of the processed signals (Ys, Yd).
 
13. Apparatus (1) of any one of claims 2 to 12, wherein the separator (6) is configured for providing a separated signal (Y1, Y2) based on a sum of two of the processed signals (Ys, Yd).
 
14. Apparatus (1) of any one of claims 2 to 13, wherein the separator (6) is configured for providing N separated signals (Y1, Y2) by applying an inverse mixing matrix (g-1) having dimensions N times N to a matriced signal based on the N processed signals (Ys, Yd), and
wherein N is a number of channel signals (x1, x2) comprised by the multichannel audio signal (100).
 
15. Apparatus (1) of any one of claims 1 to 14, wherein the apparatus (1) comprises a corrector (8), and
wherein the corrector (8) is configured to modify the separated signals (Y1, Y2) by replacing amplitudes of the separated signals (Y1, Y2) with amplitudes based on amplitudes of the channel signals (x1, x2).
 
16. Apparatus (1) of claim 15, wherein the corrector (8) is configured to replace the amplitudes of the separated signals (Y1, Y2) with amplitudes of transformed signals (X1, X2) provided by a transformer (3).
 
17. Apparatus (1) of any one of claims 1 to 16, wherein the apparatus (1) comprises an inverse transformer (7), and
wherein the inverse transformer (7) is configured for providing modified channel signals (y1, y2) based on the separated signals (Y1, Y2) by applying an inverse transformation.
 
18. Apparatus (1) of claim 15 or 16, wherein the apparatus (1) comprises an inverse transformer (7), and
wherein the inverse transformer (7) is configured for providing modified and corrected channel signals (z1, z2) based on corrected signals (Z1, Z2) provided by the corrector (8) by applying an inverse transformation.
 
19. Apparatus (1) of claim 17 or 18, wherein the inverse transformer (7) is configured for applying an inverse Short Time Fourier Transformation.
 
20. Apparatus (1) of any one of claims 1 to 19, wherein the apparatus (1) comprises an extractor (2), and
wherein the extractor (2) is configured for providing channel signals (x1, x2) comprised by the multichannel audio signal (100).
 
21. Apparatus (1) of any one of claims 1 to 20, wherein the apparatus (1) is configured to perform steps on combinations of channel signals (x1, x2) based on a difference between signals with less precision than on different combinations of signals.
 
22. Method for processing a multichannel audio signal (100) comprising a plurality of channel signals (x1, x2), wherein the method is configured for performing a time scale modification of the multichannel audio signal (100), the method comprising:

providing combined signals (Xs, Xd) based on the channel signals (x1, x2), wherein the combined signals comprise a sum signal (Xs) calculated from a sum of two channel signals (x1, x2) and a difference signal (Xd) calculated from a difference of said two channel signals (x1, x2),

providing at least one processed signal (Ys) to perform time scale modification by modifying a phase of the sum signal (Xs), and optionally a processed difference signal (Yd) by modifying a phase of the difference signal, wherein the providing the processed signal comprises modifying the phase by applying a phase vocoder method, or modifying the phase by applying a phase locking, or modifying the phase so that a horizontal phase coherence is maintained, and

providing separated signals (Y1, Y2) based on the at least one processed signal (Ys) and the difference signal (Xd) or the optionally processed difference signal (Yd), wherein

providing the separated signals (Y1, Y2) comprises reversing the combination of the channel signals, applied by the providing the combined signals (Xs, Xd).


 
23. Method of claim 22, further comprising:
modifying amplitudes of modified channel signals by replacing the amplitudes of modified channel signals with amplitudes based on amplitudes of corresponding channel signals.
 
24. Computer program for performing, when running on a computer or a processor, the method of claim 22 or 23.
 


Ansprüche

1. Vorrichtung (1) zum Verarbeiten eines Mehrkanalaudiosignals (100), das eine Mehrzahl von Kanalsignalen (x1, x2) aufweist, wobei die Vorrichtung (1) dazu konfiguriert ist, eine Zeitskalenmodifikation des Mehrkanalaudiosignals (100) durchzuführen, und wobei die Vorrichtung (1) folgende Merkmale aufweist:

einen Kombinierer (4) zum Bereitstellen kombinierter Signale (Xs, Xd) basierend auf den Kanalsignalen (x1, x2), wobei der Kombinierer (4) dazu konfiguriert ist, durch Berechnen einer Summe von zwei Kanalsignalen (x1, x2) ein Summensignal (Xs) bereitzustellen, und durch Berechnen einer Differenz der zwei Kanalsignale (x1, x2) ein Differenzsignal (Xd) bereitzustellen;

einen Phasenadapter (5), wobei der Phasenadapter (5) dazu konfiguriert ist, zumindest ein verarbeitetes Signal (Ys), um durch Modifizieren einer Phase des Summensignals (Xs) eine Zeitskalenmodifikation durchzuführen, und optional durch Modifizieren einer Phase des Differenzsignals ein verarbeitetes Differenzsignal (Yd) bereitzustellen, wobei der Phasenadapter (5) dazu konfiguriert ist, die Phase durch Anwenden eines Phasenvocoderverfahrens zu modifizieren, wobei der Phasenadapter (5) dazu konfiguriert ist, die Phase durch Anwenden einer Phasenverriegelung zu modifizieren, oder wobei der Phasenadapter (5) dazu konfiguriert ist, die Phase zu modifizieren, so dass eine horizontale Phasenkohärenz beibehalten wird und

einen Separator (6), wobei der Separator (6) dazu konfiguriert ist, basierend auf dem zumindest einen verarbeiteten Signal (Ys) und dem Differenzsignal (Xd) oder dem optional verarbeiteten Differenzsignal (Yd) getrennte Signale (Y1, Y2) bereitzustellen, wobei der Separator (6) dazu konfiguriert ist, die Kombination der Kanalsignale, die durch den Kombinierer (4) angewendet wird, umzukehren.


 
2. Vorrichtung (1) gemäß Anspruch 1, bei der der Phasenadapter (5) dazu konfiguriert ist, durch Modifizieren von Phasen von N Signalen (Xs, Xd) basierend auf Kombinationen von N Kanalsignalen (x1, x2) N verarbeitete Signale (Ys, Yd) bereitzustellen, wobei der Separator (6) dazu konfiguriert ist, basierend auf dem verarbeiteten Signal (Ys, Yd) N getrennte Signale (Y1, Y2) bereitzustellen und
wobei N eine Anzahl von Kanalsignalen (x1, x2) ist, die in dem Multikanalaudiosignal (100) enthalten ist.
 
3. Vorrichtung (1) gemäß Anspruch 2, bei der die N Kombinationen der Kanalsignale (x1, x2) lineare Kombinationen der Kanalsignale (x1, x2) sind.
 
4. Vorrichtung (1) gemäß einem der Ansprüche 1 bis 3, wobei die Vorrichtung (1) eine Transformationseinrichtung (3) aufweist und
wobei die Transformationseinrichtung (3) dazu konfiguriert ist, durch Transformieren von Signalen von dem Zeitbereich in den Frequenzbereich transformierte Signale (X1, X2) bereitzustellen.
 
5. Vorrichtung (1) gemäß Anspruch 4, bei der die Transformationseinrichtung (3) dazu konfiguriert ist, eine Kurzzeit-Fourier-Transformation anzuwenden.
 
6. Vorrichtung (1) gemäß einem der Ansprüche 4 oder 5,
bei der der Kombinierer (4) dazu konfiguriert ist, basierend auf den Kanalsignalen (x1, x2) kombinierte Signale (Xs, Xd) bereitzustellen und
wobei die Transformationseinrichtung (3) dazu konfiguriert ist, durch Anwenden einer Transformation basierend auf den kombinierten Signalen transformierte Signale bereitzustellen.
 
7. Vorrichtung (1) gemäß Anspruch 4 oder 5,
bei der der Kombinierer (4) dazu konfiguriert ist, basierend auf den transformierten Signalen (X1, X2), die durch die Transformationseinrichtung (3) bereitgestellt werden, kombinierte Signale (Xs, Xd) bereitzustellen.
 
8. Vorrichtung (1) gemäß Anspruch 7, bei der der Kombinierer (4) dazu konfiguriert ist, durch Berechnen einer Summe von zwei transformierten Signalen (X1, X2) ein Summensignal (Xs) bereitzustellen.
 
9. Vorrichtung (1) gemäß Anspruch 7 oder 8, bei der der Kombinierer (4) dazu konfiguriert ist, durch Berechnen einer Differenz zwischen zwei transformierten Signalen (X1, X2) ein Differenzsignal (Xd) bereitzustellen.
 
10. Vorrichtung (1) gemäß Anspruch 7, bei der der Kombinierer (4) dazu konfiguriert ist, durch Anwenden einer Mischmatrix (g) mit den Abmessungen N mal N an die N transformierten Signale (X1, X2), basierend auf den N Kanalsignalen, die zu dem Mehrkanalaudiosignal (100) gehören, die kombinierten Signale (Xs, Xd) bereitzustellen und
wobei N eine Anzahl von Kanalsignalen (x1, x2) ist, die in dem Mehrkanalaudiosignal (100) enthalten ist.
 
11. Vorrichtung (1) gemäß einem der Ansprüche 4 oder 5, bei der die Transformationseinrichtung (3) eine Analyse-Sprunggröße Ra aufweist,
wobei die Vorrichtung ferner eine inverse Transformationseinrichtung (7) aufweist, wobei die inverse Transformationseinrichtung (7) eine Synthese-Sprunggröße Rs aufweist und
wobei die Synthese-Sprunggröße Rs und die Analyse-Sprunggröße Ra auf unterschiedliche Werte eingestellt sind, so dass die Zeitskalenmodifikation ein Signal ist, das sich in der Zeit dehnt, wenn Ra < Rs, oder so, dass die Zeitskalenmodifikation ein Signal ist, das in der Zeit schrumpft, wenn Ra > Rs.
 
12. Vorrichtung (1) gemäß einem der Ansprüche 2 bis 11, bei der der Separator (6) dazu konfiguriert ist, basierend auf einer Differenz zwischen zwei der verarbeiteten Signale (Ys, Yd) ein getrenntes Signal (Y1, Y2) bereitzustellen.
 
13. Vorrichtung (1) gemäß einem der Ansprüche 2 bis 12, bei der der Separator (6) dazu konfiguriert ist, basierend auf einer Summe von zwei der verarbeiteten Signale (Ys, Yd) ein getrenntes Signal (Y1, Y2) bereitzustellen.
 
14. Vorrichtung (1) gemäß einem der Ansprüche 2 bis 13, bei der der Separator (6) dazu konfiguriert ist, durch Anwenden einer inversen Mischmatrix g-1 mit den Abmessungen N mal N an ein matrixiertes Signal, basierend auf den N verarbeiteten Signalen (Ys, Yd), N getrennte Signale (Y1, Y2) bereitzustellen und
wobei N eine Anzahl von Kanalsignalen (x1, x2) ist, die in dem Mehrkanalaudiosignal (100) enthalten ist.
 
15. Vorrichtung (1) gemäß einem der Ansprüche 1 bis 14, wobei die Vorrichtung (1) einen Korrektor (8) aufweist und
wobei der Korrektor (8) dazu konfiguriert ist, die getrennten Signale (Y1, Y2) durch Ersetzen von Amplituden der getrennten Signale (Y1, Y2) mit Amplituden basierend auf Amplituden der Kanalsignale (x1, x2) zu modifizieren.
 
16. Vorrichtung (1) gemäß Anspruch 15,
bei der der Korrektor (8) dazu konfiguriert ist, die Amplituden der getrennten Signale (Y1, Y2) mit Amplituden von transformierten Signalen (X1, X2), die durch eine Transformationseinrichtung (3) bereitgestellt werden, zu ersetzen.
 
17. Vorrichtung (1) gemäß einem der Ansprüche 1 bis 16, wobei die Vorrichtung (1) eine inverse Transformationseinrichtung (7) aufweist und
wobei die inverse Transformationseinrichtung (7) dazu konfiguriert ist, basierend auf den getrennten Signalen (Y1, Y2) durch Anwenden einer inversen Transformation modifizierte Kanalsignale (y1, y2) bereitzustellen.
 
18. Vorrichtung (1) gemäß Anspruch 15 oder 16, wobei die Vorrichtung (1) eine inverse Transformationseinrichtung (7) aufweist und
wobei die inverse Transformationseinrichtung (7) dazu konfiguriert ist, basierend auf korrigierten Signalen (Z1, Z2), die durch den Korrektor (8) bereitgestellt werden, durch Anwenden einer inversen Transformation modifizierte und korrigierte Kanalsignale (z1, z2) bereitzustellen.
 
19. Vorrichtung (1) gemäß Anspruch 17 oder 18, bei der die inverse Transformationseinrichtung (7) dazu konfiguriert ist, eine inverse Kurzzeit-Fourier-Transformation anzuwenden.
 
20. Vorrichtung (1) gemäß einem der Ansprüche 1 bis 19, wobei die Vorrichtung (1) einen Extrahierer (2) aufweist und
wobei der Extrahierer (2) dazu konfiguriert ist, Kanalsignale (x1, x2) bereitzustellen, die in dem Mehrkanalaudiosignal (100) enthalten sind.
 
21. Vorrichtung (1) gemäß einem der Ansprüche 1 bis 20, wobei die Vorrichtung (1) dazu konfiguriert ist, Schritte an Kombinationen von Kanalsignalen (x1, x2) basierend auf einer Differenz zwischen Signalen mit geringerer Genauigkeit durchzuführen als an unterschiedlichen Kombinationen von Signalen.
 
22. Verfahren zum Verarbeiten eines Mehrkanalaudiosignals (100), das eine Mehrzahl von Kanalsignalen (x1, x2) aufweist, wobei das Verfahren dazu konfiguriert ist, eine Zeitskalenmodifikation des Mehrkanalaudiosignals (100) durchzuführen, wobei das Verfahren folgende Schritte aufweist:

Bereitstellen kombinierter Signale (Xs, Xd) basierend auf den Kanalsignalen (x1, x2), wobei die kombinierten Signale ein Summensignal (Xs), das von einer Summe von zwei Kanalsignalen (x1, x2) berechnet wird, und ein Differenzsignal (Xd) aufweisen, das von einer Differenz der zwei Kanalsignale (x1, x2) berechnet wird,

Bereitstellen zumindest eines verarbeiteten Signals (Ys), um durch Modifizieren einer Phase des Summensignals (Xs) eine Zeitskalenmodifikation durchzuführen, und optional eines verarbeiteten Differenzsignals (Yd) durch Modifizieren einer Phase des Differenzsignals, wobei das Bereitstellen des verarbeiteten Signals das Modifizieren der Phase durch Anwenden eines Phasenvocoderverfahrens, oder Modifizieren der Phase durch Anwenden einer Phasenverriegelung oder Modifizieren der Phase, so dass eine horizontale Phasenkohärenz beibehalten wird, aufweist und Bereitstellen getrennter Signale (Y1, Y2) basierend auf dem zumindest einen verarbeiteten Signal (Ys) und dem Differenzsignal (Xd) oder dem optional verarbeiteten Differenzsignal (Yd), wobei

Bereitstellen der getrennten Signale (Y1, Y2) ein Umkehren der Kombination der Kanalsignale umfasst, die durch das Bereitstellen der kombinierten Signale (Xs, Xd) angewendet wird.


 
23. Verfahren gemäß Anspruch 22, das ferner folgenden Schritt aufweist:
Modifizieren von Amplituden von modifizierten Kanalsignalen durch Ersetzen der Amplituden der modifizierten Kanalsignale mit Amplituden basierend auf Amplituden entsprechender Kanalsignale.
 
24. Computerprogramm zum Durchführen, wenn dasselbe auf einem Computer oder einem Prozessor läuft, des Verfahrens gemäß Anspruch 22 oder 23.
 


Revendications

1. Appareil (1) de traitement d'un signal audio multicanal (100) comprenant une pluralité de signaux de canal (x1, x2), dans lequel l'appareil (1) est configuré pour effectuer une modification d'échelle temporelle du signal audio multicanal (100), et dans lequel l'appareil (1) comprend:

un combineur (4) destiné à fournir des signaux combinés (Xs, Xd) sur base des signaux de canal (x1, x2), où le combineur (4) est configuré pour fournir un signal de somme (Xs) en calculant une somme de deux signaux de canal (x1, x2) et pour fournir un signal de différence (Xd) en calculant une différence desdits deux signaux de canal (x1, x2);

un adaptateur de phase (5), où l'adaptateur de phase (5) est configuré pour fournir au moins un signal traité (Ys) pour effectuer une modification d'échelle temporelle en modifiant une phase du signal de somme (Xs), et optionnellement un signal de différence traité (Yd) en modifiant une phase du signal de différence, où l'adaptateur de phase (5) est configuré pour modifier la phase en appliquant un procédé de vocodeur de phase, où l'adaptateur de phase (5) est configuré pour modifier la phase en appliquant un verrouillage de phase, ou bien où l'adaptateur de phase (5) est configuré pour modifier la phase de sorte que soit maintenue une cohérence de phase horizontale, et

un séparateur (6), où le séparateur (6) est configuré pour fournir des signaux séparés (Y1, Y2) sur base d'au moins un signal traité (Ys) et du signal de différence (Xd) ou du signal de différence optionnellement traité (Yd), où le séparateur (6) est configuré pour inverser la combinaison des signaux de canal, appliquée par le combineur (4).


 
2. Appareil (1) selon la revendication 1, dans lequel l'adaptateur de phase (5) est configuré pour fournir N signaux traités (Ys, Yd) en modifiant les phases de N signaux (Xs, Xd) sur base de combinaisons de N signaux de canal (x1, x2),
dans lequel le séparateur (6) est configuré pour fournir N signaux séparés (Y1, Y2) sur base du signal traité (Ys, Yd), et
où N est un nombre de signaux de canal (x1, x2) compris dans le signal audio multicanal (100).
 
3. Appareil (1) selon la revendication 2, dans lequel les N combinaisons des signaux de canal (x1, x2) sont des combinaisons linéaires des signaux de canal (x1, x2).
 
4. Appareil (1) selon l'une quelconque des revendications 1 à 3, dans lequel l'appareil (1) comprend un transformateur (3), et
dans lequel le transformateur (3) est configuré pour fournir des signaux transformés (X1, X2) en transformant les signaux du domaine temporel au domaine de la fréquence.
 
5. Appareil (1) selon la revendication 4, dans lequel le transformateur (3) est configuré pour appliquer une Transformation de Fourier de Courte Durée.
 
6. Appareil (1) selon l'une quelconque des revendications 4 ou 5,
dans lequel le combineur (4) est configuré pour fournir des signaux combinés (Xs, Xd) sur base des signaux de canal (x1, x2), et
dans lequel le transformateur (3) est configuré pour fournir des signaux transformés sur base des signaux combinés en appliquant une transformation.
 
7. Appareil (1) selon la revendication 4 ou 5,
dans lequel le combineur (4) est configuré pour fournir des signaux combinés (Xs, Xd) sur base des signaux transformés (X1, X2) fournis par le transformateur (3).
 
8. Appareil (1) selon la revendication 7, dans lequel le combineur (4) est configuré pour fournir un signal de somme (Xs) en calculant une somme de deux signaux transformés (X1, X2).
 
9. Appareil (1) selon la revendication 7 ou 8, dans lequel le combineur (4) est configuré pour fournir un signal de différence (Xd) en calculant une différence entre deux signaux transformés (X1, X2),
 
10. Appareil (1) selon la revendication 7, dans lequel le combineur (4) est configuré pour fournir les signaux combinés (Xs, Xd) en appliquant une matrice de mélange (g) présentant des dimensions de N fois N aux N signaux transformés (X1, X2) sur base des N signaux de canal (x1, x2) appartenant au signal audio multicanal (100), et
où N est un nombre de signaux de canal (x1, x2) compris dans le signal audio multicanal (100).
 
11. Appareil (1) selon l'une quelconque des revendications 4 ou 5, dans lequel le transformateur (3) présente une grandeur de saut d'analyse Ra,
dans lequel l'appareil comprend par ailleurs un transformateur inverse (7), dans lequel le transformateur inverse (7) présente une grandeur de saut de synthèse Rs, et
dans lequel la grandeur de saut de synthèse Rs et la grandeur de saut d'analyse Ra sont réglées à des valeurs différentes, de sorte que la modification d'échelle temporelle soit un signal s'allongeant dans le temps lorsque Ra < Rs, ou de sorte que la modification d'échelle temporelle soit un signal se contractant dans le temps lorsque Ra > Rs.
 
12. Appareil (1) selon l'une quelconque des revendications 2 à 11, dans lequel le séparateur (6) est configuré pour fournir un signal séparé (Y1, Y2) sur base d'une différence entre deux des signaux traités (Ys, Yd).
 
13. Appareil (1) selon l'une quelconque des revendications 2 à 12, dans lequel le séparateur (6) est configuré pour fournir un signal séparé (Y1, Y2) sur base d'une somme de deux des signaux traités (Ys, Yd).
 
14. Appareil (1) selon l'une quelconque des revendications 2 à 13, dans lequel le séparateur (6) est configuré pour fournir N signaux séparés (Y1, Y2) en appliquant une matrice de mélange inverse (g-1) présentant des dimensions de N fois N à un signal matriciel sur base des N signaux traités (Ys, Yd), et
où N est un nombre de signaux de canal (x1, x2) compris dans le signal audio multicanal (100).
 
15. Appareil (1) selon l'une quelconque des revendications 1 à 14, dans lequel l'appareil (1) comprend un correcteur (8), et
dans lequel le correcteur (8) est configuré pour modifier les signaux séparés (Y1, Y2) en remplaçant les amplitudes des signaux séparés (Y1, Y2) par des amplitudes basées sur les amplitudes des signaux de canal (x1, x2).
 
16. Appareil (1) selon la revendication 15,
dans lequel le correcteur (8) est configuré pour remplacer les amplitudes des signaux séparés (Y1, Y2) par des amplitudes des signaux transformés (X1, X2) fournis par un transformateur (3).
 
17. Appareil (1) selon l'une quelconque des revendications 1 à 16, dans lequel l'appareil (1) comprend un transformateur inverse (7), et
dans lequel le transformateur inverse (7) est configuré pour fournir des signaux de canal modifiés (y1, y2) sur base des signaux séparés (Y1, Y2) en appliquant une transformation inverse.
 
18. Appareil (1) selon la revendication 15 ou 16, dans lequel l'appareil (1) comprend un transformateur inverse (7), et
dans lequel le transformateur inverse (7) est configuré pour fournir des signaux de canal modifiés et corrigés (z1, z2) sur base des signaux corrigés (Z1, Z2) fournis par le correcteur (8) en appliquant une transformation inverse.
 
19. Appareil (1) selon la revendication 17 ou 18, dans lequel le transformateur inverse (7) est configuré pour appliquer une Transformation de Fourier de Courte Durée inverse.
 
20. Appareil (1) selon l'une quelconque des revendications 1 à 19, dans lequel l'appareil (1) comprend un extracteur (2), et
dans lequel l'extracteur (2) est configuré pour fournir les signaux de canal (x1, x2) compris dans le signal audio multicanal (100).
 
21. Appareil (1) selon l'une quelconque des revendications 1 à 20, dans lequel l'appareil (1) est configuré pour effectuer les étapes sur des combinaisons de signaux de canal (x1, x2) sur base d'une différence entre les signaux avec moins de précision que sur différentes combinaisons de signaux.
 
22. Procédé de traitement d'un signal audio multicanal (100) comprenant une pluralité de signaux de canal (x1, x2), dans lequel le procédé est configuré pour effectuer une modification d'échelle temporelle du signal audio multicanal (100), le procédé comprenant le fait de:

fournir des signaux combinés (Xs, Xd) sur base des signaux de canal (x1, x2), où les signaux combinés comprennent un signal de somme (Xs) calculé à partir d'une somme de deux signaux de canal (x1, x2) et un signal de différence (Xd) calculé à partir d'une différence desdits deux signaux de canal (x1, x2),

fournir au moins un signal traité (Ys) pour effectuer une modification d'échelle temporelle en modifiant une phase du signal de somme (Xs), et optionnellement un signal de différence traité (Yd) en modifiant une phase du signal de différence, où fournir le signal traité comprend le fait de modifier la phase en appliquant un procédé de vocodeur de phase, ou de modifier la phase en appliquant un verrouillage de phase, ou de modifier la phase de sorte que soit maintenue une cohérence de phase horizontale, et

fournir des signaux séparés (Y1, Y2) sur base de l'au moins un signal traité (Ys) et du signal de différence (Xd) ou du signal de différence optionnellement traité (Yd),
dans lequel

fournir des signaux séparés (Y1, Y2) comprend inverser la combinaison des signaux de canal, appliquée par fournir des signaux combinés (Xs, Xd).


 
23. Procédé selon la revendication 22, comprenant par ailleurs le fait de:
modifier les amplitudes des signaux de canal modifiés en remplaçant les amplitudes des signaux de canal modifiés par des amplitudes basées sur les amplitudes des signaux de canal correspondants.
 
24. Programme d'ordinateur pour réaliser, lorsqu'il est exécuté sur un ordinateur ou un processeur, le procédé selon la revendication 22 ou 23.
 




Drawing




















Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description




Non-patent literature cited in the description