The present application is based on and claims priority to Japanese patent application No. 2025-066855, filed on April 15, 2025, with the Japanese Patent Office.

The present disclosure relates to an acoustic signal processing apparatus.

Recently, a technique called immersive sound or immersive audio has attracted attention. Immersive sound is an acoustic technique that can give a sense of presence and immersion as if a listener is in a place with a sound source by creating three-dimensional (3D) sound, 3D surround sound, or the like from which sound is heard from all directions of the listener. While conventional surround sound expands a sound field in a planar manner, immersive sound differs in that the immersive sound can expand a sound field in a three-dimensional manner.

In the immersive sound, the trend is to use multi-channel surround sound such as 7.1.4 channels, but most of the existing sound sources are sound sources having two channels or a relatively small number of channels. Therefore, there is a need to achieve an acoustic effect that exceeds the output sound of a limited number of channels by using existing sound sources with the limited number of channels. Some techniques are known for audio signal processing to impart an acoustic effect (see, for example, Patent Documents 1, 2, and 3).

The audio processing apparatus disclosed in Patent Document 1 determines a converted spatial audio signal from an input spatial audio signal. The input spatial audio signal has an input audio representation and an input direction of arrival, and the converted spatial audio signal has an omnidirectional audio component and at least one direction audio component. The audio processing apparatus estimates a wave representation including a wave field quantity and a wave direction-of-arrival quantity based on the input audio representation and the input direction of arrival, and processes the wave field quantity and the wave direction-of-arrival quantity to obtain the omnidirectional audio component and at least one direction component.

The audio processing apparatus disclosed in Patent Document 2 generates an enhanced downmix signal based on a multi-channel microphone signal. The audio processing apparatus includes a spatial analyzer, a filter calculator, and a filter. The spatial analyzer calculates, based on a multi-channel microphone signal, a set of spatial cue parameters including directional information describing the direction of arrival of direct sound, direct sound power information, and diffuse sound power information. The filter calculator is for calculating enhanced filter parameters depending on: the directional information describing the direction of arrival of direct sound, the direct sound power information, and the diffuse sound power information. The filter is for filtering a microphone signal or a signal derived from the microphone signal using the enhanced filter parameters to obtain the enhanced downmix signal.

The acoustic processing system disclosed in Patent Document 3 divides a stereo signal into stereo time-frequency signal segments corresponding to frequency domain samples in a given time segment, determines a similarity index indicating the degree of similarity of the stereo time-frequency signal segments for each stereo time-frequency signal segment pair, and decomposes the time-frequency signal segments by generating a sum segment, a center segment, and a side stereo segment pair using the similarity index. Then, a multi-channel signal having a center signal generated from the sum segment and a side signal generated from the side stereo segments is generated.

The technique disclosed in Patent Document 1 relates to a technique of resynthesizing sound recorded in a specific format called directional audio coding into components for each direction, which is applicable only to the specific format and is difficult to apply to general two-channel stereo audio signals.

The technique disclosed in Patent Document 2 converts a multi-channel microphone signal into an enhanced downmix signal but does not convert an audio signal having two channels or a relatively small number of channels into multiple channels.

The technique disclosed in Patent Document 3 generates a multi-channel signal from a stereo signal, but it is necessary to separate the stereo signal into a plurality of spatial segments by calculating a parameter called a similarity index.

• Patent Document 1: JP 2011-530915A
• Patent Document 2: JP 2013-520691A
• Patent Document 3: JP 2013-527727A

An object of the present disclosure is to enable realization of an acoustic effect that provides a listener with spatial perception exceeding that of conventional surround sound by using an audio signal with a small number of channels.

The present disclosure relates to an acoustic signal processing apparatus according to the appended claims. Embodiments are disclosed in the dependent claims. An acoustic signal processing apparatus according to an aspect includes a spatial separation processing circuit configured to generate, from input audio signals of a plurality of channels, spatial component audio signals for each of a plurality of spatial components corresponding to a plurality of virtual sound sources set at different spatial positions; and an integration processing circuit configured to add, to the input audio signals of the plurality of channels, the spatial component audio signals of the plurality of channels generated by the spatial separation processing circuit for each of the plurality of spatial components.

• FIG. 1 is a diagram showing a configuration example of an acoustic signal processing apparatus according to a first embodiment.
• FIG. 2 is a diagram showing a configuration example of a spatial separation processing unit according to the first embodiment.
• FIG. 3 is a diagram showing a configuration example of a filter coefficient calculation circuit according to the first embodiment.
• FIG. 4 is a diagram showing a configuration example of the acoustic signal processing apparatus in a modification.
• FIG. 5 is a diagram showing a configuration example of the acoustic signal processing apparatus according to a second embodiment.
• FIG. 6 is a diagram showing a configuration example of the spatial separation processing unit according to a second embodiment.
• FIG. 7 is a diagram showing a general arrangement position of surround speakers.
• FIG. 8A is a diagram showing a configuration example of the filter coefficient calculation circuit according to the second embodiment.
• FIG. 8B is a diagram showing a configuration example of the filter coefficient calculation circuit according to the second embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of an acoustic signal processing apparatus 10 according to a first embodiment. As shown in FIG. 1, the acoustic signal processing apparatus 10 according to the first embodiment includes a spatial separation processing unit 1, an integration processing unit 2, a delay circuit 3, and correction processing units 4 and 5.

The spatial separation processing unit 1 generates spatial component audio signals for a plurality of spatial components corresponding to a plurality of virtual sound sources set at different spatial locations, from input audio signals of a plurality of channels. In the first embodiment, input audio signals of a plurality of channels are two-channel stereo audio signals consisting of Lch (left channel) and Rch (right channel). In the following description, the term "two channels " means Lch and Rch. In the first embodiment, the plurality of spatial component audio signals corresponding to a plurality of virtual sound sources are N spatial component audio signals for Lch (N is an any integer of two or greater) and N spatial component audio signals for Rch. The plurality of virtual sound sources will be described later. The spatial component audio signals will be described below with reference to FIG. 2.

FIG. 2 is a diagram showing a configuration example of the spatial separation processing unit 1. In FIG. 2, only the configuration of a portion for generating a first spatial component audio signal is shown, and the configuration of a portion for generating the second to N-th spatial component audio signals is simplified. The configurations for generating the first to N-th spatial component audio signals are the same except that the filter coefficients to be set are different.

As shown in FIG. 2, the spatial separation processing unit 1 generates the first to N-th spatial component audio signals (also referred to as spatial component 1 to N audio signal) of two channels from the input audio signals of two channels. In the configuration for generating the first spatial component audio signal of Lch, the spatial separation processing unit 1 includes an adder 11_L, a subtractor 12_L, a delay circuit 13_L, an FIR filter 14_L, and a subtractor 15_L. In the configuration for generating the first spatial component audio signal of Rch, the spatial separation processing unit 1 includes an adder 11_R, a subtractor 12_R, a delay circuit 13_R, an FIR filter 14_R, and a subtractor 15_R.

In the configuration for generating the first spatial component audio signal of Lch (corresponding to one channel), the spatial separation processing unit 1 generates a sum signal and a difference signal from the input audio signals of Lch and Rch by using the adder 11_L and the subtractor 12_L. Also, the spatial separation processing unit 1 applies, by using the FIR filter 14_L, filter processing based on the first spatial component to each of the input audio signal of Rch (corresponding to an input audio signal other than the one channel), the sum signal, and the difference signal. Then, the subtractor 15_L subtracts a plurality of audio signals that have passed through the FIR filter 14_L from the input audio signal of Lch delayed by the delay circuit 13_L in accordance with the processing time of the FIR filter 14_L, thereby generating the first spatial component audio signal of Lch.

In the signal processing of Lch, the filter processing based on the first spatial component means performing filtering in accordance with the first spatial component by using the filter coefficients W_L1, W_L2, and W_L3 set for Lch. That is, the FIR filter 14_L filters the input audio signal of Rch by using the filter coefficient W_L1, filters the sum signal output from the adder 11_L by using the filter coefficient W_L2, and filters the difference signal output from the subtractor 12_L by using the filter coefficient W_L3. The three filter coefficients W_L1, W_L2, and W_L3 are set as described later with reference to FIG. 3.

With respect to a configuration for generating the first spatial component audio signal of Rch (corresponding to the one channel), the spatial separation processing unit 1 generates a sum signal and a difference signal from the input audio signals of Lch and Rch by using the adder 11_R and the subtractor 12_R. The spatial separation processing unit 1 also applies, by using the FIR filter 14_R, filter processing based on the first spatial component to each of the input audio signal of Lch (corresponding to an input audio signal other than the one channel), the sum signal, and the difference signal. Then, the subtractor 15_R subtracts a plurality of audio signals that have passed through the FIR filter 14_R from the input audio signal of Rch delayed by the delay circuit 13_R in accordance with the processing time of the FIR filter 14_R, thereby generating the first spatial component audio signal of Rch.

The filter processing based on the first spatial component in the signal processing of Rch means filtering in accordance with the first spatial component by using the filter coefficients W_R1, W_R2, and W_R3 set for Rch. That is, the FIR filter 14_R filters the input audio signal of Lch by using the filter coefficient W_R1, filters the sum signal output from the adder 11_R by using the filter coefficient W_R2, and filters the difference signal output from the subtractor 12_R by using the filter coefficient W_R3. The three filter coefficients W_R1, W_R2, and W_R3 are set as described later with reference to FIG. 3.

Referring back to FIG. 1, description will be provided. The delay circuit 3 (delay circuit 3_L for Lch and delay circuit 3_R for Rch) delays the two-channel input audio signals in accordance with the processing time of the spatial separation processing unit 1.

The correction processing unit 4 performs desired correction processing on the two-channel input audio signals that have passed through the delay circuit 3 and the two-channel spatial component audio signals that are generated for each of the first to N-th spatial components by the spatial separation processing unit 1. The correction processing unit 4 may include, for example, an equalizer that adjusts the sound quality by increasing or decreasing a specific frequency band, a reverb circuit that imparts an arbitrary sound effect, a bandpass filter, a delay circuit, and/or the like. The correction processing unit 4 is optional.

The integration processing unit 2 (adder 2_L for Lch and adder 2_R for Rch) adds the two-channel spatial component audio signals generated for each of the first to N-th spatial components by the spatial separation processing unit 1 to the two-channel input audio signals. In the configuration example shown in FIG. 1, the integration processing unit 2 adds the first to N-th spatial component audio signals of Lch that have passed through the correction processing unit 4 to the input audio signals of Lch that have passed through the correction processing unit 4. Also, the integration processing unit 2 adds the first to N-th spatial component audio signals of Rch that have passed through the correction processing unit 4 to the input audio signals of Rch that have passed through the correction processing unit 4.

The correction processing unit 5 performs desired correction processing on the two-channel input audio signals that have passed through the integration processing unit 2. The correction processing unit 5 may include, for example, an equalizer, a reverb circuit, a bandpass filter, a delay circuit, and/or the like. The correction processing unit 5 may be omitted.

FIG. 3 is a diagram showing a configuration example of a filter coefficient calculation circuit 20 that calculates filter coefficients W_L1, W_L2, W_L3, W_R1, W_R2, and W_R3 to be set in the FIR filters 14_L and 14_R of FIG. 2. FIG. 3 shows a configuration example for calculating the filter coefficients W_L1, W_L2, W_L3, W_R1, W_R2, and W_R3 to be set in the FIR filters 14_L and 14_R (hereinafter, abbreviated as the first spatial component FIR filters 14_L and 14_R) used for generating the first spatial component audio signal. A configuration for calculating filter coefficients to be set in the second to N-th spatial component FIR filters (also referred to as spatial component 2 to N FIR filters, not shown in FIG. 2) is also the same as that of FIG. 3.

As shown in FIG. 3, it is assumed that there are N virtual sound sources S₁ to S_N at arbitrary positions in a virtual acoustic space, and that sounds output from the virtual sound sources S₁ to S_N are collected by a microphone 101_L for Lch and a microphone 101_R for Rch installed in the same virtual acoustic space. Here, a sound emitted from the virtual sound source S₁ reaches the microphone 101_L for Lch under the influence of a spatial transfer function C_1L, and reaches the microphone 101_R for Rch under the influence of a spatial transfer function C_1R. These spatial transfer functions C_1L and C_1R can be calculated in advance. Spatial transfer functions from the other virtual sound sources S₂ to S_N to the microphones 101_L and 101_R can also be calculated in advance.

As shown in FIG. 3, the filter coefficient calculation circuit 20 includes a first signal processing unit 201, a second signal processing unit 202, and an error signal calculation unit 203. The first signal processing unit 201 generates two-channel spatial component virtual audio signals from two-channel virtual audio signals input to the microphones 101_L and 101_R from the plurality of virtual sound sources S₁ to S_N. The configuration of the first signal processing unit 201 is substantially the same as that of the spatial separation processing unit 1 shown in FIG. 2. However, the first spatial component FIR filters 14_L and 14_R shown in FIG. 2 are set with filter coefficients of fixed values, whereas the first spatial component FIR filters 24_L and 24_R shown in FIG. 3 are configured by adaptive filters having variable filter coefficients. Hereinafter, these filters will be referred to as FIR adaptive filters 24_L and 24_R.

With respect to the configuration for generating the first spatial component virtual audio signal of Lch, the first signal processing unit 201 includes an adder 21_L, a subtractor 22_L, a delay circuit 23_L, an FIR adaptive filter 24_L, and a subtractor 25_L. As described above, except that the FIR adaptive filter 24_L differs from the FIR filter 14_L of FIG. 2, the adder 21_L, the subtractor 22_L, the delay circuit 23_L, and the subtractor 25_L have the same configuration as the adder 11_L, the subtractor 12_L, the delay circuit 13_L, and the subtractor 15_L of FIG. 2.

With respect to the configuration for generating the first spatial component virtual audio signal of Rch, the first signal processing unit 201 includes an adder 21_R, a subtractor 22_R, a delay circuit 23_R, an FIR adaptive filter 24_R, and a subtractor 25_R. As described above, except that the FIR adaptive filter 24_R differs from the FIR filter 14_R of FIG. 2, the adder 21_R, the subtractor 22_R, the delay circuit 23_R, and the subtractor 25_R have the same configuration as the adder 11_R, the subtractor 12_R, the delay circuit 13_R, and the subtractor 15_R of FIG. 2.

The second signal processing unit 202 generates two-channel virtual audio signals input to the microphones 101_L and 101_R from one virtual sound source (the first virtual sound source S₁ in the case of FIG. 3). As shown in FIG. 3, the second signal processing unit 202 includes FIR filters 26_L and 26_R in which filter coefficients implementing spatial transfer functions C_1L and C_1R from the first virtual sound source S₁ to the microphones 101_L and 101_R are set, and delay circuits 27_L and 27_R.

The error signal calculation unit 203 calculates error signals E_1L and E_1R between the spatial component virtual audio signal generated by the first signal processing unit 201 and the virtual audio signal from the virtual sound source S₁ generated by the second signal processing unit 202. The FIR adaptive filters 24_L and 24_R of the first signal processing unit 201 operate to minimize errors indicated by the error signals E_1L and E_1R, and determine filter coefficients by which the errors are minimized. The filter coefficients thus determined are set as filter coefficients W_L1, W_L2, W_L3, W_R1, W_R2, and W_R3 of the first spatial component FIR filters 14_L and 14_R shown in FIG. 2.

That is, the filter coefficients W_L1, W_L2, and W_L3 to be set in the first spatial component FIR filter 14_L of Lch shown in FIG. 2 are coefficients determined by the FIR adaptive filter 24_L so as to minimize errors between the virtual audio signal of Lch input to the microphone 101_L from the first virtual sound source S₁ and the spatial component virtual audio signal generated for Lch from the two-channel virtual audio signals input to the microphones 101_L and 101_R from the plurality of virtual sound sources S₁ to S_N, as shown in FIG. 3.

Similarly, the filter coefficients W_R1, W_R2, and W_R3 to be set in the first spatial component FIR filter 14_R of Rch shown in FIG. 2 are coefficients determined by the FIR adaptive filter 24_R so as to minimize errors between the virtual audio signal of Rch input to the microphone 101_R from the first virtual sound source S₁ and the spatial component virtual audio signal generated for Rch from the two-channel virtual audio signals input to the microphones 101_L and 101_R from the plurality of virtual sound sources S₁ to S_N, as shown in FIG. 3.

In the acoustic signal processing apparatus 10 according to the first embodiment configured as described above, the two-channel spatial component audio signals generated for each of a plurality of spatial components by the spatial separation processing unit 1 are added to the two-channel input audio signals by the integration processing unit 2, and thus the audio is emphasized for each of the spatial components, and it is possible to realize an acoustic effect that provides a listener with spatial perception exceeding that of conventional surround sound.

Next, a modification of the first embodiment will be described with reference to the drawings. FIG. 4 is a diagram showing a configuration example of an acoustic signal processing apparatus 10A according to the modification. In FIG. 4, components denoted by the same reference numerals as those shown in FIG. 1 have the same functions, and therefore, redundant description thereof will be omitted.

As shown in FIG. 4, the acoustic signal processing apparatus 10A according to the modification further includes a multi-channel processing unit 6. Further, the acoustic signal processing apparatus 10A according to the modification includes an integration processing unit 2A and correction processing units 4A and 5A, instead of the integration processing unit 2 and correction processing units 4 and 5 shown in FIG. 1.

The multi-channel processing unit 6 generates audio signals of other channels from two-channel input audio signals. In the example shown in FIG. 4, the multi-channel processing unit 6 generates audio signals of Cch (center channel), LFE channel (subwoofer channel), LS channel (left surround channel), and RS channel (right surround channel) using stereo audio signals of Lch and Rch.

As a method for generating surround audio signals of four other channels from two-channel stereo audio signals, a known method can be used. For example, the multi-channel processing unit 6 can generate four-channel surround audio signals by performing processing based on the following calculation using an adder, a subtractor, a multiplier, and a filter. $$\mathrm{C}=\left(\mathrm{L}+\mathrm{R}\right)/2$$ $$\mathrm{LFE}=\mathrm{LPF}\left(\left(\mathrm{L}+\mathrm{R}\right)/2\right)$$ $$\mathrm{LS}=\left(\mathrm{L}-\mathrm{R}\right)/2$$ $$\mathrm{RS}=\left(\mathrm{R}-\mathrm{L}\right)/2$$

The correction processing unit 4A performs desired correction processing on the two-channel input audio signals that have passed through the delay circuit 3, the four-channel surround audio signals generated by the multi-channel processing unit 6, and the two-channel spatial component audio signals generated for each of the first to N-th spatial components by the spatial separation processing unit 1. The correction processing unit 4A may include, for example, an equalizer, a reverb circuit, a bandpass filter, a delay circuit, and/or the like. The correction processing unit 4A may be omitted.

The integration processing unit 2A includes an adder 2_L for Lch, an adder 2_R for Rch, an adder 2_C for Cch, an adder 2_LFE for LFEch, an adder 2_LS for LSch, and an adder 2_RS for RSch, and adds the two-channel spatial component audio signals generated for each of the first to N-th spatial components by the spatial separation processing unit 1 to the two-channel input audio signals and the four-channel surround audio signals.

In the configuration example shown in FIG. 4, the integration processing unit 2A adds the first to N-th spatial component audio signals of Lch that have passed through the correction processing unit 4A to the input audio signals of Lch that have passed through the correction processing unit 4A. Also, the integration processing unit 2A adds the first to N-th spatial component audio signals of Rch that have passed through the correction processing unit 4A to the input audio signals of Rch that have passed through the correction processing unit 4A. Also, the integration processing unit 2A adds the first to N-th spatial component audio signals of Lch and Rch that have passed through the correction processing unit 4A to the input audio signal of Cch that has passed through the correction processing unit 4A. Also, the integration processing unit 2A adds the first to N-th spatial component audio signals of Lch and Rch that have passed through the correction processing unit 4A to the input audio signal of LFEch that has passed through the correction processing unit 4A. The integration processing unit 2A adds the first to N-th spatial component audio signals of Lch that have passed through the correction processing unit 4A to the input audio signal of LSch that has passed through the correction processing unit 4A. The integration processing unit 2A adds the first to N-th spatial component audio signals of Rch that have passed through the correction processing unit 4A to the input audio signal of RSch that has passed through the correction processing unit 4A.

The correction processing unit 5A performs desired correction processing on the two-channel input audio signals and the four-channel surround audio signals that have passed through the integration processing unit 2A. The correction processing unit 5A may include, for example, an equalizer, a reverb circuit, a bandpass filter, a delay circuit, and/or the like. The correction processing unit 5A may be omitted.

As in the modification shown in FIG. 4, by adding the two-channel spatial component audio signals generated for each of a plurality of spatial components by the spatial separation processing unit 1 to the two-channel input audio signals and the four-channel surround audio signals generated from the input audio signals, it is possible to emphasize each spatial component of a total of six-channel surround audio signals. That is, the embodiment shown in FIG. 1 can be extended to the multi-channel surround shown in FIG. 4.

In the first embodiment and the modification described above, an example in which the input audio signals to the acoustic signal processing apparatuses 10 and 10A are two-channel stereo audio signals has been described. On the other hand, a second embodiment described below shows a configuration example in which the input audio signals are four-channel surround audio signals (Lch, Rch, LSch, and RSch). In the second embodiment, the four-channel surround audio signals mean audio signals of Lch, Rch, LSch, and RSch.

FIG. 5 is a diagram showing a configuration example of an acoustic signal processing apparatus 10B according to the second embodiment. As shown in FIG. 5, the acoustic signal processing apparatus 10B according to the second embodiment includes a spatial separation processing unit 1B, an integration processing unit 2B, a delay circuit 3B, correction processing units 4B and 5B, and a multi-channel processing unit 6B.

The spatial separation processing unit 1B generates spatial component audio signals for each of a plurality of spatial components corresponding to a plurality of virtual sound sources set at different spatial positions from each of the input audio signals of the four channels.

FIG. 6 is a diagram showing a configuration example of the spatial separation processing unit 1B. In FIG. 6, only the configuration of a portion for generating the first spatial component audio signal is shown, and the configuration of a portion for generating the second to N-th spatial component(also referred to as spatial component 2 to N) audio signals is omitted. The configurations for generating the first to N-th spatial component audio signals are the same except that the filter coefficients to be set are different.

As shown in FIG. 6, the spatial separation processing unit 1B generates four-channel first spatial component audio signals from four-channel input audio signals. Here, when generating a spatial component audio signal of one channel among the four channels, the spatial component audio signal is generated by using the input audio signals of three channels including the one channel. Among the input audio signals of three channels, the input audio signals of two channels other than the one channel are the input audio signals of two channels adjacent to the one channel.

The adjacency relationship between channels is determined based on the general arrangement positions of surround speakers as shown in FIG. 7. That is, when one channel is LSch, the two channels other than the one channel are Lch and RSch, which are adjacent to LSch. When one channel is Lch, the two channels other than the one channel are Rch and LSch, which are adjacent to Lch. When one channel is Rch, the two channels other than the one channel are Lch and RSch, which are adjacent to Rch. When one channel is RSch, the two channels other than the one channel are Rch and LSch, which are adjacent to RSch.

For example, the spatial separation processing unit 1B generates a first spatial component audio signal of LSch from input audio signals of three channels of LSch, Lch, and RSch. As shown in FIG. 6, with respect to a configuration for generating the first spatial component audio signal of LSch, the spatial separation processing unit 1B includes adders 31_LS and 32_LS, subtractors 33_LS and 34_LS, a delay circuit 35_LS, an FIR filter 36_LS, and a subtractor 37_LS.

With respect to a configuration for generating the first spatial component audio signal of LSch (corresponding to one channel), the spatial separation processing unit 1B generates a sum signal and a difference signal from input audio signals of LSch and Lch by using the adder 31_LS and the subtractor 33_LS, and generates a sum signal and a difference signal from input audio signals of LSch and RSch by using the adder 32_LS and the subtractor 34_LS. The spatial separation processing unit 1B applies, by using the FIR filter 36_LS, filter processing based on the first spatial component to each of the input audio signals of Lch and RSch (corresponding to input audio signals other than the one channel), the sum signals, and the difference signals. Then, the subtractor 37_LS subtracts a plurality of audio signals that have passed through the FIR filter 36_LS from the input audio signal of LSch delayed by the delay circuit 35_LS in accordance with the processing time of the FIR filter 36_LS, thereby generating the first spatial component audio signal of LSch.

In the signal processing of LSch, the filter processing based on the first spatial component means that filtering is performed by using filter coefficients W_LS1 to W_LS6 set for LSch in accordance with the first spatial component. That is, the FIR filter 36_LS filters the input audio signal of Lch by using the filter coefficient W_LS1, filters the sum signal output from the adder 31_LS by using the filter coefficient W_LS2, filters the difference signal output from the subtractor 33_LS by using the filter coefficient W_LS3, filters the input audio signal of RSch by using the filter coefficient W_LS4, filters the sum signal output from the adder 32_LS by using the filter coefficient W_LS5, and filters the difference signal output from the subtractor 34_LS by using the filter coefficient W_LS6. The six filter coefficients W_LS1 to W_LS6 are set as described later with reference to FIG. 8A.

The spatial separation processing unit 1B generates the first spatial component audio signal of Lch from input audio signals of three channels of Lch, Rch, and LSch. With respect to a configuration for generating the first spatial component audio signal of Lch, as shown in FIG. 6, the spatial separation processing unit 1B includes adders 31_L and 32_L, subtractors 33_L and 34_L, a delay circuit 35_L, an FIR filter 36_L, and a subtractor 37_L.

With respect to the configuration for generating the first spatial component audio signal of Lch (corresponding to one channel), the spatial separation processing unit 1B generates a sum signal and a difference signal from input audio signals of Lch and LSch by using the adder 31_L and the subtractor 33_L, and generates a sum signal and a difference signal from input audio signals of Lch and Rch by using the adder 32_L and the subtractor 34_L. Also, the spatial separation processing unit 1B applies, by using the FIR filter 36_L, filter processing based on the first spatial component to input audio signals of LSch and Rch (corresponding to input audio signals other than the one channel), the sum signals, and the difference signals. Then, the subtractor 37_L subtracts a plurality of audio signals that have passed through the FIR filter 36L from the input audio signal of Lch delayed by the delay circuit 35_L in accordance with the processing time of the FIR filter 36_L, thereby generating the first spatial component audio signal of Lch.

In the signal processing of Lch, the filter processing based on the first spatial component means that filtering is performed by using filter coefficients W_L1 to W_L6 set for Lch in accordance with the first spatial component. That is, the FIR filter 36_L filters the input audio signal of LSch by using the filter coefficient W_L1, filters the sum signal output from the adder 31_L by using the filter coefficient W_L2, filters the difference signal output from the subtractor 33_L by using the filter coefficient W_L3, filters the input audio signal of Rch by using the filter coefficient W_L4, filters the sum signal output from the adder 32_L by using the filter coefficient W_L5, and filters the difference signal output from the subtractor 34_L by using the filter coefficient W_L6. The six filter coefficients W_L1 to W_L6 are set as described later with reference to FIG. 8A.

The spatial separation processing unit 1B generates the first spatial component audio signal of Rch from input audio signals of three channels of Rch, Lch, and RSch. With respect to a configuration for generating the first spatial component audio signal of Rch, as shown in FIG. 6, the spatial separation processing unit 1B includes adders 31_R and 32_R, subtractors 33_R and 34_R, a delay circuit 35_R, an FIR filter 36_R, and a subtractor 37_R.

With respect to the configuration for generating the first spatial component audio signal of Rch (corresponding to one channel), the spatial separation processing unit 1B generates a sum signal and a difference signal from input audio signals of Rch and RSch by using the adder 31_R and the subtractor 33_R, and generates a sum signal and a difference signal from input audio signals of Rch and Lch by using the adder 32_R and the subtractor 34_R. Also, the spatial separation processing unit 1B applies, by using the FIR filter 36_R, filter processing based on the first spatial component to each of the input audio signals of Lch and RSch (corresponding to input audio signals other than the one channel), the sum signals, and the difference signals. Then, the subtractor 37_R subtracts a plurality of audio signals that have passed through the FIR filter 36_R from the input audio signal of Rch delayed by the delay circuit 35_R in accordance with the processing time of the FIR filter 36_R, thereby generating the first spatial component audio signal of Rch.

In the signal processing of Rch, the filter processing based on the first spatial component means that filtering is performed by using filter coefficients W_R1 to W_R6 set for Rch in accordance with the first spatial component. That is, the FIR filter 36_R filters the input audio signal of RSch by using the filter coefficient W_R1, filters the sum signal output from the adder 31_R by using the filter coefficient W_R2, filters the difference signal output from the subtractor 33_R by using the filter coefficient W_R3, filters the input audio signal of Lch by using the filter coefficient W_R4, filters the sum signal output from the adder 32_R by using the filter coefficient W_R5, and filters the difference signal output from the subtractor 34_R by using the filter coefficient W_R6. The six filter coefficients W_R1 to W_R6 are set as described later with reference to FIG. 8B.

The spatial separation processing unit 1B generates the first spatial component audio signal of RSch from input audio signals of three channels of RSch, Rch, and LSch. With respect to a configuration for generating the first spatial component audio signal of RSch, as shown in FIG. 6, the spatial separation processing unit 1B includes adders 31_RS and 32_RS, subtractors 33_RS and 34_RS, a delay circuit 35_RS, an FIR filter 36_RS, and a subtractor 37_RS.

With respect to the configuration for generating the first spatial component audio signal of RSch (corresponding to one channel), the spatial separation processing unit 1B generates a sum signal and a difference signal from input audio signals of RSch and Rch by using the adder 31_RS and the subtractor 33_RS, and generates a sum signal and a difference signal from input audio signals of RSch and LSch by using the adder 32_RS and the subtractor 34_RS. Also, the spatial separation processing unit 1B applies, by using the FIR filter 36_RS, filter processing based on the first spatial component to each of the input audio signals of LSch and Rch (corresponding to input audio signals other than the one channel), the sum signals, and the difference signals. Then, the subtractor 37_RS subtracts a plurality of audio signals that have passed through the FIR filter 36_RS from the input audio signal of RSch delayed by the delay circuit 35_RS in accordance with the processing time of the FIR filter 36_RS, thereby generating the first spatial component audio signal of RSch.

In the signal processing of RSch, the filter processing based on the first spatial component means that filtering is performed by using the filter coefficients W_RS1 to W_RS6 set for RSch in accordance with the first spatial component. That is, the FIR filter 36_RS filters the input audio signal of Rch by using the filter coefficient W_RS1, filters the sum signal output from the adder 31_RS by using the filter coefficient W_RS2, filters the difference signal output from the subtractor 33_RS by using the filter coefficient W_RS3, filters the input audio signal of LSch by using the filter coefficient W_RS4, filters the sum signal output from the adder 32_RS by using the filter coefficient W_RS5, and filters the difference signal output from the subtractor 34_RS by using the filter coefficient W_RS6. The six filter coefficients W_RS1 to W_RS6 are set as described later with reference to FIG. 8B.

Referring back to FIG. 5, description will be provided. The delay circuit 3B delays the input audio signals of four channels in accordance with the processing time of the spatial separation processing unit 1B.

The multi-channel processing unit 6B generates audio signals of other channels from the input audio signals of the four channels. In the example shown in FIG. 5, the multi-channel processing unit 6B generates audio signals of Cch (center channel), LFE channel (subwoofer channel), LBS channel (left rear channel), RBS channel (right rear channel), and BS channel (center rear channel) by using the input audio signals of the four channels. The multi-channel processing unit 6B may be omitted.

As a method for generating surround audio signals of five other channels from the input audio signals of the four channels, a known method can be used. For example, the multi-channel processing unit 6B can generate surround audio signals of five channels by performing processing based on the following calculation using an adder, a subtractor, a multiplier, and a filter. $$\mathrm{C}=\left(\mathrm{L}+\mathrm{R}\right)/2$$ $$\mathrm{LFE}=\mathrm{LPF}\left(\left(\mathrm{L}+\mathrm{R}\right)/2\right)$$ $$\mathrm{LBS}=\left(\mathrm{LS}-\mathrm{R}\right)/2$$ $$\mathrm{RBS}=\left(\mathrm{RS}-\mathrm{L}\right)/2$$ $$\mathrm{BS}=\left(\mathrm{LBS}+\mathrm{RBS}\right)/2$$

The correction processing unit 4B performs desired correction processing on nine-channel audio signals that have passed through the multi-channel processing unit 6B and the four-channel spatial component audio signals that are generated for each of the first to N-th spatial components by the spatial separation processing unit 1B. The correction processing unit 4B may include, for example, an equalizer, a reverb circuit, a bandpass filter, a delay circuit, and/or the like. The correction processing unit 4B may be omitted.

The integration processing unit 2B adds the four-channel spatial component audio signals generated for each of the first to N-th spatial components by the spatial separation processing unit 1B to the nine-channel audio signals that have passed through the correction processing unit 4B. In the configuration example shown in FIG. 5, the integration processing unit 2B adds the first to N-th spatial component audio signals of Lch to the audio signal of Lch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of Rch to the audio signal of Rch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of Lch and Rch to the audio signal of Cch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of Lch and Rch to the audio signal of LFEch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of LSch and Lch to the audio signal of LSch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of RSch and Rch to the audio signal of RSch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of LSch to the audio signal of LBSch. The integration processing unit 2B also adds the first to N-th spatial component audio signals of RSch to the audio signal of RBSch. Also, the integration processing unit 2B adds the first to N-th spatial component audio signals of LSch and RSch to the audio signal of BSch.

The correction processing unit 5B performs desired correction processing on nine-channel audio signals that have passed through the integration processing unit 2B. The correction processing unit 5B may include, for example, an equalizer, a reverb circuit, a bandpass filter, a delay circuit, or the like. The correction processing unit 5B may be omitted.

FIGS. 8A and 8B are diagrams showing a configuration example of a filter coefficient calculation circuit 20B that calculates filter coefficients W_LS1 to W_LS6, W_L1 to W_L6, W_R1 to W_R6, and W_RS1 to W_RS6 to be set in the FIR filters 36_LS, 36_L, 36_R, and 36_RS shown in FIG. 6. FIGS. 8A and 8B are diagrams showing a configuration example for calculating the filter coefficients W_LS1 to W_LS6, W_L1 to W_L6, W_R1 to W_R6, and W_RS1 to W_RS6 set in the FIR filters 36_LS, 36_L, 36_R, and 36_RS (hereinafter, abbreviated as the first spatial component FIR filter 36) used for generating the first spatial component audio signal. A configuration for calculating filter coefficients to be set in the second to N-th spatial component FIR filters (not shown in FIG. 5) is also the same as those of FIGS. 8A and 8B.

As shown in FIGS. 8A and 8B, the filter coefficient calculation circuit 20B includes first signal processing units 201_LS, 201_L, 201R, and 201_RS, second signal processing units 202_LS, 202_L, 202_R, and 202_RS, and error signal calculation units 203_LS, 203_L, 203_R, and 203_RS.

The first signal processing unit 201_LS, the second signal processing unit 202_LS, and the error signal calculation unit 203_LS generate a spatial component virtual audio signal of LSch from four-channel virtual audio signals input to the microphones 101_LS, 101_L, 101_R, and 101_RS from a plurality of virtual sound sources S₁ to S_N.

The first signal processing unit 201_L, the second signal processing unit 202_L, and the error signal calculation unit 203_L generate a spatial component virtual audio signal of Lch from four-channel virtual audio signals input to the microphones 101_LS, 101_L, 101_R, and 101_RS from the plurality of virtual sound sources S₁ to S_N.

The first signal processing unit 201_R, the second signal processing unit 202_R, and the error signal calculation unit 203_R generate a spatial component virtual audio signal of Rch from four-channel virtual audio signals input to the microphones 101_LS, 101_L, 101_R, and 101_RS from the plurality of virtual sound sources S₁ to S_N.

The first signal processing unit 201_RS, the second signal processing unit 202_RS, and the error signal calculation unit 203_RS generate a spatial component virtual audio signal of RSch from four-channel virtual audio signals input to the microphones 101_LS, 101_L, 101_R, and 101_RS from the plurality of virtual sound sources S₁ to S_N.

Hereinafter, a configuration for generating the spatial component virtual audio signal of LSch will be described as a representative example, but the same applies to configurations for generating spatial component virtual audio signals of Lch, Rch, and RSch.

The configuration of the first signal processing unit 201_LS is substantially the same as the configuration for generating the first spatial component audio signal of LSch in the spatial separation processing unit 1B shown in FIG. 6. However, the first spatial component FIR filter 36_LS shown in FIG. 6 is set with filter coefficients of fixed values, whereas the first spatial component FIR filter 46_LS shown in FIG. 8A is configured by an adaptive filter having variable filter coefficients. Hereinafter, the first spatial component FIR filter 46_LS will be referred to as the FIR adaptive filter 46_LS.

The first signal processing unit 201_LS includes adders 41_LS and 42_LS, subtractors 43_LS and 44_LS, a delay circuit 45_LS, the FIR adaptive filter 46_LS, and a subtractor 47_LS. As described above, except that the FIR adaptive filter 46_LS differs from the FIR filter 36_LS shown in FIG. 6, the adders 41_LS and 42_LS, the subtractors 43_LS and 44_LS, the delay circuit 45_LS, and the subtractor 47_LS have the same configuration as the adders 31_LS and 32_LS, the subtractors 33_LS and 34_LS, the delay circuit 35_LS, and the subtractor 37_LS, as shown in FIG. 6.

The second signal processing unit 202_LS generates a virtual audio signal of the LS channel input to the microphone 101_LS from one virtual sound source (the first virtual sound source S₁ in the case of FIG. 8A). As shown in FIG. 8A, the second signal processing unit 202_LS includes an FIR filter 48_LS in which filter coefficients implementing the spatial transfer function C_1LS from the first virtual sound source S₁ to the microphone 101_LS are set, and a delay circuit 49_LS.

The error signal calculation unit 203_LS calculates an error signal E_1LS between the spatial component virtual audio signal generated by the first signal processing unit 201_LS and the virtual audio signal of LSch from the virtual sound source S₁ generated by the second signal processing unit 202_LS. The FIR adaptive filter 46_LS of the first signal processing unit 201_LS operates to minimize the error indicated by the error signal E_1LS, and determines filter coefficients by which the error is minimized. The filter coefficients thus determined are set as filter coefficients W_LS1 to W_LS6 of the first spatial component FIR filter 36_LS shown in FIG. 6.

That is, the filter coefficients W_LS1 to W_LS6 to be set in the first spatial component FIR filter 36_LS of LSch shown in FIG. 6 are coefficients determined by the FIR adaptive filter 46_LS so as to minimize the error between the virtual audio signal of LSch input to the microphone 101_LS from the first virtual sound source S₁ and the spatial component virtual audio signal generated for LSch from the four-channel virtual audio signals input to the microphones 101_LS, 101_L, 101_R, and 101_RS from the plurality of virtual sound sources S₁ to S_N, as shown in FIG. 8A.

In the acoustic signal processing apparatus 10B according to the second embodiment configured as described above, it is possible to realize further expansion to multi-channel surround compared with the above-described modification. That is, even when the input audio signals have more than two channels, since the spatial component audio signals of each channel generated for each of a plurality of spatial components by the spatial separation processing unit 1B are added to the input audio signals of each channel by the integration processing unit 2B, the audio is emphasized for each of the spatial components, and it is possible to realize an acoustic effect that provides a listener with spatial perception exceeding that of conventional surround sound.

In the second embodiment described above, the input audio signals have four channels, and when generating the spatial component audio signal of one channel among the four channels, the spatial component audio signal is generated by using the input audio signals of three channels including the one channel, and the input audio signals of two channels adjacent to the one channel are used as the input audio signals of two channels other than the one channel. However, the present disclosure is not limited to the above example. For example, when the spatial component audio signal of one channel is generated, the spatial component audio signal may be generated by using input audio signals of all four channels, and the input audio signals of three channels other than the one channel may be used as the input audio signals other than the one channel.

If the spatial component audio signal is generated by using input audio signals of all four channels, it is possible to enhance the audio for each spatial component by adding spatial component audio signals reflecting the acoustic characteristics of the input audio signals of all channels. Even in the case where the spatial component audio signal is generated by using the input audio signals of three channels, which is one channel fewer than four channels, as in the second embodiment, since the spatial component audio signal is generated by using the input audio signals of one channel and two channels adjacent to the one channel, it is possible to enhance the audio for each spatial component by adding spatial component audio signals reflecting the acoustic characteristics having a large influence on the one channel, and it is also advantageous in that an increase in the circuit scale can be suppressed.

Further, in the second embodiment, an example of inputting four-channel audio signals consisting of Lch, Rch, LSch, and RSch to the acoustic signal processing apparatus 10B has been described, but the input audio signals are not limited to this example. For example, the multi-channel processing unit 6B may be omitted, and nine-channel audio signals may be input.

The configurations of the spatial separation processing units 1 and 1B shown in the above embodiments and modifications are merely examples, and are not limited to these configurations. That is, in the spatial separation processing units 1 and 1B, filtering is performed on the audio signals generated by adding or subtracting the audio signals of each channel, but the method of addition or subtraction may be different from that in the above example.

Clauses applicable to the present disclosure are summarized below.

An acoustic signal processing apparatus includes:

• a spatial separation processing circuit configured to generate, from input audio signals of a plurality of channels, spatial component audio signals for each of a plurality of spatial components corresponding to a plurality of virtual sound sources set at different spatial positions; and
• an integration processing circuit configured to add, to the input audio signals of the plurality of channels, the spatial component audio signals of the plurality of channels generated by the spatial separation processing circuit for each of the plurality of spatial components.

In an acoustic signal processing apparatus in clause 1, the spatial separation processing circuit may be further configured to:

• generate a sum signal and a difference signal from the input audio signals of the plurality of channels, and
• generate a spatial component audio signal of one channel of the plurality of channels by subtracting, from an input audio signal of the one channel, a plurality of audio signals that are obtained by applying filter processing based on each of: the plurality of spatial components to at least one input audio signal of at least one channel other than the one channel; the sum signal; and the difference signal.

In an acoustic signal processing apparatus in clause 1 or 2, filter coefficients of the filter processing may be coefficients determined by an adaptive filter to minimize an error between a virtual audio signal of the one channel that is input from one virtual sound source of the plurality of virtual sound sources, and a spatial component virtual audio signal generated for the one channel from virtual audio signals of the plurality of channels that are input from the plurality of virtual sound sources.

In an acoustic signal processing apparatus in any one of clauses 1 to 3, the input audio signals of the plurality of channels may be two-channel stereo audio signals.

In an acoustic signal processing apparatus in any one of clauses 1 to 3, the input audio signals of the plurality of channels may be four-channel surround audio signals.

In an acoustic signal processing apparatus in clause 2 or 3, the input audio signals of the plurality of channels may be four-channel surround audio signals, the at least one input audio signal of the at least one channel other than the one channel may include input audio signals of two channels other than the one channel, and the input audio signals of the two channels may be input audio signals of two channels adjacent to the one channel.

In an acoustic signal processing apparatus in clause 2 or 3, the input audio signals of the plurality of channels may be four-channel surround audio signals, and the at least one input audio signal of the at least one channel other than the one channel may include input audio signals of three channels other than the one channel.

An acoustic signal processing apparatus in any one of clauses 1 to 7 may further include a multi-channel processing circuit configured to generate, from the input audio signals of the plurality of channels, audio signals of multiple channels other than the plurality of channels. The integration processing circuit may be further configured to add the spatial component audio signals of the plurality of channels generated by the spatial separation processing circuit for each of the plurality of spatial components, to the input audio signals of the plurality of channels, and the audio signals of the multiple channels generated by the multi-channel processing circuit.

The above embodiments are merely examples of the present disclosure, and do not intend to limit the scope of the present disclosure. Various modifications, combinations, and the like can be made without departing from the scope of the claims.

In the present disclosure, since spatial component audio signals of a plurality of channels generated for each of a plurality of spatial components are added to input audio signals of the plurality of channels, the audio is emphasized for each of the spatial components, thereby realizing an acoustic effect that provides a listener with spatial perception exceeding that of conventional surround sound.