The invention is directed to a hands-free system for speech signal acquisition in a vehicular cabin which shows an improved signal quality.
Hands-free systems are used in vehicular cabins, particularly of cars, for different purposes. For example, a hands-free system can be used to enable hands-free telephony. As another example, control of devices such as a navigation system or an air conditioning system might rely on a hands-free system. For this, the acquired signals are processed using a speech recognition device to determine control signals as input by a user.
In most common hands-free system, at least one microphone is provided to pick up speech signals from a user. These microphones are usually fixedly mounted inside the vehicular cabin. The place for the microphones is chosen based on different criteria. On the one hand, the arrangement and orientation of the microphones with respect to an expected speaker has to be optimized. On the other hand, usually, there are constructional and/or aesthetic or design constraints restricting the number of possible positions.
A hands-free system offers a high comfort to a user because no headset or handset is needed. However, the quality of the signal, in particular, the signal-to-noise ratio, acquired by a microphone is rather poor because the acoustic level of the speech signal decreases with the distance from the speaker's mouth to the microphone. In view of this, there is a need to enhance the signal quality of the acquired signals and to improve/increase the signal-to-noise ratio.
One possibility to obtain noise suppression in a signal is to use a microphone array containing several microphones in combination with spatial filtering. In this case, usually omni-directional microphones are used in the microphone arrays. However, due to the arrangement of the microphones in an array and the subsequent filtering, a desired directivity of the array can be achieved with the consequence of a noise suppression. In particular, the directivity of this system is controlled such that its sensitivity is highest in the direction to an expected speaker. Noise signals coming from other directions, for example, from other speaking persons or from loudspeakers, are suppressed. Possible realizations of such microphone array systems are described in
In all these microphone array systems, the microphones of the array are used to both pick up the speech signal and the noise signal. The separation of the noise signal from the speech signal is achieved by processing the entire signals acquired by the microphone array system. However, as all signals to be processed always contain both speech and noise parts, the processing of the signals is quite complex, and the resulting signal still contains noise parts.
Thus, it is the problem underlying the invention to provide a simplified hands-free system yielding a further improved signal-to-noise ratio in an acquired signal. This problem is solved by a hands-free system in accordance with claim 1.
Accordingly, a hands-free system for speech signal acquisition in a vehicular cabin is provided, comprising:
Such a hands-free system allows to acquire a signal comprising a speech part and a noise part with the at least one first directional microphone, on the one hand, and to acquire a noise signal with the at least one second directional microphone, on the other hand, as the at least one second directional microphone points in a different direction than the at least one first microphone. Therefore, a signal picked up by the at least one second microphone containing (almost) only noise can be used to remove a noise part from the signal picked up by the at least one first microphone.
In this context, it is pointed out that the term "speech" is used to denote any type of wanted signal whereas "noise" denotes an unwanted signal. In other words, "noise" can also comprise signals comprising speech elements, for example, emanating from a loudspeaker or another speaker present in the vehicular cabin; however, these signals are not wanted for a specific purpose.
The use of directional microphones has the advantage that the first microphones can be arranged so as to point to a position of an expected speaker so that the second microphones, due to their different pointing direction, only or predominately acquire a noise signal not stemming from the speaker. Thus, this hands-free system enables to enhance an acquired signal with a wanted speech part already without additional sophisticated filter methods.
The angle between the pointing directions of the first and the second directional microphones can lie between 20° and 180°, preferably between 90° and 180°, most preferred between 130° and 180°.
The noise reduction device can be configured to subtract a signal based on the signal from the at least one second directional microphone from the signal from the at least one first directional microphone.
In this way, the calculation of the noise part from the signal acquired by the at least one first directional microphone in an advantageous and simple way. In particular, the subtrahend, i.e. the signal based on the signal from the at least one second microphone, can be based on or can be equal to the sum or the average of the signals acquired by all second directional microphones.
The noise reduction device can be further configured to cancel a noise part in a signal from the at least one second directional microphone using a signal from the at least one first directional microphone.
In this way, the hands-free system is enabled to acquire enhanced speech signals or wanted signals from two different directions. In particular, in a vehicular cabin, the at least one first and second microphones can be arranged so that the at least one first microphone points in the direction of driver and the at least one second microphone in the direction of a front seat passenger. Then, if the driver is speaking, the first microphones are responsible for picking up the speech signals whereas the second microphones acquire the noise reference signals. If the front seat passenger is speaking, the roles of the first and second microphones are swept.
In other words, the noise reduction device is configured symmetrically to cancel a noise part in signals acquired either by the first or the second microphones based on noise reference signals acquired by the other at least one microphone (i.e., the second or the first microphones, respectively).
In the above embodiment, the hands-free system can further comprise a control means for controlling the signal processing of the noise reduction device according to a predetermined criterion in such a way that a noise part in a signal from the at least one first microphone and/or a noise part in a signal from the at least one second microphone is cancelled.
Such a control means allows to use the noise reduction system together with the first and second microphones in an advantageous way to acquire a speech signal from one of the two pointing directions of the microphone and suppressing noise parts therein using the microphones pointing in the other direction.
Several criteria are possible to be used by this control means. For example, always if a speech signal is detected in the signals from the first microphones, the noise reduction device is controlled such that a noise part in the signal from the at least one first microphone is cancelled using the signal from the second microphones as noise reference signal. In this way, a speaker to which the at least one first microphone is pointing is prioritized.
Alternatively, the control means can be configured such that a noise part in a signal from those microphones is cancelled for which a speech signal is detected first. In this case, each time one speaker stops talking, both the first and the second microphones are ready to acquire a speech signal.
In the previously described hands-free systems, the at least one first directional and/or the at least one second directional microphone can be provided as a microphone array with at least two directional microphones, and the noise reduction system can comprise a beamformer for processing the signals from each microphone array.
Using a microphone array and a corresponding beamformer, particularly for picking up the wanted signal, further increases the signal-to-noise ratio in the signal containing the wanted signal. This leads to a more enhanced resulting signal after cancellation of the noise part. If the at least one second microphone is also used to pick up wanted signals, such an advantageous enhancement of the resulting signal is obtained when using a microphone array and a corresponding beamformer in the case of the second microphones as well.
If both the first and the second microphones are provided as a microphone array comprising at least two directional microphones, a beamformer can be provided for processing the signals from both microphone arrays. Alternatively, for each microphone array, a beamformer can be provided.
According to a further aspect of the above described hands-free systems, the number of first microphones can equal the number of second microphones. Such a symmetrical case makes it possible to use both the first and the second microphones for picking up a wanted speech signal with similar signal quality.
The at least one first and at least one second microphones can be provided pairwise such that one first microphone and one second microphone are arranged on a common support frame, respectively.
By such a pairwise arrangement, a dual microphone is obtained comprising a first and a second directional microphone, each pointing in a different direction. Dual microphones of this kind can be provided in a very compact form, which has the advantage that not much space is required when mounting the microphones or microphone arrays, in particular, in a vehicular cabin. Each first and second microphone can further be provided in a common housing.
In the previously described hands-free systems, each first microphone can have a first output, each second microphone can have a second output and the noise reduction device can comprise a first input for each first microphone output, a second input for each second microphone output, a cancellation means, a first signal path connecting the at least one first input and the cancellation means, a second signal path connecting the at least one second input and the cancellation means, wherein the cancellation means can be configured to cancel the noise part from the signal received via the first signal path using the signal received via the second signal path, wherein each first microphone output is connected to a first input of the noise reduction device and each second microphone output is connected to a second input of the noise reduction device.
In this way, a particularly advantageous configuration of the noise reduction device and its connection via corresponding signal paths to the at least one first and second microphones is obtained.
In particular, the cancellation means can be configured to subtract the signal based on the signal received by the second signal path from the signal received by the first signal path.
In the above hands-free systems, an adaptive filter, in particular, based on an LMS, NLMS or RLS algorithm, can be provided on the second signal path between the cancellation means and each second input.
Such adaptive filters result in a further enhancement of the speech signal. The adaptation of the adaptive filter can be based on the output signal of the cancellation means. Each adaptive filter can be configured to be optimized adaptively to estimate the residual noise in the signal received by the cancellation means via the first signal path; thus, the adaptive filters are adapted to maximally cancel out such a residual noise.
In the previously described hands-free systems, the at least one first directional microphone can be provided as a microphone array with at least two directional microphones, and a beamformer can be provided on the first signal path, wherein for each first input of the noise reduction device, the first signal path comprises a sub-path connecting the first input and the beamformer.
Such an arrangement with a microphone array and a beamformer results in a further enhancement of the signal acquired by the first microphones, particularly with respect to its signal-to-noise ratio.
The beamformer can be a fixed beamformer. As an example, the beamformer can be a delay-and-sum beamformer in which the signals coming from the different microphones are delayed in such a way that signals entering the microphones from a preferred direction can be added in phase. Then, a summation of the delayed microphone signals is performed. According to another alternative, a filter-and-sum beamformer ("superdirective beamformer") can be used. Furthermore, instead of a fixed beamformer, it is also possible to use an adaptive beamformer.
The above hands-free systems can further comprise a third signal path connecting each first input and the cancellation means, wherein a blocking matrix and a subsequent adaptive filter in the direction of signal flow is provided on the third signal path, wherein for each first input of the noise reduction device, the third signal path comprises a sub-path connecting the first input and the blocking matrix and a sub-path connecting the blocking matrix and the adaptive filter.
In this way, an adaptive structure is obtained which is sometimes called "generalized side-lobe canceller" (GSC). In this way, the signal quality can be further improved.
The blocking matrix serves to block out parts of the signal so that a noise reference signal is obtained. The blocking matrix may be realized in different ways. For example, the blocking matrix may have a form so that a pairwise subtraction of neighboring input channels is performed. Another possibility is described in the article by
The previously described hands-free systems may further comprise subtraction means on the second signal path between each second input of the noise reduction device and the cancellation means and a fourth signal path comprising sub-paths connecting the first signal path and each subtraction means, wherein an adaptive filter is provided on each sub-path connecting the first signal path and each subtraction means.
With such a configuration, a reduction of wanted signal parts in the signals received from the at least one second directional microphones is obtained. As a result, these wanted signal parts will not be removed from the signal received from the at least one first directional microphone.
The adaptation of the adaptive filters on the sub-paths can be based on the output of the corresponding cancellation means on the second signal path. With these additional adaptive filters, it is possible to compensate for wanted signal portions that leak into the noise reference signals.
The above-described hands-free systems can further comprise an adaptation controller with a speech detector for controlling adaptation of an adaptive filter in accordance with the detection of speech. In particular, the hands-free systems can comprise an adaptation control for adaptation of all adaptive filters in accordance with the detection of speech.
In particular, the adaptation controller can be configured to initiate adaptation of an adaptive filter on the second signal path if no speech is detected on the first signal path. The adaptation controller can be configured to initiate adaptation of an adaptive filter on a signal path connecting the first signal path and a subtraction means on the second signal path, if wanted speech is detected in the signal on the first signal path. Such an adaptation control further prevents wanted signal cancellation.
In the above hands-free systems, each first microphone output can be connected to a second input of the noise reduction device and each second microphone output can be connected to a first input of the noise reduction device, and the hands-free system can comprise a control means for controlling the signal processing of the noise reduction device according to a predetermined criterion in such a way that a noise part in a signal from the at least one first microphone and/or a noise part in a signal from the at least one second microphone is cancelled by the cancellation means.
Such a configuration leads to a symmetrical arrangement. It has the advantage that canceling a noise part in a signal received from the at least one second directional microphone using a signal received from the at least one first directional microphone is enabled.
In particular, the cancellation means can be further configured to subtract a signal based on the signal received by the first signal path from a signal received by the second signal path. Furthermore, a signal path with an adaptive filter connecting the first input and the cancellation means and/or a signal path with a beamformer connecting the second input and the cancellation means can be provided. These signal paths can be identical to the first and second signal paths mentioned above. The adaptive filter and the beamformer can be configured as those on the first and the second signal path.
It is possible to use the adaptive filters and the beamformer described above for processing the signals acquired by the other microphones. For this, the second input can be connected with the beamformer on the first signal path and the first input can be connected with the adaptive filters on the second signal path. In this case, the control means is configured to control the signal flow in the noise reduction device accordingly. Thus, depending on which microphones (the first or the second microphones) are used to pick up a desired signal, the beamformer and the cancellation means in the noise reduction device are used in the appropriate way.
According to another aspect, the noise reduction device of the above-described hands-free systems can be a first noise reduction device to provide a first enhanced output signal, and the hands-free system can further comprise a second noise reduction device configured to cancel a noise part in a signal from the at least one second directional microphone using a signal from the at least first second directional microphone to provide a second enhanced output signal.
This allows to independently enhance signals coming from the first and the second directional microphones, respectively. In particular, two enhanced output signals (corresponding to the first and the second directional microphones, respectively) can be provided.
In such a case, the hands-free system can further comprise a signal mixer for combining the first and the second enhanced output signal to provide a combined enhanced output signal.
The signal mixer can be configured such that the mixing weights are controlled in a time-varying way depending on a speech activity detected for the at least one first and/or the at least one second directional microphone. Alternatively, the signal mixer can be configured such that one of the mixing weights is set to zero according to a predetermined criterion. As an example, if speech activity is detected for one of the first or the second microphones, the mixing weight for the other of the first or second microphones is set to zero. In this way, the speaker who starts first is prioritized.
In the above hands-free systems, the second noise reduction device can be configured to process a signal from the at least one first directional microphone and a signal from the at least one second directional microphone at least partly in the same way as the first noise reduction device processes a signal from the at least one second directional microphone and a signal from the at least one first directional microphone, respectively.
Thus, an at least partly symmetrical configuration is obtained. In particular, the second noise reduction device can at least partly comprise the same components as the first noise reduction device as described above. For example, the second noise reduction device can comprise a first input for each second microphone output, a second input for each first microphone output, a cancellation means, a first signal path connecting the at least one first input and the cancellation means, a second signal path connecting the at least one second input and the cancellation means, wherein the cancellation means can be configured to cancel the noise part from the signal received via the first signal path using the signal received via the second signal path, wherein each first microphone output is connected to a second input of the noise reduction device and each second microphone output is connected to a first input of the noise reduction device. As a further example, the second noise reduction device can comprise a beamformer and/or a cancellation means, similar to the case of the first noise reduction device.
In addition, the above-described hands-free systems can further comprise an acoustic echo canceller (AEC).
Such an acoustic echo canceller can use a loudspeaker excitation signal as a basis to cancel a further noise part in a signal received from the at least one first directional microphone. In this way, an acoustic feedback from loudspeakers into the microphones will also be compensated for. Preferably, if more than one loudspeaker is present, the acoustic echo canceller can comprise one filter for each loudspeaker.
The invention further provides the use of a dual microphone comprising a first directional microphone being oriented in a first direction and a second directional microphone being oriented in a second direction different from the first direction, the first and the second microphone being arranged on a common support frame, for forming a microphone array for a hands-free system for speech signal acquisition in a vehicular cabin, in particular, as described above.
Furthermore, a vehicular cabin is provided comprising a hands-free system as described above.
Further features and advantages of the invention will be described in the following with reference to the figures.
In the embodiment shown, the first microphone array is pointing in the direction of an expected speaker. The output of these first microphones is used to generate a primary signal xP(k).
The hands-free system further comprises a noise reduction device 4. A cancellation means 5 forms part of this noise reduction device 4. This cancellation means 5 serves to cancel a noise part from the primary signal xP. The noise reduction device 4 comprises a first input 6 for each first microphone 1.
A first signal path 7 connects the first inputs 6 and the cancellation means 5. On this first signal path 7, a beamformer 8 is provided that is a fixed beamformer (FBF) in the present example. As an example, the beamformer can be a delay-and-sum beamformer or a filter-and-sum ("superdirective") beamformer. However, an adaptive beamformer can be used as well. The first signal path 7 comprises sub-paths 7' connecting the first inputs 6 of the noise reduction device 4 and the beamformer 8.
Whereas the first microphones 1 are used for picking up a wanted speech signal, denoted by xS,1(k) and xS,2(k), the second microphones 3 are used for picking up noise signals only (or at least predominantly), denoted by xN,1(k) and xN,2(k). The noise reduction device 4 has inputs 9 for receiving the signals from the second microphones 3.
A second signal path 10 connects the second inputs 9 and the cancellation means 5. On the second signal path 10, adaptive filters 11 having transfer functions hN,1(k) and hN,2(k) are provided between both second inputs 9 and the cancellation means 5.
The adaptive filters 11 are used to filter the noise reference signals, wherein these filters are adapted to maximally cancel out residual noise parts of the primary signal xP. There are a variety of adaptation methods to update the adaptive filters 11. As an example, LMS, NLMS or RLS algorithms can be used.
A main advantage of the hands-free system as shown in
In the example shown in
Furthermore, the number of microphones in the first and the second microphone array need not be the same. Particularly, if the first directional microphones are intended for wanted speech signal acquisition whereas the second directional microphones for noise signal acquisition only, a microphone array with two or more microphones can be provided as the first microphones and less microphones, particularly only one directional microphone, as the at least one second directional microphone.
In the example illustrated in
In this figure, a dual microphone is schematically depicted. Such a dual microphone comprises a first directional microphone 1 and a second directional microphone 3, both microphones pointing in a different direction. Their pointing direction is indicated by arrows 2. The angle α between the pointing directions preferably lies between 90° and 180°, in particular, between 130° and 180°.
In this dual microphone, the two directional microphones are mounted on a common substrate (not shown). Furthermore, the illustrated example also comprises a common housing 12. In this way, a very compact arrangement of two directional microphones is obtained.
As shown in
The distance d between two neighboring dual microphones is to be chosen depending on different parameters such as type of beamformer used, distance between the microphone array and the expected position of a speaker's (such as a driver's) mouth, space restrictions for mounting the microphone arrays, etc. According to a typical example, the dual microphones can be arranged with a spacing of d = 7cm.
The angle α between the pointing directions can be chosen depending on the intended use of the microphone arrays. For example, if both microphone arrays are used to pick up speech in a car, the directional microphones are to be oriented in the direction of the speaker and the front seat passenger.
In this example, a third signal path 13 is provided which connects the first inputs 6 and the cancellation means 5. On this third signal path 13, a blocking matrix (BM) 14 and, in the direction of signal flow, a subsequent adaptive filter 15 with the transfer function hGSC(k) is provided. The blocking matrix 14 is configured to block wanted speech parts in the speech microphone signals xS,1(k) and xS,2(k). The resulting signal xB(k) is used as a noise reference signal.
Thus, together with the two microphone signals xN,1(k) and xN,2(k), three noise reference signals are present in total. Such a structure comprising a blocking matrix and a subsequent adaptive filter is called "generalized sidelobe canceller" (GSC). In cancellation means 5, the three noise reference signals are subtracted from the primary signal xP.
In the case of leakage of the speech signal into the noise reference signals, not only the noise portions of the reference signals are correlated with the primary signal, but also some speech portions of the noise reference signals. In this case, there is the risk that the adaptive filtering as illustrated in
In order to overcome this problem, a noise reduction device as shown in
Preferably, the filters 18 for reducing speech portions should be updated when the desired speaker is active. On the other hand, the noise filters 11 should not be updated when the desired speaker is active in order to prevent further signal cancellation. In view of this, an adaptation controller (not shown) can be provided comprising a speech detector for providing speech of a desired speaker to freeze or update the adaptive filters, respectively. Then, the speech filters 18 adapted only during speech activity of the desired speaker ("speech adaptive") whereas the noise filters 11 are adapted during noise periods ("noise adaptive").
A further example of a hands-free system is depicted in
It is to be pointed out that the primary signal and/or noise reference signals used in the examples can be weighted depending on the filtering along the respective signal path.
In the case of a multi-channel playback over several loudspeakers, an extra filter 19 is preferred for each playback channel in the echo canceller. For example, two filters are advantageous for stereo playback.
In the examples described above, the structure of the noise reduction device has always been depicted such that the first microphones are used to provide a primary signal, whereas the second microphones are responsible for yielding the noise reference signals. However, as the microphones can be provided in a symmetrical way, the noise reduction device can also be configured accordingly. In other words, the signal processing can be applied for signals either coming from the first or the second microphone array. In the case of a hands-free system mounted in a vehicular cabin, this allows to use the system for enhancing speech signals from both the driver and a front seat passenger, for example'.
When configuring the noise reduction device in this way, one can use the same beamformer and adaptive filters for both microphone arrays. In this case, a control means is to be provided controlling the signal flow from each microphone array to the appropriate elements of the noise reduction device for the specific case. For example, if the driver is speaking, a structure as shown in the figures would be present whereas if the front passenger is speaking, the roles of the microphone arrays and the corresponding connections within the noise reduction device to the filters are to be swept. In other words, in the latter case, the microphones pointing to the driver are used to pick up the noise reference signal, whereas the microphones pointing to the front seat passenger are used to pick up speech.
The control of the signal flow of the microphone signals to the respective filters of the noise reduction device may depend on a predetermined criterion. For example, the control may depend on the speech activity of both speakers; the microphone array picking up speech first will be switched to the beamformer, then, this configuration can be maintained until the first speech pause occurs. Alternatively, this signal flow configuration can be maintained permanently (for a predetermined time or until a manual reset is performed).
Furthermore, the hands-free systems shown in the figures and described above can comprise a second noise reduction being configured similarly as the first noise reduction device with the difference that the signals from the second directional microphones are fed to the first inputs and the signals from the first directional microphones are fed to the second inputs of the second noise reduction device. Particularly if the components and signal paths in the second noise reduction device are identical to those of the first noise reduction device, the first and second microphone signals are processed in the second noise reduction device in a converse way compared to the case of the first noise reduction device.
Preferably, such a hands-free system comprises also a signal mixer to combine the output signals of the first and the second noise reduction device to obtain a combined enhanced output signal. The mixing weights, i.e. the weights with which the signals are multiplied before being summed, can be controlled in a time-varying way depending on the speech activity detected for the first and second microphones. As an alternative example, the weights can be controlled such that a speaker who starts speaking first is put through permanently whereas the other speaker is blocked by setting the corresponding mixing weight to zero.
Further modifications and variations of the present invention will be apparent to those skilled in the art in view of this description. Accordingly, the description is to be construed as illustrative only and is for the purpose of teaching those skilled in the art on the general manner of carrying out the present invention. It is to be understood that the forms of the invention form and described herein are to be taken as the presently preferred embodiments.
This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.