|
(11) | EP 2 988 527 A1 |
| (12) | EUROPEAN PATENT APPLICATION |
|
|
|
|
||||||||||||||||||||
|
|||||||||||||||||||||
| (54) | System and method for detecting location of sound sources in a three-dimensional space |
| (57) System and method for detecting location of sound sources in a three-dimensional
space using linear microphone arrays comprising a plurality of microphones wherein
the microphones are located in at least two groups of at least two microphones wherein
each group has a different spacing of the respective microphones. Three such linear
microphone arrays are used wherein first ends of all three microphone arrays, comprising
the same arrangement of microphones, are in proximity or adjacent to each other; and
the separate microphone arrays are positioned in different planes in three-dimensional
space.
|
[0001] The present invention relates to a system and method for detecting location of sound sources in a three-dimensional space. In particular the present invention relates to detecting an angle at which a sound source is located with respect to a linear microphone array.
[0002] Prior art defines US 7090117 B2 entitled "Microphone array and digital signal processing system" disclosing a digital microphone array is configured in an open geometry such as a sphere with a large number of inexpensive microphone elements mounted in opposite-facing pairs. The microphone array with DSP is intended to be placed in a three-dimensional sound field, such as a concert hall or film location, and to completely isolate all sound sources from each other while maintaining their placement in a coherent sound field including reverberance.
[0003] It would be advantageous to provide a simpler microphone array capable of executing sound localization.
[0004] A publication of EP2592846A1 discloses a method and an apparatus for processing signals of a spherical microphone array on a rigid sphere used for generating an Ambisonics representation of the sound field, wherein an equalisation filter is applied to the inverse microphone array response.
[0005] The aim of the development of the present invention is an improved, more accurate and resources cost effective system and method for detecting location of sound sources in a three-dimensional space.
[0006] An object of the present invention is a linear microphone array comprising a plurality of microphones wherein the microphones are located in at least two groups of at least two microphones whereas each group has a different spacing of the respective microphones.
[0007] Preferably, there are five groups of microphones each comprising at least two microphones wherein spacing of respective microphones in groups is such that in a subsequent group the spacing is twice of that of the preceding group.
[0008] Preferably, there are five groups of microphones and that the first group comprises seventeen microphones, while the remaining four groups comprise eight microphones each.
[0009] Another object of the present invention is a linear microphone system using three linear microphone arrays according to the first object of the present invention, the system being having first ends of all three microphone arrays, comprising the same arrangement of microphones, are in proximity or adjacent to each other; and the separate microphone arrays are positioned in different planes in three-dimensional space.
[0010] Preferably, the other ends of the microphone arrays linearly extend on X, Y and Z axis respectively.
[0011] Further object of the present invention is a method for sound source localization using a microphone array, the method comprising the steps of: positioning the system according to the second object of the present invention in a detection area; assigning each group of microphones within each microphone array to a non-overlapping frequency band wherein the higher the frequency the lower the spacing of microphones; for each of the microphone arrays executing the steps of: filtering sounds from each microphone with band-pass filters into sub-bands; selecting active microphones depending on the selected sub-band that is associated with microphones spacing; selecting, for the selected active microphones and sub-band, appropriate samples wherein the higher the sub-band frequency the more samples are selected whereas sampling frequency is greater than the frequency of the sampled sub-band having the highest frequency; selecting angular sampling density based on band frequency; calculating a delay, for each value of angle α within a range of -90° to +90°, with which sound will arrive to each microphone from a given direction assuming a distance from a sound source is infinite; calculating a sample of sound for a given direction by adding sound of all active microphones taking delays into account for a given angle; calculating signal strength arriving from each of the tested directions by summing absolute values of N directional samples thereby obtaining signal strength curve values; detecting local maxima and assuming them as detection result; using sound source angles obtained for each of the microphone arrays in order to determine sound source localization by means of triangulation.
[0012] Another object of the present invention is a computer program comprising program code means for performing all the steps of the computer-implemented method according to the present invention when said program is run on a computer.
[0013] Another object of the present invention is a computer readable medium storing computer-executable instructions performing all the steps of the computer-implemented method according to the present invention when executed on a computer.
[0014] These and other objects of the invention presented herein are accomplished by providing a system and method for detecting location of sound sources in a three-dimensional space. Further details and features of the present invention, its nature and various advantages will become more apparent from the following detailed description of the preferred embodiments shown in a drawing, in which:
Fig. 1 shows a microphone array;
Fig, 2 shows activation scheme of microphone arrays;
Figs. 3A-B depict a microphone array system;
Fig. 4 presents a diagram of the method according to the present invention;
Fig. 5 depicts an exemplary division of an acoustic band;
Fig. 6 presents an example of an answer of the detector;
Fig. 7 presents a diagram of the system according to the present invention; and
Fig. 8 shows an installation of the system in a room.
NOTATION AND NOMENCLATURE
[0015] Some portions of the detailed description which follows are presented in terms of data processing procedures, steps or other symbolic representations of operations on data bits that can be performed on computer memory. Therefore, a computer executes such logical steps thus requiring physical manipulations of physical quantities.
[0016] Usually these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. For reasons of common usage, these signals are referred to as bits, packets, messages, values, elements, symbols, characters, terms, numbers, or the like.
[0017] Additionally, all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Terms such as "processing" or "creating" or "transferring" or "executing" or "determining" or "detecting" or "obtaining" or "selecting" or "calculating" or "generating" or the like, refer to the action and processes of a computer system that manipulates and transforms data represented as physical (electronic) quantities within the computer's registers and memories into other data similarly represented as physical quantities within the memories or registers or other such information storage.
[0018] A computer-readable (storage) medium, such as referred to herein, typically may be non-transitory and/or comprise a non-transitory device. In this context, a non-transitory storage medium may include a device that may be tangible, meaning that the device has a concrete physical form, although the device may change its physical state. Thus, for example, non-transitory refers to a device remaining tangible despite a change in state.
DESCRIPTION OF EMBODIMENTS
[0019] A microphone array according to the present invention comprises, as shown in Fig. 1, a supporting body 101 and linearly, spatially located microphones 102A-L wherein the microphones are located in at least two groups 103A-C of at least two microphones whereas each group has a different spacing of the respective microphones.
[0020] The microphones 102 are preferably located on a straight line such that a first group of microphones comprises microphones spaced by for example 6,25mm, the second group of microphones comprises microphones spaced by for example 12,5mm, the third group of microphones comprises microphones spaced by for example 25mm, the fourth group of microphones comprises microphones spaced by for example 50mm and the fifth group of microphones comprises microphones spaced by for example 100mm. Therefore, there are five groups each comprising at least two microphones wherein spacing of respective microphones in groups is such that in subsequent group the spacing is for example twice of that of the preceding group.
[0021] Preferably, the first group comprises 17 microphones, while the remaining four groups comprise eight microphones each. This number is a preferred arrangement as shown by experiments and evaluation of response curve at different numbers of microphones in arrays.
[0022] Such an arrangement is due to different geometry spacing of the microphones for different sound frequencies. A general rule is that the frequency bands of the microphones groups are non-overlapping and the higher the frequency the lower the spacing of microphones. Therefore, for sounds up to 1 KHz there are used 17 microphones spaced by 100 mm. For band 1..4kHz there may be used 17 microphones spaced by 50 mm, for band 4..8kHz there may be used 17 microphones spaced by 25mm, for band 8..15kHz there may be used 17 microphones spaced by 12.5 mm and for band over 15kHz there may be used 17 microphones spaced by 6.25 mm.
[0023] Fig. 2 shows five microphone arrays 203A - 203E according to the present invention, wherein active microphones are marked with a thick border 201 and inactive microphones are marked with a thin border 202. For the sake of simplicity and readability of the drawing, the microphones are equally spaced on the drawing but in reality are spaced according to the spacing factor X to 16X as indicated in the figure.
[0024] The microphone array 203A on the bottom is configured for band up to 1 kHz, the microphone array 203B is configured for a band 1..4kHz, the microphone array 203C is configured for a band 4..8kHz, the microphone array 203D is configured for a band 8..15kHz and the microphone array 203E is configured for a band over 15kHz. The configuration denotes herein a selection of particular active microphones whereas being active means that samples from this microphone will be taken into account in signal analysis.
[0025] A single linear microphone array according to the present invention allows for determining an angle, at which sound sources are located with respect to this linear microphone array. In order to obtain a full information regarding location of sound sources in 3D space, it is necessary to apply at least three microphone arrays.
[0026] The microphone arrays must be spaced for example by 90 degrees wherein first ends of all microphone arrays (comprising the same arrangement of microphones) are in proximity or adjacent to a virtual center of a circle as shown in Fig. 3A. Fig. 3A shows a view in a single plane but the separate microphone arrays must be positioned in different planes in 3D space. Preferably, the other ends of microphone arrays linearly extend on X, Y and Z axis respectively (for example forming three edges of a cube as shown in Fig. 3B). Such a microphone system may be located in a corner of a room near the ceiling.
[0027] Having three detected angles (one per each microphone array), such a microphone system is able to determine location of each detected sound source by means of triangulation.
[0028] Depending on frequency of sounds to be detected it is necessary to apply appropriate sampling frequency, which is greater than the high frequency of the sampled band in case the frequency is specific or greater than the low frequency of the sampled band in case the band is and open range eg. above 15KHz. Hence for sounds up to 8kHz the sampling frequency is for example 48kHz, for sounds in band 8..15kHz the sampling frequency is for example 96kHz and for sounds above 15kHz the sampling frequency is for example 192kHz.
[0029] Typically, the analysis is executed in a full band, which requires a sampling frequency of 192kHz. For a band over 15kHz all samples will be analyzed, for a band in range of 8..15kHz every second sample will be analyzed while for a band below 8kHz every fourth sample will be analyzed (in order to decrease samples frequency). Similarly, all microphones will be sampled but only samples from the active microphones will be subject to sub-band analysis (in case of sampling frequency there is a different division into groups that in case of microphones spatial location).
[0030] Fig. 4 presents a diagram of the method according to the present invention wherein the system of Fig. 3B is applied. The method starts at step 401 from filtering sounds from each microphone with band-pass filters. Fig. 5 depicts an exemplary division of an acoustic band. All bands are analyzed simultaneously, therefore sampled. However, for a given sub-band analysis there are selected only microphones that are appropriate for the selected band. Samples from some microphones will be used only for a single sub-band while some other will be used for more sub-bands, for example the edge microphone, spaced by 6,25 mm from another, will be used in all sub-bands. The appropriate microphones are selected at step 402.
[0031] Selection of appropriate microphones depends on the selected sub-band that is associated with microphones spacing. In Fig. 2 there are five bands into which a sub-band falls thereby defining the active microphones. For example a sub-band 12 of 10-11 KHz falls into the band 8..15kHz therefore spacing of active microphones is assumed as 203D and these microphones are appropriate for this sub-band.
[0032] Next, at step 403, for the selected set of microphones and sub-band there are selected appropriate samples (eg. all in case of over 15kHz band, every second sample for a band of 8..15kHz and every fourth sample for a band below 8kHz).
[0033] Further, at step 404 there is selected angular sampling density based on the band frequency. In particular, for each sub-band there is executed detection of sound sources. A round angle is divided into kk sections (selection of an angle between samples). A division coefficient kk may have a value of 360 except for band 0..500Hz wherein the preferred value of kk=60 and 7..8kHz band wherein the preferred value of kk = 120.
[0034] For each value of angle α within a range of -90° to 90° there is, at step 405, calculated a delay, with which sound will arrive to each microphone from a given direction (it is assumed at this stage that a distance from a sound source is infinite; under such an assumption the delay of sound between microphones depends only on direction). A reference point is the leftmost microphone. For angle -90° the delay will have a value calculated by the following equation:
where dx is a distance between a given microphone and a reference microphone and Vs is a speed of sound. Next there is calculated a coefficient of delay dependent on sampling:
where Fs is a sampling frequency and subsequently:
where α is an angle selected for evaluation.
[0035] In order for the values of delay to be above zero, for sounds from the left the leftmost microphone shall be taken as a reference microphone and for sounds from the right, the rightmost microphone shall be taken as a reference microphone.
[0036] Subsequently, at step 406, there is calculated a sample of sound for a given direction by adding sound of all 17 microphones taking delays into account (only 17 microphones are active regardless of the sampled frequency). For the reference microphone the delay equals 0 and for the remaining microphones it is derived from their distance to the reference microphone and the angle at which sound approaches.
where Mt is a sound sample, i is the number of the microphone (wherein the first microphone is the reference microphone (leftmost or rightmost) and the second is the next active microphone etc.) and t is the number of a sound sample for the reference microphone.
[0037] For i-th microphone there is taken into account, as the added value, a sample delayed by dt3 samples with respect to the reference microphone.
[0038] In this manner there is obtained, for each direction, a set of sound samples. It is equivalent of a directional microphone separately for each direction.
[0039] A further step is addition of absolute values of sound samples for all directions. The number of samples should be assumed as a result of experiments (verifying accuracy of detection), for example it may be N=200 samples.
[0040] As a result there is obtained, at step 407, a vector of data comprising information on signal strength arriving from each of the tested directions. A sample graphical representation of the signal strength data is shown in Fig. 6. The plot 601 shows two local maxima 602, 603 that indicate two sound sources: one for an angle of about 45° and the other for an angle of about 135°.
[0041] The curve 601 may be filtered in order to easier find the local maxima. The bottom part of the plot may be discarded as noise. It may be assumed that data for further analysis shall be above 1/3 of the maximum value.
[0042] Subsequently, it is determined for which angles local maxima are present. These angles are the result of identification of directions from which sound source emit sound (this process determines direction for a single microphone array), step 408.
[0043] It needs to be taken into account that typical sound sources eg. humans emit sounds having different frequencies. Since the analysis is executed in sub-bands it may be confirmed that a given detection is accurate by comparing results in sub-bands 409. In case a sound source is present in a plurality of sub-bands where the angles are similar, a resulting angle may be calculated as a weighted average where the lowest sub-band has a lower weight than the other sub-bands. It is to be noted that typically not all sub-bands will be taken into account as a sound source typically does not emit sound in all of them.
[0044] The method according to the present invention allows for detection of an angle at which a sound source is present with respect to a microphone array. Since the quality of detection depends on microphones placement and sound frequencies, it is necessary to apply division into sub-bands and selection of microphones for each sub-band. Three such microphone arrays allow to detect a sound source in a three-dimensional space with respect to the microphone system.
[0045] After obtaining three angles: alpha, beta and gamma a final sound source location may be determined. For the purpose of this example, there is assumed an arrangement in proximity to the ceiling (Fig. 8 shows an installation of the system in a room) of a room where one microphone array is parallel to the floor in X axis and the angle it detects is alpha. Similarly, the second microphone array is also parallel to the floor but in the Y axis and the angle it detects is beta. Lastly the third microphone array is perpendicular to the floor and hence the other microphone arrays and the angle it detects is gamma. Thereby the system operates on Cartesian coordinate system. The alpha, beta and gamma angles point a location in 3D space from which sound arrives. The zero point is the location of the microphone arrays system. Each of the alpha, beta, gamma angles denotes a plane in a 3D space whereas the planes intersect at a point wherein the sound source is located.
[0046] Fig. 7 presents a diagram of the system according to the present invention. The system comprises the microphone array arrangement 702 shown in Fig. 3 and an appropriate sampling module 703 managed by a controller 705.
[0047] The system may be realized using dedicated components or custom made FPGA or ASIC circuits. The system comprises a data bus 701 communicatively coupled to a memory 704. Additionally, other components of the system are communicatively coupled to the system bus 701 so that they may be managed by the controller 705.
[0048] The memory 704 may store computer program or programs executed by the controller 705 in order to execute steps of the method according to the present invention.
[0049] Therefore, the controller 705 is configured to executed step of the method described with reference to Fig. 4.
[0050] The present invention results in a useful determination of sound location that may for example be used in surveillance systems. Such results are concrete and tangible thus not abstract. Therefore, the invention provides a useful, concrete and tangible result.
[0051] According to the present invention data acquired by different microphones are processed within a dedicated machine. Hence, the machine or transformation test is fulfilled and that the invention is not abstract.
[0052] It can be easily recognized, by one skilled in the art, that the aforementioned method for detecting location of sound sources in a three-dimensional space may be performed and/or controlled by one or more computer programs. Such computer programs are typically executed by utilizing the computing resources in a computing device. Applications are stored on a non-transitory medium. An example of a non-transitory medium is a non-volatile memory, for example a flash memory or volatile memory, for example RAM. The computer instructions are executed by a processor. These memories are exemplary recording media for storing computer programs comprising computer-executable instructions performing all the steps of the computer-implemented method according the technical concept presented herein.
[0053] While the invention presented herein has been depicted, described, and has been defined with reference to particular preferred embodiments, such references and examples of implementation in the foregoing specification do not imply any limitation on the invention. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader scope of the technical concept. The presented preferred embodiments are exemplary only, and are not exhaustive of the scope of the technical concept presented herein.
• the microphones are located in at least two groups (103A-C) of at least two microphones whereas each group has a different spacing of the respective microphones.
• first ends of all three microphone arrays, comprising the same arrangement of microphones, are in proximity or adjacent to each other; and
• the separate microphone arrays are positioned in different planes in three-dimensional space.
• positioning the system according to claim 4 in a detection area;
• assigning each group of microphones within each microphone array to a non-overlapping frequency band wherein the higher the frequency the lower the spacing of microphones;
• for each of the microphone arrays executing the steps of:
• filtering (401) sounds from each microphone with band-pass filters into sub-bands;
• selecting active microphones (402) depending on the selected sub-band that is associated with microphones spacing;
• selecting (403), for the selected active microphones and sub-band, appropriate samples wherein the higher the sub-band frequency the more samples are selected whereas sampling frequency is greater than the frequency of the sampled sub-band having the highest frequency;
• selecting (404) angular sampling density based on band frequency;
• calculating a delay (405), for each value of angle α within a range of -90° to +90°, with which sound will arrive to each microphone from a given direction assuming a distance from a sound source is infinite;
• calculating (406) a sample of sound for a given direction by adding sound of all active microphones taking delays into account for a given angle;
• calculating (407) signal strength arriving from each of the tested directions by summing absolute values of N directional samples thereby obtaining signal strength curve values;
• detecting (408) local maxima and assuming them as detection result;
• using sound source angles obtained for each of the microphone arrays in order to determine sound source localization by means of triangulation.
where dx is a distance between a given microphone and a reference microphone and Vs is a speed of sound, whereas a coefficient of delay dependent on sampling equals:
where Fs is a sampling frequency; whereas the calculated delay equals:
where α is an angle selected for evaluation.
• a microphone array;
• a data bus (701) communicatively coupling components of the system;
• a memory (704) for storing data;
• a controller (705);
• a sampling module (703);
the system being characterized in that it comprises:
• the microphone array system (702) according to claim 2;
• whereas the controller (705) is configured to control the sampling module (703) and to execute all steps of the method according to claim 6.
Amended claims in accordance with Rule 137(2) EPC.
• the microphones are located in at least two groups (103A-C) of at least two microphones whereas each group has a different spacing of the respective microphones;
• wherein spacing of respective microphones in groups is such that in a subsequent group the spacing is twice of that of the preceding group;
• wherein the first group comprises seventeen microphones, while the remaining groups comprise eight microphones each;
• whereas the microphones are configured to form at least two non-overlap-ping, frequency subbands, thereby forming at least two subband microphone groups (203A - 203E), each consisting of seventeen microphones, such that each of the subsequent subband microphone groups includes at least one microphone from each of the preceding subband microphone groups whereas spacing of microphones, assigned to a given subband microphone group, is equal.
• 6,25mm if its associated subband is higher than 15kHz or
• 12.5mm its associated subband is above 8kHz but below 15kHz or
• 25mm if its associated subband is above 4KHz but below 8kHz or
• 50mm if its associated subband is above 1 KHz but below 4kHz.
• first ends of all three microphone arrays, comprising the same arrangement of microphones, are in proximity or adjacent to each other; and
• the separate microphone arrays are positioned in different planes in three-dimensional space.
• positioning the system according to claim 4 in a detection area;
• assigning each group of microphones within each microphone array to a non-overlapping frequency subband wherein the higher the frequency the lower the spacing of microphones;
• for each of the microphone arrays executing the steps of:
• filtering (401) sounds from each microphone with band-pass filters into sub-bands and for each non-overlapping frequency sub-band:
• selecting active microphones (402) depending on the selected non-overlapping frequency sub-band that is associated with respective microphones spacing;
• selecting (403), for the selected active microphones and sub-band, appropriate audio samples, from respective microphones, wherein the higher the sub-band frequency the more samples are selected whereas sampling frequency is greater than the frequency of the sampled non-overlapping frequency sub-band having the highest frequency;
• selecting (404) angular sampling density based on band frequency;
• calculating a delay (405), for each value of angle α within a range of -90° to +90°, with which sound will arrive to each microphone from a given direction assuming a distance from a sound source is infinite;
• calculating (406) a sample of sound for a given direction by adding sound of all active microphones, for the selected non-overlapping frequency sub-band, taking delays into account for a given angle;
• calculating (407) signal strength arriving from each of the tested directions by summing absolute values of N samples thereby obtaining signal strength curve values;
• detecting (408) local maxima and assuming them as detection result;
• using sound source angles obtained for each of the microphone arrays in order to determine sound source localization by means of triangulation.
where dx is a distance between a given microphone and a reference microphone and Vs is a speed of sound, whereas a coefficient of delay dependent on sampling equals:
where Fs is a sampling frequency; whereas the calculated delay equals:
where α is an angle selected for evaluation.
• a microphone array;
• a data bus (701) communicatively coupling components of the system;
• a memory (704) for storing data;
• a controller (705);
• a sampling module (703);
the system being characterized in that it comprises:• the microphone array system (702) according to claim 1;
• whereas the controller (705) is configured to control the sampling module (703) and to execute all steps of the method according to claim 6.
REFERENCES CITED IN THE DESCRIPTION
This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.
Patent documents cited in the description