TECHNICAL FIELD
[0001] The present invention relates to wearable microphone array apparatus comprising at
least two microphones within a garment, further comprising a control unit within the
garment connected with the microphones and adapted to receive signals from the microphones,
wherein the control unit is adapted to provide an output signal related to the localization
of a sound source upon reception of a sound signal by at least one of the microphones.
PRIOR ART
[0003] The publication from Ravi Shenoy discloses a way to implement a wearable microphone
array for sound localization. Five microphones are distributed over the surface of
a garment to provide electrical signals upon reception of a sound by the different
microphones. These electrical signals are sent to a low integrated electronic for
amplification of these signals. Analog electrical signals are used for signal transmission
between microphones and board. The amplified signals are then transmitted to an external
analog to digital conversion board connected to a desktop computer for future processing.
[0005] Although it is not mentioned within these documents, one important application of
such garments is providing a hearing aid for impaired people. These people more and
more request unnoticeable hearing aids which are especially not viewable by third
persons.
[0006] Such hearing aids that are able to filter the sound according to their incoming direction
are as such inherently limited by the spatial extent of the device, which means the
distance between microphones should compare with main wavelengths of a filtered sound,
which results in limited performance improvement.
[0007] In order to circumvent this physical limitation
EP 1 540 991 discloses hearing spectacles, but the use of microphones in spectacles limits the
possibilities of signal treatment.
[0008] EP 2 350 683 provides a wearable sound source localization device which is limited to gunshot
detection and is not included in a fabric but enclosed in an element placed on the
users shoulder.
SUMMARY OF THE INVENTION
[0009] Based on this prior art, the invention is intended to provide an improved approach
for a sound improvement device, which can especially also be used as hearing aid.
However, the application of the invention is not limited to such hearing aids.
[0010] The apparatus according to the invention comprises a reproduction device connected
with the control unit and adapted to reproduce a real-time signal to a user of the
apparatus. Said user can be the wearer or a distant user with remote access to the
information.
[0011] The invention comprises a piece of clothing, made of, for example, flexible material,
which can also be the fabric of an underwear or the felt of a headdress, that is used
as a holding structure for other elements. These other elements of the system are
arranged on the surface or inside the fabric and include two or more microphones collecting
sound (a mechanical wave-field of frequency from 1Hz to 10MHz) measures directly as
a digital signals or comprising an analog to digital converter for received analog
sound signals, a power source, a processing unit and a communication unit for a wired
or wireless transmittal of the data to other systems. Said microphones can be replaced
in the invention by any device able to measure the sound, notably but not limited
to, ultrasound-transducers or accelerometers. The connection between these elements
on the fabric are provided via conductive wires, weaved or hidden in the fabric itself.
All the elements interact in a way that the signal coming from the microphones are
processed in real time and can be sent to the user via the communication unit with
a minimal delay that is not noticeable for a human recipient. This is especially important
within the hearing aid application. It is also important for other applications where
the reaction of the human user who wears the apparatus or who remotely monitors the
output signal of such a device upon detection of a sound is intended.
[0012] The described invention can be used for a sound reproduction device which outputs
the received sound signal to an ear of the user. Such sound reproduction unit can
be placed near a user ear, inside the ear canal or implanted. Said sound reproduction
devices receive the sound signal from the transmission unit preferably wirelessly
but not limited to.
[0013] When the described invention is used as an input for a hearing aid then the processing
unit outputs one or two sound signals that are transmitted e.g. wireless by the communication
unit to the hearing aid usually placed inside or near the user's ear. The processing
unit is then able to detect the sound sources and sort them out by their direction
of arrival. The processing unit could also select a subset of sound sources that are
considered relevant for the user in the present situation and could selectively extract
the sound produced by the sources from the input sound, increasing by that means the
relevant signal-to-noise ratio (SNR) of the reproduced sound.
[0014] It is an advantage over known wearable microphone arrays, as mentioned above, for
implementing acoustic applications, that the present device is embedded in the fabric
in a way that the fabric can be worn under clothes without being noticeable.
[0015] The processing, powering and communication elements are embedded in the garment whereas
within the prior art documents the recording, processing and powering is mainly done
externally. Also these documents do not provide communication units integrated into
the system. The invention is autonomous and can thus be worn all day long by a user
without inconvenience.
[0016] The overall processing scheme satisfies the real time constraint; whereas the prior
art documents make no reference to real-time computing: the sound as received is recorded
and processed later.
[0017] In contrary to state of the art multi-microphone hearing aids where microphones are
distant of few centimetres, the large spatial extension of the disclosed apparatus
enables covering a wide spectrum of frequencies which improves the directionality
of the array and the intelligibility of recorded voices. A greater number of microphones,
e.g. 20 microphones, can be used, especially more than 5 to 10.
[0018] The distance between the microphones and the hearing aid in such an application avoids
the problem of feedback even when providing a highly amplified output signal for severely
hearing impaired people.
[0019] Within further preferred embodiments of the invention, such a device is used as a
sound sources localization device that enables the user wearing the system or a remote
user to monitor in real time the direction of arrival of a given sound. The communication
device sends the signals to a distance receiver and/or to a local reproduction device.
This reproduction device can be a touch screen worn by the user that displays the
localization of a sound source and characteristics or give a direct feedback, e.g.
through providing a stimulation signal in the direction of the sound. Such a stimulation
signal can preferably be combined with the electronic placement of the microphones
and may comprise a vibrating unit or a shock inducing unit, which alerts the user
from the direction of a sound source.
[0020] A device using the advantages of the invention comprises a fabric, a plurality of
transducers placed on, under or inside the fabric, one or more power supplies placed
on, under or inside the fabric that each powers one or more of the transducers and
at least one control unit as computing and signal processing unit which processes
in real-time at least one signal from the said transducers and outputs at least one
processed signals. The transducers are microphone units. The fabric can be a garment
and especially an underwear. A communication unit for transferring the processed signal
can be a RF unit, an infrared unit, Bluetooth unit or a Wifi unit. Such communication
can comprise a transmittal to a sound reproduction device, e.g, a hearing aid and
the communication can comprise the transmittal of information relating to the location
of the sound source to the user. In case of a hearing aid, the computing unit can
use the input microphone signals to generate an output sound signal that has preferred
directivity characteristics.
[0021] The reproduction device can comprise one or more actuators adapted to transmit a
tactile feedback signal on the inside of the garment towards the user. Such a signal
can be a vibration when the actuator is a vibrator. It can be an electric discharge,
if the reproduction device is an electrode. The actuator can also comprise an embossment
creating or pin extending unit, so that the user perceives this tactile feedback.
[0022] On the other hand, reproduction device can comprise a hearing aid, especially a wireless
hearing aid, which optionally receives control signals through a wireless interface
from a transmission unit connected with the control unit.
[0023] The control unit can comprise a speech-to-text conversion unit adapted to provide
an output signal based on speech or specific predetermined sound signals to a reproduction
device. Specific predetermined sound signals can be the horn signal of an ambulance,
the acoustic signal of a police car, the sound of an incoming shell, the ringing sound
of a telephone, of a door bell or the sound of any security relevant source.
[0024] The reproduction device can comprise a visual display screen, e.g. on a tablet computer,
which can be connected by wire or wireless to the control unit. Said control unit
then comprises a coding unit providing an output signal for a visual display of the
speech or specific predetermined sound signals.
[0025] The reproduction device can comprise an array of actuators forming a Braille cell,
e.g. an array of 2 times 3 or 2 times 4 actuators, which can be controlled at the
same time to transmit Braille coded information. Then of course the control unit comprises
a coding unit providing an output signal for a Braille representation of the speech
or specific predetermined sound signals within the Braille cell.
[0026] The reproduction device can comprise at least or only one actuator and then the control
unit comprises a coding unit providing an output signal for a Morse or another sequentially
coded representation of the speech or specific predetermined sound signals.
[0027] The reproduction device can also comprise a plurality of at least two, preferably
four or more actuators provided in a preferably similar or equal angularly spaced
relationship on the circumference of the garment, wherein the control unit comprises
a source localization unit and is adapted to actuate in real-time the actuator being
closest in view of the direction of a specific sound event, so that the user is incited
to turn himself towards the source of speech, especially if unwanted noise sources
are filtered out.
[0028] The apparatus according to the invention, especially when the reproduction unit is
either worn by the wearer or located in a distant place (e.g. at the command unit
in a military application case) comprises within the reproduction unit a visual display
screen that displays information related to the location of sound events optionally
together with characteristics of said sound events. It is also interesting to provide
for one reproduction unit that it receives the signal from several devices and merges
them to display an improved map of the sound events.
BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Preferred embodiments of the invention are described in the following with reference
to the drawings, which are for the purpose of illustrating the present preferred embodiments
of the invention and not for the purpose of limiting the same. In the drawings,
- Fig. 1
- shows a front side view of a shirt as schematical representation of a device according
to an embodiment of the invention;
- Fig. 2
- shows the shirt of Fig. 1 from the back side of the shirt;
- Fig. 3
- shows a front side view of a collar as schematical representation of a device according
to a further embodiment of the invention;
- Fig. 4
- shows the collar of Fig. 3 from the back side of the collar;
- Fig. 5
- shows a front side view of a shirt as schematical representation of a device according
to a further embodiment of the invention, based on the array distribution of Fig.
1;
- Fig. 6
- shows the shirt of Fig. 5 from the back side of the shirt;
- Fig. 7
- shows a block diagram of the connections of the device according to Fig. 1 as the
basic device; and
- Fig. 8
- shows a block diagram of the connections of the device according to Fig. 1 with control
elements realized within the signal processing unit for additional processing steps.
DESCRIPTION OF PREFERRED EMBODIMENTS
[0030] Fig. 1 shows a front side view of a shirt 10 as garment as a schematical representation
of a device according to an embodiment of the invention. The device comprises a plurality
of microphone units 20 distributed on the front side of shirt 10. They are provided
in three lines of three microphones 20 on the height of the torso area of the shirt
as well as comprising two microphone units 20 on the shoulder areas of the shirt.
[0031] A microphone unit 20 preferably comprises a microphone as such, as well as signal
processing elements to provide a pre-transformed electrical signal relating to the
detected sound as usual. The transmission is preferably done digitally in order to
improve noise immunity of the sound signal. This is especially important if the user
has a mobile communication device e.g. cell phone, wifi transmitter. Further noise
immunity can be obtained using differential signal e.g LVDS. The microphones or the
signal processing element part of the microphone units are connected via electrical
wires 21, shown as solid lines, with a specific signal combining signal processing
unit 30, which provides e.g. a localisation information of a specific sound from e.g.
a speaker as known from the prior art. The signal transmission is then prepared to
be transmitted to the transmission unit 40 through wire 31. Transmission unit 40 comprises
a wireless transponder, shown as the symbol 41 for wireless transfer to the remote
hearing aid, which is usually an ear loudspeaker, not shown in Fig. 1.
[0032] Power lines 22 connect a battery 23 with each of the microphones 20 as well as the
signal processing unit 30.
[0033] Fig. 2 shows the shirt 10 of Fig. 1 from the back side of shirt 10. As usual all
identical or similar features of the various embodiments receive the same of similar
reference numerals. The back side of shirt 10 also comprises eleven microphone units
20, three columns with three lines forming almost a rectangle on the back of shirt
10 as well as two additional shoulder microphone elements. These two elements are
provided almost on the shoulder but separate from the elements connected with the
front side microphones 20. It could be the case that these microphones on the shoulder
parts are provided quite near one another. The signal handling can be separated between
the signals issued by the back side and the front side of the shirt reducing the localization
effort of the signal processing unit 30 through previous filtering. It is preferred
that there is nevertheless one single battery for both arrays, the front and the back
array, and one single signal processing unit 30.
[0034] Fig. 3 shows a front side view of a collar 110 as schematical representation of a
device according to a further embodiment of the invention; and Fig. 4 shows the collar
110 of Fig. 3 from the back side of the collar 110;
[0035] Collar 110 has a superposing area 111, which can be used, e.g. with a velcro type
connection to close the collar 110 around the neck of a user, as shown in Fig. 3 and
4.
[0036] Here one array of microphones 20 is provided on the front side of the collar 110
in two lines of 4 and 6 microphone units 20. The microphone units are interconnected
and connected with the signal processing unit 30 via lines 21 as well as via power
lines 22 with the battery 23.
[0037] Fig. 5 shows a front side view of a shirt 10 as schematical representation of a device
according to a further embodiment of the invention, based on the array distribution
of Fig. 1. Fig. 6 shows the shirt of Fig. 5 from the back side of the shirt. Beside
the microphone units 20 which are disposed identically to the embodiment of Fig. 1
there are provided actuators 50.
[0038] The embodiment as shown in Fig. 5 and 6 shows a combination of two embodiments which
can be used separately or together. One embodiment uses two times two actuators 50
distributed on the front as well as on the back of the garment 10, as shown in Fig.
6. Beside these upper actuators 50 or replacing these actuators, within another embodiment,
there is provided a line of here 9 plus 9 actuator elements 50 around the waist line
of the garment 10. The actuator units 50 in the lower area provide a complete 360
degree coverage with actuators 50 in view of an user of this garment 10, almost in
regularly spaced intervals in a horizontal direction. Eighteen actuators 50 allow
an angle resolution of approximately 360/18 = 20 degrees with a slightly higher value
on the side of the body. It is also possible to use a different number of actuators,
like 4 plus 4 which gives roughly an angle resolution of 45 degrees (=360/8).
[0039] The advantage of the use of these actuators 50 becomes clear in view of possible
applications. Profoundly hearing impaired people are not able to understand speech
if they cannot use side channel information as lipreading. The microphone array with
microphone units 20 analyses the incoming sound, detects the interesting source like
speech with prior art techniques of sound analysis and can identify the direction
where the speaker is standing. Then the actuator 50 in the direction of the source
is activated and performs his function.
[0040] Actuators 50 can comprise a vibrator and as such give a tactile feedback on the skin
or the under garment of the user. It can also provide a different answer, like a heat
pulse, a pin moving in direction of the skin of the user extending over the casing
of the actuator 50. A contact actuator 50 is a transceiver that can transmit information
to the wearer being in contact with its skin, e.g. a vibration device or an electrode
are two preferred embodiments. When the two times two actuators 50 are used, it is
also possible to provide height information of the sound source, when the upper or
lower actuators 50 are activated.
[0041] Within one embodiment the sound source is analysed for the presence of speech and
the speech is converted into the corresponding text message by the signal processing
unit 30, 38 based on the different incoming signals from the microphone units 20 over
the wires 21. This text message is then transmitted to the actuators 50 with a predetermined
coding. This can be a Morse code on one or more specific actuators 50 or an array
of two times three actuators 50 (two columns, three lines; not shown in the drawings;
or eight actuators with a fourth additional line) is used as the usual known Braille
coding to communicate the spoken words. It is also possible that the signal processing
unit 30 provides a cable or wireless 41 link to a screen, e.g. on a tablet computer
for a visual representation of the spoken words.
[0042] The aim of the actuators 50 is therefore at least twofold. The above mentioned possibility
provides identification of a speaker, indication of the direction to the user and
providing a coded (Braille, Morse as tactile or by visual means) signal on the spoken
information. The processing unit 30 can also ensure that the speaker in front of the
user wearing the garment 10 is the sound source which is used for speech decoding,
since the microphone units 20 allow to distinguish this source in view of the orientation
of the garment 10.
[0043] A further advantage of the system incorporated in the processing unit 30 is the localization
of possibly dangers. The microphone units on the front, the back and different heights
on the garment 10 allow a better analysis of incoming sound signals for the processing
unit 30 which can also be programmed to discern different noise patterns, e.g. the
signal of a police or ambulance, the specific signal of a tramway to attract the user's
attention to such surrounding dangers. This can also include shouting people to warn.
This is specifically useful, if such a sound source is localized at the back of the
user so that activation of the backwards oriented actuators 50 provides a direct indication
to a possibly problematic sound source in this direction.
[0044] There are also other applications as sound localization on the battlefield. Sound
information is vital for soldiers on the field as it may enable to localize a hidden
enemy who is shooting. This localization can be especially problematic with the ears
because the direction of the shock wave created by a supersonic bullet does not directly
point towards the shooter, inducing a moment of confusion that can be fatal. Additional
problems are created by reverberant environment that prevent the localization by human
ears. Electronics means can enable an accurate and instantaneous localization through
the use of the array of microphone units 20. However, the spatial extension of the
microphone array (radius bigger than 50cm) required for an accurate localization is
unacceptable to mount on a firearm, or to carry on a battlefield. The present innovation
can be used to overcome this problem by embedding the array of microphones 20 into
the garments 10 of the soldier as the user so as to benefit from a large spatial extent
as well as a reduced space requirement.
[0045] The device includes a reproduction part that enables a real-time and easy understanding
of the acoustic scene. The preferred embodiment here includes a set of incorporated
contact actuators 50 arranged on a circular way so as to cover all directions in view
of a standing person. When a sound is localized by the signal processing unit 30,
the actuator 50 that corresponds to the direction is activated.
[0046] Another application related to sound localization on the battlefield is to distantly
monitor the position of the enemies. The localization of the gunshots given by the
device worn by a soldier is transmitted to a control operation centre together with
soldier location information e.g. GPS position of the soldier, and displayed on a
real-time updated map. In this case, the reproduction device of the disclosed apparatus
includes e.g. a screen displaying e.g. a map of the battlefield. The information from
several apparatus can be further combined to be displayed on a single map.
[0047] It is noted that the signal wires 51 for the actuators 50 can use or be identical
to the signal wires 21 of the microphone units 20, since they are both connected to
the signal processing unit 30. It is also possible to combine the power wires 22 and
52.
[0048] In an embodiment not shown in the drawings, microphone units 20 and actuators 50
are combined on the same support, i.e. a printed circuit board. The actuators 50 can
also be provided within the microphone units 20 within the same chip element.
[0049] Fig. 7 shows a block diagram of the connections of the device according to Fig. 1
as a basic device and Fig. 8 shows a block diagram of the connections of the device
according to Fig. 1 with control elements realized within the signal processing unit
30 for additional processing steps. Same reference numerals are used in the block
diagram of Fig. 7 and 8 for elements provided in the actual garment 10.
[0050] The power unit 23 comprises usually a battery and sometimes a converter. The power
unit 23 is connected via power wires 22 to the array of microphone units 20, the processing
unit 30 and, if provided the transmission unit 40. The signal transmission is realized
via wires 21 from the microphones 20 to the signal processing unit 30 and from the
signal processing unit 30 towards the optional transmission unit 40 via wire 31. The
signal wires 21, 31 (and 51) can be replaced by wireless transmission means.
[0051] Within Fig. 8, the signal wires 21 from the microphone units 20 are provided to the
signal processing unit 30 of Fig. 7 which comprises a source localization unit 35,
a relevance estimation unit 36, a beamformer 37, a speech-to-text conversion unit
38, a coding unit 39 and a reproduction device 130.
[0052] As explained above the signal processing unit 30 comprises a source localization
unit 35 which provides as an output on signal wire 135 an information relative to
the angular position of the sound source e.g. azimuth and altitude angles relatively
to the user wearing the garment 10. A relevance estimation unit 36 also receives the
source sound signal and the source localization signal. The relevance estimation unit
36 determines the relative relevance of the signals, depending on the conditions.
The voice of an interlocutor should be magnified whereas other conversations from
other groups of person should be suppressed (known in the prior art as the cocktail
party problem). Outside of buildings an emergency signal from an ambulance is an important
relevant sound source whereas indoor, it is not the case. This relevancy signal is
transmitted via wire 136 to the beamformer unit 37. The beamformer unit or spatial
filtering unit 37 is a signal processing technique used in sensor arrays for directional
signal transmission or reception. This is achieved by combining elements in the array
in such a way that signals at particular angles experience constructive interference
while others experience destructive interference. Beamforming is used on the receiving
end in order to achieve spatial selectivity and provides an important receive/transmit
gain. The combination of units 35, 36 and 37 relates to an adaptive beamforming used
to detect and estimate the signal-of interest at the output of the microphone units
20 by means of a spatial filtering and interference rejection. For the spatial filtering
a least-squares method can be used.
[0053] The thus optimised incoming sound signal over wires 21 is forwarded via wire 137
to the speech-to-text conversion unit providing the spoken words in a computer understandable
form. These elements are then forwarded to a coding unit 39, which codes the information
in a way which is understood by the reproduction device 130. This can comprise a 2x3
or 2x4 Braille coding for transmittal to the actuators 50 or a simpler Morse-code
transfer to a single or a specific number of actuators 50 for sequential transmittal
of the spoken words or as a coding for a visual transfer into ASCII-codes for display
on a screen as reproduction device 130. Furthermore hearing aids for the ear, either
implanted or to be introduced into the ear canal are such reproduction devices 130.
In such a case, preferably a wireless transmitter unit 40 is used but a wire transfer
as line 139 is also possible.
LIST OF REFERENCE SIGNS
[0054]
| 10 |
garment |
41 |
wireless transfer |
| 20 |
microphone unit |
50 |
actuator |
| 21 |
signal wire |
51 |
signal wire |
| 22 |
power wire |
52 |
power wire |
| 23 |
power source |
110 |
Collar |
| 30 |
signal processing unit |
111 |
superposing area |
| 31 |
transmission transfer wire |
130 |
reproduction device |
| 35 |
source localization unit |
135 |
signal wire |
| 36 |
relevance estimation unit |
136 |
signal wire |
| 37 |
beamformer |
137 |
signal wire |
| 38 |
speech-to-conversion unit |
138 |
signal wire |
| 39 |
coding unit |
139 |
signal wire |
| 40 |
transmission unit |
|
|
1. Wearable transducer array apparatus comprising at least two transducers (20), within
a garment (10), further comprising a control unit (30) within the garment (10) connected
with the transducers (20) and adapted to receive signals (21) from the transducers
(20), wherein the control unit (30) is adapted to provide an output signal related
to the localization (31; 35, 36, 37; 135) of a sound source upon reception of a sound
signal by at least one of the transducers (20), characterized in that apparatus comprises at least one reproduction device (50; 130) connected (31, 40,
41; 139; 51) with the control unit (30) and adapted to reproduce a real-time signal
to a user of the apparatus.
2. Apparatus according to claim 1, wherein transducers are microphones (20).
3. Apparatus according to claim 1 or 2, wherein the transmission of the signals (21)
between the transducers (20) and the control unit (30) is digital, either differential
or not differential.
4. Apparatus according to one of claims 1 to 3, wherein the reproduction device (50;
130) comprises at least one actuator (50) provided within the garment (10) adapted
to transmit a tactile feedback signal on the inside of the garment (10), wherein optionally
the actuator (50) is a vibrator or comprises an embossment creating or pin extending
unit.
5. Apparatus according to any one of claims 1 to 4, wherein the reproduction device (130)
comprises at least one sound reproduction device which outputs the received sound
signal to one or both ears of the user.
6. Apparatus according to claim 5, wherein the sound reproduction device is a hearing
aid, especially a wireless hearing aid, optionally receiving control signals through
a wireless (41) interface from a transmission unit (40) connected with the control
unit (30).
7. Apparatus according to any one of claims 1 to 6, wherein the reproduction unit comprises
a visual display screen that displays information related to the location of sound
events optionally together with characteristics of said sound events.
8. Apparatus according to any one of claims 1 to 6, wherein the reproduction unit is
adapted to receive signals from several devices and merges them to display an improved
map of the sound events.
9. Apparatus according to any one of claims 1 to 8, wherein the control unit (30) comprises
a speech-to-text conversion unit (38) adapted to provide an output signal (138) based
on speech or specific predetermined sound signals to a reproduction device (130).
10. Apparatus according to claim 9, wherein reproduction device (130) comprise a visual
display screen and wherein the control unit (30) comprises a coding unit (39) providing
an output signal (139) for a visual display of the speech or specific predetermined
sound signals.
11. Apparatus according to claim 9 or 10, wherein the reproduction device (50; 130) comprises
an array of actuators (50) forming a Braille cell, and wherein the control unit (30)
comprises a coding unit (39) providing an output signal (139) for a Braille representation
of the speech or specific predetermined sound signals within the Braille cell.
12. Apparatus according to claim 9 or 10, wherein the reproduction device (50; 130) comprises
at least one actuator (50) and wherein the control unit (30) comprises a coding unit
(39) providing an output signal (139) for a Morse or another sequentially coded representation
of the speech or specific predetermined sound signals.
13. Apparatus according to any one of claims 1 to 12, wherein the reproduction device
(50; 130) comprises a plurality of at least two, preferably eight actuators (50) provided
in a preferably similar or equal angularly spaced relationship on the circumference
of the garment, wherein the control unit (30) comprises a source localization unit
(35) and is adapted to actuate in real-time some of the actuators (50) depending on
the location of specific sound events, especially those being closest in view of the
direction of a specific sound event.