TECHNICAL FIELD
[0001] The present application relates to the field of hearing aids.
SUMMARY
[0002] In an aspect of the present application, a method of improving environmental awareness
in a hearing aid is provided. The method can comprise determining, based on movement
data, a movement parameter indicative of a movement of the hearing aid. The method
can comprise determining whether the movement parameter meets a movement criterion.
The method can comprise determining whether an output unit of the hearing aid is outputting
an audio. The method can comprise obtaining, by at least one microphone of the hearing
aid, audio data indicative of an environment around the hearing aid. In accordance
with a determination that the movement parameter meets the movement criterion and
that the output unit is outputting audio, the method can comprise determining, based
on the audio data, an object parameter indicative of a moving object in the environment.
In accordance with a determination that the movement parameter meets the movement
criterion and that the output unit is outputting audio, the method can comprise determining
whether the object parameter meets a danger criterion. In accordance with a determination
that the object parameter meeting a danger criterion, the method can include modifying
the audio being output by the output unit.
[0003] In accordance with a determination that the movement parameter does not meet the
movement criterion or that the output unit is outputting audio, the method can comprise
not determining, based on the audio data, an object parameter indicative of a moving
object in the environment. In accordance with a determination that the movement parameter
does not meet the movement criterion or that the output unit is outputting audio,
the method can comprise not determining whether the object parameter meets a danger
criterion. In accordance with a determination that the object parameter does not meet
a danger criterion, the method can include not modifying the audio being output by
the output unit.
[0004] Embodiments of the disclosed methods can improve environmental awareness for a user
of a hearing aid. Embodiments of the disclosed methods can improve safety for a user
of a hearing aid. For example, embodiments of the disclosed methods can aim to grab
a user's attention (e.g., signal to a user), such as by providing a change in stream/microphone
mix ratio, to ensure environmental awareness when streaming on the move. In one or
more embodiments, the disclosed method will not disrupt the streaming experience unnecessarily
when in a busy environment.
[0005] More and more people listen to music, podcasts, and audiobooks while in transit or
out enjoying nature. For most hearing aid users, their main focus is on the streamed
content, while environmental sound is attenuated due to factors such as a negative
ratio of mic to streamed input, occluding acoustics and raised hearing thresholds.
As the user is on the move, their eyes are usually trained in front, entailing a loss
of awareness to either side. This is a perfect storm which presents a risk in situations
where more environmental awareness is needed, e.g. biking in traffic. Advantageously,
embodiments of the disclosed method can be used to improve the environmental awareness
of a hearing aid user. Further, embodiments of the disclosed method can improve safety
of a hearing aid user. For example, embodiments of the disclosed method can alleviate
dampening in the environment caused by streaming.
[0006] In particular, embodiments of the disclosed methods can provide signaling to a user
of a hearing aid about a potentially dangerous situation, or another situation which
may require the user's awareness. A potentially dangerous situation can be, for example,
the approach of a vehicle and/or bicycle. The method can utilize different cues from
the hearing aid to provide an alert to a user upon meeting certain criteria.
[0007] Embodiments of the disclosed method can be performed by a hearing aid. For example,
a hearing aid can include a processor and/or algorithm configured to perform the disclosed
method. Embodiments of the disclosed method can be performed by a processor, such
as in a mobile telephone, in communication with the hearing aid. Embodiments of the
disclosed method can be performed by a server, such as a cloud server, in communication
with the hearing aid and/or an intermediate device in communication with the hearing
aid.
[0008] In one or more example methods, the method can be performed by a hearing aid in part
and a mobile telephone in part. For example, determining the movement parameter can
be performed in a mobile telephone. In other words, the mobile telephone may have
an accelerometer, whereas the hearing aid may not.
[0009] In one or more example methods, the method can include obtaining movement data. For
example, the method can include obtaining the movement data from one or more sensors.
The one or more sensors can be integrated into the hearing aid. The one or more sensors
can be in communication with the hearing aid. The one or more sensors can be in communication
with an intermediate electronic device, which in turn is in communication with the
hearing aid. In one or more example methods, the method can include determining movement
data.
[0010] In one or more example methods, the movement data is accelerometer data and/or global
positioning system (GPS) data and/or sound scene analysis. The method can include
obtaining and/or determining the movement data from one or more sensors. The one or
more sensors can include, for example, one or more of an accelerometer, a global positioning
system, and a motion sensor (e.g., motion detector).
[0011] In one or more example methods, the method further comprises obtaining the movement
data from the hearing aid. In one or more example methods, the method further comprises
obtaining the movement data from another electronic device. The another electronic
device can be a mobile phone, a cloud server, or a physical server.
[0012] The movement data can be indicative of the movement of the hearing aid. The movement
data can be indicative of the movement of a user of the hearing aid.
[0013] Based on the movement data, the method can include determining a movement parameter.
The movement parameter is indicative of a movement of the hearing aid. The movement
parameter can be indicative of whether the hearing aid is moving or is stationary
(e.g., not moving). For example, the movement parameter can be a "true" if the movement
data indicates that the hearing aid is moving or a "false" if the movement data indicates
that the hearing aid is stationary (e.g., not moving).
[0014] The method can include determining whether the movement parameter meets the movement
criterion. In accordance with the movement parameter being indicative of the hearing
aid moving, the method determines that the movement parameter meets the movement criterion.
In accordance with the movement parameter being indicative of the hearing aid is not
moving, the method determines that the movement parameter does not meet the movement
criterion. In other words, the movement parameter can be used to determine whether
the hearing aid is moving or not. The method can include certain actions when the
hearing aid is moving and other actions when the hearing aid is not moving.
[0015] In one or more example methods, determining whether the movement parameter meets
the movement criterion comprises determining whether the movement parameter is indicative
of the hearing aid moving. In accordance with the determination that the movement
parameter is indicative of the hearing aid moving, the method comprises determining
that the movement parameter meets the movement criterion. In accordance with the determination
that the movement parameter is not indicative of the hearing aid moving, the method
comprises determining that the movement parameter does not meet the movement criterion.
[0016] In other words, the method can determine that the movement parameter meets the movement
criterion if the movement parameter is indicative of the hearing aid moving. However,
if the movement parameter is indicative of the hearing aid not moving (e.g., being
stationary), the method can determine that the movement parameter does not meet the
movement criterion.
[0017] In one or more example methods, determining whether the movement parameter meets
the movement criterion comprises determining whether the audio data is indicative
of the hearing aid being outdoors. In accordance with the determination that audio
data is indicative of the hearing aid being outdoors, the method includes determining
that the movement parameter meets the movement criterion. In accordance with the determination
that audio data is not indicative of the hearing aid being outdoors, the method includes
determining that the movement parameter does not meet the movement criterion.
[0018] The embodiments of the disclosed method may not be necessary for indoor use as there
is less potential for harm to a user of the hearing aid. Further, by not using the
disclosed method indoors, battery life can be improved by the hearing aid.
[0019] For example, determining whether the audio data is indicative of the hearing aid
being outdoors can include determining the echo (e.g., reverberation, reverb) levels
in the audio data. Indoor audio data can have a higher level of echo, due to the enclosed
space, than an outdoor location. In one or more examples, determining whether the
audio data is indicative of the hearing aid being outdoors includes determining whether
the audio data is equal to or below an echo threshold. In accordance with the determination
that the audio data is equal to or below an echo threshold, the method includes determining
that the audio data is indicative of the hearing aid being outdoors. In accordance
with the determination that the audio data is above an echo threshold, the method
includes determining that the audio data is indicative of the hearing aid being not
outdoors. The echo threshold can be a preset threshold.
[0020] In one or more example methods, determining whether the audio data is indicative
of the hearing aid being outdoors can include determining whether outdoor noises are
found in the audio data. For example, in accordance with the determination that the
audio data includes one or more of: wind sounds, bird sounds, vehicle sounds, and
nature sounds, the method can include determining that the audio data is indicative
of the hearing aid being outdoors. In accordance with the determination that the audio
data does not include one or more of: wind sounds, bird sounds, vehicle sounds, and
nature sounds, the method can include determining that the audio data is indicative
of the hearing aid being not outdoors. Other audio data cues indicative of the hearing
aid being outdoors can be used as well.
[0021] In one or more example methods, determining whether the audio data is indicative
of the hearing aid being outdoors can include applying one or more algorithm using
acoustic spatial clues to determine whether the audio data is indicative of the hearing
aid being outdoors.
[0022] In one or more example methods, determining whether the audio data is indicative
of the hearing aid being outdoors can include obtaining global positioning system
(GPS) data. For example, in accordance with the determination that the GPS data is
indicative of the user being outside, the method can include determining that the
audio data is indicative of the hearing aid being outdoors. In accordance with the
determination that the GPS data is indicative of the user not being outside, the method
can include determining that the audio data is not indicative of the hearing aid being
outdoors.
[0023] In one or more example methods, determining whether the audio data is indicative
of the hearing aid being outdoors can include obtaining temperature data. For example,
in accordance with the determination that the temperature data is indicative of a
change in temperature, the method can include determining that the audio data is indicative
of the hearing aid being outdoors. In accordance with the determination that the temperature
data is not indicative of a change in temperature, the method can include determining
that the audio data is not indicative of the hearing aid being outdoors.
[0024] In other words, the method can include at least two criteria for meeting the movement
criterion in certain embodiments. First, the hearing aid is moving. Second, the hearing
aid is outdoors.
[0025] In one or more example methods, determining whether the movement parameter meets
the movement criterion comprises determining whether the movement data is indicative
of the hearing aid moving within a time threshold. In accordance with the determination
that the movement data is indicative of the hearing aid moving within the time threshold,
the method includes determining that the movement parameter meets the movement criterion.
In accordance with the determination that the movement data is not indicative of the
hearing aid moving within the time threshold, the method includes determining that
the movement parameter does not meet the movement criterion.
[0026] For example, it can be advantageous for the method to take into account that a person
may be stopped temporarily, such as at an intersection, while continuing to monitor
the surroundings. The user of the hearing aid may quickly start moving at a dangerous
location after being stopped for a period of time.
[0027] The time threshold may be set for a particular time. Different locations may have
different time thresholds. Example time thresholds include 10 seconds, 30 seconds,
60 seconds, 90 seconds, and/or 120 seconds.
[0028] In one or more examples, the method includes determining whether the movement data
is indicative of the hearing aid throughout the time threshold. In other words, until
the method determines that the movement data is indicative of the hearing aid not
moving for the entire time threshold, the method can determine that the movement parameter
meets the movement parameter.
[0029] The method can begin using the time threshold when the movement data is indicative
of the hearing aid not moving. Advantageously, each time the hearing aid stops moving
the method can utilize the time threshold as a timer and continue monitoring the environment
around the hearing aid.
[0030] The method can include determining whether an output unit of the hearing aid is outputting
an audio. The audio can include, for example, music, speech, streaming, etc. In one
or more example methods, determining whether the output unit of the hearing aid is
outputting an audio includes determining whether the output unit (e.g., speaker) of
the hearing aid is outputting an audio not from the at least one microphone. In other
words, the method may only take action if audio outside of the hearing aid forward
path is being used. For example, determining whether an output unit of the hearing
aid is outputting an audio can include determining whether the hearing aid is receiving
audio data from another device. For example, determining whether an output unit of
the hearing aid is outputting an audio can include determining whether the hearing
aid is receiving audio data using Bluetooth Low Energy (BLE). In accordance with the
determination that the hearing aid is receiving audio data using BLE, the method includes
determining that the output unit of the hearing aid is outputting an audio.
[0031] The method can include obtaining audio data indicative of an environment around the
hearing aid. In one or more example methods, the method can include obtaining, by
at least one microphone of the hearing aid, audio data indicative of an environment
around the hearing aid. The method can include obtaining, by at least one input unit
of the hearing aid, audio data indicative of an environment around the hearing aid.
[0032] The method can determine whether the hearing aid is moving and whether the hearing
aid is playing an audio to a user. The method can determine whether the hearing aid
is moving and whether the hearing aid is playing a streaming audio to a user. In one
or more example embodiments, the method can include determining whether the movement
parameter meets the movement criterion and whether the output unit is outputting audio.
In accordance with a determination that the movement parameter meets the movement
criterion and that the output unit is outputting audio, the method includes determining,
based on the audio data, an object parameter indicative of a moving object in the
environment.
[0033] For example, the audio data can be indicative of certain object sounds in the environment.
The audio data can be indicative of an object that can potentially move in the environment.
[0034] The audio data can be indicative of a vehicle (e.g., a car, scooter, truck, motorcycle)
in the environment. The audio data can be indicative of a bicycle in the environment.
The method can determine that the object parameter is indicative of a moving object
in the environment in accordance with the audio data being indicative of certain objects
in the environment that are capable of movement. In one or more example methods, the
method can include obtaining a plurality of audio data indicative of the environment
around the hearing aid. Based on the plurality of audio data, the method can include
determining an object parameter indicative of a moving object in the environment.
By obtaining a plurality of audio data, it may provide more audio data for the determination
the object parameter.
[0035] The method can include determining whether the object parameter meets a danger criterion.
The danger criterion may also be known as an awareness criterion. For example, the
method determines that the object parameter meets the danger criterion in accordance
with the object parameter being indicative of the moving object travelling towards
the hearing aid. For example, the method determines that the object parameter meets
the danger criterion in accordance with the object parameter being indicative of the
moving object's path intersecting with the path of the hearing aid. In other words,
the method can be used to avoid dangerous situations (e.g., via the danger criterion)
or to make the user of the hearing aid aware of a potential interaction.
[0036] For example, the method determines that the object parameter does not meet the danger
criterion in accordance with the object parameter not being indicative of the moving
object travelling towards the hearing aid. For example, the method determines that
the object parameter does not meet the danger criterion in accordance with the object
parameter being indicative of the moving object's path not intersecting with the path
of the hearing aid.
[0037] In one or more example methods, determining whether the object parameter meets the
danger criterion comprises determining whether the object parameter is indicative
of the moving object moving towards the hearing aid. In accordance with the determination
that the object parameter is indicative of the moving object moving towards the hearing
aid, the method includes determining that the object parameter meets the danger criterion.
In accordance with the determination that the object parameter is not indicative of
the moving object moving towards the hearing aid, the method includes determining
that the object parameter does not meet the danger criterion.
[0038] In other words, the method is only concerned with a moving object moving towards
a hearing aid. If the moving object were moving away from the hearing aid, there would
be no potential danger and no need to alert the user.
[0039] In one or more examples, determining whether the object parameter is indicative of
the moving object moving towards the hearing aid from the sides of the user. In some
situations, it can be assumed that a moving object moving towards the hearing aid
from the front of the user of the hearing aid would be noticed by the user. However,
moving objects moving toward the users from the sides can be much more problematic,
and therefore it can be advantageous to alert the user via the hearing aid.
[0040] In one or more example methods, determining whether the object parameter meets the
danger criterion further comprises determining whether the object parameter is indicative
of the moving object moving at or above a velocity threshold. In accordance with the
determination that the object parameter is indicative of the moving object moving
at or above the velocity threshold, the method includes determining that the object
parameter meets the danger criterion. In accordance with the determination that the
object parameter is indicative of the moving object not moving at or above the velocity
threshold, the method includes determining that the object parameter does not meet
the danger criterion.
[0041] The velocity threshold can be, for example, 10 km/h, 20 km/h, or 38 km/h. The velocity
threshold can be a velocity in which the user would be harmed by interacting with
the moving object.
[0042] In one or more example methods, determining whether the object parameter meets the
danger criterion further comprises determining whether the object parameter is indicative
of the moving object moving at a zero or positive acceleration. In accordance with
the determination that the object parameter is indicative of the moving object moving
at a zero or positive acceleration, the method includes determining that the object
parameter meets the danger criterion. In accordance with the determination that the
object parameter is indicative of the moving object not moving at a negative acceleration,
the method includes determining that the object parameter does not meet the danger
criterion.
[0043] In other words, if the moving object is slowing down the method does not have to
alert the user of the hearing aid.
[0044] In one or more example methods, determining whether the object parameter meets the
danger criterion comprises inputting the audio data into a neural network. In one
or more example methods, determining whether the object parameter meets the danger
criterion comprises determining, by the neural network, the object parameter. In one
or more example methods, determining whether the object parameter meets the danger
criterion comprises outputting, by the neural network, whether the object parameter
meets the danger criterion.
[0045] In other words, the method can utilize a neural network (such as artificial intelligence,
machine learning, etc.) for determination of whether the object parameter meets the
danger criteria. The neural network can include one or more algorithms for interpretation
of audio data to determine whether the object parameter meets the danger criterion.
[0046] In one or more example methods, determining, by the neural network, the object parameter
comprises classifying, by the neural network, the moving object.
[0047] For example, the neural network can be configured to output a classification of the
moving object. Classification can include one or more of vehicle, bicycle, animal,
pedestrian, and non-factor. The method can be configured to determine, based on the
classification output by the neural network, whether the object parameter meets the
danger criterion.
[0048] In accordance with the object parameter meeting a danger criterion, the method includes
modifying the audio being output by the output unit. Modifying can include attenuating
the audio. Modifying can include reducing the volume of the audio. Modifying can include
turning off noise cancellation. Modifying can include any change to the audio that
can provide some sort of signalling to the user of the hearing aid.
[0049] In certain examples, a user may be wearing two hearing aids (e.g., a binaural system).
For example, a user may be wearing a first hearing aid and a second hearing aid. Modifying
the audio being output by the output unit can include modifying the audio being output
by the output unit of the first hearing aid and/or modifying the audio being output
by the output unit of the second hearing aid. This may allow for directional alert
to a user of the hearing aid (e.g., modifying the audio of the hearing aid in the
direction of the approaching object).
[0050] In one or more example methods, modifying the audio comprises attenuating the audio.
In other words, the method can include reducing the loudness of the audio data, providing
an audio clue to the user of the hearing aid. Modifying the audio can include fully
attenuating the audio. Modifying the audio can include attenuating the audio by, for
example, 1, 2, 3, 4, or 5 dBs. Modifying the audio can include attenuating the audio
to create awareness for a hearing aid user without fundamentally changing the streaming
experience.
[0051] In one or more example methods, modifying the audio comprises adjusting a mix ratio
of the audio. In other words, the method includes changing the parameters of the audio,
providing an audio clue to the user.
[0052] In one or more example methods, modifying the audio comprises outputting an audio
tone. In other words, the method an include adding further audio to alert the user.
The audio tone can include a chime, a tone, or any audio separate from the audio data
currently being output.
[0053] In one or more example methods, in accordance with the object parameter meeting the
danger criterion, the method comprises vibrating the hearing aid. In one or more example
methods, in accordance with the object parameter not meeting the danger criterion,
the method comprises not vibrating the hearing aid. In other words, the method can
include providing a physical cue to the user of the hearing aid.
[0054] Further disclosed herein are example hearing aids having a memory and a processor
configured to perform any and/or all of the method steps disclosed herein.
A hearing aid:
[0055] The above-disclosed method(s) can be performed partially or fully by a hearing aid.
[0056] The hearing aid may be adapted to provide a frequency dependent gain and/or a level
dependent compression and/or a transposition (with or without frequency compression)
of one or more frequency ranges to one or more other frequency ranges, e.g. to compensate
for a hearing impairment of a user. The hearing aid may comprise a signal processor
for enhancing the input signals and providing a processed output signal.
[0057] The hearing aid may comprise an output unit for providing a stimulus perceived by
the user as an acoustic signal based on a processed electric signal. The output unit
may comprise a number of electrodes of a cochlear implant (for a CI type hearing aid)
or a vibrator of a bone conducting hearing aid. The output unit may comprise an output
transducer. The output transducer may comprise a receiver (loudspeaker) for providing
the stimulus as an acoustic signal to the user (e.g. in an acoustic (air conduction
based) hearing aid). The output transducer may comprise a vibrator for providing the
stimulus as mechanical vibration of a skull bone to the user (e.g. in a bone-attached
or bone-anchored hearing aid). The output unit may (additionally or alternatively)
comprise a (e.g. wireless) transmitter for transmitting sound picked up-by the hearing
aid to another device, e.g. a far-end communication partner (e.g. via a network, e.g.
in a telephone mode of operation, or in a headset configuration).
[0058] The hearing aid may comprise an input unit (e.g., at least one microphone) for providing
an electric input signal representing sound. The input unit may comprise an input
transducer, e.g. a microphone, for converting an input sound to an electric input
signal. The input unit may comprise a wireless receiver for receiving a wireless signal
comprising or representing sound and for providing an electric input signal representing
said sound.
[0059] The wireless receiver and/or transmitter may e.g. be configured to receive and/or
transmit an electromagnetic signal in the radio frequency range (3 kHz to 300 GHz).
The wireless receiver and/or transmitter may e.g. be configured to receive and/or
transmit an electromagnetic signal in a frequency range of light (e.g. infrared light
300 GHz to 430 THz, or visible light, e.g. 430 THz to 770 THz).
[0060] The hearing aid may comprise a directional microphone system adapted to spatially
filter sounds from the environment, and thereby enhance a target acoustic source among
a multitude of acoustic sources in the local environment of the user wearing the hearing
aid. The directional system may be adapted to detect (such as adaptively detect) from
which direction a particular part of the microphone signal originates. This can be
achieved in various different ways as e.g. described in the prior art. In hearing
aids, a microphone array beamformer is often used for spatially attenuating background
noise sources. The beamformer may comprise a linear constraint minimum variance (LCMV)
beamformer. Many beamformer variants can be found in literature. The minimum variance
distortionless response (MVDR) beamformer is widely used in microphone array signal
processing. Ideally the MVDR beamformer keeps the signals from the target direction
(also referred to as the look direction) unchanged, while attenuating sound signals
from other directions maximally. The generalized sidelobe canceller (GSC) structure
is an equivalent representation of the MVDR beamformer offering computational and
numerical advantages over a direct implementation in its original form.
[0061] The hearing aid may comprise antenna and transceiver circuitry allowing a wireless
link to an entertainment device (e.g. a TV-set), a communication device (e.g. a telephone),
a wireless microphone, a separate (external) processing device, or another hearing
aid, etc. The hearing aid may thus be configured to wirelessly receive a direct electric
input signal from another device. Likewise, the hearing aid may be configured to wirelessly
transmit a direct electric output signal to another device. The direct electric input
or output signal may represent or comprise an audio signal and/or a control signal
and/or an information signal.
[0062] In general, a wireless link established by antenna and transceiver circuitry of the
hearing aid can be of any type. The wireless link may be a link based on near-field
communication, e.g. an inductive link based on an inductive coupling between antenna
coils of transmitter and receiver parts. The wireless link may be based on far-field,
electromagnetic radiation. Preferably, frequencies used to establish a communication
link between the hearing aid and the other device is below 70 GHz, e.g. located in
a range from 50 MHz to 70 GHz, e.g. above 300 MHz, e.g. in an ISM range above 300
MHz, e.g. in the 900 MHz range or in the 2.4 GHz range or in the 5.8 GHz range or
in the 60 GHz range (ISM=Industrial, Scientific and Medical, such standardized ranges
being e.g. defined by the International Telecommunication Union, ITU). The wireless
link may be based on a standardized or proprietary technology. The wireless link may
be based on Bluetooth technology (e.g. Bluetooth Low-Energy technology, e.g. LE audio),
or Ultra WideBand (UWB) technology.
[0063] The hearing aid may be constituted by or form part of a portable (i.e. configured
to be wearable) device, e.g. a device comprising a local energy source, e.g. a battery,
e.g. a rechargeable battery. The hearing aid may e.g. be a low weight, easily wearable,
device, e.g. having a total weight less than 100 g, such as less than 20 g, such as
less than 5 g.
[0064] The hearing aid may comprise a 'forward' (or `signal') path for processing an audio
signal between an input and an output of the hearing aid. A signal processor may be
located in the forward path. The signal processor may be adapted to provide a frequency
dependent gain according to a user's particular needs (e.g. hearing impairment). The
hearing aid may comprise an 'analysis' path comprising functional components for analyzing
signals and/or controlling processing of the forward path. Some or all signal processing
of the analysis path and/or the forward path may be conducted in the frequency domain,
in which case the hearing aid comprises appropriate analysis and synthesis filter
banks. Some or all signal processing of the analysis path and/or the forward path
may be conducted in the time domain.
[0065] An analogue electric signal representing an acoustic signal may be converted to a
digital audio signal in an analogue-to-digital (AD) conversion process, where the
analogue signal is sampled with a predefined sampling frequency or rate f
s, f
s being e.g. in the range from 8 kHz to 48 kHz (adapted to the particular needs of
the application) to provide digital samples x
n (or x[n]) at discrete points in time t
n (or n), each audio sample representing the value of the acoustic signal at t
n by a predefined number N
b of bits, N
b being e.g. in the range from 1 to 48 bits, e.g. 24 bits. Each audio sample is hence
quantized using N
b bits (resulting in 2
Nb different possible values of the audio sample). A digital sample x has a length in
time of 1/f
s, e.g. 50 µs, for
fs = 20 kHz. A number of audio samples may be arranged in a time frame. A time frame
may comprise 64 or 128 audio data samples. Other frame lengths may be used depending
on the practical application.
[0066] The hearing aid may comprise an analogue-to-digital (AD) converter to digitize an
analogue input (e.g. from an input transducer, such as a microphone) with a predefined
sampling rate, e.g. 20 kHz. The hearing aids may comprise a digital-to-analogue (DA)
converter to convert a digital signal to an analogue output signal, e.g. for being
presented to a user via an output transducer.
[0067] The hearing aid may comprise a number of detectors configured to provide status signals
relating to a current physical environment of the hearing aid (e.g. the current acoustic
environment), and/or to a current state of the user wearing the hearing aid, and/or
to a current state or mode of operation of the hearing aid. Alternatively or additionally,
one or more detectors may form part of an
external device in communication (e.g. wirelessly) with the hearing aid. An external device
may e.g. comprise another hearing aid, a remote control, and audio delivery device,
a telephone (e.g. a smartphone), an external sensor, etc.
[0068] One or more of the number of detectors may operate on the full band signal (time
domain). One or more of the number of detectors may operate on band split signals
((time-) frequency domain), e.g. in a limited number of frequency bands.
[0069] The number of detectors may comprise a movement detector, e.g. an acceleration sensor.
The movement detector may be configured to detect movement of the user's facial muscles
and/or bones, e.g. due to speech or chewing (e.g. jaw movement) and to provide a detector
signal indicative thereof.
[0070] The hearing aid may comprise a classification unit configured to classify the current
situation based on input signals from (at least some of) the detectors, and possibly
other inputs as well. In the present context `a current situation' may be taken to
be defined by one or more of
- a) the physical environment (e.g. including the current electromagnetic environment,
e.g. the occurrence of electromagnetic signals (e.g. comprising audio and/or control
signals) intended or not intended for reception by the hearing aid, or other properties
of the current environment than acoustic);
- b) the current acoustic situation (input level, feedback, etc.), and
- c) the current mode or state of the user (movement, temperature, cognitive load, etc.);
- d) the current mode or state of the hearing aid (program selected, time elapsed since
last user interaction, etc.) and/or of another device in communication with the hearing
aid.
[0071] The classification unit may be based on or comprise a neural network, e.g. a recurrent
neural network, e.g. a trained neural network.
[0072] The hearing aid may further comprise other relevant functionality for the application
in question, e.g. compression, noise reduction, etc.
[0073] The hearing aid may comprise a hearing instrument, e.g. a hearing instrument adapted
for being located at the ear or fully or partially in the ear canal of a user, e.g.
a headset, an earphone, an ear protection device or a combination thereof. A hearing
system may comprise a speakerphone (comprising a number of input transducers (e.g.
a microphone array) and a number of output transducers, e.g. one or more loudspeakers,
and one or more audio (and possibly video) transmitters e.g. for use in an audio conference
situation), e.g. comprising a beamformer filtering unit, e.g. providing multiple beamforming
capabilities.
[0074] It is intended that some or all of the structural features of the hearing aid described
above, in the `detailed description of embodiments' or in the claims can be combined
with embodiments of the method(s), when appropriately substituted by a corresponding
process and vice versa. Embodiments of the method have the same advantages as the
corresponding devices.
A computer readable medium or data carrier:
[0075] In an aspect, a tangible computer-readable medium (a data carrier) storing a computer
program comprising program code means (instructions) for causing a data processing
system (a computer) to perform (carry out) at least some (such as a majority or all)
of the (steps of the) method described above, in the `detailed description of embodiments'
and in the claims, when said computer program is executed on the data processing system
is furthermore provided by the present application.
[0076] By way of example, and not limitation, such computer-readable media can comprise
RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other
magnetic storage devices, or any other medium that can be used to carry or store desired
program code in the form of instructions or data structures and that can be accessed
by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc,
optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks
usually reproduce data magnetically, while discs reproduce data optically with lasers.
Other storage media include storage in DNA (e.g. in synthesized DNA strands). Combinations
of the above should also be included within the scope of computer-readable media.
In addition to being stored on a tangible medium, the computer program can also be
transmitted via a transmission medium such as a wired or wireless link or a network,
e.g. the Internet, and loaded into a data processing system for being executed at
a location different from that of the tangible medium.
[0077] In an aspect of the present application, a method of improving environmental awareness
in a hearing aid via a tangible computer-readable medium is provided. The a tangible
computer-readable medium (a data carrier) storing a computer program can comprise
program code means (instructions) for causing a data processing system (a computer)
to perform (carry out) determining, based on movement data, a movement parameter indicative
of a movement of the hearing aid. The tangible computer-readable medium (a data carrier)
storing a computer program can comprise program code means (instructions) for causing
a data processing system (a computer) to perform (carry out) determining whether the
movement parameter meets a movement criterion. The tangible computer-readable medium
(a data carrier) storing a computer program can comprise program code means (instructions)
for determining whether an output unit of the hearing aid is outputting an audio.
The tangible computer-readable medium (a data carrier) storing a computer program
can comprise program code means (instructions) for obtaining, by at least one microphone
of the hearing aid, audio data indicative of an environment around the hearing aid.
In accordance with a determination that the movement parameter meets the movement
criterion and that the output unit is outputting audio, the tangible computer-readable
medium (a data carrier) storing a computer program can comprise program code means
(instructions) for determining, based on the audio data, an object parameter indicative
of a moving object in the environment. In accordance with a determination that the
movement parameter meets the movement criterion and that the output unit is outputting
audio, the tangible computer-readable medium (a data carrier) storing a computer program
can comprise program code means (instructions) for determining whether the object
parameter meets a danger criterion.
A computer program:
[0078] A computer program (product) comprising instructions which, when the program is executed
by a computer, cause the computer to carry out (steps of) the method described above,
in the `detailed description of embodiments' and in the claims is furthermore provided
by the present application.
[0079] In an aspect of the present application, a method of improving environmental awareness
in a hearing aid is provided. The method can comprise determining, based on movement
data, a movement parameter indicative of a movement of the hearing aid. The method
can comprise determining whether the movement parameter meets a movement criterion.
The method can comprise determining whether an output unit of the hearing aid is outputting
an audio. The method can comprise obtaining, by at least one microphone of the hearing
aid, audio data indicative of an environment around the hearing aid. In accordance
with a determination that the movement parameter meets the movement criterion and
that the output unit is outputting audio, the method can comprise determining, based
on the audio data, an object parameter indicative of a moving object in the environment.
In accordance with a determination that the movement parameter meets the movement
criterion and that the output unit is outputting audio, the method can comprise determining
whether the object parameter meets a danger criterion.
A data processing system:
[0080] In an aspect, a data processing system comprising a processor and program code means
for causing the processor to perform at least some (such as a majority or all) of
the steps of the method described above, in the `detailed description of embodiments'
and in the claims is furthermore provided by the present application.
A hearing system:
[0081] In a further aspect, a hearing system comprising a hearing aid as described above,
in the `detailed description of embodiments', and in the claims, AND an auxiliary
device is moreover provided.
[0082] The hearing system may be adapted to establish a communication link between the hearing
aid and the auxiliary device to provide that information (e.g. control and status
signals, possibly audio signals) can be exchanged or forwarded from one to the other.
[0083] The auxiliary device may be constituted by or comprise a remote control, a smartphone,
or other portable or wearable electronic device, such as a smartwatch or the like.
[0084] The auxiliary device may be constituted by or comprise a remote control for controlling
functionality and operation of the hearing aid(s). The function of a remote control
may be implemented in a smartphone, the smartphone possibly running an APP allowing
to control the functionality of the audio processing device via the smartphone (the
hearing aid(s) comprising an appropriate wireless interface to the smartphone, e.g.
based on Bluetooth or some other standardized or proprietary scheme).
[0085] The auxiliary device may be constituted by or comprise an audio gateway device adapted
for receiving a multitude of audio signals (e.g. from an entertainment device, e.g.
a TV or a music player, a telephone apparatus, e.g. a mobile telephone or a computer,
e.g. a PC, a wireless microphone, etc.) and adapted for selecting and/or combining
an appropriate one of the received audio signals (or combination of signals) for transmission
to the hearing aid.
[0086] The auxiliary device may be constituted by or comprise another hearing aid. The hearing
system may comprise two hearing aids adapted to implement a binaural hearing system,
e.g. a binaural hearing aid system.
Definitions:
[0087] In the present context, a hearing aid, e.g. a hearing instrument, refers to a device,
which is adapted to improve, augment and/or protect the hearing capability of a user
by receiving acoustic signals from the user's surroundings, generating corresponding
audio signals, possibly modifying the audio signals and providing the possibly modified
audio signals as audible signals to at least one of the user's ears. Such audible
signals may e.g. be provided in the form of acoustic signals radiated into the user's
outer ears, acoustic signals transferred as mechanical vibrations to the user's inner
ears through the bone structure of the user's head and/or through parts of the middle
ear as well as electric signals transferred directly or indirectly to the cochlear
nerve of the user.
[0088] The hearing aid may be configured to be worn in any known way, e.g. as a unit arranged
behind the ear with a tube leading radiated acoustic signals into the ear canal or
with an output transducer, e.g. a loudspeaker, arranged close to or in the ear canal,
as a unit entirely or partly arranged in the pinna and/or in the ear canal, as a unit,
e.g. a vibrator, attached to a fixture implanted into the skull bone, as an attachable,
or entirely or partly implanted, unit, etc. The hearing aid may comprise a single
unit or several units communicating (e.g. acoustically, electrically or optically)
with each other. The loudspeaker may be arranged in a housing together with other
components of the hearing aid, or may be an external unit in itself (possibly in combination
with a flexible guiding element, e.g. a dome-like element).
[0089] A hearing aid may be adapted to a particular user's needs, e.g. a hearing impairment.
A configurable signal processing circuit of the hearing aid may be adapted to apply
a frequency and level dependent compressive amplification of an input signal. A customized
frequency and level dependent gain (amplification or compression) may be determined
in a fitting process by a fitting system based on a user's hearing data, e.g. an audiogram,
using a fitting rationale (e.g. adapted to speech). The frequency and level dependent
gain may e.g. be embodied in processing parameters, e.g. uploaded to the hearing aid
via an interface to a programming device (fitting system), and used by a processing
algorithm executed by the configurable signal processing circuit of the hearing aid.
[0090] A `hearing system' refers to a system comprising one or two hearing aids, and a `binaural
hearing system' refers to a system comprising two hearing aids and being adapted to
cooperatively provide audible signals to both of the user's ears. Hearing systems
or binaural hearing systems may further comprise one or more `auxiliary devices',
which communicate with the hearing aid(s) and affect and/or benefit from the function
of the hearing aid(s). Such auxiliary devices may include at least one of a remote
control, a remote microphone, an audio gateway device, an entertainment device, e.g.
a music player, a wireless communication device, e.g. a mobile phone (such as a smartphone)
or a tablet or another device, e.g. comprising a graphical interface. Hearing aids,
hearing systems or binaural hearing systems may e.g. be used for compensating for
a hearing-impaired person's loss of hearing capability, augmenting or protecting a
normal-hearing person's hearing capability and/or conveying electronic audio signals
to a person. Hearing aids or hearing systems may e.g. form part of or interact with
public-address systems, active ear protection systems, handsfree telephone systems,
car audio systems, entertainment (e.g. TV, music playing or karaoke) systems, teleconferencing
systems, classroom amplification systems, etc.
[0091] The invention is set out in the appended set of claims.
BRIEF DESCRIPTION OF DRAWINGS
[0092] The aspects of the disclosure may be best understood from the following detailed
description taken in conjunction with the accompanying figures. The figures are schematic
and simplified for clarity, and they just show details to improve the understanding
of the claims, while other details are left out. Throughout, the same reference numerals
are used for identical or corresponding parts. The individual features of each aspect
may each be combined with any or all features of the other aspects. These and other
aspects, features and/or technical effect will be apparent from and elucidated with
reference to the illustrations described hereinafter in which:
FIG. 1 shows an example flowchart of one or more embodiments of the disclosed method,
FIG. 2 illustrates an example situation according one or more embodiments of the disclosed
method,
FIG. 3 illustrates an example situation according one or more embodiments of the disclosed
method, and
FIG. 4 shows an example hearing aid incorporating one or more elements of the disclosed
method.
[0093] The figures are schematic and simplified for clarity, and they just show details
which are essential to the understanding of the disclosure, while other details are
left out. Throughout, the same reference signs are used for identical or corresponding
parts.
[0094] Further scope of applicability of the present disclosure will become apparent from
the detailed description given hereinafter. However, it should be understood that
the detailed description and specific examples, while indicating preferred embodiments
of the disclosure, are given by way of illustration only. Other embodiments may become
apparent to those skilled in the art from the following detailed description.
DETAILED DESCRIPTION OF EMBODIMENTS
[0095] The detailed description set forth below in connection with the appended drawings
is intended as a description of various configurations. The detailed description includes
specific details for the purpose of providing a thorough understanding of various
concepts. However, it will be apparent to those skilled in the art that these concepts
may be practiced without these specific details. Several aspects of the apparatus
and methods are described by various blocks, functional units, modules, components,
circuits, steps, processes, algorithms, etc. (collectively referred to as "elements").
Depending upon particular application, design constraints or other reasons, these
elements may be implemented using electronic hardware, computer program, or any combination
thereof.
[0096] The electronic hardware may include micro-electronic-mechanical systems (MEMS), integrated
circuits (e.g. application specific), microprocessors, microcontrollers, digital signal
processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices
(PLDs), gated logic, discrete hardware circuits, printed circuit boards (PCB) (e.g.
flexible PCBs), and other suitable hardware configured to perform the various functionality
described throughout this disclosure, e.g. sensors, e.g. for sensing and/or registering
physical properties of the environment, the device, the user, etc. Computer program
shall be construed broadly to mean instructions, instruction sets, code, code segments,
program code, programs, subprograms, software modules, applications, software applications,
software packages, routines, subroutines, objects, executables, threads of execution,
procedures, functions, etc., whether referred to as software, firmware, middleware,
microcode, hardware description language, or otherwise.
[0097] The present application relates to the field of hearing aids.
[0098] FIG. 1 shows an example flowchart of one or more embodiments of the disclosed method.
As shown, the method 100 is for improving environmental awareness in a hearing aid,
such as the hearing aid 400 shown in FIG. 4.
[0099] The method 100 includes determining 102, based on movement data, a movement parameter
indicative of a movement of the hearing aid. The method 100 includes determining 104
whether the movement parameter meets a movement criterion. The method 100 includes
determining 106 whether an output unit of the hearing aid is outputting an audio.
The method 100 includes obtaining 108, by at least one microphone of the hearing aid,
audio data indicative of an environment around the hearing aid.
[0100] In accordance with a determination 110 that the movement parameter meets the movement
criterion and that the output unit is outputting audio, the method 100 includes determining
112, based on the audio data, an object parameter indicative of a moving object in
the environment and determining 114 whether the object parameter meets a danger criterion.
[0101] In accordance with a determination 116 that the object parameter meeting a danger
criterion, the method 100 includes modifying 118 the audio being output by the output
unit.
[0102] In accordance with a determination 110 that the movement parameter does not meet
the movement criterion or that the output unit is outputting audio, the method 100
includes not determining 120, based on the audio data, an object parameter indicative
of a moving object in the environment and not determining 122 whether the object parameter
meets a danger criterion.
[0103] In accordance with a determination 116 that the object parameter does not meet a
danger criterion, the method 100 includes not modifying 124 the audio being output
by the output unit.
[0104] In one or more example methods determining 104 whether the movement parameter meets
the movement criterion includes determining whether the movement parameter is indicative
of the hearing aid moving, and in accordance with the determination that the movement
parameter is indicative of the hearing aid moving, determining that the movement parameter
meets the movement criterion. In accordance with the determination that the movement
parameter is indicative of the hearing aid not moving, the method 100 includes determining
that the movement parameter does not meets the movement criterion.
[0105] In one or more example methods, determining 104 whether the movement parameter meets
the movement criterion includes determining whether the audio data is indicative of
the hearing aid being outdoors, and in accordance with the determination that audio
data is indicative of the hearing aid being outdoors, determining that the movement
parameter meets the movement criterion. In accordance with the determination that
audio data is indicative of the hearing aid being not being outdoors, the method 100
includes determining that the movement parameter does not meet the movement criterion.
[0106] The method 100 can further include obtaining the movement data from the hearing aid.
The movement data can be accelerometer data and/or global positioning system (GPS)
data and/or sound scene analysis.
[0107] In one or more example methods, modifying 118 the audio comprises one or more of
attenuating the audio, adjusting a mix ratio of the audio, and outputting an audio
tone. In accordance with the object parameter meeting the danger criterion, the method
comprises vibrating the hearing aid. In accordance with the object parameter not meeting
the danger criterion, the method comprises not vibrating the hearing aid.
[0108] Determining 114 whether the obj ect parameter meets the danger criterion includes
determining whether the object parameter is indicative of the moving object moving
towards the hearing aid, and in accordance with the determination that the object
parameter is indicative of the moving object moving towards the hearing aid, determining
that the object parameter meets the danger criterion. In accordance with the determination
that the object parameter is not indicative of the moving object moving towards the
hearing aid, the method 100 includes determining that the object parameter does not
meet the danger criterion.
[0109] Determining 114 whether the object parameter meets the danger criterion further includes
determining whether the object parameter is indicative of the moving object moving
at or above a velocity threshold, and in accordance with the determination that the
object parameter is indicative of the moving object moving at or above the velocity
threshold, determining that the object parameter meets the danger criterion. In accordance
with the determination that the object parameter is indicative of the moving object
moving at or above the velocity threshold, the method 100 includes determining that
the object parameter does not meet the danger criterion.
[0110] Determining 114 whether the object parameter meets the danger criterion can include
inputting the audio data into a neural network, determining, by the neural network,
the object parameter, and outputting, by the neural network, whether the obj ect parameter
meets the danger criterion. Determining, by the neural network, the object parameter
can include classifying, by the neural network, the moving object.
[0111] Determining 114 whether the movement parameter meets the movement criterion includes
whether the movement data is indicative of the hearing aid moving within a time threshold,
an in accordance with the determination that the movement data is indicative of the
hearing aid moving within the time threshold, determining that the movement parameter
meets the movement criterion. In accordance with the determination that the movement
data is indicative of the hearing aid not moving within the time threshold, determining
that the movement parameter does not meet the movement criterion.
[0112] Further disclosed herein is a hearing aid, such as hearing aid 400, having a memory
and a processor configure to perform the method 100 of any one of the previous claims.
[0113] Further disclosed is a data processing system comprising a processor and program
code means for causing the processor to perform at least some of the steps of the
method 100 disclosed herein.
[0114] Further disclosed is a computer program product comprising instructions which, when
the program is executed by a computer, cause the computer to carry out the steps of
the method 100 disclosed herein.
[0115] FIG. 2 illustrates an example situation 200 according one or more embodiments of
the disclosed method, such as method 100 discussed with respect to FIG. 1.
[0116] As shown, a user 202 of a hearing aid is walking outside (e.g., the movement parameter
is indicative of movement of the hearing aid). The hearing aid of the user 202 is
also connected to their mobile telephone via BLE in order to stream music 204 (e.g.,
the output unit of the hearing aid is outputting an audio).
[0117] Further, the hearing aid of user 202 is configured to obtain, by at least one microphone,
audio data 206 indicative of the environment around the hearing aid. Accordingly,
the movement parameter meets the movement criterion and the output unit is outputting
the audio, so further steps are taken. If the user 202 was not moving, or if the hearing
aid was not outputting audio, no further steps would be taken.
[0118] In the situation 200 shown in FIG. 2, the user 202 of the hearing aid is approaching
an intersection 208. In this particular intersection 208, a vehicle 210 is travelling
towards the user 202 of the hearing aid. Based on the audio data 206, the object parameter
indicative of a moving object in the environment can be determined (e.g., the moving
vehicle 210).
[0119] Further, it can be determined that the object parameter meets a danger criterion
(e.g., the vehicle 210 will hit the user 202 of the hearing aid when crossing the
intersection 208). Accordingly, the hearing aid can modify the audio being output
which, as shown in FIG. 2, is fully attenuating the audio. This provides an audio
clue to the user 202 of the hearing aid to be aware of their surroundings to avoid
being hit by the vehicle 210.
[0120] FIG. 3 illustrates an example situation according one or more embodiments of the
disclosed method.
[0121] In particular, FIG. 3 illustrates the dB levels of sound being produced by the hearing
aid. In the situation shown in FIG. 3, the streamed audio level 302 has a higher dB
output than the microphone output 304.
[0122] Time 306 illustrates a moment where, according to the disclosed method, it is determined
that an object parameter meets a danger criterion. Accordingly, the method modifies
the audio being output by the output unit of the hearing aid. This is shown in FIG.
3 as slight reduction of the dB level of the streamed audio level 302 and a slight
increase in the dB level of the microphone output 304. Even a minor change in the
input ratio and/or the output ratio can create awareness to a user of the hearing
aid. For example, only a few dBs of change (e.g., not enough to change the streaming
experience, but enough to create awareness) could cause a hearing aid user's head
turn. At time 308, the situation has been resolved the dB levels can return to what
they were previously.
[0123] FIG. 4 shows an example embodiment of a hearing aid according to the disclosure.
As shown, the hearing aid 400 includes the housing 414. The hearing aid 400 is configured
to be worn behind the user's ears and comprises a behind-the-ear part 402 and an in-the-ear
part 404. The behind-the-ear part 402 is connected to the in-the-ear part 404 via
connecting member 406. However, the hearing aid 400 may be configured in other ways
e.g., as completely-in-the-ear hearing aids. In some examples, the hearing aid 300
is in communication with another hearing aid via a short-range wireless link, e.g.,
an inductive wireless link.
[0124] It is intended that the structural features of the devices described above, either
in the detailed description and/or in the claims, may be combined with steps of the
method, when appropriately substituted by a corresponding process.
[0125] As used, the singular forms "a," "an," and "the" are intended to include the plural
forms as well (i.e. to have the meaning "at least one"), unless expressly stated otherwise.
It will be further understood that the terms "includes," "comprises," "including,"
and/or "comprising," when used in this specification, specify the presence of stated
features, integers, steps, operations, elements, and/or components, but do not preclude
the presence or addition of one or more other features, integers, steps, operations,
elements, components, and/or groups thereof. It will also be understood that when
an element is referred to as being "connected" or "coupled" to another element, it
can be directly connected or coupled to the other element, but an intervening element
may also be present, unless expressly stated otherwise. Furthermore, "connected" or
"coupled" as used herein may include wirelessly connected or coupled. As used herein,
the term "and/or" includes any and all combinations of one or more of the associated
listed items. The steps of any disclosed method are not limited to the exact order
stated herein, unless expressly stated otherwise.
[0126] It should be appreciated that reference throughout this specification to "one embodiment"
or "an embodiment" or "an aspect" or features included as "may" means that a particular
feature, structure or characteristic described in connection with the embodiment is
included in at least one embodiment of the disclosure. Furthermore, the particular
features, structures or characteristics may be combined as suitable in one or more
embodiments of the disclosure. The previous description is provided to enable any
person skilled in the art to practice the various aspects described herein. Various
modifications to these aspects will be readily apparent to those skilled in the art.
[0127] The claims are not intended to be limited to the aspects shown herein but are to
be accorded the full scope consistent with the language of the claims, wherein reference
to an element in the singular is not intended to mean "one and only one" unless specifically
so stated, but rather "one or more." Unless specifically stated otherwise, the term
"some" refers to one or more.