Field
[0001] The present invention relates generally to security monitoring system installations
for premises.
Background
[0002] Security monitoring systems for monitoring premises, often referred to as alarm systems,
typically provide a means for detecting the presence and/or actions of people at the
premises and reacting to detected events. Commonly such systems include sensors to
detect the opening and closing of doors and windows, movement detectors to monitor
spaces (both within and outside buildings) for signs of movement, microphones to detect
sounds such as breaking glass, and image sensors to capture still or moving images
of monitored zones. Such systems may be self-contained, with alarm indicators such
as sirens and flashing lights that may be activated in the event of an alarm condition
being detected. Such installations typically include a control unit (which may also
be termed a central unit or local management device), generally mains powered, that
is coupled to the sensors, detectors, cameras, etc. ("nodes"), and which processes
received notifications and determines a response. The local management device or central
unit may be linked to the various nodes by wires, but increasingly is instead linked
wirelessly, rather than by wires, since this facilitates installation and may also
provide some safeguards against sensors/detectors effectively being disabled by disconnecting
them from the central unit. Similarly, for ease of installation and to improve security,
the nodes of such systems typically include an autonomous power source, such as a
battery power supply, rather than being mains powered.
[0003] As an alternative to self-contained systems, a security monitoring system may include
an installation at a premises, domestic or commercial, that is linked to a remotely
located monitoring station where, typically, human operators manage the responses
required by different alarm and notification types. These monitoring stations are
often referred to as Central Monitoring Station (CMS) because they may be used to
monitor a large number of security monitoring systems distributed around the monitoring
station, the CMS located rather like a spider in a web. In such centrally monitored
systems, the local management device or central unit at the premises installation
typically processes notifications received from the nodes in the installation and
notifies the Central Monitoring Station of only some of these, depending upon the
settings of the system - in particular whether it is fully or only partially armed,
and the nature of the detected events. In such a configuration, the central unit at
the installation is effectively acting as a gateway between the nodes and the Central
Monitoring Station. Again, in such installations the central unit may be linked by
wires, or wirelessly, to the various nodes of the installation, and these nodes will
typically be battery rather than mains powered.
[0004] Despite the ready availability of professionally installed security monitoring systems,
many homes remain unprotected by such systems, possibly because people are put off
by the expected cost of installing such systems and subscribing to monitoring services.
[0005] In recent years many people have installed Wi-Fi enabled video cameras in their homes
- from suppliers such as Ring and Arlo, enabling users to check up on the status of
their homes remotely. These cameras typically include a PIR motion sensor that activates
the camera when someone comes in view of the sensor. Captured video can be stored
in the cloud - typically requiring payment of a subscription fee - for viewing at
users' leisure, and push alerts can be sent to the user's smartphone (and those of
others, if desired) so that the user can receive real-time notification of presence.
Typically, these cameras are installed more for simple peace of mind, than with a
view to creating a stand-alone security monitoring system. Indeed, it is not uncommon
for occupants of premises that already have a professionally installed security monitoring
system also to install such cameras so that they can be reassured about the situation
at home when they are away. In particular, parents of school-age children often install
such cameras so that they can see when their children get home from school - and also
so that they can "keep an eye on" their kids in the hours before a parent returns
home. Such cameras are known by the term "convenience cameras" because it is convenient
to use them for a kind of self-monitored surveillance, without needing to worry about
someone in an alarm surveillance centre being able to look in on private life in the
home - something about which people feel very sensitive, especially when they are
parents of young children. The suppliers of such convenience cameras also offer "monitoring
packages" which typically involve installing extra devices such as a control unit
and PIR motion sensors to create in effect a burglar alarm installation, and paying
a significantly enhanced subscription fee. Many owners of convenience cameras are
put off by the cost of the extra equipment that needs to be installed, as well as
by the typically hefty annual subscription. Yet many of these people, especially if
they are parents of school age children, or part-time carers for elderly parents or
relatives would welcome more extensive reassurance about conditions in the homes already
provided with a convenience camera.
[0006] The present invention enables this more extensive reassurance, and this may be made
possible without the need to install in effect a complete burglar alarm system.
[0007] Embodiments of the invention are based on the insight that it may be possible to
combine a radio-based location sensing arrangement to detect human presence and location
based on detecting perturbations of radio signals, for example such a system based
on use of Wi-Fi signals, with a convenience camera it is possible to provide enhanced
security without the need to install a lot of additional components or pay significant
annual fees.
[0008] More generally, a non-alarm sensor, which is preferably a convenience camera but
could be another source of information such as a microphone, can be combined with
such a radio-based location sensing arrangement to provide a system such that in the
event of the non-alarm sensor detecting an "event" the radio-based location sensing
arrangement is checked to determine whether there is a human presence when/where no
human presence is expected, and triggering an alarm event and push/stream video/images
(information) to a user or other remote consumer in the event that the radio-based
location sensing arrangement detects human presence.
[0009] For the purposes of the present application, "burglary" should be understood in the
sense that the offence is defined under English law: entering a building or part of
a building as a trespasser intent to commit theft, grievous bodily harm, or criminal
damage; or having entered as a trespasser, stealing, or inflicting/attempting to inflict
grievous bodily harm. There is no requirement for the entry to involve "breaking in",
simply entering through an open or unlocked entrance is sufficient. Throughout the
specification we may refer to someone intent on committing burglary as a burglar,
intruder, or villain, as distinct from those resident at the property (on a temporary
or permanent basis) who do not fit within the definition of burglar - who we will
refer to a residents. In terms of detecting presence, movement, and location, both
of these classes of people fall within the term "occupant".
Summary
[0010] In a first aspect there is provided a security monitoring system for premises, the
system including: a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including
a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations
being configured to operate a radio-based location sensing arrangement to detect human
presence and location based on detecting perturbations of radio signals of the Wi-Fi
network, one of the stations of the Wi-Fi network being a motion-triggered video camera,
the video camera being configured to transmit a notification to a remote entity in
the event that it is triggered, the Wi-Fi access point being configured to: respond
to the transmission of a notification by the video camera by using the radio-based
location sensing arrangement to determine whether and where there is human presence
within the premises; and, in the event that human presence is detected, generate a
report, that includes details of where and when human presence has been detected;
and to notify the report to one or more designated recipients.
[0011] Optionally, the Wi-Fi access point is configured to receive a push notification from
the remote entity in the event of a notification being transmitted to the remote entity
as a result if the video camera having been triggered. Such push notification are
typically sent from the camera supplier's / operator's "cloud" servers to designated
recipients whose contact details (email address or mobile phone number, for example)
have been provided to the camera supplier/operator by the user of the camera - the
Wi-Fi access point can have its own email address and be provided with logic to enable
it to respond to the content of emails received from the service, and/or have a SIM
so that it can receive push notifications directly. All we need is for the Wi-Fi access
point to be able to recognise alert messages from the camera operator/supplier and
to distinguish these from other types of message.
[0012] Optionally, the Wi-Fi access point is configured to recognise notifications sent
by the video camera to the remote entity. That is, the Wi-Fi access point can be configured
to recognise characteristics of addressing, format or size, for example, of the Wi-Fi
signals that the camera transmits when it is reporting an event to the remote entity.
Again, the Wi-Fi access point doesn't need to read these messages as long as there
is some characteristic that distinguishes these messages from, for example, housekeeping
or "checking in" messages.
[0013] Optionally, the Wi-Fi access point has access to a database of locations within the
premises, and optionally, the Wi-Fi access point is configured to use the database
to determine whether a detected human presence is within a location included in the
database. A database of locations may be provided to enable the Wi-Fi access to point
to label the locations where human presence is detected (main bedroom, office, Juliet's
room, etc.), and can also be used to identify particular locations where human presence
should always be flagged up - for example the office, the gun room, the master bedroom,
etc., and to indicate other areas where human presence should only be flagged up at
certain times of day, for example, and yet other areas where human presence may be
acceptable at any time or at any time during the day - or for a certain duration,
for example on the doorstep of the front door, where temporary presence at any time
during the day is probably acceptable, but may be a concern at night or if the presence
extends to more than a few minutes.
[0014] Optionally, the Wi-Fi access point has access to a calendar or timetable for use
in determining whether detected human presence should be notified or not. By having
access to a calendar, timetable or schedule of the occupant(s) of the premises - e.g.
the household schedule, it is possible for the Wi-Fi access point to adapt its behaviour
to account for human presence, its count, and its location.
[0015] According to a second aspect, there is provided a method performed by a premises
security monitoring system, the system including:
a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of
Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured
to operate a radio-based location sensing arrangement to detect human presence and
location based on detecting perturbations of radio signals of the Wi-Fi network, one
of the stations of the Wi-Fi network being a motion-triggered video camera, the video
camera being configured to transmit a notification to a remote entity in the event
that it is triggered,
the method comprising:
the video camera transmitting a notification to the remote entity upon being triggered;
the Wi-Fi access point, in response to the transmission of the notification by the
video camera, determining using the radio-based location sensing arrangement whether
and where there is human presence within the premises; and, in the event that human
presence is detected,
generating a report, that includes details of where and when human presence has been
detected; and
notifying the report to one or more designated recipients.
[0016] Optionally, the determining whether and where there is human presence within the
premises is triggered by the Wi-Fi access point receiving a push notification from
the remote entity in the event of a notification being transmitted to the remote entity
as a result if the video camera having been triggered.
[0017] Optionally, the determining whether and where there is human presence within the
premises is triggered by the Wi-Fi access point recognising a notification sent by
the video camera to the remote entity.
[0018] Optionally, the method, further comprising the Wi-Fi access point accessing a database
of locations within the premises and using the database to determine whether a detected
human presence is within a location included in the database.
[0019] According to a third aspect, there is provided a method of detecting the presence
of an intruder in premises, the method comprising:
detecting an event at the premises using a non-alarm sensor, such as a "convenience"
camera;
receiving notification of the event and, based on receiving the notification, checking
a radio-based location sensing arrangement to detect human presence and location based
on detecting perturbations of radio signals to determine whether there is a human
presence where no human presence should be, and if such human presence is detected
creating an alert.
Brief description of the drawings:
[0020] Embodiments of the invention will now be described, by way of example only, with
reference to the accompanying drawings, in which:
Figure 1 illustrates schematically a plan of a single floor of premises in which a
security monitoring system has been provided using a video camera and a Wi-Fi access
point that provides radio-based presence and location sensing; and
Figure 2 illustrates schematically the principles of radio-based presence and location
sensing;
Specific description
[0021] Figure 1 is a schematic plan of a single storey dwelling 100 which we will use to
illustrate aspects of the invention.
[0022] The dwelling has a front door 102, the dwelling's main entrance, that is accessed
externally by a path 104. A doorbell 105 is provided adjacent the front door, preferably
in the form of a video doorbell which should preferably be arranged to provide a view
of the whole of the front approach of the house (by means of which the front door
can be accessed) including the path 104. The main entrance 102 leads into a hall 106
by means of which all the rooms of the house may be accessed. The dwelling has a rear
door 108 that leads out from a kitchen 110 onto a terrace 112. Each of the rooms includes
at least one window 114, or in the case of the living room 116 and master bedroom
118 a pair of French Windows 120 that open out onto the terrace 112. The living room's
French Windows permit access to the terrace and a rear garden, but are not intended,
or used, for regular access to the interior of the premises.
[0023] A "convenience" video camera (for example as provided by Ring as "stick up cameras")
130 is located in the hall 106 at a point where it provides a good view of the entrance
door 102, of the door of the study 136, as well as of a large expanse of the hall.
This camera is Wi-Fi enabled and is part of a Wi-Fi network provided by Wi-Fi access
point 128 which is located in the living room. The Wi-Fi access point 128 may connect
to the internet via gateway/router 600 which provides a cables broadband connection,
or optionally directly to the internet using a PLMN, such as a UMTS, 4G or 5G network
using an appropriate internal transceiver and SIM.
[0024] The video camera 130 is operatively connected, via the Wi-Fi network to a remote
entity 200, typically a cloud server provided by the supplier/operator of the camera
(e.g managed by Ring or Arlo, for example) to which events (including video clips)
are communicated when the camera is triggered (typically by means of an internal motion
sensor such as a PIR). These events may be onward reported through the cloud (the
manufacturer's or operator's back end systems) to designated individuals - for example
by sending push notifications and/or email notifications.
[0025] The front door is also provided with a doorbell 105, and if this is a video doorbell
this too may report incidents to the remote entity 200 via the Wi-Fi network provided
by access point 128. Likewise, a garden camera 144, if present, may also report incidents
to the remote entity 200 via the Wi-Fi network provided by access point 128.
[0026] Figure 1 also shows the presence of a plurality of devices that have Wi-Fi functionality,
and that are all stations, STA, in the Wi-Fi network provided by Access Point 128.
The observant reader will have noticed that this includes features whose presence
has not yet been mentioned, and these are elements that may play a role in providing
radio-based presence and location sensing (which we will hereafter refer to a Wi-Fi
sensing, or WFS, for convenience). The access point 128 functions as a WFS receiver,
and various Wi-Fi devices are positioned around the premises both to perform their
normal role but also to act as illuminators for WFS. Within the house there are several
Wi-Fi extenders 150 which are convenient because they can readily be positioned in
any vacant electrical socket, provide good signal strength, and typically have a small
form factor. These may also be used outside - if weatherproof or protected from the
weather. The external Wi-Fi camera 144, and video doorbell, if present, can also function
as outdoor illuminators to improve the reach of the WFS. In the kitchen 110 a smart
speaker 152 and a Wi-Fi extender 150 are provided as illuminators, while broadband
router or gateway 600 provides illumination from within the dining room. In the study
136 a "smart plug" 154 acts as illuminator, while in the hall the video camera 130
acts as an illuminator. It will be appreciated that this choice of Wi-Fi sources,
and their disposition, is given merely to illustrate a suitable approach - the number
of illuminators and the positioning required very much depend upon the area of cover
required for WFS and on the size and type of construction of the property being monitored.
It may not always be necessary to use external Wi-Fi illuminators in order to extend
Wi-Fi sensing to beyond the external walls of a building, but by providing some suitably
positioned external illuminated, as shown schematically here, it should be possible
to extend the range of WFS to achieve the desired results. With the ubiquity and wide
penetration of Wi-Fi devices into the lives of a large proportion of the population
of the developed world, it may often the case that no, or very few, Wi-Fi devices
need to be added to a home in order to provide a satisfactory level of WFS cover -
although that may not always be the case with the elderly.
[0027] We will now provide a brief introduction to radio-based presence detection, which
may for example be based on analysing the signal dynamics and signal statistics of
radio signals and/or detecting changes in channel state information (CSI). A radio
(or wireless) signal as used herein refers to a signal transmitted from a radio transmitter
and received by a radio receiver, wherein the radio transmitter and radio receiver
operate according to a standard or protocol. Such standards include, but are not limited
to, IEEE 802.11. (which includes the Wi-Fi standards), IEEE 802.15 (which includes
Zigbee), Bluetooth SIG, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 180,
GSM 19011, GPM ITU-R 5.13, GPM ITU-R 5.150, ITU-R 5.280, 3GPP 4G (including LTE),
3GPP 5G, 3GPP NR, AND IMT-2000. However, the radio transmitters and receivers providing
and using radio signals for WFS may operate in non-telecommunications or Industrial,
Scientific and Medical (ISM) spectral regions without departing from the scope of
the invention.
[0028] Essentially the idea is to use radio signals to probe a zone or zones of interest,
and to analyse and extract statistics from these signals, in particular looking at
the physical layer and/or data link layer such as MAC address measurements that expose
the frequency response of a radio channel (e.g., CSI or RSSI measurements). These
measurements are processed to detect anomalies and variations over time, and in particular
to detect changes signifying the entrance of a person and/or movement of a person
within a monitored zone. The zone(s) to be monitored need to be covered sufficiently
by radio signals, but the sources of the radio signals may either already be present
before a monitoring system is established - for example from the plurality of Wi-Fi
or Bluetooth capable devices that are now dotted around the typical home or office,
or the sources may be added specifically to establish a monitoring system. Often some
established (i.e., already located or installed) radio devices are supplemented by
some extra devices added as part of establishing a radio-based presence detection
system. Among the types of devices (pre-installed or specifically added) that may
be used as part of such a detection system are Wi-Fi access points, Wi-Fi routers,
smart speakers, Wi-Fi repeaters, as well as video cameras and video doorbells, smart
bulbs, etc. Because presence (or intrusion) is detected by detecting a change in the
properties or character of radio signals compared to some previous reference signal(s),
it is preferred to establish what might be termed the monitoring network between radio
devices that are essentially static (i.e., that remain in the same position for extended
periods) rather than relying on devices that are repeatedly moved - such as smart
phones, headphones, laptops, and tablet devices. It is not strictly speaking essential
for all the devices whose signals are used by the monitoring system to be part of
the same network - for example, signals from Wi-Fi access points of neighbouring premises
could be used as part of a monitoring system in different premises. Again, a primary
consideration is the stability of the signals from the signal sources that are used.
Wi-Fi access points provided by broadband routers are seldom moved and rarely turned
off, consequently they can generally be relied upon as a stable signal source - even
if they are in properties neighbouring the property containing the zone or zones to
be monitored.
[0029] The idea is illustrated very schematically in Figure 2, here with an installation
200 including just a single source (or illuminator) 202 and just a single receiver
204, for simplicity, although in practice there will typically be multiple sources
(illuminators) and sometimes plural receivers. The installation 200 has been established
to monitor a monitored zone 206. In Figure 2A we see that in steady state, and in
the absence of a person, radio signals are transmitted from the source 202, spread
through the monitored zone 206, and are received by the receiver 204. Of course, in
most installations there will be walls, ceilings, floors, and other structures that
will tend to reflect, at least in part, signals from the source. Furniture and other
objects may block and attenuate the signals, the reflected signals will give rise
to multiple paths, and the signals may interfere with each other, and there may be
scattering and other behaviours, such as phase shifts, frequency shifts, all leading
to complexity in the channels experienced by the radio signals that arrive at the
receiver 204. But while the environment is static and unchanging, the receiver will
tend to see a consistent pattern of radio signals. And this is true whether or not
the source transmits continuously or transmits periodically. But this consistent pattern
of received signals is changed by the arrival of an intruder 208, as shown in Figure
2B. From Figure 2B we see that, at the very least, the presence of a person in the
monitored zone blocks at least some of the signals from the source, and that affects
the pattern of radio signals received by the receiver 204. The changed pattern of
signals received by the receiver enables the presence of the intruder to be detected
by a presence monitoring algorithm that is supplied with information derived from
the received signals. It will be appreciated that the nature and extent of the perturbation
of the signals passing from the source 202 to the receiver 204 is likely to change
as the intruder 208 enters, passes through, and leaves the monitored area 206, and
that this applies also to reflected, refracted, and attenuated signals. These changes
may enable the location of a person within the zone, and their speed of movement,
to be determined. Indeed, these techniques have been shown even to be capable of detecting
gestures, and patterns of human respiration, as well as enabling "people counting".
[0030] It will be realised that signals that are received from an illuminator device (or
from more than one illuminator device) after having passed through a monitored space
(or volume), have in effect been filtered by the environment to which they have been
exposed. We can therefore imagine the monitored volume as a filter having a transfer
coefficient, and we can see that a received signal is at least in part defined by
the properties, or channel response, of the wireless channel through which it propagated.
If the environment provided by the monitored volume changes, for example by the addition
of a person, then the transfer coefficient of the filter, and the channel response
or properties, will also change. The changes in the transfer coefficient, and in the
channel response, consequent on the change in the environment of the monitored space,
can be detected and quantified by analysing radio signals received by the wireless
sensing receiver(s). Both the introduction of an object, e.g. a person, into the monitored
space, and movement of that object within the monitored space will change the environment
and hence change the effective transfer coefficient and the channel response.
[0031] The radio-based sensing system can be trained by establishing a base setting in which
the monitored zone is unoccupied, which is then labelled as unoccupied for example
using a smartphone app or the like, and then training occupied states by a person
entering, standing, and then walking through each of the zones one by one. Presence
at different locations in each of the zones may be captured and labelled in the system
in the same way. This process may be repeated with two people, and then optionally
with more people. In essence this is a supervised machine learning approach, but other
approaches to training may be used.
[0032] The system may need to be retrained for the base setting if bulky furniture or other
large objects (particularly if made of metal) are added to or moved within the monitored
space, because these can be expected to change the propagation properties of the relevant
zone/space. The data for unoccupied states are preferably retained within a database
of "unoccupied" states, even when there are changes to the arrangement of furniture
etc. It may not be necessary to retrain for the occupied states if the system can
determine a delta function between the previous base state and the new one, because
the delta function may also be applicable in occupied states. But if not, it may be
sufficient to retrain only a subset of the occupied states previously learnt. The
system may also be configured to self-learn to accommodate changes in the characteristics
of the zones when unoccupied, and to add newly determined unoccupied state data to
the database.
[0033] Although the Figure 2 example uses just a single source (illuminator) and a single
receiver, as already mentioned generally multiple sources (illuminators) will be used
in order to achieve satisfactory coverage of the zone or zones to be monitored. Multiple
zones may be monitored by a single receiver through the use of multiple strategically
placed sources, but each zone, or some zones of multiples zones may have a dedicate
receiver that does not serve other zones. Likewise, a radio signal source (illuminator)
may provide illuminating signals for a single monitored zone or for multiple monitored
zones. Also, a presence monitoring system (and a security monitoring system including
such a presence monitoring system) may use mesh network arrangement, for example a
Wi-Fi mesh network, in which multiple devices act as receivers for illuminating signals
- either for a single monitored zone or for multiple monitored zones.
[0034] In a first aspect there is provided a security monitoring system for premises, the
system including: a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including
a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations
being configured to operate a radio-based location sensing arrangement to detect human
presence and location based on detecting perturbations of radio signals of the Wi-Fi
network, one of the stations of the Wi-Fi network being a motion-triggered video camera,
the video camera being configured to transmit a notification to a remote entity in
the event that it is triggered, the Wi-Fi access point being configured to: respond
to the transmission of a notification by the video camera by using the radio-based
location sensing arrangement to determine whether and where there is human presence
within the premises; and, in the event that human presence is detected, generate a
report, that includes details of where and when human presence has been detected;
and to notify the report to one or more designated recipients.
[0035] Optionally, the Wi-Fi access point is configured to receive a push notification from
the remote entity in the event of a notification being transmitted to the remote entity
as a result if the video camera having been triggered. Such push notification are
typically sent from the camera supplier's / operator's "cloud" servers to designated
recipients whose contact details (email address or mobile phone number, for example)
have been provided to the camera supplier/operator by the user of the camera - the
Wi-Fi access point can have its own email address and be provided with logic to enable
it to respond to the content of emails received from the service, and/or have a SIM
so that it can receive push notifications directly. All we need is for the Wi-Fi access
point to be able to recognise alert messages from the camera operator/supplier and
to distinguish these from other types of message.
[0036] Optionally, the Wi-Fi access point is configured to recognise notifications sent
by the video camera to the remote entity. That is, the Wi-Fi access point can be configured
to recognise characteristics of addressing, format or size, for example, of the Wi-Fi
signals that the camera transmits when it is reporting an event to the remote entity.
Again, the Wi-Fi access point doesn't need to read these messages as long as there
is some characteristic that distinguishes these messages from, for example, housekeeping
or "checking in" messages.
[0037] Optionally, the Wi-Fi access point has access to a database of locations within the
premises, and optionally, the Wi-Fi access point is configured to use the database
to determine whether a detected human presence is within a location included in the
database. A database of locations may be provided to enable the Wi-Fi access to point
to label the locations where human presence is detected (main bedroom, office, Juliet's
room, etc.), and can also be used to identify particular locations where human presence
should always be flagged up - for example the office, the gun room, the master bedroom,
etc., and to indicate other areas where human presence should only be flagged up at
certain times of day, for example, and yet other areas where human presence may be
acceptable at any time or at any time during the day - or for a certain duration,
for example on the doorstep of the front door, where temporary presence at any time
during the day is probably acceptable, but may be a concern at night or if the presence
extends to more than a few minutes.
[0038] Optionally, the Wi-Fi access point has access to a calendar or timetable for use
in determining whether detected human presence should be notified or not. By having
access to a calendar, timetable or schedule of the occupant(s) of the premises - e.g.
the household schedule, it is possible for the Wi-Fi access point to adapt its behaviour
to account for human presence, its count, and its location.
[0039] According to a second aspect, there is provided a method performed by a premises
security monitoring system, the system including:
a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of
Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured
to operate a radio-based location sensing arrangement to detect human presence and
location based on detecting perturbations of radio signals of the Wi-Fi network, one
of the stations of the Wi-Fi network being a motion-triggered video camera, the video
camera being configured to transmit a notification to a remote entity in the event
that it is triggered,
the method comprising:
the video camera transmitting a notification to the remote entity upon being triggered;
the Wi-Fi access point, in response to the transmission of the notification by the
video camera, determining using the radio-based location sensing arrangement whether
and where there is human presence within the premises; and, in the event that human
presence is detected,
generating a report, that includes details of where and when human presence has been
detected; and
notifying the report to one or more designated recipients.
[0040] Optionally, the determining whether and where there is human presence within the
premises is triggered by the Wi-Fi access point receiving a push notification from
the remote entity in the event of a notification being transmitted to the remote entity
as a result if the video camera having been triggered.
[0041] Optionally, the determining whether and where there is human presence within the
premises is triggered by the Wi-Fi access point recognising a notification sent by
the video camera to the remote entity.
[0042] Optionally, the method, further comprising the Wi-Fi access point accessing a database
of locations within the premises and using the database to determine whether a detected
human presence is within a location included in the database.
[0043] According to a third aspect, there is provided a method of detecting the presence
of an intruder in premises, the method comprising:
detecting an event at the premises using a non-alarm sensor, such as a "convenience"
camera;
receiving notification of the event and, based on receiving the notification, checking
a radio-based location sensing arrangement to detect human presence and location based
on detecting perturbations of radio signals to determine whether there is a human
presence where no human presence should be, and if such human presence is detected
creating an alert.
[0044] The access point 128 may be further configured to use data from the radio-based location
sensing arrangement to perform people counting, and optionally to use determine the
presence of one or more intruders based on a detected change in the people count when
the system is in the nocturnal armed mode. For example, the techniques and methods
described in
US2020/0302187A1, assigned to Origin Wireless, can be used to count occupants and determine their
locations in installations, systems and methods according to embodiments of the invention.
The access point 128 can, for example use people counting to keep track of the number
of people in the house - and this may include being aware of the number of people
who have, for example, gone into the rear garden or who are on the terrace.
[0045] The access point 128 may also be configured to receive and store a number corresponding
to the number of usual residents - something that is particularly useful if the dwelling
is the home of someone who lives alone. In such a case, the access point 128 can improve
its confidence in determining whether detected human presence (and count) constitutes
a reportable event or not. The access point 128 could also be programmed to deal with
days or periods when more residents than usual are expected - for example the weekly
visit of a cleaner, carer, or heath visitor, or the monthly visits of family or friends.
Such information could be entered by the resident or an authorised user or installer
using, for example an app on a device or online, or via spoken commands or touch screen
input at the access point 128, or entered via a web app or a smartphone app.
[0046] A brief explanation will now be given of how Wi-Fi Sensing works, and how Wi-Fi Sensing
can be integrated into a security monitoring system, and in particular how WFS can
be integrated into a central unit of a security monitoring system.
[0047] Wi-Fi Sensing can be performed with any Wi-Fi device and can be used on any available
communication path. Each communication path between two devices gives the chance to
extract information about the surrounding environment. Wi-Fi sensing is based on an
ability to estimate the wireless channel and hence the surrounding environment. Because
Wi-Fi networks comprise many devices spread throughout a geographical area, they are
well suited to exploiting these devices' transmissions in effect to provide a radar
system. Depending on the number of devices, the radar system may be monostatic, bistatic,
or multistatic. In monostatic WFS, a single device measures its own transmitted Wi-Fi
signals. In bistatic WFS, the receiver and transmitter are two different devices (for
instance, an AP and a STA in infrastructure mode). In multistatic WFS, the received
signals from multiple Wi-Fi transmitters are used to learn about a shared environment.
[0048] At least one Wi-Fi transmitter and one Wi-Fi receiver are required to perform WFS
measurements, and these can be located in the same device (to create a kind of monostatic
radar) or in different devices. The measurement is always performed by a Wi-Fi Sensing-enabled
receiver on the Wi-Fi signal transmitted by a transmitter, and which may or may not
originate from a Wi-Fi sensing-capable device. The device that transmits the signal
that is used for measurements is called the "illuminator," as its transmissions enable
collection of information about the channel - that is, it illuminates the channel.
[0049] Different modes of Wi-Fi Sensing measurements are recognised - Passive, Triggered,
Invoked, and Pushed, and these depend upon what triggers the illuminator device to
transmit a Wi-Fi signal. Preferably the agent improves the usefulness of the standard
beacon interval by using optimised timings.
[0050] In passive mode, WFS relies on transmissions that are part of regular Wi-Fi communication.
The Wi-Fi Sensing receiver(s) rely only on transmissions between itself and the illuminator
device(s). Passive transmissions do not introduce overhead, but the Wi-Fi sensing
device lacks control over the rate of transmissions, transmission characteristics
(bandwidth, number of antennas, use of beamforming), or environmental measurements.
[0051] Triggered measurement happen when a Wi-Fi Sensing device is triggered to transmit
a Wi-Fi packet for the purpose of WFS measurements, either in response to a received
Wi-Fi packet or by the higher layers (for instance, in WFS software).
[0052] Invoked measurement involves utilizing a packet transmission that is in response
to a packet received from the Wi-Fi Sensing receiver device.
[0053] In pushed mode, a transmission is initiated by the illuminator device for measurement.
A pushed transmission can be either a unicast or a multicast/broadcast message. Multicast/broadcast
messages can be used for measurements by multiple WFS receivers simultaneously if
the devices are not in power-save mode. Triggered transmissions introduce overhead
because additional over-the-air transmissions are required. Pushed transmissions introduce
less overhead compared to invoked transmissions, because the exchange is unidirectional
rather than bidirectional. Triggered transmissions allow for a system to control both
the rate and occurrence of measurements.
[0054] A WFS network is made up of one or more WFS illuminators and one or more WFS receivers.
A WFS system is made up of three main components and that are present in Wi-Fi Sensing
illuminators and receivers:
first is the Wi-Fi radio, which encompasses the radio technology specified in IEEE
802.11 standards, the interfaces and the APIs connecting the radio to the higher layers;
second is the Wi-Fi Sensing software agent, consisting of a signal processing algorithm
and interfaces, the agent interacting with the Wi-Fi environment, and turning radio
measurement data into motion or context-aware information; and
thirdly, an application layer operates on the Wi-Fi sensing output and forms the services
or features which are ultimately presented to an end user - such as a security monitoring
service provided by a security monitoring system that detects presence using WFS.
[0055] A WFS system can be built based on existing Wi-Fi standards, hardware, software and
infrastructure.
[0056] The fundamental component required to enable Wi-Fi sensing on the radio is the interface
to enable control and extraction of periodic channel or environmental measurement
data. Regardless of device type, operating band or Wi-Fi generation, the core APIs
to enable Wi-Fi sensing are similar, as the required data and control are common.
[0057] The WFS software Agent can reside on any Wi-Fi device; for example, in the infrastructure
mode, the agent may reside on the AP, in which case channel measurements from all
the STAs associated with the AP can be collected. The software agent may also be located
on a STA. But in the security management system applications this would mean that
the STA would either need to be the controller of the security management system (e.g.
the CU), or would have to be reporting to the controller of the security management
system (e.g. the CU). Generally, we therefore prefer to run the software agent on
the CU, and given that the CU is conveniently also an access point, it makes sense
for us to run the software agent on the CU acting as AP rather than merely as an STA.
[0058] The WFS software Agent uses the WFS radio APIs to interact with the Wi-Fi radio,
the APIs enabling extraction of desired channel environment measurement information
and providing the ability to assert any related controls to configure WFS features.
[0059] The WFS Agent has two main subsystems: Configuration and Control; and a Sensing Algorithm.
The Configuration and Control subsystem interact with the radio, using a standard
set of APIs. The Configuration and Control subsystem performs tasks including sensing
capability identification, pushed illumination coordination, and radio measurement
configuration. The sensing algorithm subsystem includes intelligence needed to extract
the desired features from the radio measurement data and may differ according to the
desired sensing application.
[0060] The WFS software Agent is needed on any sensing receiver but is merely optional on
an illuminator - only being required if the illuminator also acts as a receiver. If
included on an illuminator, only the configuration and control subsystem is needed.
By having the agent on the illuminator, additional enhancements are enabled, including
sensing capability identification and co-ordinated pushed illumination. If the illuminator
is not running an agent, it is still technically able to participate in the sensing
network, but only the most basic features that currently exist in Wi-Fi standards
will be supported.
[0061] The WFS software Agent processes and analyses the channel measurement information
and makes sensing decisions, such as detecting motion. This information is then shared
with the application layer via the Wi-Fi Sensing agent I/O interface. As well as interfacing
with the radio and the application layer, the Wi-Fi Sensing agent also interfaces
with the existing Wi-Fi services on the system. This interface is necessary for the
agent to provide feedback for sensing optimizations that can be used in radio resource
management decisions, such as band steering or AP selection requests.
[0062] The application layer of a WFS system creates the sensing service and in effect presents
the information to the end user (in our case to the security management system).
[0063] The application layer can potentially reside on any networked device: in some embodiments
of the present invention, it will reside in the central unit 122 along with the WFS
agent, but in other embodiments the application layer may exist in an external server
or even in the central monitoring station. We prefer, however, to provide the application
layer on the central unit to avoid potential problems with signalling delays (for
example due to accidental or deliberate network interruption) between the central
unit (or other WFS receiver) and a remotely located entity. The application layer
receives input from one or multiple Wi-Fi sensing software agents. It combines the
information and delivers it to the security management system which may then in turn
provide it to the CMS and/or to a cloud service by means of which push notifications
may be sent to a registered user device such as a smartphone - allowing users to receive
real-time notifications and the ability to view historic data.
[0064] A typical Wi-Fi home network follows one of two common deployment scenarios. The
first consists of a single AP that serves as the internet gateway for all the devices
in the house. The second consists of multiple APs forming an ESS and extending coverage
throughout the home. Depending on the use case, the Wi-Fi Sensing receiver may be
the AP and/or other devices in the network. Not all the devices in a home deployment
need to be Wi-Fi Sensing capable.
[0065] Wi-Fi Sensing can be deployed in all types of Wi-Fi networks and topologies, operating
in different frequency bands (2.4, 5, 6, and 60 GHz) and different bandwidths. The
sensing resolution and performance depends on the use case requirements. In general,
it is enhanced with the increase in the number of participating devices and higher
bandwidths. Applications that require lower resolutions and longer range, such as
home monitoring, can be deployed using Wi-Fi networks operating in 2.4GHz and 5GHz.
Applications that require higher resolutions and lower range, such as gesture recognition,
require 60GHz Wi-Fi networks.
[0066] In multi-AP and/or multi-band deployments, there may be an advantage to having a
Wi-Fi sensing device connected to a specific AP or operating in a specific frequency
band. Radio resource management (RRM) events, such as AP and/or band steering, should
be conducted in coordination with the Wi-Fi Sensing agent/operation.
[0067] The devices involved with Wi-Fi Sensing will depend upon the deployment environment
and the specific use case. The sensing measurements also need to be processed by the
device with enough computation power. The coordination of sensing, including participating
devices, is a role particularly suited to an AP. Typically the central unit of a security
monitoring system will have ample processing power, as well as being able to function
as an AP, to handle this task efficiently and speedily.
[0068] The nature of Wi-Fi networks is such that it should be possible able to add additional
Wi-Fi sensing capable devices to the network to enhance accuracy, coverage and/or
localization. These additional devices do not necessarily need to be Wi-Fi Sensing
capable or dedicated Wi-Fi sensing devices to participate; however, optionally they
may also identify their Wi-Fi sensing capabilities and supported features to the AP.
Internet of Things (IoT) devices for home deployment can typically also be used as
part of a WFS installation supporting a WFS-enabled security monitoring system: example
include Wi-Fi controllable plugs and sockets, light bulbs, thermostats, smart speakers,
and video door bells. However, even when a device connects to the AP and reports that
it is Wi-Fi sensing capable, the Wi-Fi Sensing agent may elect not to make use of
that device.
[0069] WFS for a security monitoring system may be run over a dedicated Wi-Fi network, the
premises having at least one other Wi-Fi network for other purposes. But for reasons
of simplicity and economy it may often be preferred to operate a single Wi-Fi network
to serve all a household's (or small business's) needs including WFS for a security
monitoring service. If a single-network solution is adopted, performance degradation
due to airtime usage and sensing overhead must be minimized and hence Wi-Fi transactions
required for conducting sensing measurements and sensing management and processing
must be optimized for efficiency.
[0070] For each Wi-Fi Sensing application, at least one network device executes the sensing
software, or Wi-Fi Sensing Agent. The Wi-Fi Sensing agent is typically placed on the
AP, but it can be placed on any STA (although, as previously mentioned, we prefer
to run the Wi-Fi Sensing agent on the AP). Following authentication and association
of a device with the Wi-Fi network, the Wi-Fi Sensing agent should discover the device
and its sensing capabilities. Depending on the capabilities of the device, its role
in the Wi-Fi sensing network would be determined. If the new device is another Wi-Fi
Sensing-capable AP, then coordination among the agents is required.
[0071] The WFS agent needs to have a mechanism to determine which devices are capable and
needs to participate in the sensing for each application on a device-specific basis.
[0072] A WFS agent also needs to be capable of configuring the radio for measurements and
triggering transmissions on a periodic basis for sensing measurements, and to enable/disable
measurements or adjust configuration parameters for Wi-Fi sensing-capable devices.
Optionally, the Wi-Fi Sensing agent is also able to request specific radio resource
management operations, such as AP or band steering. The WFS agent is also preferably
able to detect and process specific sensing events and communicate the relevant information
to the application layer (e.g., the security monitoring system) for specific handling
and user presentation.
[0073] One of the parameters that impacts the quality of the received signal in a wireless
network is the amount of interference present. Interference can be caused by other
Wi-Fi devices operating in the same band, which causes cochannel interference, or
in an adjacent channel, which causes adjacent channel interference. It can also be
caused by non-W-Fi devices, which can be other communication systems or unintentional
transmissions that create electromagnetic noise in the band. Interference can impact
Wi-Fi Sensing performance in two ways. Firstly, it may interfere with the sensing
transmissions and thereby reduce the number of measurements made in a given time interval.
As such, it introduces jitter in time instants during which the measurements are made.
Secondly channel-state measurements may capture the impact of transient interference,
such as for a non-Wi-Fi device, as opposed to motion in the environment.
[0074] Wireless systems deploy various techniques to avoid or reduce the impact of interference,
and these techniques also help to improve WFS performance. These techniques aim at
maximizing the reuse of spectrum, while minimizing the overlap of spectrum used by
nearby networks: for example, Dynamic Channel Allocation (DCA); Auto Channel Selection
(ACS); optimized RF planning; (e.g., non-overlapping channels and use of reduced channel
width when applicable), and power control.
[0075] As already mentioned, increasing the number of illuminators may result in a higher
sensing performance: with more transmitters that are located sufficiently apart from
one another, motion in a larger area can be detected; when motion is detected using
transmissions on one or more transmitters, information is provided that can be used
to determine localization of the motion; and sensing accuracy is improved with a higher
number of measurements taken across a larger number of transmitters in most scenarios.
[0076] The IEEE 802.11a preamble is useful for Wi-Fi Sensing. The preamble contains a short
training field (STF), a guard interval and a long training field (LTF). The STF is
used for signal detection, automatic gain control (AGC), coarse frequency adjustment
and timing synchronization. The LTF is used for fine frequency adjustment and channel
estimation. Since only 52 subcarriers are present, the channel estimation will consist
of 52 frequency points. Newer OFDM PHY versions (HT/VHT/HE) maintain the IEEE 802.11a
preamble for backward compatibility and refer to it as the legacy preamble. The legacy
preamble spans a 20MHz bandwidth and consists of a legacy STF (L-STF) and legacy LTF
(L-LTF). As more recently defined OFDM PHY versions (HT/VHT/HE) introduce wider channel
bandwidths (up to 160MHz) for backward compatibility, the legacy preamble is duplicated
on each 20MHz channel. This allows the receiver to compute 52, 104, 208 or 416 valid
L-LTF frequency points, which represent the channel estimation between the two devices.
Also potentially useful for Wi-Fi Sensing are the MIMO training fields present in
HT, VHT and HE LTFs. The MIMO fields are modulated using the full bandwidth (20MHz
to 160MHz) and are traditionally used by the receiver to estimate the mapping between
the constellation outputs and the receive chains. Since these fields span the full
bandwidth, they provide more frequency points. For example, a 20MHz L-LTF contains
52 subcarriers, while a 20MHz HT/VHT-LTF contains 56 subcarriers. The latest introduction
of the HE PHY has the potential to enhance Wi-Fi Sensing. In addition to enabling
operation in the 6GHz spectrum, the HE PHY has increased the number of subcarriers
per 20MHz bandwidth by 4x, which effectively allows for better object resolution.
[0077] The IEEE 802.11ad amendment defines a Directional-Multi-Gigabit (DMG) PHY for operation
in the 60GHz band. While there are three different modulation schemes (Control, Single-Carrier
and OFDM) defined, Control and the Single Carrier PHY are the primary PHY used in
802.11ad (and is also part of the subsequent 802.11ay amendment). Regardless of the
modulation scheme, every packet starts with a preamble that consists of a short training
field (STF) and a channel estimation field (CEF). The STF is used for timing estimation
and AGC adjustment. CEF is used for channel estimation. Similar to the OFDM-based
PHYs, the necessary channel estimation for Wi-Fi Sensing is available following successful
reception and processing of the preamble of a packet and can be provided to the higher
layers. The wide channel bandwidth available in 802.11ad/ay can significantly improve
the performance of Wi-Fi Sensing in terms of the resolution; however, the limited
communication range in 60GHz band restricts the sensing range and coverage. As such,
in many situations the central unit of a security monitoring system may relay instead
on frequency bands with longer range, sufficient to cover the majority of households.
However, for smaller-scale installations the use of the 60GHz band may be attractive
and therefore embodiments of the invention may use this band for WFS.
[0078] When it comes to identifying peer devices in a WFS installation, the MAC layer mechanisms
may be used to obtain information about the connected devices and the roles they play
in Wi-Fi sensing. The MAC layer also initiates and drives transmissions required for
channel estimation among the devices in the Wi-Fi Sensing network.
[0079] Various aspects of peer identification arise with Wi-Fi Sensing. The first is identifying
the devices and the channel estimation mapped to the physical environment between
any two devices. Typically, an STA is identified by a 48-bit MAC address. A MAC address
is sufficient identification for STAs associated with a Wi-Fi network; however, if
the association is lost during the lifetime of the application, then randomized MAC
addresses may be used. In this case, a different or more involved mechanism would
be required to identify each STA. This identification must match the corresponding
channel estimate measurement obtained from the PHY. The second is identifying the
device network role and its connection type, such as whether it is an AP or an STA,
or whether it is part of a mesh or a P2P connection. This information is used by the
Wi-Fi Sensing agent to decide the best method for conducting measurements.
[0080] The third aspect is the identification of WFS device capabilities, such as sensing
capabilities, supported measurement rate, and the availability and willingness of
the device to participate in sensing measurements. This information is required from
all devices in the network for the Wi-Fi Sensing agent to select devices participating
in the sensing measurements.
[0081] As already noted, there are different types of transmissions that can be used for
illumination of the Wi-Fi channel and obtaining measurements between two devices.
Passive transmissions rely on existing Wi-Fi traffic and do not introduce any new
MAC layer requirements. Triggered transmissions, however, rely on additional transmissions.
Depending on whether existing packet exchange procedures are used for triggered transmissions
or new exchanges are defined, the requirements on the MAC layer will be different.
An example of one existing packet exchange that can be used for triggering invoked
transmissions is null data packet (NDP) and ACK exchange. NDP transmission by the
Wi-Fi Sensing receiver can be used to invoke a Wi-Fi Sensing transmitter to respond
with an ACK, which may then be used to compute a channel estimation. The disadvantage
of using ACK packets for channel estimation, in 2.4/5GHz bands, is that the ACKs are
only transmitted in legacy mode. Another example of how an invoked measurement can
be triggered is by use of the implicit unidirectional beamforming procedure, first
defined in the IEEE 802.11n standard. In this procedure, an STA requests beamforming
training by sending a MAC frame with the training request (TRQ) bit set to 1. This
triggers the receiving device to send an NDP announcement, followed by an NDP to illuminate
the channel. The benefit of this invoked measurement is that it is not limited to
the legacy preamble for channel measurements and uses the MIMO training fields, as
well.
[0082] In pushed measurements, a transmission is triggered by the illuminator to be received
by one or multiple Wi-Fi Sensing receivers. Beacon frames are an example of using
existing MAC packet exchanges for pushed measurements.
[0083] Also as already noted, to support different use cases, either the AP or STA may take
the role of sensing receiver; additionally, there may be multiple sensing receivers
required to support the application. Moreover, there may be multiple illuminators
involved in the measurements. MAC layer coordination is used to coordinate the sensing
transmissions among the illuminators and the sensing receivers in an efficient way.
MAC layer scheduling may also be used to enable periodic measurements on which some
use cases rely. Coordination and scheduling at the MAC layer should enable different
options for conducting sensing measurements among multiple illuminators and sensing
receivers, with minimal added overhead, while accounting for the power save state
of the devices.
[0084] To interact with the MAC and PHY, the WFS agent has an interface to pass the WFS
control information to the radio and extract the measurement data. The interface should
be PHY agnostic and is, therefore, defined in a generic manner and extendable to cover
different radio driver implementations, including drivers from different chipset vendors.
The interface definition should allow for potential additional features or capabilities
provided by a specific PHY or a chipset, as well as a path for growing the technology.
Definition of a standard interface/API enables radio firmware and driver developers
to ensure compliance and enables reuse of components or common codes, which may be
placed into a library. Most Wi-Fi drivers are based on either the wireless-extensions
framework or the more recent and actively developed cfg80211 / nl80211 framework.
As the system integration components are largely provided, these frameworks enable
Wi-Fi driver developers to focus on the hardware aspects of the driver. These frameworks
also offer significant potential as a location for defining a WFS API. The WFS interface
should provide the WFS agent with STA identification and enable the WFS agent to track
the physical device in the network (i.e., the AP to which it is connected), as well
as the device's capability and availability to participate in the measurements.
[0085] The WFS agent requires control of the STAs that will participate in the sensing measurements,
as well as what measurement type (passive vs triggered) will be performed. The WFS
interface should provide such control, either on a global system scale or on a per
STA basis so that the WFS agent can conduct WFS measurements in the most efficient
manner.
[0086] Based on the specific WFS application or use case, different measurement rates may
be required. The measurement rate is typically decided by the WFS agent, and the interface
should support its control. However, to provide the lowest jitter and best efficiency
possible, it is best to rely on the MAC layer for scheduling. WFS applications may
have different measurement parameter requirements (bandwidth, antenna configuration,
etc.). The configuration of measurement parameters allows the application to obtain
only the data it requires to maintain efficiency. The measurement parameters should
be configurable independently for each STA.
[0087] The WFS interface should be flexible enough for the radio to specify whether the
data payload is in time-domain or frequency-domain, the numerical format, etc. By
having this knowledge, the Wi-Fi Sensing agent can correctly interpret the data.