|
(11) | EP 0 805 426 A2 |
| (12) | EUROPEAN PATENT APPLICATION |
|
|
|
|
|||||||||||||||||||
| (54) | A surveillance system and method of operation thereof |
| (57) A surveillance system comprises at least one remote device (2) and a surveillance
device (4) including camera means (6). The camera means are adapted to produce a series
of images of a static zone to be monitored. An image processor including comparison
means arranged to compare each of said images with a previously produced reference
image is also included in order to detect any difference therebetween. The image processor
is further arranged to select any portion of any of said images which differs from
the reference image. Communication means (12) are arranged to transmit image data
characteristic of said portion to said at least one remote device, the surveillance
system operating under the control of a control means (14). |
[0001] This invention relates to a surveillance system and method of operation thereof, and in particular to a system in which data relating to trespass on a monitored zone is transmitted to a remote location.
[0002] The security and surveillance industry is one which is in the process of rapid growth. This is due partly to a general feeling of insecurity on behalf of members of the public and owners of commercial sites coupled with improved technology. Consequently both private homes and commercial locations are being fitted with surveillance systems including close circuit television or burglar alarms. The burglar alarms tend to include an audio or visual alarm device located on the premises and/or a communication link to a remote location.
[0003] A significant problem which has arisen with this increasing number of alarm systems is the increasing number of false alarms being raised. This problem has reached such major proportions that police authorities are refusing to respond to a burglar alarm which has not been verified as being an actual alarm as opposed to an error of some sort. Thus, mere audio or visual alarms are not being responded to at all and alarms connected to a police station are required to be connected via a central operations control system. Thus, if an alarm is triggered a signal is sent to a central location wherein a operator endeavours to determine whether or not the alarm was raised in error. This is achieved by contacting a number of named persons via telephone and requesting information on the cause of the alarm coupled with a security code or pass word. If this verification can not be obtained the central location contacts the police who will then respond to the alarm. However, this is a very expensive process which is not always successful and can lead to a failure to detect false alarms merely because a designated person is not available.
[0004] Therefore, there is a need in this field for a method and apparatus which would alleviate the aforementioned problems.
[0005] GB 2260880A addresses the problem in relation to personal alarm systems and includes a camera and a transmitter. The camera when activated either by a loud noise or by manual activation captures a photograph of an intruder and transmits that photograph to a remote location. This results in a deterrent effect, as evidence of the intrusion is sent to a remote site, be it a police station or other site. On this action an alarm can also be raised. However, this prior art system has a substantial problem in that the amount of data required to be sent to transmit a full image including superfluous background is significant. This can lead to a relatively long period of time being taken up in the transmission. Also a significant bandwidth is required for such a transmission system.
[0006] Therefore, it is an object of the present invention to produce a surveillance system and method of operation thereof which enables transmission with reduced bandwidth and reduced transmission times.
[0007] According to a first aspect of the present invention there is provided a surveillance system comprising at least one remote device and a surveillance device including camera means adapted to produce a series of images of a static zone to be monitored, an image processor including comparison means arranged to compare each of said images with a previously produced reference image, in order to detect any difference therebetween, said image processor being further arranged to select any portion of any of said images which differs from said reference image, and communication means arranged to transmit image data characteristic of said portion or portions to said at least one remote device, the surveillance system operating under the control of a control means.
[0008] According to a second aspect of the present invention there is provided a surveillance means for use in the system as described above.
[0009] According to a third aspect of the present invention there is provided a remote device for use in the system as described above.
[0010] According to a fourth aspect of the present invention there is provided a method of surveillance of a static zone including producing a series of images of the zone, comparing each of said images with a reference image in order to detect a difference therebetween, selecting any portion of any of said images which differs from said reference image and transmitting image data characteristic of said portion or portions to at least one remote device arranged to receive said image data.
[0011] An embodiment of the present invention will now be described, by way of example, with reference to the enclosed Figure.
[0012] The Figure illustrates a block diagram of a surveillance system comprising a plurality of remote devices 2 in the form of GSM compatible mobile telephones each of which is fitted with a visual display. Other telephones such as land line telephones and other display means such as facsimile means may be utilised in alternate embodiments of the present invention. The surveillance system also includes a surveillance device 4 including camera means 6 in the form of a CCD camera. Such camera typically would cover a field of view of 60°, equalling approximately an area of 4.6X3.5m at a range of 4m. Resolution of 320X240 pixels with a fixed focus of 200mm to infinity is provided. Other video cameras such as infra-red cameras may be utilised in an alternate embodiment of the present invention. The system also includes an image processor 8 including a comparison means 10 arranged to compare each of the images with the previously produced reference image. The reference image is normally produced after the production of every thirty images of the zone. Each of the images of the zone is compared, in the comparator 10, to the reference image and any difference therebetween is detected. The image processor 8 is adapted to select any portion of any of said images which differs from the reference image. Communication means 12 are arranged to transmit the image data characteristic of the portion or portions which differ from the reference image to at least one of said remote devices 2. The system is arranged such that the remote devices are ranked in order of importance. If the system fails to make contact with the first device a subsequent device 2 is contacted. If the system fails to contact any of the devices a taped message to this effect is transmitted by the communication means 12 to a further remote site such as a local police station.
[0013] Prior to the image data being transmitted it is compressed. In an alternate embodiment vertical resolution may be utilised instead of compression. When each of the images is compared with a reference image if more than 2% of the image data in the compared image differs from that of the image data for the reference image then a determination is made that the image includes a difference. Image data relating to that difference only is then compressed and transmitted to the remote devices. Thus the system in accordance with the present invention requires significantly less band width and transmits data significantly quicker than systems in accordance with the prior art. Throughout the comparison corresponding pixels in each image are compared.
[0014] Once an alarm is determined the system initiates an alarm sequence which initiates a pre-programmed call via either a terrestrial telephone or GSM mobile phone equipped with a modem and data board to a suitable mobile phone. The mobile phone is also equipped with a modem data board and small display screen. The data sent to the mobile unit upon answering includes, as stated above, only the data relating to the difference between the reference and detected images. A means in the mobile unit receives checks and puts the data in order and converts it to a video with sync signal under the control of a local processor. Each image in turn may be selected for viewing, controlled by bi-directional modem link with a viewer's control buttons and data block acknowledgement. The viewer may initiate suitable responses to the alarm by means of the mobile telephone incorporated in the system. This may include calling the police or initiating an alarm.
[0015] The image processing manipulation and transmission steps required in the method in accordance with the present invention may be implemented as follows:
(a) producing a train of sequential image frames;
(b) selecting a reference image from the train of images in (a) above that defines information regarding stationary objects within a scene represented by the sequence of images in (a) above;
(c) comparing the (temporally) next image frame to the said reference frame to detect a motion image representing information regarding changes in the scene;
(d) repeating (c) above 30 times;
(e) repeating steps (b), (c) and (d) above;
(f) on (c) above providing said motion image, calculating the percentage of change in the current (next) image as compared to the then reference image;
(g) on (f) above proving to produce a percentage of change greater than 2% then suspending the selections, comparisons and analysis of the current (next) images with reference images as in (b), (c), (d) and (e) above;
(h) on (g) above, saving the next 30 current images and the then reference image to frame store device;
(i) on (h) above, selecting the next image from the frame store device and removing the parts of the image which are common to the reference image, storing the resultant image in a frame store device;
(j) on (g) above, a telephone connection is established automatically, either by terrestrial equipment or by GSM mobile link via modem and data card to a GSM mobile apparatus, equipped with a local processor, modem, data card and screen, together with control functions;
(k) on (j) above, the first treated current image from the frame store as in (i) above is sent to the mobile unit using data compression to JPEG standard;
(l) on (k) above, the first treated current image receiving at the mobile unit, is decompressed and displayed on the screen, described in (j) above;
(m) on (l) above, pre-programmed information text is added to the screen explaining time, date and origin of image.
(n) on (l) and (m) above, manually cycling to the next image as required, repeating steps (k), (l) and (m) above.
(o) establishing voice communication from the mobile unit to the fixed unit; establishing reset, saving images, or close-down procedure for the fixed unit directed from the mobile unit as desired;
(p) using the mobile phone unit to give an alarm if appropriate;
(q) on (j) above if the mobile unit is engaged, or out of range, or has a timed-out or no reply state, the fixed unit reverting to a fall-back routine, auto dialling any desired telephone number.
[0016] Technical specifications of equipment suitable for use in a system in accordance with the present invention are provided in Annex A enclosed herewith.
[0017] Modifications may be introduced without departing from the invention as now claimed. For example, the degree of difference between the image and reference may be increased or decreased and the number of images taken prior to up dating the reference image may be increased or decreased.
ANNEX A
SURVEILLANCE SYSTEM
[0020] The fixed monitor has a camera, processor board, GSM phone or transmitter with modem, terrestrial phone connection with a modem and a power supply.
[0021] The mobile display unit has a GSM phone with modem, processor board, local display, control buttons and a power source.
Product specification
[0022] Visual
Field of view 60° (scene - 4.6X3.5m at 4m range) of camera
Resolution 320X240 pixels
Focus, fixed 200 mm to infinity
Minimum lighting 25 lux (artificial)
[0023] Processing board and associated elements
Transmission time 2 seconds for a 2% change of view
LCD display and keypad for number and parameter setting, controls and reference frame control/store.
Alarm state detection - change of more than 2% of the picture (adjustable)
Audible message store - integral microphone. Recording controls on LCD panel
Local alarm relay - controlled from mobile
Power supply - universal mains supply with battery backup for camera, processor, GSM phone and both modems
[0024] Physical parameters of first unit
Box for electronics, integral GSM phone or transmitter & modem, short cable to camera, cable to standard phone plug, terminals for alarm relay, and mains inlet
Mobile display
[0025] GSM phone with modem
Display - miniature television type
control buttons to download new or modified pictures and to operate alarm relay
Power supply - local mains charge and battery
Other options
[0027] One or two way audio - for viewer to hear activity at site or for viewer to make an announcement to site.
Selective vertical resolution in field of viewer, giving a horizontal band of high resolution. This might be set on site or selectable by the viewer to send improved resolution with a specific horizontal band, e.g. where an intruder's head might be.
Approved persons in view - inhibits call sequence to others.
Audio trigger - confirming video detect.
PIR trigger -confirming video detect.
[0028] The security of any data sent has to be considered. For data received viz instructions from the viewer asking for different picture data requires security. For any remote settings e.g. system ON or OFF then security would also be needed.
System design
Key points
[0030] Viewed scene - resolution - lighting - video compression - task of motion sensor - GSM phone and modem -nature and quantity of data to be sent to mobile - picture transmission time - mobile unit.
Viewed scene
[0031] There are an increasing number of TV cameras available for direct connection to a PC. Intended for video conferencing or multimedia work, they convert the picture to a digital form to connect straight into the computer's parallel port, frame grabbing cards etc not being required. Although being constrained to force picture data through an 8 bit port it is unlikely that manufacturers use a common standard at this hardware level although they do achieve it after software drivers at the PC. However, the digitised picture data from any of them will be suitable for this project and if, at a later data, a change in camera supplier is required then any changes will be straight forward.
[0032] Two units have been identified as preferable
1) Connectix QuickCam - monochrome, 320X240 pixels, 64 bit grey scale, 65° field of view, fixed focus 18" to infinity.
2) Vivitar MPP2 - colour, 320X240 pixels, 60° field of view, 128 bit grey scale when colour not used, 200mm to infinity focus.
Resolution
[0034] With 60° lens an area of 4.6X3.5m is viewed at 4m distance. The face on a body at that distance resolves into 11X15 pixels - not enough for great recognition but more than enough for intruder detection.
Lighting
[0035] Irrespective of the cameras sensitivity some thought to the scene lighting is required. Ideally the scene is lit with artificial light with constant level. These cameras operate to "the level required for flash photography" - Connectix or 25 lux - Vivitar. That corresponds to twilight levels so should not require anything special at the scene. Connectix mention a degree of picture noise when operating under poor light conditions, noted for consideration in motion detection.
[0036] Operation with natural light with its level, colour temperature and source direction changes would require a more sophisticated camera and the processing for every possible light condition likely to be error prone. Also if the scene has windows etc to let the natural light in then consideration has to be given to other external sources, e.g. car headlights, street lights, bright sunlight streaming in and so on. For this project it is intended to remain with artificial light.
[0037] The lighting is best arranged to give the greatest contrast between the static and moving images. This can be determined with some simple experiments once the motion software is running and details ultimately include in operating specification/instructions.
Video compression
[0038] Connectix also make a colour camera with built in video data compression. This reduces by a factor of 16 the data to be sent whilst still maintaining picture quality. Use of this has been investigated, since it looks as if it could make transmission onward to the mobile rather easy. This might be the case if the entire picture was to be sent on and there was no motion detection to do. However, to convert the compressed data, carry out analysis and then only to be able to send a complete picture will not achieve the aim of getting a view of just the intruder to the viewer. To send part of the picture would need data decompression, selection and recompression, implying a knowledge of the compression technique.
[0039] In general terms there are two types video compression systems, one uses information from proceeding pictures and is used for films, conferencing & video. The other is for single images and is found widely on PCs. Of the many types available, the JPEG standard is popular. More to the point, compression and decompression software is available in "C" code and this will work on the type of processors envisaged for the final product rather than on a PC. (In fact if JPEG is used it will determine what type of processors have to be used).
[0040] JPEG compression is rather good in that picture data can be transmitted at a selectable degrees of compression. For example send an initial view at maximum compression and minimum transmit time. After viewer assessment this could be repeated but more slowly and with greater resolution.
[0041] To give an idea of data reduction, figures for JPEG are, source data 64000 bits [a little less than one picture from 320X240 camera], compressed 5094 bits.
Motion sensor
[0042] With a field of view 4.6X3.5m a typical body, side on, occupies around 1.7X0.2m or 2%. It would be reasonable to be looking for a change of 1% or more in the pictures before starting an alarm. It should be recorded that there should be no change in picture in the normal, unintruded, state. Performance under minimum lighting conditions has to be confirmed when any noise in the picture is at maximum. The type of noise under "poor" light is of "sparkles" or single pixels arriving at higher than zero levels. If necessary this will require a software filter to remove them.
GSM phone and modem
[0043] Like the cameras, the number of GSM phones/accessories with data facilities is constantly improving. For the present Nokia are recommended as the best manufacturer for GSM with data. The Nokia 2110l with Data Card and Card Hosts Pro would be suitable.
[0044] Both processor boards will have to include a PCMIA connection point, avoiding the cost of two Card Host Pro units.
Nature and quantity of data to be sent to mobile and picture transmission time
[0046] Digital video and single image transmission can be quite specific in that the data for each pixel represents its brightness and that the information is sent one pixel at a time starting from the first and automatically starting a new line - unlike TV pictures which contain line and frame sync pluses. This means that the picture data is a continuous stream of bits, for 320X240 at 64 grey shades it is 460800 bits.
[0047] Under an alarm condition it is intended to send those bits representing a picture change. For the present example that might be 2% or 9216 bits. However, we need to add further information to locate the data on the screen. This can be done by a "skip" instruction which sends the screen writing point onwards. Given that the change information is a solid block, i.e. for each of the 240 "lines" there is only one point along the line from which picture data will subsequently be written [to be followed by the next "skip"] one extra value per line is needed. This increase the number of bits to 12576.
[0048] For any transmission system there are errors and if no account was taken of these then the resultant picture could be distorted - lines would be misaligned and make the view impossible to assess. A usable scheme is to send the data in blocks with a checksum. If not received correctly then the mobile asks for a resend. Taken a little further it is intended to add additional data so that the blocks can be sent and received in any order and realigned prior to writing out to the display. Ideal block size has yet to be determined, smaller blocks increase sending time with an error free channel but reduce overall time if any blocks need re-sending.
[0049] For a block size of 256 bits this increases the number of bits to 1400. For a GSM modem of nominal 9600 band rate this can be sent in around 1.5 seconds, possibly 2 seconds if any blocks needed sending again. This time is ore than acceptable, but for a larger felon of say 5% of field of view the time will approach 4 seconds and be longer for still larger picture changes.
[0050] How long would a full picture take to send? On the same basis, namely sent in blocks but without any "skips" (not needed with the complete picture", it is a figure of 52.2 seconds. This is where compression comes in, for a JPEG compression of quality 5 (negligible change) the original picture is compressed by 90%, i.e. only 46080 bits to send, taking 5.25 seconds. In practice, this time is increased by having to repeat corrupted blocks and by processing time at the receiver to decompress the data.
[0051] This data compression is possible but quite complex to achieve. It will significantly increase the processing requirements of both processors and associated components. As an alternative it will be useful to use variable vertical resolution.
[0052] This works by sending that part of the field of view that should contain an intruders face at full resolution and the remainder at reduced resolution. This could be set up for a specific site in the box or as an option under the viewers control.
Mobile Unit
[0053] The modem link between the two stations may be bi-directional. The return data is for the viewers control buttons and data block acknowledgement. Reception of and reaction to this data is straight-forward at the transmitting end. In a later development data might be interchanged between stations for security purposes.
[0054] Picture data, once received, checked and put in order is converted to a video with sync signal, this all being under control of a local processor. If compression is found to be necessary it would significantly affect choice of processor and associated components and have some bearing on the power consumption.
[0055] A miniature television may be used for the display, modified to receive the video signals, . Alternatively, the display technology can be integrated with the processing board to optimize size and power consumption.
[0056] There is another display type which may be used namely a computer type 1/4VGALCD. This has the same resolution as the camera but each pixel is either on or off, no grey scale. If the pictures are adequate to determine the presence of an intruder then its use would reduce the amount of picture data to be sent, and hence the transmission time. The display considered was 130X93mm with a viewing area 104X75mm.
1. A surveillance system comprising at least one remote device (2) and a surveillance
device (4) including camera means (6) adapted to produce a series of images of a static
zone to be monitored, an image processor (8) including comparison means (10) arranged
to compare each of said images with a previously produced reference image, in order
to detect any difference therebetween, said image processor (8) being further arranged
to select any portion of any of said images which differs from said reference image,
and communication means (12) arranged to transmit image data characteristic of said
portion or portions to said at least one remote device, the surveillance system operating
under the control of a control means (14).
2. A system as claimed in Claim 1, wherein the at least one remote device is a telephone
with a visual display.
3. A system as claimed in Claim 1, wherein said remote device is a GSM compatible mobile
telephone.
4. A system as claimed in any preceding claim, wherein said communication means (12)
includes means for compressing the image data prior to transmission.
5. A system as claimed in any preceding claim, wherein a selective vertical resolution
system is utilised prior to transmission of said image data.
6. A system as claimed in any preceding claim, wherein the camera means (6) is a video
camera preferably a charged-couple device (CCD) camera or an infra-red camera.
7. A system as claimed in any preceding claim, wherein the image processor (8) and camera
means (6) are arranged to update the reference image periodically preferably after
approximately every thirty subsequent images are produced.
8. A surveillance means (4) for use in a system as claimed in any preceding claim.
9. A remote device for use in a system as claimed in any of Claims 1 to 7.
10. A method of surveillance of a static zone including producing a series of images of
the zone, comparing each of said images with a periodically updated reference image
in order to detect a difference therebetween, selecting any portion of any of said
images which differs from said reference image and transmitting image data characteristic
of said portion or portions to at least one remote device arranged to receive said
image data.
11. A method as claimed in Claim 10, wherein said determination of any difference between
any of said images and said reference image is made if the image data characteristic
of the produced image differs by more than 2% from the image data characteristic of
the reference image.
12. A method as claimed in any of Claims 10 to 11, wherein the reference image is up dated
after each series of approximately thirty images are produced.
13. A method as claimed in any of Claims 10 to 12 wherein an operator of any one of said
remote devices on receipt of said image data can determine whether or not the surveillance
area has been intruded into by a non-authorised person and can take action accordingly.
14. A method as claimed in Claim 13, wherein said person causes an alarm to be raised
at said zone and/or at a further remote station.
15. A method as claimed in any of Claims 10 to 13, wherein if said data cannot be successfully
transmitted to one of said remote devices a message to this effect is transmitted
to a still further remote device.