(19)
(11)EP 3 300 653 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
03.07.2019 Bulletin 2019/27

(21)Application number: 16191827.1

(22)Date of filing:  30.09.2016
(51)International Patent Classification (IPC): 
A61B 3/113(2006.01)
G02B 27/00(2006.01)

(54)

HEAD/EYE TRACKING WITH LIGHT SOURCE PREHEATING

KOPF-/AUGENVERFOLGUNG MIT LICHTQUELLENVORERWÄRMUNG

SUIVI OCULAIRE/DE TÊTE AVEC PRÉCHAUFFAGE DE SOURCE LUMINEUSE


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(43)Date of publication of application:
04.04.2018 Bulletin 2018/14

(73)Proprietor: Smart Eye AB
413 27 Göteborg (SE)

(72)Inventor:
  • SÖRNER, Per
    413 23 GÖTEBORG (SE)

(74)Representative: AWA Sweden AB 
P.O. Box 11394
404 28 Göteborg
404 28 Göteborg (SE)


(56)References cited: : 
EP-A1- 0 883 195
JP-A- 2010 258 350
US-A1- 2016 117 554
WO-A2-2015/034801
US-A1- 2011 304 281
  
      
    Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


    Description

    Field of the invention



    [0001] The present invention relates to the field of head or eye tracking, i.e. monitoring and tracking the head or eyes of a user by means of image acquisition and processing.

    Background of the invention



    [0002] Most head/eye tracking systems employ illumination of the object (i.e. the eyes of a user) in order to improve performance of the system. In order for such illumination to be distinguishable, the system should, as far as possible, be able to suppress ambient light.

    [0003] In order to eliminate or minimize interference from ambient light, the illumination of a head/eye tracking system can be restricted to a narrow wavelength range, preferably outside the visible light spectrum. Typically, a light source having a light emission spectrum concentrated around a distinct center wavelength outside the visible spectrum is used in combination with a band-pass filter having a pass-band centered around the center wavelength.

    [0004] The light source can be a solid state light source, such as a LED with a center wavelength in the near infra red (NIR) region, e.g. 840 nm or 940 nm. The filter has a pass-band enabling capturing of most light emitted by the light source, while at the same time blocking most ambient light. As an example, the pass-band of the filter is ± 25 nm.

    [0005] One particular challenge is sunlight, which has a relatively high irradiance (flux per area) over a broad spectrum, including NIR. In particular some automotive applications present challenges with strong sunlight.

    [0006] In the NIR region, the spectral irradiance (flux per area per wavelength) of sunlight is approximately 1 W/m2/nm, so that the total irradiance of sunlight admitted by the filter mentioned above is around 50 W/m2. In order to distinguish illumination from the light source from sunlight, the irradiance of the light source (flux per area) thus needs to be in the same order of magnitude (50 W/m2). The required electrical power can be limited by using pulsed light, and in a typical implementation the light source has an electrical power as low as 1-2 W.

    [0007] One issue in this context is that light sources such as LEDs have a temperature drift, i.e. the center wavelength will shift slightly when the temperature changes. As an example, for a typical LED the temperature drift is a few tenth nm/K. The pass-band of the filter therefore needs to be chosen to correspond to the LED emission spectrum in the expected operating temperature. For many applications, e.g. indoor applications without significant temperature variations, this will not present any significant problem. However, in some applications, such as automotive applications, there will be significant variations in the operating temperature of the LED.

    [0008] In a typical automotive installation, the LED temperature may change from an initial temperature, which may be as low as minus 30 degrees Celsius or less, up to a steady-state operating temperature of the LED circuitry. Depending on the ambient temperature and installation-specific thermal resistance, this steady-state operating temperature may be as low as 20 degrees Celsius and as high as 90 degrees Celsius. This corresponds to a shift of the LED center wavelength of around 30-40 nm, i.e. in the same order of magnitude as the pass-band of the filter. In worst case, most of the emitted light will be lost. If also the initial phase is accounted for, i.e. the time period before the LED reaches the steady state operating temperature, the active operating temperature range becomes even greater.

    [0009] In order to compensate for this temperature drift, and ensure satisfactory system performance also in a worst-case scenario, the pass-band of the filter may be wider, so as to ensure that most of the power emitted by the LED will pass the filter at all expected temperatures. However, as a consequence, the filter will also allow more sunlight to pass, and therefore the power of the LED needs to be increased, typically by a factor of two or three.

    General disclosure of the invention



    [0010] It is an object of the present invention to enable the illumination of a head/eye tracking system to suppress ambient light (in particular sunlight) also under conditions of large temperature variations, without requiring increased power.

    [0011] This and other objects are achieved by a head/eye tracking system comprising an image sensor for acquiring images of a user, a narrow band light source for illuminating the user, and a band-pass filter arranged between the user and the image acquisition device and having a pass-band corresponding to the emission spectrum of the light source at a predefined operating temperature. The system further comprises a heat source arranged in proximity to the light source and configured to preheat the light source to reach a temperature range including the predefined operating temperature.

    [0012] The invention is based on the realization that the variation in steady-state operating temperature can be reduced by preheating the light source to (or close to) a predefined operating temperature using a heat source. By means of the preheating, lower steady-state temperatures, which could result from low ambient temperatures, can be avoided. This means that the operating temperature becomes more predictable.

    [0013] As an example, the temperature range can be +/- 15 degrees or smaller. The important thing is that the variation of the steady-state operating temperature of the light source will be significantly smaller than for a system without preheating.

    [0014] Another advantage with the preheating according to the invention is that the start-up phase, i.e. the time from start-up until the light source reaches its steady-state operating temperature, may be significantly shortened. Without preheating, this period may be several minutes, and during this period the matching of the filter and light source may be relatively poor. With preheating according to the invention, the start-up period can be as short as 30 seconds or less.

    [0015] The predefined operating temperature can be selected to be in the upper range of expected operating temperatures. As an example, the predefined operating temperature is more than 75 degrees Celsius, preferably more than 80 degrees Celsius. It is noted that it may be possible that extreme conditions result in an operating temperature above the predefined operating temperature. However, even for such unlikely operating conditions, the resulting operating temperature will not be significantly higher than the predefined operating temperature for which the filter is configured.

    [0016] It is noted that the preheating does not need to be very accurately controlled. As long as the operating temperature is within a given range of the specified operating temperature, such as ±15 degrees, the temperature drift of the light source will not cause significant problems.

    [0017] In some embodiments, the predefined operation temperature is a function of parameters of the light source and/or the filter, so that the system performance is optimized with respect to the individual components used.

    [0018] In a simple implementation, the heat source can be self-regulating, e.g. a PTC thermistor. Alternatively, the heat source is provided with active temperature feedback control.

    [0019] The expression "narrow band" light source is intended a light source having a light emission spectrum concentrated within 100 nm, or preferably within 50 nm. The light emission spectrum of the light source is further preferably outside the visible spectrum.

    Brief description of the drawings



    [0020] The present invention will be described in more detail with reference to the appended drawings, showing currently preferred embodiments of the invention.

    Figure 1 shows a schematic view of an eye tracking system in which the present invention may be applied.

    Figure 2 shows a schematic block diagram of a first embodiment of the present invention.

    Figure 3 shows a schematic block diagram of a second embodiment of the present invention.

    Figure 4 is a diagram showing temperature of a LED over time.


    Detailed description of preferred embodiments



    [0021] Embodiment of the present invention will now be discussed with reference to an eye tracking system. However, the principles of the invention are equally applicable to a head tracking system, or indeed any application where a band-pass filtered image acquisition needs to be matched to a light source emission spectrum.

    [0022] The eye tracking system in figure 1 includes an image acquisition device, or camera 1, including an image sensor 2, e.g. a CMOS image sensor, and suitable optics 3. The camera 1 is arranged to acquire images of a user 4, more specifically the eyes of the user. The system further comprises a light source 5, typically configured to emit light outside the visible range such as infra red (IR) or near infra red (NIR). The light source may be a solid state light source, such as a LED. In the illustrated example, the light source 5 is a LED configured to emit light with a light spectrum concentrated in a 50 nm band centered around 850 or 940 nm (NIR). Further, an optical band-pass filter 6, e.g. an interference filter, is arranged between the user and the camera 1. The filter 6 is configured to have a pass-band substantially corresponding to the light emission spectrum of the light source 5. So, in the above mentioned example, the filter 6 should have a pass-band of around 825 - 875 nm, or 915 - 965 nm.

    [0023] A controller 7 is connected to the camera 1 and LED 5, and programmed to control the sensor 2 to acquire consecutive images under illumination by the LED 5. Typically, the LED 5 is driven with a given duty cycle, and the controller 7 then controls the sensor 1 to acquire images in synchronization with the light pulses from the LED 5.

    [0024] The system further comprises processing circuitry 8 connected to receive images acquired by the sensor 2. The processing circuitry is further programmed to track movement in the eyes, in order to acquire various information. For example, the processing circuitry may be programmed to obtain the direction of gaze of the user, or to detect drowsiness of the user.

    [0025] The system in figure 1 has many different applications, including automotive applications where the eyes of a driver are tracked e.g. for safety reasons. In such applications, the user 4 may be exposed to strong ambient light from the sun, sometimes direct sunlight.

    [0026] During operation, the user 4 is illuminated by the light source, and light reflected from the object passes the filter and is received by the camera optics 3 and stored in the sensor 2. It is noted that most ambient light, including all visible light, will be blocked by the filter. However, in a situation where the spectrum of ambient light includes the pass-band of the filter 6, such light will also be received by the camera 1. As mentioned above, sunlight does include such wavelengths, and typically has a spectral irradiance (flux per area per wavelength) of about 1 W/m2/nm in the relevant wavelength range. The power of the light source is therefore configured to provide sufficient spectral power to be distinguishable from sunlight. As an example, the LED 5 has an electrical power of 1-2 W.

    [0027] In accordance with the present invention, and with reference to figure 2-3, the LED 5 forms part of a lighting device 10, 10', further including a heat source 11, 11'. The heat source is arranged in thermal contact with the LED 5, either in direct contact or by mounting on a common circuit board, in order to enable preheating of the LED 5. The purpose of the preheating is to quickly heat the LED to a predefined operating temperature, or at least within a given range (e.g. +/- 15 degrees) of that predefined operating temperature. This avoids a large variation in the steady-state operating temperature of the LED, thus enabling a better matching of the filter pass band. Also, the start-up time, i.e. the time when the LED is heated from an initial temperature to the steady-state temperature can be shortened. As an example, the start-up period can be shortened from several minutes to less than 30 seconds, or even less than 15 seconds.

    [0028] With reference to figure 2, the heat source 11 is a self-regulating heat source, i.e. a positive temperature thermistor. A PTC thermistor will exhibit an increasing resistance with temperature, such that its temperature will stabilize at a given ambient temperature when connected to a given voltage. In this case, there is no actual feedback from the LED 5 itself, but the resulting heating of the LED 5 will be determined by the ambient temperature surrounding the PTC 13, the thermal exchange between the LED and the ambient, and of course the heating of the LED caused by its operation. By an appropriate selection of self-regulating heat source 11 and mounting of the heat source 11 and LED 5, a desired preheating of the LED 5 can be achieved.

    [0029] Alternatively, as shown in figure 3, the lighting device 10' further comprises active control circuitry 12, configured to detect the temperature of the LED 5, and to apply feedback control of the heat source 11' in order to heat the LED 5 until it reaches a predefined temperature.

    [0030] Figure 4 illustrates a possible outcome of preheating according to the present invention expressed as a diagram showing temperature over time.

    [0031] In a conventional system, the light source will typically be gradually heated from an initial temperature up to a steady-state operating temperature. The initial temperature will depend primarily on the ambient temperature, which, in the case of an automobile, can be very low, potentially as low as minus 30 degrees Celsius or lower. The steady-state operating temperature will also depend on the ambient temperature, but also on the power of the LED, heat exchange between the LED and the mechanical mounting structure (e.g. the presence of any heat sinks such as heat fins), as well as the amount of heat dissipation (e.g. air transport of heat). As an example, in a typical installation, the steady-state operating temperature of the LED may be up to 50 degrees higher than the ambient temperature. The time period required to reach the steady-state operating temperature will of course also depend on various factors, but is typically in the order of minutes.

    [0032] In figure 4, curves 41 and 42 indicate the temperature of a LED in a conventional eye tracking system for two different ambient temperatures T0 and T1. As discussed above, the curve exhibits two separate phases. In a first, initial phase, the temperature gradually increases from the initial temperature T1, T2 to a steady-state operating temperature range Top1, Top2. During a second phase, the temperature remains within this range. As indicated in figure 4, the steady-state operating temperature range will be lower for a lower initial temperature.

    [0033] Curves 43 and 44 indicate the temperature of a LED provided with preheating according to en embodiment of the present invention, again for the two different ambient temperatures T1 and T2. In this case, the preheating causes the LED to approach the predefined operating temperature Tpre within a relatively short time period. It is noted that the difference in ambient temperature (as well as possibly other factors) may result in a slight difference in steady state operating temperature in the two curves 43 and 44. In the example in figure 4, this steady state operating temperature for curve 44 is slightly higher than for curve 43.

    [0034] Figure 4 clearly illustrates the two main advantages of the present invention. To begin with, the preheating ensures that the steady state operating temperature for both ambient temperatures T1 and T2 is equal to (or close to) the predefined operating temperature Tpre. The pass band of the filter can thus be chosen to correspond to the light emission spectrum at this temperature. Further, the preheating reduces the initial phase, i.e. the time during which the LED temperature rises from the initial temperature to the steady-state operating temperature. This ensures that any period during which the filter is not matched with the emitted light spectrum will be minimized.

    [0035] The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. For example, the eye tracking system may comprise more than one light source, e.g. for purposes of glare or reflex reduction. Further, different solutions may be chosen to implement the heat source, without departing from the present invention as defined by the claims.


    Claims

    1. An head/eye tracking system comprising:

    an image sensor (1) for acquiring images of a user (4),

    a narrow band light source (5) for illuminating the user, and

    a band-pass filter (6) arranged between the user and the image acquisition device and having a pass-band corresponding to the emission spectrum of the light source for a predefined operating temperature (Tpre),

    characterized by

    a heat source (11) arranged in proximity to the light source and configured to preheat the light source to a temperature range including the predefined operating temperature (Tpre).


     
    2. The head/eye tracking system of claim 1, wherein the heat source (11) is self-regulating.
     
    3. The head/eye tracking system of claim 1, wherein the heat source is provided with active temperature feedback control (12).
     
    4. The head/eye tracking system of claim 2, wherein the heat source is a PTC thermistor.
     
    5. The head/eye tracking system of any one of the preceding claims, wherein the heat source (11) is configured such that the light source reaches the predefined operating temperature in less than 30 s.
     
    6. The head/eye tracking system of any one of the preceding claims, wherein the predefined operating temperature is above 70 degrees Celsius, preferably above 80 degrees Celsius.
     
    7. The head/eye tracking system of any one of the preceding claims, wherein the narrow band light source has a light emission spectrum concentrated within 100 nm, preferably within 50 nm.
     
    8. The head/eye tracking system of any one of the preceding claims, wherein the narrow band light source has a light emission spectrum outside the visible spectrum.
     
    9. The head/eye tracking system of any one of the preceding claims, wherein the temperature range is +/- 15 degrees of the predefined operating temperature (Tpre).
     
    10. The head/eye tracking system of any one of the preceding claims, wherein the predefined operation temperature (Tpre) is a function of parameters of the light source (5) and/or filter (6)
     
    11. A method for head/eye tracking comprising:

    illuminating a user using a narrow band light source (5),

    acquiring band-pass filtered images of the user (4), where a pass-band of the filtering corresponds to the emission spectrum of the light source for a predefined operating temperature (Tpre),

    characterized by

    preheating the light source to a temperature range including the predefined operating temperature (Tpre) using a heat source (11) arranged in proximity to the light source.


     
    12. The method of claim 11, wherein the temperature range is +/- 15 degrees of the predefined operating temperature (Tpre).
     
    13. The method of claim 11 or 12, wherein the heat source (11) is configured such that the light source reaches the predefined operating temperature in less than 30 s.
     
    14. The method of one of claims 11-13, wherein the predefined operating temperature is above 70 degrees Celsius, preferably above 80 degrees Celsius.
     


    Ansprüche

    1. Kopf-/Augenverfolgungssystem, umfassend:

    einen Bildsensor (1) zum Erfassen von Bildern eines Benutzers (4),

    eine Schmalband-Lichtquelle (5) zum Beleuchten des Benutzers, und

    einen Bandpassfilter (6), der zwischen dem Benutzer und der Bilderfassungsvorrichtung angeordnet ist und ein Durchlassband aufweist, das dem Emissionsspektrum der Lichtquelle für eine vordefinierte Betriebstemperatur (Tpre) entspricht,

    gekennzeichnet durch

    eine Wärmequelle (11), die in der Nähe der Lichtquelle angeordnet und konfiguriert ist, um die Lichtquelle auf einen Temperaturbereich vorzuwärmen, der die vordefinierte Betriebstemperatur (Tpre) beinhaltet.


     
    2. Kopf-/Augenverfolgungssystem nach Anspruch 1, wobei die Wärmequelle (11) selbstregulierend ist.
     
    3. Kopf-/Augenverfolgungssystem nach Anspruch 1, wobei die Wärmequelle mit einer aktiven Temperatur-Rückkopplungssteuerung (12) versehen ist.
     
    4. Kopf-/Augenverfolgungssystem nach Anspruch 2, wobei die Wärmequelle ein PTC-Thermistor ist.
     
    5. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei die Wärmequelle (11) derart konfiguriert ist, dass die Lichtquelle die vordefinierte Betriebstemperatur in weniger als 30 s erreicht.
     
    6. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei die vordefinierte Betriebstemperatur oberhalb von 70 Grad Celsius, vorzugsweise oberhalb von 80 Grad Celsius liegt.
     
    7. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei die Schmalband-Lichtquelle ein Lichtemissionsspektrum aufweist, das innerhalb von 100 nm, vorzugsweise innerhalb von 50 nm, konzentriert ist.
     
    8. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei die Schmalband-Lichtquelle ein Lichtemissionsspektrum außerhalb des sichtbaren Spektrums aufweist.
     
    9. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei der Temperaturbereich +/-15 Grad der vordefinierten Betriebstemperatur (Tpre) beträgt.
     
    10. Kopf-/Augenverfolgungssystem nach einem der vorhergehenden Ansprüche, wobei die vordefinierte Betriebstemperatur (Tpre) eine Funktion von Parametern der Lichtquelle (5) und/oder des Filters (6) ist.
     
    11. Verfahren zur Kopf-/Augenverfolgung, umfassend:

    Beleuchten eines Benutzers unter Verwendung einer Schmalband-Lichtquelle (5),

    Erfassen von bandpassgefilterten Bildern des Benutzers (4), wobei ein Durchlassband der Filterung dem Emissionsspektrum der Lichtquelle für eine vordefinierte Betriebstemperatur (Tpre) entspricht,

    gekennzeichnet durch

    Vorwärmen der Lichtquelle auf einen Temperaturbereich, der die vordefinierte Betriebstemperatur (Tpre) einschließt, unter Verwendung einer Wärmequelle (11), die in der Nähe der Lichtquelle angeordnet ist.


     
    12. Verfahren nach Anspruch 11, wobei der Temperaturbereich +/-15 Grad der vordefinierten Betriebstemperatur (Tpre) beträgt.
     
    13. Verfahren nach Anspruch 11 oder 12, wobei die Wärmequelle (11) derart konfiguriert ist, dass die Lichtquelle die vordefinierte Betriebstemperatur in weniger als 30 s erreicht.
     
    14. Verfahren nach einem der Ansprüche 11 bis 13, wobei die vordefinierte Betriebstemperatur oberhalb von 70 Grad Celsius, vorzugsweise oberhalb von 80 Grad Celsius, liegt.
     


    Revendications

    1. Système de suivi de tête/oculaire comprenant :

    un capteur d'images (1) pour acquérir des images d'un utilisateur (4),

    une source lumineuse à bande étroite (5) pour éclairer l'utilisateur, et

    un filtre passe-bande (6) disposé entre l'utilisateur et le dispositif d'acquisition d'images et comportant une bande passante correspondant au spectre d'émission de la source lumineuse pour une température de fonctionnement prédéfinie (Tpre),

    caractérisé par

    une source de chaleur (11) disposée à proximité de la source lumineuse et conçue pour préchauffer la source lumineuse jusqu'à la plage de température incluant la température de fonctionnement prédéfinie (Tpre).


     
    2. Système de suivi de tête/oculaire selon la revendication 1, dans lequel la source de chaleur (11) est autorégulatrice.
     
    3. Système de suivi de tête/oculaire selon la revendication 1, dans lequel la source de chaleur est pourvue d'une commande de rétroaction de température active (12).
     
    4. Système de suivi de tête/oculaire selon la revendication 2, dans lequel la source de chaleur est un thermistor PTC.
     
    5. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la source de chaleur (11) est conçue de manière à ce que la source lumineuse atteigne la température de fonctionnement prédéfinie en moins de 30 secondes.
     
    6. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la température de fonctionnement prédéfinie est supérieure à 70 degrés Celsius, de préférence supérieure à 80 degrés Celsius.
     
    7. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la source lumineuse à bande étroite a un spectre d'émission lumineuse concentré dans 100 nm, de préférence dans 50 nm.
     
    8. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la source lumineuse à bande étroite a un spectre d'émission lumineuse en dehors du spectre visible.
     
    9. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la plage de température est de +/- 15 degrés de la température de fonctionnement prédéfinie (Tpre).
     
    10. Système de suivi de tête/oculaire selon l'une quelconque des revendications précédentes, dans lequel la température de fonctionnement prédéfinie (Tpre) est fonction de paramètres de la source lumineuse (5) et/ou du filtre (6).
     
    11. Procédé de suivi de tête/oculaire, comprenant :

    l'éclairage d'un utilisateur en utilisant une source lumineuse à bande étroite (5),

    l'acquisition d'images filtrées par bande passante de l'utilisateur (4), une bande passante du filtrage correspondant au spectre d'émission de la source lumineuse pour une température de fonctionnement prédéfinie (Tpre),

    caractérisé par

    le préchauffage de la source lumineuse jusqu'à une plage de température comprenant la température de fonctionnement prédéfinie (Tpre) en utilisant une source de chaleur (11) disposée à proximité de la source lumineuse.


     
    12. Procédé selon la revendication 11, dans lequel la plage de température est de +/- 15 degrés de la température de fonctionnement prédéfinie (Tpre).
     
    13. Procédé selon la revendication 11 ou 12, dans lequel la source de chaleur (11) est conçue de manière à ce que la source lumineuse atteigne la température de fonctionnement prédéfinie en moins de 30 secondes.
     
    14. Procédé selon une des revendications 11 à 13, dans lequel la température de fonctionnement prédéfinie est supérieure à 70 degrés Celsius, de préférence supérieure à 80 degrés Celsius.
     




    Drawing