(19)
(11) EP 2 477 177 B1

(12) EUROPEAN PATENT SPECIFICATION

(45) Mention of the grant of the patent:
21.06.2017 Bulletin 2017/25

(21) Application number: 12151258.6

(22) Date of filing: 16.01.2012
(51) International Patent Classification (IPC): 
G09G 3/34(2006.01)

(54)

Local dimming boost up using depth map

Lokale Dimmerverstärkung unter Verwendung von Tiefenkarten

Augmentation d'atténuation locale utilisant une carte de profondeurs


(84) Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30) Priority: 18.01.2011 TR 201100485

(43) Date of publication of application:
18.07.2012 Bulletin 2012/29

(73) Proprietor: Vestel Elektronik Sanayi ve Ticaret A.S.
45030 Manisa (TR)

(72) Inventor:
  • Tasli, Emrah
    45030 Manisa (TR)

(74) Representative: Cayli, Hülya 
Paragon Consultancy Inc. Koza Sokak No: 63/2 GOP
06540 Ankara
06540 Ankara (TR)


(56) References cited: : 
WO-A1-2010/039419
US-A1- 2010 066 752
   
  • ROSIN ET AL: "A simple method for detecting salient regions", PATTERN RECOGNITION, ELSEVIER, GB, vol. 42, no. 11, 1 November 2009 (2009-11-01), pages 2363-2371, XP026250835, ISSN: 0031-3203, DOI: 10.1016/J.PATCOG.2009.04.021 [retrieved on 2009-05-09]
   
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description

Related Technical Field



[0001] The present invention relates to a system and method for dimming backlights of LCDs (liquid crystal displays) using depth map.

Prior Art



[0002] With the help of recent developments in semiconductor technologies, it has become possible to use LED (Light Emitting Diode) backlights in LCDs (Liquid Crystal Displays). LED backlights enable much better contrast ratios, since multiple LEDs or LED regions are arranged to have locally controllable backlights.

[0003] The contrast improvement arises from the technique of "local dimming". Local dimming comprises dimming or boosting backlights in accordance with intensity histogram of the displayed image. The backlights corresponding to dark regions of the image are dimmed as to make these regions darker, and backlights of bright regions are boosted to make brighter. Hence the overall contrast is increased, improving the image quality significantly.

[0004] The common technique for obtaining backlight dimming values is to use histogram of intensities of the regions of image. A histogram is created based on the intensities, that is; brightness or darkness of the regions of the image. The created histogram is analyzed and dimming values are obtained by weighing nominal LED driving values with histogram values and determining boosted or dimmed LED driving values.

[0005] Related to the known local dimming techniques known from the prior art, the patent document US 2009/0278867 A1 discloses a method in which a gamut mapping is utilized to control and normalize dimming values of a display. Within the method of the document, time continuity and smooth light transition is determined. It is obvious that the method is merely based on the colouring or intensity of the image. Another patent document WO 2009/057723 A1 discloses a backlight control method in which the backlight level is adjusted by extracting a tone map of an image and identifying mostly used colour code in each region of the image. Stated method is also based on colouring or intensity of the image and lacks determination of a true perception.

[0006] However, intensity based dimming may lead to image quality degradation since human perception does not merely depend on brightness. Since a 3D perception is always created by human brain, perspective is also important for a perceived image. For human vision system, a bigger and/or a closer object always draws attention independent from its colour. For an image that is displayed on a locally dimmed display, an image darker but closer to the view of camera may appear on the screen. Although the object draws attention of the viewer, since the region on which the object appeared is darkened by dimming system of the display, the perceived image creates disturbance because of inadequate perception.

[0007] Another patent document WO2010039419A1 discloses a method for reducing the power consumption of an electronic device by adjusting backlight levels. According to this document, backlight level of a display region, containing a background object, is reduced in order to reduce the power consumption of the electronic device.
Regarding the problems of the prior art, a system and method for local dimming boost up for LCDs should be realized.

Brief Description of the Invention



[0008] Present invention discloses a local dimming boost up method for liquid crystal displays which comprises extracting a depth map of an image which is displayed on the display; calculating a histogram of said depth map for each locally controllable backlight regions of said display and adjusting backlight driving values of said backlight regions in accordance with said calculated histogram. The invention also discloses system of local dimming boost up which uses the stated method.

[0009] The method and system of the present invention utilizes depth map, which gives true information about the significance levels of the objects on an image in terms of human perception, to obtain a far better and true image perception on the displays with the local dimming capability. The system and method enables elimination of misperceptions of image due to different significance levels of intensity and perspective position of the objects on the image.

Object of the Invention



[0010] The object of the invention to provide a system and method for local dimming boost up for liquid crystal displays (LCDs).

[0011] Another object of the invention is to provide a system and method for local dimming boost up for LCDs, using depth map of a displayed image.

[0012] Another object of the invention is to provide a system and method for local dimming boost up for LCDs, which create a depth map using linear mapping.

[0013] Another object of the invention is to provide a system and method for local dimming boost up for LCDs, which boosts up led driving values using a weighing parameter.

[0014] Another object of the invention is to provide a system and method for local dimming boost up for LCDs, which limits the boost up value.

[0015] Another object of the invention is to provide a system and method for local dimming boost up for LCDs, which limits the boost up value in accordance with maximum led driving value.

Brief Description of the Drawings



[0016] 

Figure 1 shows an image with three objects having different depths.

Figure 2 shows led driving values of display on which the image on figure 1 is shown.

Figure 3 shows led driving values boosted up using depth map of the image given in figure 1.

Figure 4 a block diagram of the method of the present invention.

Figure 5 a graph of a linear weighing function based on intensity values.

Figure 6 a graph of a limiting function for boost up values.



[0017] The reference numbers as used in figures may possess the following meanings;
Image (1)
First object (1a)
Second object (1b)
Third object (1c)
Intensity based backlight dimming (2)
Depth map extraction (3)
Histogram analysis (4)
Boost up region classification (5)
Threshold (6a)
First maximum led driving value (6b)
Second maximum led driving value (6c)
First boost up limit (6d)
Second boost up limit (6e)

Detailed Description of the Invention



[0018] Present invention comprises a system and method for local dimming boost up of backlight for LCDs (Liquid Crystal Displays).

[0019] Recent LCDs use LEDs (Light Emitting Diodes) as backlight element. LED backlights can be positioned either behind the LCD panel or along the edges of the panel. LED backlight occupies much smaller area than conventional fluorescent backlights and therefore it is possible to use many LEDs as backlight for LCDs.

[0020] Since LCD comprises a plurality of LEDs for backlight, the LEDs can be controlled independently or in groups, illuminating a certain region of the LCD. Usage of LEDs in LCDs also enables utilization of backlight for enhancement of the image. The most widely used method for LED backlights is enhancement of image contrast by dimming or boosting LED driving values to alter LED light intensity. The stated method extracts image intensities of an image that is currently displayed on the LCD. An intensity histogram is created by using said intensities. Then LED driving values are adjusted in accordance with created histogram. Within the used method, the LEDs illuminating a region on which a dark region of the image is displayed are dimmed as to create a darker perception. Also the LEDs illuminating a region on which a bright region of the image is displayed are boosted as to create a brighter perception. Consequently, a better quality vision is created having an improved contrast ratio.

[0021] Darker regions of an image are less noticeable than brighter regions of the image in the dimming methods known from the state of the art. However, the human visual perception does not merely depend on colour and brightness of the image. An important aspect of human visual perception is position and size of an object in relation to the point of view. As an object is either closer to the viewing point or is bigger than surrounding objects, it draws attention due to human visual perception system.

[0022] Since the colour and/or brightness of an object are independent from its position and size, it is not possible to obtain a true image perception via intensity based dimming or boosting. The present invention offers a method and system as a solution to stated problem.

[0023] Present invention comprises a method of local dimming boost up for liquid crystal displays comprising the steps of;
  • Extracting a depth map of an image which is displayed on the display
  • Calculating a histogram of said depth map for each locally controllable backlight regions of said display
  • Adjusting backlight driving values of said backlight regions in accordance with said calculated histogram.


[0024] The method of said invention adjusts the dimming values in accordance with depth map of the image. Hence the perceived image is enhanced in proportion to the position and/or size of the objects in the image.

[0025] An exemplary image (1) on which the method is applicable is given in figure 1. The image (1) comprises three objects (1a, 1b, 1c). Here the first object (1a) has the most significance and the third object (1c) has the least significance. The first object (1a) may have a dark colour with apparent boundaries.

[0026] The LED driving values corresponding to said objects (1 a, 1 b, 1 c) and nearby regions are given in figure 2. The maximum driving value in this embodiment is 128. The value corresponding to the region of the first object (1a) is 64, which is a medium driving level. Although the third object (1c) is much less significant than the first object (1a), the driving value corresponding to region of third object (1 c) is 50, which is close to that of first object (1a). Therefore, an emphasis on the first object is not determined with the stated driving values.

[0027] Figure 3 shows backlight driving values for the same image (1) which are adjusted by the method of the present invention. Since the significance of the first object (1a) is higher, this significance appears on the depth map of the said image. Hence, the driving level of the regions corresponding to the first object (1a) is boosted to emphasis said object. Since other objects (1b, 1c) also have significance with respect to surrounding environment, driving levels of regions corresponding to these objects (1b, 1c) are also boosted. However, the boost levels are consistent with the significance of the objects (1a, 1b, 1c). Hence while the driving values for the first object (1a) is boosted from 64 to 82, the driving values for the third object (1c) is boosted from 40 to 52.

[0028] Figure 4 shows the block diagram of the method in an embodiment of the present invention. An input image can first be processed to have an intensity based backlight dimming (2) as known from the prior art (for example as disclosed in patent document US2010066752A1). Meanwhile, a depth map of the image is extracted (3) (for example according to saliency map detection method as disclosed in ROSIN ET AL: "A simple method for detecting salient regions"). The histogram of the extracted depth map is then analyzed (4) to determine boost up values. Then the intensity based backlight dimming is evaluated with the histogram of the depth map and boost up region classifications are made (5) to determine final backlight driving values.

[0029] In an embodiment of the invention, said adjusting backlight driving values of said backlight is done by weighing said histogram of depth map with a histogram of intensity map of said image using a weighing parameter. Since local dimming techniques known from the state of the art usually employ image intensity, it is preferable to use a weighing equation to obtain a true adjustment by utilizing image intensity in boost up technique.

[0030] Preferably, the weighing parameter is further adjusted in relation to intensity of the image. A graph of an exemplary weighting parameter function in relation to intensity is given in figure 5, wherein minimum weighing parameter is 1 and K denotes maximum weighing parameter. Since a static weighting parameter results in limited boost up, a non linear boost level can be obtained by adjusting weighting parameter. Consequently, as a region of the image gets brighter, the effect of depth map based boost up is increased to preserve true image perception.

[0031] In an another embodiment of the invention said adjustment of backlight driving values is limited in accordance with maximum backlight driving value of the display. To preserve the true image perception, it is important not to boost up a region on which a darker portion of the image is shown up to a point that the region becomes brighter than a brighter region of the image. A common practice in this method can be limiting the boost up value at a maximum value which is half of the maximum backlight driving value of the display, as it is shown on the figure 6. In the given graph in figure 6, a first boost up limit (6d) is set to 64 while a first maximum backlight driving value (6b) is 128. While a second maximum backlight driving value (6c) is 255, a second boost up limit (6e) is set to 128. A threshold (6a) can be defined so that a boost up can be inhibited when the maximum backlight driving value is lower than the defined threshold (6a). The inhibition of boost up prevents a bright disturbing region on a dark scene shown on the display.

[0032] Present invention further comprises a system for local dimming boost up using depth map. The system comprises;
  • Means for extracting a depth map of an image which is displayed on the display
  • Means for calculating a histogram of said depth map for each locally controllable backlight regions of said display
  • Means for adjusting backlight driving values of said backlight regions in accordance with said calculated histogram.


[0033] Preferably the system of present invention further comprises means for weighing said histogram of depth map with a histogram of intensity map of said image using a weighing parameter.

[0034] Within the method and system of the present invention, the local dimming is further adjusted to create a better true image perception based on the image depth and significance.


Claims

1. A local dimming boost up method for liquid crystal displays comprising the steps of;

- Calculating a histogram of intensity map of an image which is displayed on the display,

- Calculating dimming or boosting backlight driving values per each locally controllable backlight region according to said histogram of intensity map such that the regions determined to be bright are boosted and the regions determined to be dark are dimmed; characterized by,

- Extracting a depth map of said image, wherein said depth map gives true information about the significance levels of the objects on an image in terms of human perception;

- Calculating a histogram of said depth map for each locally controllable backlight regions of said display and according to the result determining driving values per region such that the driving values are boosted in regions presenting objects with higher significance;

- Evaluating the driving values per region calculated according to the histogram of intensity map with the driving values per region calculated according to the histogram of the depth map to determine the final driving values per each locally controllable backlight region.


 
2. A local dimming boost up method according to claim 1 characterized in that said adjusting backlight driving values of said backlight regions further comprises weighing said histogram of depth map with a histogram of intensity of said image using a weighing parameter.
 
3. A local dimming boost up method according to claim 2 characterized in that said weighing parameter is adjusted in relation to said intensity map of the image.
 
4. A local dimming boost up method according to claim 2 characterized in that said adjusting backlight driving values of said backlight regions comprises limiting backlight driving values in accordance with maximum backlight driving value of the display.
 
5. A local dimming boost up method according to claim 4 characterized in that said limiting comprises defining a threshold and inhibiting boost up when a maximum backlight driving value of the display is lower than said threshold.
 
6. A local dimming boost up system for liquid crystal displays comprising;

- Means for calculating a histogram of intensity map of an image which is displayed on the display,

- Means for calculating dimming or boosting backlight driving values per each locally controllable backlight region according to said histogram of intensity map such that the regions determined to be bright are boosted and the regions determined to be dark are dimmed; characterized by further comprising,

- Means for extracting a depth map of said image, wherein said depth map gives true information about the significance levels of the objects on an image in terms of human perception;

- Means for calculating a histogram of said depth map for each locally controllable backlight regions of said display and according to the result determining driving values per region such that the driving values are boosted in regions presenting objects with higher significance;

- Means for evaluating the driving values per region calculated according to the histogram of intensity map with the driving values per region calculated according to the histogram of the depth map to determine the final driving values per each locally controllable backlight region.


 
7. A local dimming boost up system according to claim 6 characterized in that the system further comprises means for weighing said histogram of depth map with a histogram of intensity map of said image using a weighing parameter.
 


Ansprüche

1. Verfahren der lokalen Dimmverstärkung (dimming boost up) für Liquid Crystal Displays, umfassend die Schritte;

- Berechnen eines Histogramms einer Intensitätskarte eines Bildes, das auf dem Display angezeigt wird,

- Berechnen von Dimm- oder Verstärkungs-Hintergrundbeleuchtungsansteuerungswerten pro jeweils lokal steuerbarem Hintergrundbeleuchtungsbereich nach dem Histogramm der Intensitätskarte, so dass die Bereiche, die als hell bestimmt wurden, verstärkt werden und die Bereiche, die als dunkel bestimmt wurden, gedimmt werden; gekennzeichnet durch,

- Extrahieren einer Tiefenkarte des Bildes, wobei die Tiefenkarte eine hinsichtlich der menschlichen Wahrnehmung tatsächliche Information über die Signifikanzniveaus der Objekte auf einem Bild liefert;

- Berechnen eines Histogramms der Tiefenkarte für jeden lokal kontrollierbaren Hintergrundbeleuchtungsbereich des Displays und, entsprechend dem Ergebnis, Bestimmen von Ansteuerungswerten pro Bereich, so dass die Ansteuerungswerte in Bereichen verstärkt werden, in denen Objekte mit höherer Signifikanz dargestellt werden;

- Evaluieren der nach dem Histogramm der Intensitätskarte berechneten Ansteuerungswerte pro Bereich mit den nach dem Histogramm der Tiefenkarte berechneten Ansteuerungswerte pro Bereich zum Bestimmen der endgültigen Ansteuerungswerte pro lokal steuerbarem Hintergrundbeleuchtungsbereich.


 
2. Verfahren der lokalen Dimmverstärkung nach Anspruch 1, dadurch gekennzeichnet, dass das Anpassen von Hintergrundbeleuchtungsansteuerungswerten der Hintergrundbeleuchtungsbereiche weiter umfasst: Gewichten des Histogramms der Tiefenkarte mit einem Histogramm der Intensität des Bildes unter Verwendung eines Gewichtungsparameters.
 
3. Verfahren der lokalen Dimmverstärkung nach Anspruch 2, dadurch gekennzeichnet, dass der Gewichtungsparameter in Relation zu der Intensitätskarte des Bildes angepasst wird.
 
4. Verfahren der lokalen Dimmverstärkung nach Anspruch 2, dadurch gekennzeichnet, dass das Anpassen der Hintergrundbeleuchtungsansteuerungswerte der Hintergrundbeleuchtungsbereiche umfasst: Limitieren der Hintergrundbeleuchtungsansteuerungswerte entsprechend des maximalen Hintergrundbeleuchtungsansteuerungswerts des Displays.
 
5. Verfahren der lokalen Dimmverstärkung nach Anspruch 4, dadurch gekennzeichnet, dass das Limitieren umfasst: Definieren eines Grenzwertes und Verhindern der Verstärkung, wenn ein maximaler Hintergrundbeleuchtungsansteuerungswert des Displays größer ist als der Grenzwert.
 
6. Lokales Dimmverstärkungs-System für Liquid Crystal Displays umfassend;

- Mittel zum Berechnen eines Histogramms einer Intensitätskarte eines Bildes, das auf dem Display angezeigt wird,

- Mittel zum Berechnen von Dimm- oder Verstärkungs-Hintergrundbeleuchtungsansteuerungswerten pro jeweils lokal steuerbarem Hintergrundbeleuchtungsbereich gemäß dem Histogramm der Intensitätskarte, so dass die Bereiche, die als hell bestimmt wurden, verstärkt werden und die Bereiche, die als dunkel bestimmt wurden, gedimmt werden; dadurch gekennzeichnet, dass es weiter umfasst,

- Mittel zum Extrahieren einer Tiefenkarte des Bildes, wobei die Tiefenkarte eine hinsichtlich der menschlichen Wahrnehmung tatsächliche Information über die Signifikanzniveaus der Objekte auf einem Bild liefert;

- Mittel zum Berechnen eines Histogramms der Tiefenkarte für jeden lokal kontrollierbaren Hintergrundbeleuchtungsbereich des Displays und, entsprechend dem Ergebnis, Bestimmen von Ansteuerungswerten pro Bereich, so dass die Ansteuerungswerte in Bereichen verstärkt werden, in denen Objekte mit höherer Signifikanz dargestellt werden;

- Mittel zum Evaluieren der nach dem Histogramm der Intensitätskarte berechneten Ansteuerungswerte pro Bereich mit den nach dem Histogramm der Tiefenkarte berechneten Ansteuerungswerte pro Bereich zum Bestimmen der endgültigen Ansteuerungswerte pro lokal steuerbarem Hintergrundbeleuchtungsbereich.


 
7. Lokales Dimmverstärkungs-System nach Anspruch 6, dadurch gekennzeichnet, dass das System Mittel zum Gewichten des Histogramms der Tiefenkarte mit einem Histogramm der Intensitätskarte des Bildes unter Verwendung eines Gewichtungsparameters umfasst.
 


Revendications

1. Procédé d'augmentation d'atténuation locale pour des affichages à cristaux liquides comprenant les étapes de :

- le calcul d'un histogramme de carte d'intensité d'une image qui est affichée sur l'affichage,

- le calcul de valeurs de commande d'atténuation ou d'augmentation de rétroéclairage pour chaque région de rétroéclairage pouvant être commandée localement en fonction dudit histogramme de carte d'intensité de sorte que les régions déterminées comme étant brillantes soient augmentées et les régions déterminées comme étant sombres soient atténuées ; caractérisé par :

- l'extraction d'une carte de profondeur de ladite image, dans lequel ladite carte de profondeur donne des informations réelles en ce qui concerne les niveaux d'importance des objets dans une image en termes de perception humaine ;

- le calcul d'un histogramme de ladite carte de profondeur pour chaque région de rétroéclairage pouvant être commandée localement dudit affichage et, en fonction du résultat, la détermination de valeurs de commande pour chaque région de sorte que les valeurs de commande soient augmentées dans des régions présentant des objets avec une plus grande importance ;

- l'évaluation des valeurs de commande pour chaque région calculées en fonction de l'histogramme de carte d'intensité avec les valeurs de commande pour chaque région calculées en fonction de l'histogramme de la carte de profondeur pour déterminer les valeurs de commande définitives pour chaque région de rétroéclairage pouvant être commandée localement.


 
2. Procédé d'augmentation d'atténuation locale selon la revendication 1, caractérisé en ce que ledit ajustement de valeurs de commande de rétroéclairage desdites régions de rétroéclairage comprend en outre la pondération dudit histogramme de carte de profondeur avec un histogramme de carte d'intensité de ladite image en utilisant un paramètre de pondération.
 
3. Procédé d'augmentation d'atténuation locale selon la revendication 2, caractérisé en ce que ledit paramètre de pondération est ajusté en relation avec ladite carte d'intensité de l'image.
 
4. Procédé d'augmentation d'atténuation locale selon la revendication 2, caractérisé en ce que ledit ajustement de valeurs de commande de rétroéclairage desdites régions de rétroéclairage comprend la limitation de valeurs de commande de rétroéclairage en fonction d'une valeur de commande de rétroéclairage maximale de l'affichage.
 
5. Procédé d'augmentation d'atténuation locale selon la revendication 4, caractérisé en ce que ladite limitation comprend la définition d'un seuil et l'interdiction d'une augmentation lorsqu'une valeur de commande de rétroéclairage maximale de l'affichage est inférieure audit seuil.
 
6. Système d'augmentation d'atténuation locale pour des affichages à cristaux liquides comprenant :

- des moyens pour le calcul d'un histogramme de carte d'intensité d'une image qui est affichée sur l'affichage,

- des moyens pour le calcul de valeurs de commande d'atténuation ou d'augmentation de rétroéclairage pour chaque région de rétroéclairage pouvant être commandée localement en fonction dudit histogramme de carte d'intensité de sorte que les régions déterminées comme étant brillantes soient augmentées et les régions déterminées comme étant sombres soient atténuées ; caractérisé en ce qu'il comprend en outre :

- des moyens pour l'extraction d'une carte de profondeur de ladite image, dans lequel ladite carte de profondeur donne des informations réelles en ce qui concerne les niveaux d'importance des objets dans une image en termes de perception humaine ;

- des moyens pour le calcul d'un histogramme de ladite carte de profondeur pour chaque région de rétroéclairage pouvant être commandée localement dudit affichage et, en fonction du résultat, la détermination de valeurs de commande pour chaque région de sorte que les valeurs de commande soient augmentées dans des régions présentant des objets avec une plus grande importance ;

- des moyens pour l'évaluation des valeurs de commande pour chaque région calculées en fonction de l'histogramme de carte d'intensité avec les valeurs de commande pour chaque région calculées en fonction de l'histogramme de la carte de profondeur pour déterminer les valeurs de commande définitives pour chaque région de rétroéclairage pouvant être commandée localement.


 
7. Système d'augmentation d'atténuation locale selon la revendication 6, caractérisé en ce que le système comprend en outre des moyens pour la pondération dudit histogramme de carte de profondeur avec un histogramme de carte d'intensité de ladite image en utilisant un paramètre de pondération.
 




Drawing














Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description