A method for compensating local dimming of a backlight

Description

Technical Field

[0001] The present invention offers a method for local dimming in direct-lit and edge lit panels using salient features for required boost up in LED driving levels.

Prior Art

[0002] LED (Light Emitting Diode) backlight displays have recently become popular in the market due to their performance and energy saving characteristics. Moreover, the advantages of LED backlight can be used for improving the image quality and motion picture dynamics of the displays. Several methods have been developed in order to combine conventional LCD (Liquid Crystal Display) characteristics with LED backlight technology for image enhancement.

[0003] The perceived brightness of a pixel on the LCD depends on the cumulative effect of backlight intensity, filter transparency and LCD level. For an LCD with constant backlight intensity, the range of brightness level depends only on LCD setting. However, when a controllable backlight is used, the brightness can be increased by boosting the backlight and similarly darker regions can be achieved by backlight dimming. In this manner, if a locally controllable backlight is used with a conventional LCD, the contrast ratio (the ratio of the brightness of the brightest pixel to that of darkest pixel) can be boosted up to much higher levels. The control algorithm and methodology of backlight intensity are critical issues at this point.

[0004] Several control methods and systems are proposed to control light intensity of LED backlight. The patent document CN 201315147 proposes a method in which maximal gray value and density of this value is used to obtain the backlight driving value. The method enables use of backlight dynamics. However the proposed method is insufficient as it aims to process the whole display as a single unit and does not provide solution for locally controllable LED backlights.

[0005] The utility model document CN 201315148 offers a similar method to the utility model document CN 201315147. However, CN 201315148 offers a solution for locally controllable LED backlights in which, maximum of local gray value and its density is used to control the LED backlight. This method is better than CN 201315147 in terms of locality issues but still the information used for processing is inadequate.

[0006] The method in CN 101354875 proposes to utilize brightness statistics of different zones of the picture and to adjust corresponding local backlight intensity accordingly. The document also proposes to process the LCD control data in order to smooth the transition regions between backlight locals. The method is useful in improving image quality, yet it's insufficient as brightness statistic is inadequate for a full perception of the image.

[0007] Patent document No. US2009079768A1 discloses adjusting a backlight of a display device. According to said document, backlight of said display device is adjusted by comparing different regions shown in the display device in the spatial and temporal domains. Furthermore, during this comparison edge distributions for said regions are compared. However, said document does not provide a solution for enhancing intensity of the salient (visually attractive) regions.

[0008] Several literature works have also been published for control methods used for LED backlights. The methods depend on the histogram of the related zone to adjust light intensity of the region. The histogram based methods give better results however; they are still inadequate in terms of dark region perception.

Brief Description of the Invention

[0009] The scope of the present invention is defined by the present claims 1-8. Common methods used for the control of intensity of the LED backlight displays employ intensity or histogram of the image in which human visual system is not fully regarded. The invention proposes a method where a post processing is performed in order to compensate dimming of LED backlight based on intensity based method using salient features. The method first calculates LED driving values according to regional intensities. Meanwhile a saliency map is extracted. Then, histogram analysis of saliency map is realized and intensity based LED driving values are updated according to saliency map histogram. With the utilization of salient features, possible loss of detail in regions of the image, where intensity is low but the region is salient, are eliminated. In this manner, histogram based backlight adjustment errors are compensated and an image closer to the original is reflected, which is also more adequate for human visual perception.

Object of the Invention

[0010] The object of the invention is to compensate possible loss of detail originating from intensity based LED backlight adjustment techniques, using salient features of the image.

[0011] Another object of the invention is to improve overall contrast of the image.

[0012] Another object of the invention is to decrease power consumption while delivering a more perceptible image.

Brief Description of the Drawings

[0013] 

Figure 1; shows the saliency map of a synthetically generated image.

Figure 2; shows how the means of the small blocks are calculated hierarchically for computational efficiency.

Figure 3; shows how the mean values for the pixels that area not located at the center of the blocks are calculated by bilinear interpolation for computational efficiency.

Figure 4; shows the block diagram of algorithmic flow.

Figure 5; shows the histogram of the saliency map and effect of weighting on histogram.

Figure 6; shows the linear weighting function of the histogram.

Figure 7; shows how the amount of LED driving value boost is limited according to a parametric function depending on the block saliency level.

[0014] The reference numbers as used in figures may possess the following meanings.

The background region of the image which has a gray level of 120	(1)
The region of the image which has a gray level of 200	(2)
The region of the image which has a gray level of 50	(3)
The region of the image which has a gray level of 100	(4)
The background region of saliency map which has a gray level of 10	(5)
The region of saliency map which has a gray level of 200	(6)
The region of saliency map which has a gray level of 190	(7)
The region of saliency map which has a gray level of 20	(8)
The center points of 4 neighboring regions of LCD	(9, 10, 11, 12)
Saliency calculation point	(13)

Detailed Description of the Invention

[0015] The invention proposes a better control method for LED backlight displays in which intensity based and saliency based backlight level adjustment is performed, improving image details, contrast ratio and power efficiency.

[0016] Saliency concept is related with the response of the human visual system and defined by a map as response to original image. The saliency map is based on the deviation of a pixel from its surrounding pixels. If the intensity of a specific pixel is brighter than its surrounding pixels, the saliency map value of the said specific pixel is positive. The human visual perception directly depends on difference levels. A region deviating from surroundings in terms of light intensity takes visual attention regardless of local average light intensity.

[0017] Saliency is widely used in computer vision area and has applications such as coding, watermarking, feature matching, video abstraction and summarization. Various methods are proposed to extract salient features of an image or a video. Image enhancement based on saliency is used in computer graphics and biomedical engineering. In computer graphics area, computational complexity is decreased by processing the graphic such that, salient regions are processed in detail whereas regions with less saliency are processed with less detail. Biomedical engineering uses salient features to process an image to attract user attention to specific regions of the image.

[0018] An example of a frame and its corresponding saliency map is given in Figure 1. A synthetic view with 4 different 8 bit gray levels with a background level of 120 (1) and 3 regions with (2, 3 and 4) gray levels of 200, 50 and 100 respectively are given. The human visual system responds more to dissimilarity; hence a region deviating from the general background will be more attractive. In other words, attention is given to the objects diverging from the mean intensity level. On the right side of the Figure 1, the saliency map is given, where bright regions (6, 7) indicate high saliency; and corresponding area to the background (5) and region having gray level of 100 (8) are dark, that is saliency is low.

[0019] In most saliency extraction methods, Lab color space, which is a color space especially designed to approximate human vision, is widely utilized due to its uniformity in perceptual domain. In this work, since the fundamental aim is to determine LED backlight intensity based on the local properties of the pixels with respect to intensity, the saliency extraction is realized in RGB color space. In that manner, for each pixel, differences with the mean value of certain window sizes are calculated for three channels (R, G and B) independently. At that point, five different window sizes are utilized in order to determine the center-surround differences and how much the pixel is different from the surround. The window sizes are chosen to be according to equation (14). As can be seen in said equation (14), the window sizes are considerably large; the reason behind this choice is that small window sizes do not give enough information about the surrounding and provide edge detection instead of attraction detection which results in noisier feature maps.

[0020] Once the mean values within the specified windows of changing size are determined, the saliency of a pixel (C_k) is calculated as in the equation (15); where k corresponds to the index of the utilized window size, I_RGB is the RGB image; w and h are the window width and height respectively.

[0021] Large window sizes are utilized during the estimation of the saliency map, thus calculating the mean values of the window for each pixel independently requires huge processing. However, calculating mean for specific windows and performing bilinear interpolation drastically decreases the computation complexity. To achieve computationally simple processing, the input frame is divided into blocks determined by the smallest window ((frame width)/16 through the invention), and the calculated mean values of the blocks are assigned to the center pixels of these blocks as given in the following Figure 2 as dotted points. As the blocks get larger, the mean values are determined by mean operation through the smallest blocks in the corresponding large block. Once the mean values are determined for block centers of different sizes, bilinear interpolation is utilized to determine the mean values for each pixel (13) as illustrated in Figure 3. For the crossed pixel (13) which is between the block centers of (9, 10, 11 and 12) the mean value for a window size of ((frame_width)/4, (frame_height)/4) is calculated by interpolation of 4 neighboring block means determined bilinearly by the window size of ((frame_width)/4, (frame_height)/4). This operation is iterated for each window size defined in the equation (14), and the closest block centers to the corresponding pixel are utilized at each resolution. The calculated mean values for 5 different resolutions are subtracted from the corresponding pixel intensity value. Finally, these differences (C_i) are added to each other to determine the saliency of the pixel as in the equation (16).

[0022] In that way, the measure of how the pixels are different from their surroundings is obtained; in addition the darkness or brightness is also regarded such that, for darker pixels, saliency values are negative and for the brighter pixels, saliency is positive. This is an important fact and will be crucial in the refinement stage. Finally the salient values are mapped to a range of between (0, 255) by scaling the minimum and maximum values for convenience.

[0023] The general flow of the method is shown in Figure 4. Original image is taken as input and LED driving values are estimated through intensity based LED backlight algorithm. Simultaneously saliency levels are computed using saliency based algorithm. Then both outputs are used to decide final LED driving value. Saliency Map Extraction block computes the saliency map of the input image as explained previously. Saliency map is displayed as an intensity map where 0-255 levels show the amount of salient information contained in the image. 0 level represents the less salient region whereas 255 level corresponds to the most salient region. The histogram of the saliency map corresponding to the LED backlight blocks are computed in the next step. The number of individually controllable LED regions varies due to the manufacturer. The histogram of the saliency map for each region is calculated and weighted according to a parametric weighting formula (17). An exemplary block histogram of saliency map for a LED region is shown in Figure 5. A typical weighting function is given in figure 6. A linear mapping is used for convenience, other mapping functions may be used depending on the settings. The weighted histogram shown in Figure 5 is used to decide on saliency level for each block.

[0024] The final block of Boost Region Classification decides on the regions where LED driving values will be increased. The blocks that are previously assigned below a threshold (by the intensity based method) should not be boosted. Said blocks are important parts of the image where the intensity of the regions is really low. Said blocks supply mainly the contrast improvement. Increasing the intensity levels at these regions will bring no benefit since the proposed idea targets the medium intensity level regions where salient features are present. At this stage we have two sets of useful information. One is the estimated LED driving levels; other one is the weighted saliency levels for each region. The amount of boost is calculated in this block as follows.

[0025] As can be seen in the equation (18), Final Led Driving Level (LedBlockValueUpdate) is sum of product of parameter k and block saliency (BlockSaliency) and product of 1-k and intensity based calculated LED block Value (LedBlockValue).

[0026] The LED driving value estimated by the intensity based method is simply weighted with the estimated saliency level of the block. The parameter "k" is used as the ratio of saliency level, 0.9 is chosen for an effective boost-up. It is important to note that if the saliency level of the block is less than the estimated LED driving level, the equation (18) will cause a decrease in the final LED driving level. Saliency is meant to be used only for boosting, hence a decrease due to saliency is no means allowed. The final step in LED driving value decision is to check the limits of boost in order to prevent high temporal and spatial changes. Huge changes in intensity may cause lack of uniformity in time and space; hence a limiting idea has been utilized. The limits of boost in LED driving levels are shown in Figure 7. Limiting the boost level is realized depending on the value of block saliency. The higher the estimated block saliency, the higher the limit is. The limiting curve for two cases where block saliency is 255 (19) and 128 (20) are illustrated. The low threshold is the level below which no boost shall be performed.

Claims

1. A method for compensating local dimming in LED backlight displays using salient features, comprising the steps of:

- Calculating LED driving values according to corresponding regional intensities of an image,

- Dividing said image into windows with certain sizes and calculating mean intensity values of each of said windows,

- Extracting a saliency map of said image consisting of the saliencies of the pixels of the image, wherein the saliency of a pixel is based on the deviation of the pixel from its surrounding pixels in terms of intensity and is calculated using said mean intensity values of said windows,

- Calculating a histogram of said saliency map for each locally controllable LED region and a saliency level of the locally controllable LED region by carrying out a weighted sum of the calculated histogram values according to weights of a weighting function,

- Boosting calculated LED driving values according to calculated saliency levels of the corresponding locally controllable LED regions.

2. A method according to Claim 1 wherein said saliency extraction is realized in RGB color space.

3. A method according to Claim 1 wherein saliencies of pixels of the said image are calculated for three channels (R, G and B) independently.

4. A method according to Claim 1 wherein bilinear interpolation is performed for calculating the mean intensity values for said windows.

5. A method according to Claim 1 wherein said saliency map of said image are mapped to a range between 0 and 255 by scaling the minimum and maximum values of said saliency map.

6. A method according to Claim 1 wherein said saliency map is realized as an intensity map.

7. A method according to Claim 1 wherein the calculated LED driving values are boosted with the formula:


wherein LedBlockValueUpdate is the final LED driving level, k is a parameter, BlockSaliency is the saliency level of said locally controllable LED region and LedBlockValue is said calculated LED driving level.

8. A method according to Claim 1 wherein boosting said calculated LED driving values is controlled ensuring that said boosted LED driving values are below a defined limit.

Ansprüche

1. Verfahren zum Kompensieren eines lokalen Dimmens bei LED-Hintergrundbeleuchtungs- Displays (backlight displays) unter Verwendung von markanten Stellen oder salienten Eigenschaften (salient features), mit den folgenden Schritten

- Berechnen eines LED-Treiberwertes entsprechend regionalen Intensitäten eines Bildes;

- Aufteilen des Bildes in Fenster mit gegebenen Größen und Berechnen eines mittleren Intensitätswertes für jedes der Fenster;

- Extrahieren einer Salientenkarte (saliency map) des Bildes bestehend aus den markanten Stellen (Salienzen) der Pixel des Bildes, wobei die Salienz eines Pixels auf der Abweichung des Pixels von seinen Umgebungspixeln mit Blick auf Intensität beruht, und berechnet wird unter Verwendung der mittleren Intensitätswerte der Fenster;

- Berechnen eines Histogramms der Salientenkarte für jede lokal kontrollierbare LED-Region und eines Salientenpegels der lokal steuerbaren LED-Region durch Ausführen einer gewichteten Summe der berechneten Histogrammwerte entsprechend den Gewichten oder Wichtungen einer Gewichtungsfunktion;

- Verstärken der berechneten LED-Treiberwerte entsprechend den berechneten Salientenpegeln der entsprechenden lokal steuerbaren LED-Regionen.

2. Verfahren nach Anspruch 1, wobei das Extrahieren der herausragenden Merkmale (saliency extraction) in einem RGB-Farbraum erfolgt.

3. Verfahren nach Anspruch 1, wobei die herausragenden Merkmale der Pixel (saliencies of pixels) des Bildes für drei Kanäle (R, G und B) unabhängig voneinander berechnet werden.

4. Verfahren nach Anspruch 1, wobei eine bilineare Interpolation ausgeführt wird, zum Berechnen der mittleren Intensitätswerte der Fenster.

5. Verfahren nach Anspruch 1, wobei die Salientenkarte des Bildes in einen Bereich gemappt wird, der zwischen 0 und 255 liegt, durch Skalieren des minimalen und des maximalen Wertes der Salientenkarte.

6. Verfahren nach Anspruch 1, wobei die Salientenkarte als eine Intensitätskarte realisiert wird.

7. Verfahren nach Anspruch 1, wobei die berechneten LED-Treiberwerte mit der folgenden Formel verstärkt werden,


wobei gilt ...
LedBlockValueUpdate ist der finale LED-Treiberpegel,
k ist ein Parameter,
BlockSalienz ist der Salienzpegel der lokal kontrollierbaren LED-Region, LedBlockWert ist der berechnete LED-Treiberpegel.

8. Verfahren nach Anspruch 1, wobei das Anheben oder Verstärken der berechneten LEDTreiberwerte so gesteuert wird, dass die verstärkten oder angehobenen (boosted) LEDTreiberwerte unter einem vorgegebenen Grenzwert oder unter einer vorgegebenen Grenze sind.

Revendications

1. Procédé pour compenser la gradation de l'intensité lumineuse locale dans les afficheurs à rétroéclairage DEL en utilisant des traits saillants, comprenant les étapes consistant à :

- calculer des valeurs de pilotage de DEL selon des intensités régionales correspondantes d'une image,

- diviser ladite image en fenêtres avec certaines tailles et calculer des valeurs d'intensité moyennes de chacune desdites fenêtres,

- extraire une carte de saillance de ladite image se composant des saillances des pixels de l'image, où la saillance d'un pixel est basée sur la déviation du pixel de ses pixels environnants en termes d'intensité et est calculée en utilisant lesdites valeurs d'intensité moyennes desdites fenêtres,

- calculer un histogramme de ladite carte de saillance pour chaque région DEL pouvant être commandée localement et un niveau de saillance de la région DEL pouvant être commandée localement en effectuant une somme pondérée des valeurs d'histogramme calculées selon des poids d'une fonction de pondération,

- amplifier des valeurs de pilotage de DEL calculées selon des niveaux de saillance calculés des régions DEL pouvant être commandées localement correspondantes.

2. Procédé selon la revendication 1, dans lequel ladite extraction de saillance est réalisée dans l'espace de couleur RVB.

3. Procédé selon la revendication 1, dans lequel des saillances de pixels de ladite image sont calculées pour les trois canaux (R, V et B) indépendamment.

4. Procédé selon la revendication 1, dans lequel une interpolation bilinéaire est effectuée pour calculer les valeurs d'intensité moyennes pour lesdites fenêtres.

5. Procédé selon la revendication 1, dans lequel ladite carte de saillance de ladite image est mappée à une plage entre 0 et 255 en mettant à l'échelle les valeurs minimale et maximale de ladite carte de saillance.

6. Procédé selon la revendication 1, dans lequel ladite carte de saillance est réalisée comme une carte d'intensité.

7. Procédé selon la revendication 1, dans lequel les valeurs de pilotage de DEL calculées sont amplifiées avec la formule :


où LedllockValueUpdate est le niveau de pilotage de DEL final, k est un paramètre, BlockSaliency est le niveau de saillance de ladite région DEL pouvant être commandée localement et LedBlockValue est ledit niveau de pilotage de DEL calculé.

8. Procédé selon la revendication 1, dans lequel l'amplification desdites valeurs de pilotage de DEL calculées est commandée en s'assurant que lesdites valeurs de pilotage de DEL amplifiées soient au-dessous d'une limite définie.

Drawing