IMAGE DISPLAY IMPROVEMENT IN BRIGHTENED VIEWING ENVIRONMENTS

Abstract

 To get better performing adaptation of images to be displayed under variable surround illumination it is proposed to use a control processor (510) arranged to adjust the brightness of displaying on a display (520) of at least a dark sub-set of colors in an input image (IMG_in) in response to a measurement of a level of illumination (Lev_illumdrown) of an environment in which the display resides, wherein the adjustment comprises the establishment of an initial adjustment function (F_adj), for shifting one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) of pixels of the input image by at least one correction value (Adj_brightn) to obtain corresponding output color component triplets (R'_out, G'_out, B'_out), wherein the adjustment comprises multiplying for values of the one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) below a threshold value (GAM1) the initial adjustment function (F_adj) by a correction function (F_corr) to obtain a final version of the adjustment function (F_adj) for shifting the input color component triplets (R'_PQ, G'_PQ, B'_PQ), wherein the correction function (F_corr) satisfies a first property of yielding zero output for zero input, a second property of yielding an output value equal to one for an input value equal to the threshold value (GAM1) and a third property of having a zero first derivative at an input value equal to the threshold value (GAM1), or corresponding method of controlled adjustment.

Description

FIELD OF THE INVENTION

[0001] The invention relates to the displaying of in particular high dynamic range images or videos (sequences of images) which need some brightening of the displaying to adjust for non-dark viewing environments, which have an illumination above a typical illumination level as expected and for which the images were prepared by color grading.

BACKGROUND OF THE INVENTION

[0002] For more than half a century, an image representation/coding technology which is now called Low Dynamic Range (LDR) or. Standard Dynamic Range (SDR) worked perfectly fine for creating, communicating and displaying electronic images such as videos (i.e. temporally successive sequences of images). Colorimetrically, i.e. regarding the specification of the pixel colors, it was based on the technology which already worked fine decades before for photographic materials and paintings: one merely needed to be able to define, and display, most of the colors projecting out of an axis of achromatic colors (a.k.a. greys) which spans from black at the bottom end to the brightest achromatic color giving the impression to the viewer of being white. For television communication, which relies on an additive color creation mechanism at the display side, a triplet of red, green and blue color components needed to be communicated for each position on the display screen (pixel), since with a suitably proportioned triplet (e.g. 60%, 30%, 25%) one can make almost all colors, and in practice all needed colors (white being obtained by driving the three display channels to their maximum, with driving signal Rmax=Gmax=Bmax).

[0003] The earliest television standards (NTSC, PAL) communicated the color components as three voltage signals (which defined the amount of a color component between 0 and 700mV), where the time positions along the voltage signal corresponded by using a scan path with pixels on the screen.

[0004] The control signals generated at the creation side, directly instructed what the display should make as proportion (apart from their being an accidental fixed gamma pre-correction at the transmitter, because the physics of the cathode ray tube took approximately a square power of the input voltage, which would have made the dark colors much blacker than they were intended e.g. as seen by a camera) at the creation side). So a 60%, 30%, 25% color (which is a dark red) in the scene being captured, would look substantially similar on the display, since it would be re-generated as a 60%, 30%, 25% color (note that the absolute brightness didn't matter much, since the eye of the viewer would adapt to the white and the average brightness of the colors being displayed on the screen). One can call this "direct link driving", without further color processing (except for arbitrary and unnecessary processing a display maker might still perform to e.g. make his sky look more blue-ish). For reasons of backwards compatibility with the older black and white television broadcasts, instead of actually communicating a red, green and blue voltage signal, a brightness signal and two color difference signals called chroma were transmitted (in current nomenclature the blue chroma Cb, and the red Cr). The relationship between RGB and YCbCr is an easy one, namely they can be calculated into each other by using a simple fixed 3x3 matrix (the coefficients of which depend on the emission spectra of the three primaries, and are standardized, i.e. also to be emulated electronically internally by LCDs which actually may have different optical characteristics, so that from image communication point of view all SDR displays are alike).

[0005] These voltage signals were later for digital television (MPEG-based et al.) digitized, as prescribed in standard Rec. 709, and one defined the various amounts of e.g. the brightness component with an 8 bit code word, 0 coding for the darkest color (i.e. black), and 255 for white. Note that coded video signals need not be compressed in all situations (although they oftentimes are). In case they are we will use the wording compression (e.g. by MPEG-HEVC, AV1 etc.), not to be confused with the act of compressing colors in a smaller gamut respectively range. With brightness we mean the part of the color definition that will impact upon the colors (to be) displayed the visual property of being darker respectively brighter. In the light of the present technologies, it is important to correctly understand that there can be two kinds of brightnesses: relative brightness (as a percentage of something, which may be undefined until a choice is made, e.g. by the consumer buying a certain display, and setting its brightness setting to e.g. 120%, which will make e.g. the backlight emit a certain amount of light, and so also the white and colored pixels), and on the other hand absolute brightness. The latter can be characterized by the universal physical quantity luminance (which is measured technically in the unit nit, which is also candela per square meter). The luminance can be stated as an amount of photons coming out of a patch on an object, such as a pixel on the screen, towards the eye (and it is related to the lighting concept of illuminance, since such patch will receive a certain illuminance, and send some fraction of it towards the viewer).

[0006] Recently two unrelated technologies have emerged, which only came together because people argued that one might as well in one go improve the visual quality of images on all aspects, but those two technologies have quite different technical aspects.

[0007] On the one hand there was a strive towards wide gamut image technology. It can be shown that only colors can be made which lie within the triangle spanned by the red, green and blue primaries, and nothing outside. But one chose (EBU, also re-standardized in Rec. 709) primaries (originally phosphors for the CRT, later color filters in the LCD, etc.) which lay relatively close to the spectral locus of all existing colors, so one could make sufficiently saturated colors for most purposes. Saturation specifies how far away a color lies from the achromatic (colorless) colors, i.e. how much "color" there is. However, recently one wanted to be able to use novel displays with more saturated primaries (e.g. DCI_P3, or Rec. 2020), so that one also needed to be able to represent colors in such wider color spaces. A color space is the mathematical 3D space to represent colors (defining geometric positions of colors by coordinate numbers, e.g. a red, green and blue value of the weighed combination of primary intensities in the produced total color), often presented in such a shape that the base is defined by the triangle of 3 primaries. Color model refers to the election of the categories of numerical values one uses to define the space, e.g. red, green and blue being a natural representation for specifying additive color generators, yet the same colors can be defined by three other coordinate ranges in the Hue, Saturation, and Value model, which model characterizes the colors in a more human-related manner. For the technical discussion we may better use the word color gamut, which is the set of all colors that can be technically defined or displayed (i.e. a space may be e.g. a 3D coordinate system going to infinity whereas the gamut may be a cube or tent-shape of some size in that space). For the brightness aspect only, we will talk about brightness or luminance range (more commonly worded as "dynamic range", spanning from some minimum brightness to its maximum brightness). The present technologies will not primarily be about chromatic (i.e. color per se, such as more specifically its saturation) aspects, but rather about brightness aspects, so the chromatic aspects will only be mentioned to the extent needed for the relevant embodiments.

[0008] A more important new technology is High Dynamic Range (HDR). This should not be construed as "exactly this high" (since there can be many variants of HDR representations, with successively higher range maximum), but rather as "higher than the reference/legacy representation: SDR". Since there are new coding concepts needed, one may also discriminate HDR from SDR by aspects from the technical details of the representation, e.g. the video signal. One difference of absolute HDR systems is that they define a unique luminance for each image pixel (e.g. a pixel in an image object being a white dog in the sun may be 550 nit), where SDR signals only had relative brightness definitions (so the dog would happen look e.g. 75 nit (corresponding to 94%) on somebody's computer monitor which could maximally show 80 nit, but it would display at 234 nit on a 250 nit SDR display (yet the viewer would not typically see any difference in the look of the image, unless having those displays side by side). The reader should not confuse luminances as they exist (ultimately) at the front of any display, with luminances as are defined (i.e. establishable) on an image signal itself, i.e. even when that is stored but not displayed. Other differences are metadata that any flavor of HDR signal may have, but not the SDR signal.

[0009] Colorimetrically, HDR images can represent brighter colors than SDR images, so in particular brighter than white colors (glowing whites e.g.). Or in other words, the dynamic range will be larger. The SDR signal can represent a dynamic range of 1000:1 (how much dynamic range is actual visible when displaying, will depend inter alia on the amount of surround light reflecting on the front of the display). So if one wants to represent e.g. 10,000:1, one must resort to making a new HDR image format definition (we may in general use the word signal if the image representation is being or to be communicated rather than e.g. merely existing in the creating IC, and in general signaling will also have its own formatting and packaging, and may employ further techniques depending on the communication mechanism such as modulation).

[0010] Depending on the situation, the human eye can easily see (even if all on a screen in front corresponding to a small glare angle) 100,000:1 (e.g. 10,000 nit maximum and 0.1 minimum, which is a good black for television home viewing, i.e. in a dim room which only support lighting of a relatively low level, such as in the evening). However, it is not necessary that all images as created by a creative go as high: (s)he may elect to make the brightest image pixel in an image or the video e.g. 1000 nit.

[0011] The luminance of SDR white for videos (a.k.a. the SDR White Point Luminance (WP) or maximum luminance (ML)), is standardized to be 100 nit (not to be confused with the reference luminance of white text in 1000 nit HDR images being 200 nit). I.e., a 1000 nit ML HDR image representation can make up to 10x brighter (glowing) object colors. What one can make with this are e.g. specular reflections on metals, such as a boundary of a metal window frame: in SDR the luminance has to end at 100 nit, making them visually on slightly brighter than the e.g. 70 nit light gray colors of the part of the window frame that does not specularly reflect. In HDR one can make those pixels that reflect the light source to the eye e.g. 900 nit, making them glow nicely giving a naturalistic look to the image as if it were a real scene. The same can be done with fire balls, light bulbs, etc. This regards the definition of images; how a display which can only display whites as bright as 650 nit (the display maximum luminance ML_D) is to actually display the images is an entirely different matter, namely one of display adaptation a.k.a. display tuning, not of image (de)coding. Also the relationship with how a camera captures HDR scene colors may be tight or loose: we will in general assume that HDR colors have already been defined in the HDR image when talking about such technologies as coding, communication, dynamic range conversion and the like. In fact, the original camera-captured colors or specifically their luminances may have been changed into different values by e.g. a human color grader (who defines the ultimate look of an image, i.e. which color triplet values each pixel color of the image(s) should have), or some automatic algorithm. So already for the fact that the chromatic gamut size may stretch with less than a factor 2, whereas the brightness range e.g. luminance range may stretch by a factor 100, one expects different technical rationales and solutions for the two improvement technologies.

[0012] A HDR image may be associated with a metadatum called mastering display white point luminance (MDWPL), a.k.a. ML_V. This value, which is typically communicated in metadata of the signal, and is a characterizer of the HDR video images (rather than of a specific display, as it is an element of a virtual display associated specifically with the video, being some ideal intended display for which the video pixel colors have been optimized to be conforming). This is an electable parameter of the video, which can be contemplated as similar to the election of the painting canvas aspect ratio by a painter: first the painter chooses an appropriate AR, e.g. 4:1 for painting a landscape, or 1:1 when he wants to make a still life, and thereafter he starts to optimally position all his objects in that elected painting canvas. In an HDR image the creator will then, after having established that the MDWPL is e.g. 5000 nit, make his secondary elections that in a specific scene this lamp shade should be at 700 nit, the flames in the hearth distributed around 500 nit, etc.

[0013] The primary visual aspect of HDR images is (since the blacks are more tricky) the additional bright colors (so one can define ranges with only a MDWPL value, if one assumes the bottom luminance to be fixed to e.g. 0.1 nit). However, HDR image creation can also involve deeper blacks, up to as deep as e.g. 0.0001 nit (although that is mostly relevant for dark viewing environments, such as in cinema theatres).

[0014] The other objects, e.g. the objects which merely reflect the scene light, will be coordinated to be e.g. at least 40x darker in a 5000 nit MDWPL graded video, and e.g. at least 20x darker in a 2000 nit video, etc. So the distribution of all image pixel luminances will typically depend on the MDWPL value (not making most of the pixels very bright).

[0015] The digital coding of the brightness, involves a technical quantity called luma (Y). We will give luminances the letter L, and (relative) brightnesses the letter B. Note that technically, e.g. for ease of definition of some operations, one can always normalize even 5000 nit WPDPL range luminances to the normalized range [0,1], but that doesn't detract from the fact that these normalized luminances still represent absolute luminances on a range ending at 5000 nit (in contrast to relative brightnesses that never had any clear associated absolute luminance value, and can only be converted to luminances ad hoc, typically with some arbitrary value).

[0016] For SDR signals the luma coding used a so-called Opto-electronic Transfer Function (OETF), between the optical brightnesses and the electronic typically 8 bit luma codes which by definition was:

[0017] If B_relative is a float number ranging from 0 to 1.0, so will Y_float.

[0018] Subsequently that signal value Y_float is quantized, because we want 8 bit digital representations, ergo, the Y_dig value that is communicated to receivers over e.g. airways DVB (or internet-supplied video on demand, or blu-ray disk, etc.) has a value between 0 and 255 (i.e. power(2;8)-1).

[0019] One can show the gamut of all SDR colors (or similarly one can show gamuts of HDR colors, which would if using the same RGB primaries defining the chromatic gamut have the same base, but after renormalization stretch vertically to a larger absolute gamut solid) as in Fig. 1B. Larger Cb and Cr values will lead to a (psychovisually more relevant color characterizer) larger saturation (sat), which moves outwards from the unsaturated or colorless colors vertical axis showing the achromatic colors in the middle, towards the maximally saturated colors on the circle (which is a transformation of the usual color triangle spanned by the RGB primaries as vertices). Hues h (i.e. the color category, yellows, versus greens, versus blues) will be angles along the circle. The vertical axis represents normalized luminances (in a linear gamut representation), or normalized lumas (in a non-linear representation (coding) of those luminances, e.g via a psychovisually uniformized OETF or its inverse the EOTF). Since after normalization (i.e. division by the respective MDWPL values, e.g. 2000 for an HDR image of a particular video and 100 for an SDR image), the common representation will become easy, and one can define luminance (or luma) mapping functions on normalized axes as shown in Fig. 1D (i.e. which function F_comp re-distributes the various image object pixel values as needed, so that in the actual luminance representation e.g. a dark object looks the same, i.e. has the same luminance, but a different normalized luminance since in one situation a pixel normalized luminance will get multiplied by 100 and in the other situation by 2000, so to get the same end luminance the latter pixel should have a normalized luminance of 1/20^th of the former). When down-grading to a smaller range of luminances, one will typically get (in the normalized to 1.0 representation, i.e. mapping a range of normalized input luminances L_in between zero and one to normalized output luminances L_out) a convex function which everywhere lies above the diagonal diag, but the exact shape of that function F_comp, e.g. how fact it has to rise at the blacks, will depend typically not only on the two MDWPL values, but (to have the most perfect re-grading technology version) also on the scene contents of various (video or still) images, e.g. whether the scene is a dark cave, and there is action happening in a shadowy area, which must still be reasonably visible even on 100 nit luminance ranges (hence the strong boost of the blacks for such a scenario, compared to a daylight scene which may employ a near linear function almost overlapping with the diagonal).

[0020] In the representation of Fig. 1B one can show the mapping from a HDR color (C_H) to an LDR color (C_L) of a pixel as a vertical shift (assuming that both colors should have the same proper color, i.e. hue and saturation, which usually is the desired technical requirement, i.e. on the circular ground plane they will project to the same point). Ye means the color yellow, and its complementary color on the opposite side of the achromatic axis of luminances (or lumas) is blue (B), and W signifies white (the brightest color in the gamut a.k.a. the white point of the gamut, with the darkest colors, the blacks being at the bottom).

[0021] So the receiving side (in the old days, or today) will know it has an SDR video, if it gets this format. The maximum white (of SDR) will be by definition the brightest color that SDR can define. So if one now wants to make brighter image colors (e.g of real luminous lamps), that should be done with a different codec (as one can show the math of the Rec. 709 OETF allows only a coding of up to 1000:1 and no more).

[0022] So one defined new frameworks with different code allocation functions (EOTFs, or OETFs). What is of interest here is primarily the definition of the luma codes.

[0023] For reasons beyond what is needed for the present discussion, most HDR codecs start by defining an Electro-optical transfer function instead of its inverse, the OETF. Then one can at least basically define brighter (and darker) colors. That as such is not enough for a professional HDR coding system, since because it is different from SDR, and there are even various flavors, one wants more (new compared to SDR coding) technical information relating to the HDR images, which will be metadata.

[0024] The property of those HDR EOTFs is that they are much steeper, to encode a much larger range of needed to be coded HDR luminances, and a significant part of that range coding specifically darker colors (relatively darker, since although one may be coding absolute luminances with e.g. the Perceptual Quantizer (PQ) EOTF (standardized in SMPTE 2084), one applies the function after normalization). In fact if one were to use exact power functions as EOTFs for coding HDR luminances as HDR lumas, one would have a power of 4, or even 7. When a receiver gets a video image signal defined by such an EOTF (e.g. Perceptual Quantizer) it will know it gets a HDR video. It will need the EOTF to be able to decode the pixel lumas in the plane of lumas spanning the image (i.e. having a width of e.g. 4000 pixels and a height of 2000), which will simply be binary numbers. Typically HDR images will also have a larger word length, e.g. 10 bit. However, one should not confuse the non-linear coding one can at will design by optimizing a non-linear EOTF shape with linear codings and the amount of bits needed for them. If one needs to drive, with a linear (bit-represented) code, e.g. a DMD pixel, indeed to reach e.g. 10000:1 modulation darkest to brightest, one needs to take the log2 to obtain the number of bits. There one would need to have at least 14 bits (which may for technical reasons get rounded upwards to 16 bits), since power(2;14)= 16384 > 10000. But being able to smartly design the shape of the EOTF, and knowing that the visual system sees not all luminance differences equally, the present applicant has shown that (surprisingly) quite reasonable HDR television signals can be communicated with only 8 bit per pixel color component (of course if technically achievable in a system, 10 bits may be better and more preferable). So the receiving side may in both situations get as input a coded pixel color (luma and Cb, Cr; or in some systems by matrixing equivalent non-linear R'G'B' component values) which lie between 0 and 255, or 0 and 1023, but it will know the kind of signal it is getting (hence what ought to be displayed) from the metadata, such as the metadata (e.g. MPEG VUI metadata) co-communicated EOTF (e.g. a value 16 meaning PQ; 18 means another OETF was used to create the lumas, namely the Hybrid LogGamma OETF, ergo the inverse of that function should be used to decode the luma plane), in many HDR codings the MDWPL value (e.g. 2000 nit), and in more advanced HDR codings further metadata (some may e.g. co-encode luminance -or luma- mapping functions to apply for mapping image luminances from a primary luminance dynamic range to a secondary luminance dynamic range, such as one function FL_enc per image).

[0025] We detail the typical needs of an already more sophisticated HDR image handling chain with the aid of simple elucidation Fig. 1 (for a typical nice HDR scene image, of a monster being fought in a cave with a flame thrower, the master grading (Mstr HDR) of which is shown spatially in Fig. 1A, and the range of occurring pixel luminances on the left of Fig. 1C). The master grading or master graded image is where the image creator can make his image look as impressive (e.g. realistic) as desired. E.g., in a Christmas movie he can make a baker's shop window look somewhat illuminated by making the yellow walls somewhat brighter than paper white, e.g. 150 nit (and real colorful yellow instead of pale yellow), and the light bulbs can be made 900 nit (which will give a really lit Christmas-like look to the image, instead of a dull one in which all lights are clipped white, and not much more bright than the rest of the image objects, such as the green of the Christmas tree).

[0026] So the basic thing one must be able to do is encode (and typically also decode and display) brighter image objects than in a typical SDR image.

[0027] Let's look at it colorimetrically now. SDR (and its coding and signaling) was designed to be able to communicate any Lambertian reflecting color (i.e. a typical object, like your blue jeans pants, which absorbs some of the infalling light, e.g. the red and green wavelengths, to emit only blue light to the viewer or capturing camera) under good uniform lighting (of the scene where the action is camera-captured). Just like we would do on a painting: if we don't add paint we get the full brightness reflecting back from the white painting canvas, and if we add a thick layer of strongly absorbing paint we will see a black stroke or dot. We can represent all colors brighter than blackest black and darker than white in a so-called color gamut of representable colors, as in Fig. 1B (the "tent"). As a bottom plane, we have a circle of all representable chromaticities (note that one can have long discussions that in a typical RGB system this should be a triangle, but those details are beyond the needs of the present teachings). A chromaticity is composed of a certain (rotation angle) hue h (e.g. bluish-green e.g. "teal"), and a saturation sat, which is the amount of pure color mixed in a grey, e.g. the distance from the vertical axis in the middle which represents all achromatic colors from black at the bottom becoming increasingly bright till we arrive at white. Chromatic colors, e.g. a half-saturated purple, can also have a brightness, the same color being somewhat darker or brighter. However, the brightest color in an additive system can only be (colorless) white, since it is made by setting all color channels to maximum R=G=B=255, ergo, there is no unbalance which would make the color clearly red (there is still a little bit of bluishness respectively yellowishness in the elected white point chromaticity, but that is also an unnecessary further discussion, we will assume D65 daylight white). We can define those SDR colors by setting MDWPL a (relative) 100% for white W (n.b., in legacy SDR white does not actually have a luminance, since legacy SDR does not have a luminance associated with the image, but we can pretend it to be X nit, e.g. typically 100 nit, which is good average representative value of the various legacy SDR tv's).

[0028] Now we want to represent brighter than Lambertian colors, e.g. the self-luminous flame object (flm) of the flame thrower of the soldier (sol) fighting the monster (mon) in this dark cave.

[0029] Let's say we define a (video maximum luminance ML_V) 5000 nit master HDR grading (master means the starting image -most important in this case best quality grading- which we will optimally grade first, to define the look of this HDR scene image, and from which we can derive secondary gradings a.k.a. graded images as needed). We will for simplicity talk about what happens to (universal) luminances, then we can for now leave the debate about the corresponding luma codes out of the discussion, and indeed PQ can code between 1/10,000 nit and 10,000 nit, so there is no problem communicating those graded pixel luminances as a e.g. 10 bit per component YCbCr pixelized HDR image, if coding according to that PQ EOTF (of course, the mappings can also be represented, and e.g. implemented in the processing IC units, as an equivalent luma mapping).

[0030] The two dotted horizontal lines represent the limitations of the SDR codable image, when associating 100 nit with the 100% of SDR white.

[0031] Although in a cave, the monster will be strongly illuminated by the light of the flames, so we will give it an (average) luminance of 300 nit (with some spread, due to the square power law of light dimming, skin texture, etc.).

[0032] The soldier may be 20 nit, since that is a nicely slightly dark value, still giving some good basic visibility.

[0033] A vehicle (veh) may be hidden in some shadowy corner, and therefore in an archetypical good impact HDR scene of a cave e.g. have a luminance of 0.01 nit. The flames one may want to make impressively bright (though not too exaggerated). On an available 5000 nit HDR range, we could elect 2500 nit, around which we could still gradually make some darker and brighter parts, but all nicely colorful (yellow and maybe some oranges).

[0034] What would now happen in a typical SDR representation, e.g. a straight from camera SDR image capturing?

[0035] The camera operator would open his iris so that the soldier comes out at "20 nit", or in fact more precisely 20%. Since the flames are much brighter (note: we didn't actually show the real world scene luminances, since master HDR video Mstr_HDR is already an optimal grading to have best impact in a typical living room viewing scenario, but also in the real world the flames would be quite brighter than the soldier, and certainly the vehicle), they would all clip to maximum white. So we would see a bright area, without any details, and also not yellow, since yellow must have a lower luminance (of course the cinematographer may optimize things so that there is still somewhat of a flame visible even in LDR, but then that is firstly never as impactful extra bright, and secondly at the detriment of the other objects which must become darker).

[0036] The same would also happen if we built an SDR (max. 100 nit) TV which would map equi-luminance, i.e. it would accurately represent all luminances of the master HDR grading it can represent, but clip all brighter object to 100 nit white.

[0037] So the usual paradigm in the LDR era was to relatively map, i.e. the brightest brightness (here luminance) of the received image to the maximum capability of the display. So as this maps 5000 nit by division by 50 on 100 nit, the flames would still be okay since they are spread as yellows and oranges around 50 nit (which is a brightness representable for a yellow, since as we see in Fig. 1B the gamut tent for yellows goes down in luminance only a little bit when going towards the most saturated yellows, in contrast to blues (B) on the other side of the slice for this hue angle B-Ye, which blues can only be made in relatively dark versions). However, this would be at the detriment of everything else becoming quite dark, e.g. the soldier 20/50 nit which is pure black (and this is typically a problem that we see in SDR renderings of such kinds of movie scene).

[0038] So, if having established a good HDR maximum luminance (i.e. ML_V) for the master grading, and a good EOTF e.g. PQ for coding it, we can in principle start communicating HDR images to receivers, e.g. consumer television displays, computers, cinema projectors, etc.

[0039] But that is only the most basic system of HDR.

[0040] The problem is that, unless the receiving side has a display which can display pixels at least as bright as 5000 nit, there is still a question of how to display those pixels.

[0041] Some (DR adaptation) luminance down-mapping must be performed in the TV, to make darker pixels which are displayable. E.g. if the display has a (end-user) display maximum luminance ML_D of 1500 nit, one could somehow try to calculate 1200 nit yellow pixels for the flame (potentially with errors, like some discoloration, e.g. changing the oranges into yellows).

[0042] This luminance down-mapping is not really an easy task, especially to do very accurately instead of sufficiently well, and therefore various technologies have been invented (also for the not necessarily similar task of luminance up-mapping, to create an output image of larger dynamic range and in particular maximum luminance than the input image).

[0043] Typically one wants a mapping function (generically, i.e. used for simplicity of elucidation) of a convex shape in a normalized luminance (or brightness) plot, as shown in Fig. 1D. Both input and output luminances are defined here on a range normalized to a maximum equaling one, but one must mind that on the input axis this one corresponds to e.g. 5000 nit, and on the output axis e.g. 200 nit (which to and fro can be easily implemented by division respectfully multiplication). In such a normalized representation the darkest colors will typically be too dark for the grading with the lower dynamic range of the two images (here for down-conversion shown on the vertical output axis, of normalized output luminances L_out, the horizontal axis showing all possible normalized input luminances L_in). Ergo, to have a satisfactory output image corresponding to the input image, we must relatively boost those darkest luminances, e.g. by multiplying by 3x, which is the slope of this luminance compression function F_comp for its darkest end. But one cannot boost forever if one wants no colors to be clipped to maximum output, ergo, the curve must get an increasingly lower slope for brighter input luminances, e.g. it may typically map input 1.0 to output 1.0. In any case the luminance compression function F_comp for down-grading will lie above the 45 degree diagonal (diag) typically.

[0044] Care must still be taken to do this correctly. E.g., some people like to apply three such compressive functions to the three red, green and blue color channels separately. Whilst this is a nice and easy guarantee that all colors will fit in the output gamut (an RGB cube, which in chromaticity-luminance (L) view becomes the tent of Fig. 1B) especially with higher non-linearities it can lead to significant color errors. A e.g. reddish orange hue is determined by the percentage of red and green, e.g. 30% green and 70% red. If the 30% now gets doubled by the mapping function, but the red stays in the feeble-sloped part of the mapping function almost unchanged, we will have a 60/70, i.e. 50/50 i.e. a yellow instead of an orange. This can be particularly annoying if it depends on (in contrast to the SDR paradigm) non-uniform scene lighting, e.g. a sports car entering the shadows suddenly turning yellow.

[0045] Ergo, whilst the general desired shape for the brightening of the colors may still be the function F_comp (e.g. determined by the video creator, when grading a secondary image corresponding to his master HDR image already optimally graded), one wants a more savvy down-mapping. As shown in Fig. 1B, for many scenarios one may desire a re-grading which merely changes the brightness of the normalized luminance component (L), but now the innate type of color, i.e. its chromaticity (hue and saturation). If both SDR and HDR are represented with the same red, green and blue color primaries, they will have a similarly shaped gamut tent, only one being higher than the other in absolute luminance representation. If one scales both gamuts with their respective MDWPL values (e.g. MDWPL1= 100 nit, and MDWPL2= 5000 nit), both gamuts will exactly overlap. The desired mapping from a HDR color C_H to a corresponding output SDR color C_L (or vice versa) will simply be a vertical shifting, whilst the projection to the chromaticity plane circle stays the same.

[0046] Although the details of such approaches are also beyond the need of the present application, we have thought examples of such color mapping mechanism before, where the three color components are processed coordinately, although in a separate luminance and chroma processing path, e.g. in WO2017157977.

[0047] If it is now possible to down-grade with one (or more) luminance mapping functions (the shape of which may be optimized by the creator of the video(s)), in case one uses invertible functions one can design a more advanced HDR codec.

[0048] Instead of just making some final secondary grading from the master image, e.g. in a television, one can make a lower dynamic range image version for communication, communication image Im_comm. We have elected in the example this image to be defined with its communication image maximum luminance ML_C equal to 200 nit. The original 5000 nit image can then be reconstructed (a.k.a. decoded) as a reconstructed image Rec_HDR (i.e. with the same reconstructed image maximum luminance ML_REC) by receivers, if they receive in metadata the decoding luminance mapping function FL_dec, which is typically substantially the inverse of the coding luminance mapping function FL_enc, which was used by the encoder to map all pixel luminances of the master HDR image into corresponding lower pixel luminances of the communication image Im_comm. So the proxy image for actually communicating an image of a higher dynamic range (DR_H, e.g. spanning from 0.001 nit to 5000 nit) is an image of a different, lower dynamic range (DR_L).

[0049] Interestingly, one can even elect the communication image to be a 100 nit LDR (i.e. SDR) image, which is immediately ready (without further color processing) to be displayed on legacy LDR images (which is a great advantage, because legacy displays don't have HDR knowledge on board). How does that work? The legacy TV doesn't recognize the MDWPL metadatum (cos that didn't exist in the SDR video standard, so the TV is also not arranged to go look for it somewhere in the signal, e.g. in a Supplemental Enhancement Information message, which is MPEG's mechanism to introduce all kinds of pre-agreed new technical information). It is also not going to look for the function. It just looks at the YCbCr e.g. 1920x1080 pixel color array, and displays those colors as usual, i.e. according to the SDR Rec. 709 interpretation. And the creator has chosen in this particular codec embodiment his FL_enc function so that all colors, even the flame, map to reasonable colors on the limited SDR range. Note that, in contrast to a simple multiplicative change corresponding to the opening or shutting of a camera iris in an SDR production (which typically leads to clipping to at least one of white and/or black), now a very complicated optimal function shape can be elected, as long as it is invertible (e.g. we have taught systems with first a coarse pre-grading and then a fine-grading). E.g. one can move the luminance (respectively relative brightness) of the car to a level which is just barely visible in SDR, e.g. 1% deep black, whilst moving the flame to e.g. 90% (as long as everything stays invertible). That may seem extremely daunting if not impossible at first sight, but many field tests with all kinds of video material and usage scenarios have shown that it is possible in practice, as long as one does it correctly (following e.g. the principles of WO2017157977).

[0050] How do we now know that this is actually a HDR video signal, even if it contains an LDR-usable pixel color image, or in fact that any HDR-capable receiver can reconstruct it to HDR: because there are also the functions FL_dec in metadata, typically one per image. And hence that signal codes what is also colorimetrically, i.e. according to our above discussion and definition, a (5000 nit) HDR image.

[0051] Although already more complex than the basic system which communicates only a PQ-HDR image, this per SDR proxy coding is still not the best future-proof system, as it still leaves the receiving side to guess how to down-map the colors if it has e.g. a 1500 nit, or even a 550 nit, tv.

[0052] Therefore we added a further technical insight, and developed so-called display tuning technology (a.k.a. display adaptation): the image can be tuned for any possible connected tv, i.e. any ML_D, because one can double the function of the coding function FL_enc as some guidance function for the up-mapping from 100 nit Im_comm not to a 5000 nit reconstructed image, but to e.g. a 1500 nit image. The concave function, which is substantially the inverse of F_comp (note, for display tuning there is no requirement of exact inversion as there is for reconstruction), will now have to be scaled to be somewhat less steep (i.e. from the reference decoding function FL_dec a display adapted luminance mapping function FL_DA will be calculated), since we expand to only 1500 nit instead of 5000 nit. I.e. an image of tertiary dynamic range (DR_T) can be calculated, e.g. optimized for a particular display in that the maximum luminance of that tertiary dynamic range is typically the same as the maximum displayable luminance of a particular display.

[0053] Techniques for this are described in WO2017108906 (we can transform a function of any shape into a similarly-shaped function which lies closer to the 45 degree diagonal, by an amount which depends on the ratio between the maximum luminances of the input image and the desired output image, versus the ratio of the maximum luminances of the input image and a reference image which would here be the reconstructed image, by e.g. using that ratio to obtain closer points on for all points on the diagonal orthogonally projecting a line segment from the respective diagonal point till it meets a point on the input function, which closer points together define the tuned output function, for calculating the to be displayed image Im_disp luminances from the Im_comm luminances).

[0054] Not only did we get more kinds of displays even for basic movie or television video content (LCD tv, mobile phone, home cinema projector, professional movie theatre digital projector), and more video different sources and communication media (satellite, streaming over the internet, e.g. OTT, streaming over 5G), but also did we get more production manners of video.

[0055] Fig. 2 shows -in general, without desiring to be limiting- a few typical creations of video where the present teachings may be usefully deployed.

[0056] In a studio environment, e.g. for the news or a comedy, there may still be a tightly controlled shooting environment (although HDR allows relaxation of this, and shooting in real environments). There will be controlled lighting (202), e.g. a battery of base lights on the ceiling, and various spot lights. There will be a number of bulky relatively stationary television cameras (201). Variations on this often real-time broadcast will be e.g. a sports show like soccer, which will have various types of cameras like near-the-goal cameras for a local view, overview cameras, drones, etc.

[0057] There will be some production environment 203, in which the various feeds from the cameras can be selected to become the final feed, and various (typically simple, but potentially more complex) grading decisions can be taken. In the past this often happened in e.g. a production truck, which had many displays and various operators, but with internet-based workflows, where the raw feeds can travel via some network, the final composition may happen at the premises of the broadcaster. Finally, when simplifying the production for this elucidation, some coding and formatting for broadcast distribution to end (or intermediate, such as local cable stations) customers will happen in formatter 204. This will typically do the conversion to e.g. PQ YCbCr from the luminances as graded as explained with Fig. 1, for e.g. an intermediate dynamic range format, calculate and format all the needed metadata, convert to some broadcasting format like DVB or ATSC, packetize in chunks for distribution, etc. (the etc. indicating there may be tables added for signaling available content, sub-titling, encryption, but at least some of that will be of lesser interest to understand the details of the present technical innovations).

[0058] In the example the video (a television broadcast in the example) is communicated via a television satellite 250 to a satellite dish 260 and a satellite signal capable set-top-box 261. Finally it will be displayed on an end-user display 263.

[0059] This display may be showing this first video, but it may also show other video feed, potentially even at the same time, e.g. in Picture-in-Picture windows (or some data of the first HDR video program may come via some distribution mechanism and other data via another).

[0060] A second production is typically an off-line production. We can think of a Hollywood movie, but it can also be a show of somebody having a race through a jungle. Such a production may be shot with other optimal cameras, e.g. steadicam 211 and drone 210. We again assume that the camera feeds (which may be raw, or already converted to some HDR production format like HLG) are stored somewhere on network 212, for later processing. In such a production we may in the last months of production have some human color grader use grading equipment 213 to determine the optimal luminances (or relative brightnesses in case of HLG production and coding) of the master grading. Then the video may be uploaded to some internet-based video service 251. For professional video distribution this may be e.g. Netflix.

[0061] A third example is consumer video production. Here the user will have e.g. when making a vlog a ring lighter 221, and will capture via a mobile phone 220, but (s)he may also be capturing in some exterior location without supplementary lighting. She/he will typically also upload to the internet, but now maybe to youtube, or tiktok, etc.

[0062] In case of reception via the internet, the display 263 will be connected via a modem, or router 262 or the like (more complicated setups like in-house wifi and the like are not shown in this mere elucidation).

[0063] Another user may be viewing the video content on a portable display (271), such as a laptop (or similarly other users may use a non-portable desktop PC), or a mobile phone etc. They may access the content over a wireless connection (270), such as Wifi, 5G, etc.

[0064] Current professional reference displays for viewing created video can go as high as ML_D = 10,000 nit. Even in the consumer arena, recently a 10,000 nit television has been demonstrated. So, if things evolve beneficially, we may see high quality HDR imagery being commonplace in various applications on not too long term. Note that cameras don't actually capture exact luminances, but rather relative brightnesses of a scene (due to the arbitrary selection of iris etc.), however, they have no problem capturing high dynamic ranges nowadays. Even standard cameras can at least faithfully capture 14 powers of two of differential brightness in a scene, and more advanced cameras with specialized sensor construction, or techniques such as multiple exposure can go well beyond this (e.g. 20 powers of two, which is a quite reasonable capturing dynamic range for most situations).

[0065] So it can be seen that today, various kinds of video, in various technical codings, can be generated and communicated in various manners, and our coding and processing systems have been designed to handle substantially all those variants.

[0066] Fig. 3 shows an example of an absolute (nit-level-defined) dynamic range conversion circuit 300 for a (HDR) image or video decoder (the encoder working similarly but with inverted functions typically, i.e. the function to be applied being the function of the other side mirrored over the diagonal). It is based on coding a primary image (e.g. a master HDR grading) with a primary luminance dynamic range (DR_Prim) as another (so-called proxy) image with a different secondary range of pixel luminances (DR_Sec). If the encoder and all its supply-able decoders have pre-agreed or know that the proxy image has a maximum luminance of 100 nit, this need not be communicated as an SDR_WPL metadatum. If the proxy image is e.g. a 200 nit maximum image, this will be indicated by filling its proxy white point luminance P_WPL with the value 200, or similarly for 80 nit etc. The maximum of the primary image (HDR_WPL= 1000), to be reconstructed by the dynamic range conversion circuit, will normally be co-communicated as metadata of the received input image, or video signal, i.e. together with the input pixel color triplets (Y_in, Cb_in, Cr_in). The various pixel lumas will typically come in as a luma image plane, i.e. the sequential pixels will have first luma Y11, second Y21, etc. (typically these will be scanned, and the dynamic range conversion circuit will convert pixel by pixel to output pixel color triplets (Y_out, Cb_out, Cr out). We will primarily focus on the brightness dimension of the pixel colors in this elucidation. Various dynamic range conversion circuits may internally work differently, to achieve basically the same thing: a correctly reconstructed output luminance L_out for all image pixels (the actual details don't matter for this innovation, and the embodiments will focus on teaching only aspects as far as needed).

[0067] The mapping of luminances from the secondary dynamic range to the primary dynamic range may happen on the luminances themselves, but also on any luma representation (i.e. according to any EOTF, or OETF), provided it is done correctly, e.g. not separately on non-linear R'G'B' components. The internal luma representation need not even be the one of the input (i.e. of Y_in), or for that manner of whatever output the dynamic range conversion circuitry or its encompassing decoder may deliver (e.g. a format luma Y_sigfm for a particular communication format or communication system, "communicating" including storage to a memory, e.g. inside a PC, a hard disk, an optical storage medium, etc.).

[0068] We have optionally (dotted) shown a luma conversion circuit 301, which turns the input lumas Y_in into perceptionally uniformized lumas Y_pc.

[0069] Applicant standardized in ETSI 103433 a useful equation to convert luminances in any range to such a perceptual luma representation:

[0070] In which the function RHO is defined as

[0071] The value WPL_inrep is the maximum luminance of the range that needs to be converted to psychovisually uniformized lumas, so for the 100 nit SDR image this value would be 100, and for the to be reconstructed output image (or the originally coded image at the creation side) the value would be 1000.

[0072] Ln_in are the luminances along that whichever range which need to be converted, after normalization by dividing by its respective maximum luminance, i.e. within range [0,1].

[0073] Once we have an input and an output range normalized to 1.0, we can apply a luminance mapping function actually in the luma domain, as shown inside the luma mapping circuit 302, which does the actual luma mapping for each incoming pixel.

[0074] In fact, this mapping function had been specifically chosen by the encoder of the image (at least for yielding good quality reconstructability, and maybe also a reduced amount of needed bits when MPEG compressing, but sometimes also fulfilling further criteria like e.g. the SDR proxy image being of correct luminance distribution for the particular scene -a dark cave, or a daytime explosion- on a legacy SDR display, etc.). So this function F_dec (or its inverse) will be extracted from metadata of the input image signal or representation, and supplied to the dynamic range conversion circuit for doing the actual per pixel luma mapping. In this example the function F_dec directly specifies the needed mapping in the perceptual luma domain, but other variants are of course possible, as the various conversions can also be applied on the functions. Furthermore, although for simplicity of explanation, and to guarantee the teaching is understood, we teach here a pure decoder dynamic range conversion, but other dynamic range conversions may use other functions, e.g. a function derived from F_dec, etc. The details of all of that are not needed for understanding the present innovative contribution to the technology.

[0075] In general one will not only change the luminances, but there will be a corresponding change in the chromas Cb and Cr. That can also be done in various manners, from strictly inversely decoding, to implementing additional features like a saturation boost, since Cb and Cr code the saturation of the pixels. Thereto another function is typically communicated in metadata (recoloring specification function FCOL), which determines the chromatic recoloring behavior, i.e. the mapping of Cb and Cr (note that Cb and Cr will typically be changed by the same multiplicative amount, since the ratio of Cr/Cb determines the hue, and generally one does not want to have hue changes when decoding, i.e. the lower and higher dynamic range image will in general have object pixels of different brightness, and oftentimes at least some of the pixels will have different saturation, but ideally the hue of the pixels in both image versions will be the same). This color function will typically specify a multiplier which has a value dependent on a brightness code Y (e.g. the Y_pc, or other codes in other variants). A multiplier establishment circuit 305 will yield the correct multiplier m for the brightness situation of the pixel being processed. A multiplier 306 will multiply both Cb_in and Cr_in by this same multiplier, to obtain the corresponding output chromas Cb_out= m*Cb_in and Cr_out=m*Cr_in. So the multiplier realizes the correct chroma processing, therefore the whole color processing of any dynamic range conversion being correctly configurable in the dynamic range conversion circuit.

[0076] Furthermore, there may typically be (at least in a decoder) a formatting circuit 310, so that the output color triplet (Y_out, Cb_out, Cr_out) can be converted to whatever needed output format (e.g. an RGB format, or a communication YCbCr format, Y_sigfm, Cb_ sigfm, Cr_sigfm). E.g. if the circuit outputs to a version of a communication channel 379 which is an HDMI cable, such cables typically use PQ-based YCbCr pixel color coding, ergo, the lumas will again be converted from the perceptual domain to the PQ domain by the formatting circuit.

[0077] It is important that the reader well understands what is a (de)coding, and how an absolute HDR image, or its pixel colors, is different from a legacy SDR image. There may be a connection to a display tuning circuit 380, which calculates ultimate pixel colors and luminances to be displayed at the screen of some display, e.g. a 450 nit tv which some consumer has at home.

[0078] However, in absolute HDR, one can establish pixel luminances already in the decoding step, at least for the output image (here the 1000 nit image).

[0079] We have shown this in Fig. 3B, for some typical HDR image being an indoors/outdoors image. Note that, whereas in the real world the outdoors luminances may typically be 100 times brighter than the indoors luminances, in an actual master graded HDR image it may be better to make them e.g. 10× brighter, since the viewer will be viewing all together on a screen, in a fixed viewing angle, even typically in a dimly illuminated room in the evening, and not in the real world.

[0080] Nevertheless, we find that when we look at the luminances corresponding to the lumas, e.g. the HDR luminances L_out, we typically see a large histogram (of counts N(L_out) of each occurring luminance in an output image of this homely scene). This spans considerably above some lower dynamic range lobe of luminances, and above the low dynamic range 100 nit level, because the sunny outdoors images have their own histogram lobe. Note that the luminance representation is drawn non-linearly, e.g. logarithmically. We can also trace what the encoder would do at the encoding side, when making the 100 nit proxy image (and its histogram of proxy luminance counts N(L_in)). A convex function, as shown in Fig. 1, or inside luma mapper 302, is used which squeezes in the brighter luminances, due to the limitations of the smaller luminance dynamic range. There is still some difference between the brightness of indoors and outdoors, and still a considerable range for the upper lobe of the outdoors objects, so that one can still make the different colors needed to color the various objects, such as the various greens in the tree. However, there must also be some sacrifices. Firstly the indoors objects will display (assuming for the moment an 100 or 200 nit display would faithfully display those luminances as coded, and not e.g. do some arbitrary beautification processing which brightens them) darker, darker than ideally desired, i.e. up to the indoors threshold luminance T_in of the HDR image. Secondly, the span of the upper lobe is squeezed, which may give the outdoors objects less contrast. Thirdly, since bright colors in the tent-shaped gamut as shown in Fig. 1 cannot have large saturation, the outdoors colors may also be somewhat pastellized, i.e. of lowered saturation. But of course if the grader at the creation side has control over all the functions (F_enc, FCOL), he may balance those features, so that some have a lesser deviation at the detriment of others. E.g. if the outdoors shows a plain blue sky, the grader may opt for making it brighter, yet less blue. If there was a beautiful sunset, he may want to retain all its colors, and make everything dimmer instead, in particular if there are no important dark corners in the indoors part of the image, which would then have their contents badly visible, especially when watching tv with all the lights on (note that there are also techniques for handling illumination differences and the visibility of the blacks, but that is too much information for this patent application's elucidation).

[0081] The middle graph shows what the lumas would look like for the proxy luminances, and that may typically give a more uniform histogram, with e.g. approximately the same span for the indoors and outdoors image object luminances. The lumas are however only relevant to the extent of coding the luminances, or in case some calculations are actually performed in the luma domain (which has advantages for the size of the word length on the processing circuitry). Note that whereas the absolute formalism can allocate luminances on the input side between zero and 100 nit, one can also treat the SDR luminances as relative brightnesses (which is what a legacy display would do, when discarding all the HDR knowledge and communicated metadata, and looking merely at the 0-255 luma and chroma codes).

[0082] There is one caveat to the direct rendering paradigm of video coding. Indeed, as we have explained, both in the SDR era and for at least absolute nit HDR codings, the idea was that the viewer got to see exactly what the creator wanted him to see because, (all hardware limitations which were counteracted besides) the display would render the various grey percentages (or more generally all colors) exactly as created (and as visible in the creation studio on a reference display which is supposed to behave largely like the home display, provided both displays have the same white point luminance that is; and approximately in the percentual brightness relationships as at least the important objects in the scene would have, and how a camera would capture them, at least in the SDR era, leaving camera specifics besides).

[0083] That is the behavior we see in Fig. 4 (with the standard display driving function TF_std), which explains something which could be done already in the days of the Cathode Ray Tube (CRT), i.e. the typical display of the SDR video times last century. If one applies a driving signal s_driv to the display (lets say one color channel to keep things simple), ignoring for the moment the square power behavior of the display to that signal, i.e. assuming it has already been converted by pre-squaring into an equivalent linear driving signal, we see we can drive the display up to the maximum of the driving signal, which we may call "1", or 100%, and which corresponds to the brightest displaying of the display, because it doesn't get to do anything brighter, already following from the fact it is directly controlled by the display. If we then say the display should e.g. display 25% of its maximum, to create some grey color, i.e. s_driv = 0.25, we will indeed on the screen see a displayed screen luminance (L_screen) of 25% of the brightest pixels the screen would see. And the eye and the brain would adjust to this displayed image, recognize what would be the illumination level and the visual percept or qualia of white, and all the other colors. So brightness (grey) driving signal s_driv "0" would show the darkest possible color on the display, and that would be black (e.g. an OLED display would have the local OLED not emitting light), 10% should give 10% output, and 100% would give 100% output, whatever that would be (relative displaying such as legacy SDR displaying would e.g. show white as 150 nit, if that was the display one bought, and the brightness settings one made if not sticking to factory settings; and absolute displays should ideally display the maximum indicated pixel luminance of an image, or some reduced version thereof).

[0084] However, in the studio one may be watching the display in a surround illumination which causes the average reflecting grey object around the display to be e.g. 10 nit. Not so in the viewing environment of the consumer: his viewing environment may change considerably (usually substantially statically on average, but potentially even dynamically if e.g. the sun gets replaced by incoming storm clouds). One user may be watching that same movie in almost pitch black circumstances, maybe only with the LEDs indicators of his appliances and some other stray light illuminating the room, and another viewer, or the same viewer at a different time, may be watching the movie during sunlit daytime. Although some extreme situations are difficult to fully compensate and then technically out of scope (one can never compensate the direct sunlight shining on the screen of a display that can go only as bright as 100 nit, then one should close the curtains), it is advantageous if the display has some controls that can correct for the situation that e.g. the partner is reading a book somewhat further in the room. Thereto already CRTs had two separately settable controls. The first one is called "contrast" and it set the contrast or differences in adjacent grey values through acting in the amount of light emitted by the brightest white, by adding a (multiplicative) amount of luminance Adj_contrast to the white Whi color (and linearly all colors below, with their respective proportions). Some displays, like older LCD displays with a fixed backlight, can electronically not adjust this. In any case, a second control setting, called "brightness" (somewhat interestingly named, since the contrast also acts on the brightness, whereas brightness primarily acts on the brightnesses of the darker colors, so it is also called "offset") can add a fixed amount which is the black offset (Adj_brightn) to all luminances. The equation modeling the controllable electronic behavior of especially an SDR display would be:

[0085] This would yield several controllable versions of an adjusted display driving function (TF_adj).

[0086] It should be noted that one may not necessarily want to drive the maximum brightness of HDR in the same manner (since there is much more sophistication going on, both in the luminance distribution of the received HDR images, and the manner to display adapt them), nor does one necessarily want to adjust the one or more mid-region contrasts in such a crude manner, however as regards the blacks, especially as in many situations there may be no need to critically focus on ultra-deep HDR blacks, it may be a reasonable technical objective to make the HDR displaying scenario as regards the blacks behave the same or similar as the SDR scenario.

[0087] So one would also in HDR image reception and display scenarios at least raise all luminances with the additive constant Adj_brightn (and maybe also do some multiplicative boost with Adj_contrast, but let's assume that is left to "1", i.e. standard value), which would primarily have a visual effect on the darker colors in the image (note also that the visual system -i.e. processing in the eyes and brain- is such that it is approximately logarithmic, ergo, the differences in "the blacks" i.e. the darkest colors, are perceived as more significant than adding a small additive change to bright colors). In particular, the darkest black is raised to a value which is in the vicinity of a typical luminance level originating from the surround illumination in the room. Although there are also some visual adaptation effects in the human eye, the primary issue here would be reflections from the lighter environments (e.g. walls) on the front screen of the display, which would mask the image detail in the darkest emitted screen luminances, at least be above zero nit. E.g. the drowning level Lev_illumdrown may be on average 5 nit, added to the displayed image as per the following equation:

[0088] In which L_screen_electronically_displayed is the amount of colored light and luminance that gets generated electronically be the display controlled by the incoming image, i.e. eq. 1, and L_screen_seen is the total luminance travelling to the viewer's eye. Ergo, a totally black pixel of the image (s_driv = 0) still gets seen as a somewhat brighter black, namely the black one would expect in such a somewhat brighter viewing environment. If one sets the Adj_brightn value around the average Lev_illumdrown value, e.g. slightly lower, the user nicely gets to see all the different very dark blacks and greys which are driven slightly above perfect black (Blk) i.e. s_drive= 0 (i.e. s_driv=1, or 2, etc.). I.e. he still gets to see all image detail of the images in their darkest areas, such as a fight going on at night time, or a monster lurking in the dark.

[0089] Although working very well for many years, it was found by the present inventors that this manner of illumination-counter-control is not the best way to realize a high quality display of HDR images.

SUMMARY OF THE INVENTION

[0090] The indicated problem is handled by control processor (510) arranged to adjust the brightness of displaying on a display (520) of at least a dark sub-set of colors in an input image (IMG_in) in response to a measurement of a level of illumination (Lev_illumdrown) of an environment in which the display resides, wherein the adjustment comprises the establishment of an initial adjustment function (F_adj), for shifting one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) of pixels of the input image by at least one correction value (Adj_brightn) to obtain corresponding output color component triplets (R'_out, G'_out, B'_out), wherein the adjustment comprises multiplying for values of the one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) below a threshold value (GAM1) the initial adjustment function (F_adj) by a correction function (F_corr) to obtain a final version of the adjustment function (F_adj) for shifting the input color component triplets (R'_PQ, G'_PQ, B'_PQ), wherein the correction function (F_corr) satisfies a first property of yielding zero output for zero input, a second property of yielding an output value equal to one for an input value equal to the threshold value (GAM1) and a third property of having a zero first derivative at an input value equal to the threshold value (GAM1).

[0091] A standard surround adjustment function (as explained with Fig. 4) would work nicely on many types of image content, since the distribution of the values suggest a reasonable black (black is a qualia in the brain, that can vary to a certain degree: besides the absolute minimum black, dark greys start looking blackish already if their brightness falls below 5% of white). One can make better or worse looking black under good dark viewing conditions, but under significant surround illumination the visibility of all the dark pixels is the dominant criterion over good looking black, and somewhat worse blacks can be accepted. That is also still true when the surround illumination is not constant over space or time. E.g., some part of the movie may be reflecting a somewhat brighter surround object on a screen that reflects specularly, say some lamp shade in the room, and also that bright spot may change if somebody walks in front of the lamp shade. For some kinds of fully black and large image objects, like e.g. the black bars of a spatial format conversion (e.g. showing a 21:9 movie on a 16:9 television), the classical solution may not be preferable, as those objects may look rather greyish instead of true black.

[0092] Therefore the inventor invented a correction strategy, that leaves most of the processing intact (since there may be highly tuned surround light compensation going on in some of the embodiments), but makes at least for the true black objects as defined in the input video a true black to be displayed.

[0093] The actual color processing can be performed in various color spaces. What is common is that additive display mechanisms typically require three color components to make substantially any color as desired. However, the basis a.k.a. primary colors may vary. E.g., when retaining SDR chromaticities and a so-called narrow gamut, one may re-use the EBU primaries, however, one may give those a substantially increased maximum brightness (compared to the classical CRT, which could make red pixels with a maximum brightness of about 30 nit, and no more, current e.g. OLEDs can make much brighter red primaries, hence make contributions in the color triplet that defines any color of the red component which are brighter than before, hence a larger gamut of brighter colors can be made, despite the components still being defined in an RGB color model). However the saturation of the colors which can be made (i.e. the color gamut) must still fall within the boundaries of the triangle spanned by the elected EBU primaries. If one wants to be able to make more saturated (still RGB) colors, one can use e.g. DCI_P3 or Rec. 2020 color primaries.

[0094] Whereas the driving of an additive display (be it an OLED display, LCD, or any other) always happens in RGB (because it must create in a location on the screen a suitably weighed proportion of lit red, green and blue sub-pixels), the calculations of that needed driving color, hence also the adjustments of the present embodiments, can happen in different color representations or models. Since the skilled person of optimized color displaying should know how to transform between color models (which is the election of the three components) respectively color spaces (which is the metric quantification of the models, giving the various definable colors specific numerical values), which typically involves using the equations of the defined color spaces, we will elucidate only two typical examples.

[0095] If colors are processed in a separate luma (Y) and chroma (Cb and Cr) processing, then typically one may apply the adjustment and its novel correction function only to the luma component (Cb and Cr may also be correctively counter-processed, but also arbitrarily processed or even left unchanged).

[0096] If one processes in an RGB color model (of any non-linear Electro-Optical Transfer Function specification, i.e. by e.g. converting the linear red, green and blue percentual or absolute contributions by a Perceptual Quantizer inverse EOTF which is standardized in SMPTE 2084), which is sometimes necessary in some color processing architectures, then the three color components will get the function-based adjustment as taught.

[0097] In some embodiments there may be only one correction value (Adj_brightn) to be applied for all possible input color component value similarly at least in the initial adjustment function (F_adj), but more advanced methods may apply a differential brightening, in which the initial adjustment function (F_adj) would shift the input component values with different amounts depending on the value of the input component, i.e. there would be more than one correction value (Adj_brightn) applied in the initially envisaged surround illumination adaptive processing.

[0098] The present improvement method can work in any scenario, as it corrects the initial adjustment function (F_adj) for dark input values (defining dark image colors) below an electable threshold value (GAM1). So in the luma processing this threshold value will be used for the input lumas, and RGB processing will use three times the same threshold value when processing each color channel

[0099] (e.g. the red one) separately (typically in parallel). The threshold value may be chosen to be a pre-fixed small value (e.g. 0.1 nit or smaller, or 0.0001 relatively), but some embodiments may elect it to be equal to a measured surround illumination level (Lev_illumdrown). This value may be evaluated once, e.g. read on startup as the previous value when watching television, and then corrected to be a new value for today's watching, after measuring for e.g. 5 minutes, and then keeping this value, or it can be continuously adapted, but not on too fine a scale as one doesn't; want to see the black level dancing around, but e.g. once every 10 minutes or less often. So the correction function multiplies with the initial adjustment function (F_adj), for all possible input values, to get a final adjustment function, which will be the one applied for doing the surround illumination adaptive brightening of mainly the darker pixels (though as explained, if one uses a pure constant addition as initial adjustment this will also change the brightest pixels, but as explained in a visually less significant manner).

[0100] Advantageously the control processor (510) applies the correction function which is strictly monotonically increasing (note that the initial adjustment function can have e.g. clipping above a maximum value). In this text, we will use the usual definition that a monotonically increasing function, when making its input higher, cannot have a lower output for a higher input than for a lower input value (though it could have a constant output for some range of increasing input values). A strictly monotonically increasing function must always have a higher output value for a higher input value than for a previous lower input value, i.e. it cannot have flat output value regions.

[0101] Advantageously the control processor (510) applies its illumination adjustment and hence its correction function on an input and output domain of psychovisually uniformized color component values. One example of such a function is the perceptual quantizer functions of SMPTE 2084, i.e. when doing processing on only a luma component one would pre-apply the inverse PQ EOTF on the luminance of a pixel, and then do the illumination adjustment on the luma in the uniformized domain. When processing three RGB components in parallel, one would apply the inverse PQ EOTF on the three linear red, green and blue component contributions to any incoming pixel color being processed, and then do the illumination adjustment on those three resulting non-linear representations of the color component (the non-linearity indicated typically by a prime, e.g. R' instead of the linear R). Another example of a psychovisually uniformized function is the Philips Uniformized (PU) function, as specified in ETSI 1034233 (hereby incorporated in its entirety), which is:

In which the function RHO is defined as

[0102] The value WPL_inrep is the maximum luminance of the range that needs to be converted to psychovisually uniformized lumas, so for the 100 nit SDR image this value would be 100, and for a 1000 nit maximum image the value would be 1000.

[0103] Ln_in are the luminances along that whichever range which need to be converted, after normalization by dividing by its respective maximum luminance, i.e. within range [0,1].

[0104] So before doing the actual processing, one pre-applies an inverse perceptual quantizer electro-optical transfer functions on linear versions of the one or more color components of input color component triplets (which by themselves in some embodiments where the surround illumination acts as a post-processor may have already undergone image processing such as display adaptation, but in embodiments where all processing happens in one step, the one or more components may be input color components as received in a e.g. broadcasted video signal).

[0105] Advantageously the control processor (510) uses an embodiment of the correction function which has a number of higher order derivatives equal to zero at an input value equal to the threshold value (GAM1).

[0106] Useful embodiments of the correction function to apply in practice are e.g. y= - 1*power(x;4)+4*power(x;3)-6* power(x;2)+4*x, wherein x is an input value between zero and a linear representation of the threshold value (GAM1) which is normalized to a maximum value equal to 1, and y is the output value of the correction function for the input value; or y= power(x;5) - 5*power(x;4)+10*power(x;3)-10* power(x;2)+5*x.

[0107] The control processor may reside in any apparatus preparing for display, such as e.g. a computer, or in a display itself.

[0108] The present innovations may also be embodied in a method of adjusting the brightness of displaying on a display (520) of at least a dark sub-set of colors in an input image (IMG_in) in response to a measurement of a level of illumination (Lev_illumdrown) of an environment in which the display resides, wherein the adjustment comprises the establishment of an initial adjustment function (F_adj), for shifting one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) of pixels of the input image by at least one correction value (Adj_brightn) to obtain corresponding output color component triplets (R'_out, G'_out, B'_out), wherein the adjustment comprises multiplying for values of the one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) below a threshold value (GAM1) the initial adjustment function (F_adj) by a correction function (F_corr) to obtain a final version of the adjustment function (F_adj) for shifting the input color component triplets (R'_PQ, G'_PQ, B'_PQ), wherein the correction function (F_corr) satisfies a first property of yielding zero output for zero input, a second property of yielding an output value equal to one for an input value equal to the threshold value (GAM1) and a third property of having a zero first derivative at an input value equal to the threshold value (GAM1).

[0109] A useful embodiment of the method of adjusting the brightness of displaying as has an embodiment of the correction function is strictly monotonically increasing.

[0110] A useful embodiment of the method of adjusting the brightness of displaying applies the correction function on an input and output domain of psychovisually uniformized color component values (i.e. both input and output values are pre-transformed from their reference linear representations to any such a domain, typically defined by a corresponding EOTF or OETF), such as by pre-applying an inverse perceptual quantizer electro-optical transfer functions on linear versions of the one or more color components of input color component triplets.

[0111] The various methods may be embodied as computer program products comprising software code for instructing a processor to execute one of the methods, and e.g. delivered over internet (e.g. a firmware upgrade for the display) etc.

BRIEF DESCRIPTION OF THE DRAWINGS

[0112] These and other aspects of the method and apparatus according to the invention will be apparent from and elucidated with reference to the implementations and embodiments described hereinafter, and with reference to the accompanying drawings, which serve merely as non-limiting specific illustrations exemplifying the more general concepts, and in which dashes are used to indicate that a component is optional, non-dashed components not necessarily being essential. Dashes can also be used for indicating that elements, which are explained to be essential, but hidden in the interior of an object, or for intangible things such as e.g. selections of objects/regions.

[0113] In the drawings:

Fig. 1 schematically explains various luminance dynamic range re-gradings, i.e. mappings between input luminances (typically specified to desire by a creator, human or machine, of the video or image content) and corresponding output luminances (of various image object pixels), of a number of steps or desirable images that can occur in a HDR image handling chain;

Fig. 2 schematically introduces (non-limiting) some typical examples of use scenarios of HDR video or image communication from origin (e.g. production) to usage (typically a home consumer);

Fig. 3 schematically illustrates how the brightness conversion and the corresponding conversion of the pixel chromas may work and how the pixel brightness histogram distributions of a lower and higher brightness, here specifically luminance, dynamic range corresponding image may look;

Fig. 4 schematically shows how a display can be controlled to show a same input image in different manners, adjusting for an amount of surround illumination in a location in which the image is displayed;

Fig. 5 schematically shows a first circuit diagram to embody the present invention (one or more of the circuits may be operating under control of software, and may be at least partially configurable);

Fig. 6 schematically shows a second circuit diagram to embody the present invention (one or more of the circuits may be operating under control of software, and may be at least partially configurable);

Fig. 7 shows numerically the operation of an embodiment of the current innovation to obtain better performing surround illumination adaptation for a scenario of a 1000 nit maximum display and a 1 nit surround illumination level (other initial adjustment function shapes are equally possible);

Fig. 8 exemplifies the same embodiment, but in a scenario with a 1000 nit maximum display and a 10 nit surround illumination level; and

Fig. 9 shows the shapes of some embodiments of correction functions according to the principles of the present innovation that can be used.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0114] Fig. 5 shows a first embodiment of how to realize the improved surround illumination compensation. In this variant, the control processor (510) is realized as a post-processor, and acts on input color component triplets (red non-linear component R'_PQ, and green and blue non-linear component G'_PQ and B'_PQ) which form pixels of an input image IMG_in of the present innovation's surround illumination adjusting stage, and are outputted from a pre-processing stage, which non-linear components in this example are non-linearized according to the inverse perceptual quantizer EOTF. The control processor comprises three function-based mapping circuits (first 511, second 512, and third 513 function-based mapping circuit) for the respective input component, to yield a respective brightened output component, e.g. non-linear red output R'_out (respectively G'_out, B'_out). In practice, after the calculation of the function, this may be realized e.g. as lookup tables. The shape of the adjustment function (F_adj), as elucidated with the examples of Figs. 7 and 8, depends on a measurement of the surround illumination (Lev_illumdrown), by illumination meter 580. How to use an illumination meter is a detail beyond the scope of the present invention, e.g. it may comprise a white light homogenizing dome before sending the incoming light to a measuring electronic component which captures the amount of incoming photons and converts them to a voltage or current, which is typically factory-calibrated into a representative digital value, and a corresponding luminance Lev_illumdrown. The meter may be e.g. on the front of a display. The pre-processing circuit 501 may typically e.g. implement the image processing as described above, i.e. decode and display optimize an input HDR image or video without taking into account a particular (higher) level of surround illumination (i.e. assuming a dark room, or some typical reference viewing illumination, of e.g. dim or dark home movie viewing). Several systems will be simplistic, and call that situation of the darkest pixel display (in any environment) pragmatically zero, irrespective the actual amount of e.g. leak light of the display, a small amount of reflected light on the front of the display screen etc. A more advanced approach, which takes all these details into account to come to a more precise characterization of the actual displayed output (and hence look) of the blacks is e.g. taught in applicant's WO2022/233612 (the teachings of which are incorporated by reference in their entirety). The present improvement technology can work on both variants, and in its various implementation embodiments (i.e. order of processing circuitry, elected color space for calculating, etc.).

[0115] The pre-processing circuit 501 will typically contain a luma processing circuit 502, which will adapt the incoming absolute luminances or relative percentual brightnesses, by using some luma mapping function, which will typically depend on characteristics of the display to be served with the images, such as the maximum luminance capability ML_D of the display. The shape of the function may also depend on properties of the image, such as global properties like its average luma or average luminance, or more detailed properties, which may be communicated as metadata (e.g. a guidance luma mapping function F_Lguid). A chroma processing circuit may do chroma processing, ideally corresponding to the luma processing, so that the colors do not look over- or undersaturated. A color transformation circuit 504 transforms the color representation consisting of the mapped luma Y_LM and the processed chromas Cb_pr, and Cr_pr into the needed input format for the control processor, i.e. PQ-based non-linear R'G'B' components.

[0116] Fig. 6 shows another possible embodiment of the control processor, now as a total color mapping circuit 5101. This circuit can again do some luma mapping in luma mapping circuit 550. E.g. by using a luma mapping function as received as an exterior guiding luma mapping F_Lguid, which may be further optimized into another function shape (e.g. taking into account metadata MET such as a maximum luminance ML_D of a connected display) before applying the present techniques, or already constitute a fully optimized initial adjustment function F_adj for the connected display capabilities and the current viewing environment characterization (i.e. the currently stored or determined level of illumination Lev_illumdrown). Before actually doing the luma processing, the function is however multiplied by the correction function to yield a new bottom part 555 of the final version of the adjustment function F_adj. That function may then be e.g. stored in a LUT, and will thereafter be applied to any of the incoming pixels sequentially scanned from the input image, until e.g. anew scene occurs, and at least the optimal luma mapping for the new scene luma distribution will require a new function shape, or, until a new measurement of the environment illumination requires a new bottom part (and/or a new additive constant Adj_brightn), even when a static mapping curve is used for the entire movie for the display adaptation of the video to the maximum luminance capabilities of the display. The chroma processing circuit 551 may similarly apply any of the chroma adaptation variants already mentioned above, either independent of the measured level of illumination Lev_illumdrown or also in dependence on that level. E.g., preferably the chroma correction may take luminance dependencies into account. One may e.g. similarly as for the RGB parallel processing, use a same or similar correction function also for the bottom part of the Cb and Cr component. Finally a color transformer 552 will be arranged to again do the static color transformation to the needed output format (R'_out, G'_out, B'_out) for driving the display, e.g. PQ-based for communication over HDMI cable, etc.

[0117] Fig. 7 shows the numerical behaviour of the processing, in an example of the Fig. 5 example. Each of the three function-based mapping circuits (511, 512, and 513) applies such a function, where Ein is the normalized value in the PQ domain of the input to the function and Eout the normalized value of the output result of the function. The initial adjustment function F_adj is here simply a linear compressive function (after having re-distributed all image object lumas of a scene corresponding to the capabilities of the connected display in the pre-processor), in the psychovisually uniformized PQ domain, which maps the zero image pixel values to an output value equalling the measured level of illumination Lev_illumdrown (i.e. as output on the vertical axis), and the maximum to the maximum of the image (i.e. retaining its value, here 1000 nit, represented in its equivalent PQ value 0.7518). F_std would again be the function without taking account of the specific surround illumination of the viewing environment i.e. it would be an identity transform (up to 1000 nit). The simple embodiment of the initial adjustment function F_adj would in this case be:

in which PQ stands for the inverse Perceptual Quantizer EOTF, GAM is in this example 1 nit (which is relatively dark illumination), and the value of the display maximum luminance ML_D is here chosen to be 1000 nit, i.e. the display can display pixels as bright as 1000 nit

[0118] To come to the final F_adj function to be applied to the various non-linear red, green and blue pixel color components of the pixels of the input image., this function of Eq. 2 is multiplied by the correction function F_BCor for the darkest pixel values, i.e. values below GAM1 on the input axis.

[0119] Ergo the definition of the final function is:

and

[0120] One can also specify the shape of the function F_BCor, so that it is normalized on the input axis, i.e. "1" corresponds to whichever elected value of GAM1.

[0121] This function, called h(x) with 0<=x<=1 will typically satisfy the properties:

first derivative of h(1) =0

and typically h(x) is strictly monotonically increasing

(and oftentimes it is also advantageous if its 2^nd, 3^rd, ... (n-1)th derivative are also zero, for any polynomial of order n>=3).

[0122] The following functions have been determined by the inventor useful functions to use as the correction function F_BCor (where the coefficients are the coefficients of powers, starting with the highest power n on the left, and the constant (i.e. multiplied with 1) on the right:

i.e. this means that a second order polynomial, should one want to use it (but a fifth or 6^th order has been researched to be more preferred), is: -x*x + 2x;

[0123] The other polynomials one may consider are:

[0124] Fig. 9 gives the shapes of these polynomials (from which one may typically want to use order 4 up to 8). We see that if one desires a steep convergence towards the initial adjustment function F_adj, one should use a higher order polynomial, e.g. order 8 or higher.

[0125] Fig. 8 gives another example of the same procedure of multiplying with a corrective function to obtain good blacks for the actual fully blacks in an image, but now for an illumination level (equivalent luminance) of 10 nit (i.e. a somewhat bright viewing environment). Now it has a higher second threshold value GAM2.

[0126] The algorithmic components disclosed in this text may (entirely or in part) be realized in practice as hardware (e.g. parts of an application specific integrated circuit) or as software running on a special digital signal processor, or a generic processor, etc. At least some of the elements of the various embodiments may be running on a fixed or configurable CPU, GPU, Digital Signal Processor, FPGA, Neural Processing Unit, Application Specific Integrated Circuit, microcontroller, SoC, etc. The images may be temporarily or for long term stored in various memories, in the vicinity of the processor(s) or remotely accessible e.g. over the internet.

[0127] It should be understandable to the skilled person from our presentation which components may be optional improvements and can be realized in combination with other components, and how (optional) steps of methods correspond to respective means of apparatuses, and vice versa. Some combinations will be taught by splitting the general teachings to partial teachings regarding one or more of the parts. The word "apparatus" in this application is used in its broadest sense, namely a group of means allowing the realization of a particular objective, and can hence e.g. be (a small circuit part of) an IC, or a dedicated appliance (such as an appliance with a display), or part of a networked system, etc. "Arrangement" is also intended to be used in the broadest sense, so it may comprise inter alia a single apparatus, a part of an apparatus, a collection of (parts of) cooperating apparatuses, etc.

[0128] The computer program product denotation should be understood to encompass any physical realization of a collection of commands enabling a generic or special purpose processor, after a series of loading steps (which may include intermediate conversion steps, such as translation to an intermediate language, and a final processor language) to enter the commands into the processor, and to execute any of the characteristic functions of an invention. In particular, the computer program product may be realized as data on a carrier such as e.g. a disk, data present in a memory, data travelling via a network connection -wired or wireless. Apart from program code, characteristic data required for the program may also be embodied as a computer program product. Some of the technologies may be encompassed in signals, typically control signals for controlling one or more technical behaviors of e.g. a receiving apparatus, such as a television. Some circuits may be reconfigurable, and temporarily configured for particular processing by software. Some parts of the apparatuses may be specifically adapted to receive, parse and/or understand innovative signals.

[0129] Some of the steps required for the operation of the method may be already present in the functionality of the processor instead of described in the computer program product, such as data input and output steps.

[0130] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention. Where the skilled person can easily realize a mapping of the presented examples to other regions of the claims, we have for conciseness not mentioned all these options in-depth. Apart from combinations of elements of the invention as combined in the claims, other combinations of the elements are possible. Any combination of elements can in practice be realized in a single dedicated element, or split elements.

[0131] Any reference sign between parentheses in the claim is not intended for limiting the claim. The word "comprising" does not exclude the presence of elements or aspects not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements, nor the presence of other elements.

Claims

1. A control processor (510) arranged to adjust the brightness of displaying on a display (520) of at least a dark sub-set of colors in an input image (IMG_in) in response to a measurement of a level of illumination (Lev_illumdrown) of an environment in which the display resides, wherein the adjustment comprises the establishment of an initial adjustment function (F_adj), for shifting one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) of pixels of the input image by at least one correction value (Adj_brightn) to obtain corresponding output color component triplets (R'_out, G'_out, B'_out), wherein the adjustment comprises multiplying for values of the one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) below a threshold value (GAM1) the initial adjustment function (F_adj) by a correction function (F_corr) to obtain a final version of the adjustment function (F_adj) for shifting the input color component triplets (R'_PQ, G'_PQ, B'_PQ), wherein the correction function (F_corr) satisfies a first property of yielding zero output for zero input, a second property of yielding an output value equal to one for an input value equal to the threshold value (GAM1) and a third property of having a zero first derivative at an input value equal to the threshold value (GAM1).

2. The control processor (510) as claimed in claim 1, wherein the correction function is strictly monotonically increasing.

3. The control processor (510) as claimed in one of the above claims, wherein the correction function is applied on an input and output domain of psychovisually uniformized color component values, such as by pre-applying an inverse perceptual quantizer electro-optical transfer functions on linear versions of the one or more color components of input color component triplets.

4. The control processor (510) as claimed in one of the above claims, wherein the threshold value (GAM1) on the input axis is set to be equal to level of illumination (Lev_illumdrown).

5. The control processor (510) as claimed in one of the above claims, wherein the correction function has a number of higher order derivatives equal to zero at an input value equal to the threshold value (GAM1).

6. The control processor (510) as claimed in one of the above claims, wherein the correction function is equal to y= -1*power(x;4)+4*power(x;3)-6* power(x;2)+4*x, wherein x is an input value between zero and a linear representation of the threshold value (GAM1) which is normalized to a maximum value equal to 1, and y is the output value of the correction function for the input value.

7. The control processor (510) as claimed in one of the claims 1 to 5, wherein the correction function is equal to:

wherein x is an input value between zero and a linear representation of the threshold value (GAM1) which is normalized to a maximum value equal to 1, and y is the output value of the correction function for the input value.

8. A display comprising the control processor as in any of the above claims.

9. A method of adjusting the brightness of displaying on a display (520) of at least a dark sub-set of colors in an input image (IMG_in) in response to a measurement of a level of illumination (Lev_illumdrown) of an environment in which the display resides, wherein the adjustment comprises the establishment of an initial adjustment function (F_adj), for shifting one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) of pixels of the input image by at least one correction value (Adj_brightn) to obtain corresponding output color component triplets (R'_out, G'_out, B'_out), wherein the adjustment comprises multiplying for values of the one or more color components of input color component triplets (R'_PQ, G'_PQ, B'_PQ) below a threshold value (GAM1) the initial adjustment function (F_adj) by a correction function (F_corr) to obtain a final version of the adjustment function (F_adj) for shifting the input color component triplets (R'_PQ, G'_PQ, B'_PQ), wherein the correction function (F_corr) satisfies a first property of yielding zero output for zero input, a second property of yielding an output value equal to one for an input value equal to the threshold value (GAM1) and a third property of having a zero first derivative at an input value equal to the threshold value (GAM1).

10. The method of adjusting the brightness of displaying as claimed in claim 9, wherein the correction function is strictly monotonically increasing.

11. The method of adjusting the brightness of displaying as claimed in claim 9 or 10, wherein the correction function is applied on an input and output domain of psychovisually uniformized color component values, such as by pre-applying an inverse perceptual quantizer electro-optical transfer functions on linear versions of the one or more color components of input color component triplets.

12. The method of adjusting the brightness of displaying as claimed in one of the above method claims, wherein the threshold value (GAM1) on the input axis is set to be equal to level of illumination (Lev_illumdrown).

13. A computer program product comprising software code for instructing a processor to execute one of the methods as claimed in claims 9 to 12.

Drawing