Sunday, November 8, 2020

Wayland Color Management and High Dynamic Range

Wayland color management (CM) has a long history of discussions, most of which could be found here:

wayland-devel-2014-March                           Niels's CM proposal with a prototype
wayland-devel-2016-December                     Niels's enhanced CM proposal
wayland-devel-2017-December                     Ville's HDR POC
wayland-devel-2017-January                         Niels's CM protocol proposal
wayland-devel-2019-March                           Graeme's HDR TM comment
                         Shashank's HDR proposal 
                               Harish's HDR implemetation
                    Sebastian's CM proposal
Sebastian's CM MR


These are very interesting yet heated debates that deserves special attention.
The most pr
oposals here are about usage of ICC profiles for colour management.

There are input and output colour spaces. Profile defines the colour space attributes and viewing requirements. Profile connection space(PCS) is used for concatenation of more than one profiles to create a mapping from source to target colour space. The above HDR proposals are not based on ICC profiles due to lack of HDR support in ICC.

I want to quote Jan Morovic ('Color Gamut Mapping',2008, p.93) that explains why HDR images are different from traditional images and as a result, ICC spec could not applied to HDR.
"The reason for this gap is two-fold:

- HDR images are not ones that the visual system experiences using a fixed state  of adaptation (e.g. imagine looking at a lawn with a tree in its middle on a sunny day: looking at the grass in the shade will show details there and make the directly lit grass appear very bright and showing less detail...and a given scene can look different depending on where a viewer focuses in it. The reason for this is that the human visual system has a relatively limited simultaneous dynamic range of around 150:1(Vos, 1984; Seetzen et al., 2004). In other words, if the difference between two sides of the edge exceeds this luminance ratio, its perceived contrast will not continue increasing. Contrast across greater dynamic ranges of up to 10^5:1 can be seen thanks to adaptation.
 - Color appearance models are derived from data collected under viewing conditions where stimuli would only come from the much more limited dynamic range of typical imaging media; therefore, their predictions of HDR stimuli are of unknown accuracy. As a consequence, there is currently separate development of gamut mapping and HDR imaging solutions that could converge if the above differences were resolved".

Undoubtedly, it is necessary to somehow combine these technologies so that you can watch the video using desktop compositor.And such function has been around for a long time.A tone mapping is a mathematical function that transforms HDR scene luminance into the luminance range that can be shown on a display. There is a variety of tone mapping algorithms: Reinhard, Mobius, Hable, etc..

Most implementations about HDR tone mapping I have found are listed below

 Blog 64

Most implementations use shaders, some do 3D LUT creation which combines all the transforms; the question is how to connect ICC profiles with tone mapping as flexible workflow.After conducting extensive research, I decided to use LMCS - colour ICC engine library, world-acclaimed, mature, evolving since 1998. The author of LCMS Marti was very supportive and helped me to write a plugin to LCMS .Pekka and Sebastian also participated in these discussions.I implemented luminance compression pipeline, known as EETF, using ITU-R BT.2390.

LCMS has a built-in operator for sophisticated viewing conditions and many other features which can be used for HDR solution.
Here is the result of this work :MR1
In addition, there is a WIP proposal about user mode slider
Rebase color management to Weston master branch WIP color-management-rebase-master

I am proposing to take as base the spec 2390 and implement the following HDR use cases

This solution has very flexible design for HDR uses cases:
The color transform looks as following :input ICC profile + Tone mapping profile + Output profile.Profile vendors or colour scientists can override the tone mapping profile as necessary to provide superior quality which can be comparable with mastering metadata or rendering intent parameter.

The solution could be used as-is for hardware composition, but not all chips have 3DLUT and LCMS can address this by using optimization plugin as DEGAM/CTM/GAMMA pipeline. The  implementation could be part of LCMS or a separate plugin( subject to discussions with Marti) that would fit to embedded and workstation systems.

There is another consideration to have flexible flow, which is easy to override. The video industry is moving to change colour space per frame; as a result, gamut mapping operator became more and more important. It is evident that gamut differences will persist and we are looking into a robust and flexible way of color reproduction to meet the fast-moving industry. 

Thanks to Pekka who inspired me to write this article.









No comments:

Post a Comment