LMTs Part 4: How do they work and how are they made? (continued...)

lmt
look-transforms
Tags: #<Tag:0x00007f3c60339e30> #<Tag:0x00007f3c60339cf0>

(Scott Dyer) #1

In Part 3, analytic LMTs were introduced by walking through the creation of simple examples. But analytic LMTs can have any number of processing steps, and this post walks through the creation of more complex analytic LMTs.

[Transforms used in this article are again provided as CTL code at the end of the article.]

Analytic LMT – Example 3 (complex)

Example 3 is the probably the most complex example, introducing vector-based secondary color correction and sequencing several independent operations into a single LMT. Structurally, this LMT consists of some primary corrections up front followed by a combination of hue-qualified adjustments to hue angles and saturation. The look was roughly modeled after a print film emulation, resulting in something high contrast and slightly “grungy” (see Figures 16 and 17)*.

Figure 16. ACES Output Transform (left) vs. LMT-Analytic 3 + ACES Output Transform (right)

Figure 17. ACES Output Transform (left) vs. LMT-Analytic 3 + ACES Output Transform (right)

[*Please keep in mind that I am not a colorist, and my goal with this look is to demonstrate the possibilities with very complex Analytic LMTs. The result might not be beautiful to you, but it should illustrate the process!]

Design Approach

As mentioned, the sequence of operations in the LMT that produced this result is quite a bit more complex than in the previous examples (Figure 18).

Figure 18. The processing chain used for example Analytic LMT 3. The LMT begins with an overall reduction of “chroma,” followed by an ASC CDL adjustment to add some yellow dinge, then a contrast boost using a gamma adjustment in linear space, and finally a series of hue-region-specific rotations to creatively skew hues and re-add “chroma” in some hues.

The sub-transforms for applying ASC-CDL and the gamma adjust are re-used from the previous examples. Each box in Figure 18 is more fully explained in the following sections, with depiction of the before & after with processing up to that individual piece of the processing chain applied.

Introducing “Yab” and “YCH”

Many of the sub-transforms make use of a a generic lightness/opponent color space, here-in referred to as “Yab.” From Yab, a lightness/chroma/hue representation, referred to as “YCH”, can also be attained. Yab and YCH are defined solely for convenience of targeting specific lightness, hues, or chroma. They are intentionally simple coordinate spaces defined by ratios of RGB and are not intended to have any correlation to perceptual luminance, chrominance, or hue.

Yab can be attained via a simple conversion from RGB; it is essentially a space where an RGB unit cube has been rotated and stood vertically on its [0,0,0] corner such that the neutral axis runs vertically (Figure 19).

Figure 19. A visualization of the relationship between RGB (bottom left) and Yab (right). The conversion between spaces can be represented as a rotational matrix (top left).

YCH is a cylindrical representation of Yab, where C represents the distance of a color from the neutral axis and H is a hue angle in degrees (Figure 20).

Figure 20. Formulas for converting Yab to YCH (left) and a visualization C and H (right).

Reduce “chroma”

To scale chroma, RGB is converted to YCH, a simple percentage is multiplied to the C channel, and then values are converted back to RGB. Figure 21 shows the before/after of this step, with the chroma scalar value set to 0.7.

Figure 21. Before chroma reduction (left) vs after chroma reduction (right).

ASC CDL (in ACEScct)

The base math for ASC CDL is reused from Analytic 1a and 1b. A fairly minor adjustment to the blue channel slope and offset was made in order to bring a touch more yellow into the overall balance (Figure 22).

Figure 22. Before ASC CDL adjustment (left) and after ASC CDL adjustment (right).

Gamma adjust (in linear)

The base math for adjusting contrast with a gamma adjustment in linear was reused from Analytic 1c. Contrast was increased around mid-gray with a gamma value of 1.5 (Figure 23).

Figure 23. Before contrast boost (left) and after contrast boost (right).

Rotate R toward Y

To select hue regions and avoid harsh transitions, a cubic basis shaper function is used. This function tapers from full power, 1.0, at the middle and falls off to zero at a desired width to allow for a narrow to wide range of hues to be affected.

Figure 24. Visualization of the cubic basis shaper function with width set to 120 degrees. At the centered hue, fH=1. FH tapers off smoothly to 0 as the magnitude of the hue distance increases away from the centered hue value.

The hue rotation formula takes as input the center hue to be targeted, the width of the hue region to target, and an amount to rotate the hues in that region. Hues that fall at the center location get a full amount of rotation. Hues that are outside of the hue region get no amount of rotation. The amount rotation for hues off of direct center but still in the affected hue region is determined by the cubic basis shaper function.

All of the values chosen were determined through continuous tweaking. Sometimes a pixel was sampled from the ACES image and converted to YCH to determine to what hue it corresponded. A trick to quickly confirm if the correct span of hues is being targeted is to set the hue rotation amount to 180˚, which should have the effect of turning the affected pixels to a hue opposite them on a hue wheel, thus making it easier to identify the affected pixels.

To make reds slightly more orang-ish by rotating toward yellow, the following values were used:
Center hue = 0˚
Hue region width = 30˚
Rotation amount = 5˚

Figure 25. Before (left) and after hue rotation of reds toward yellow (right).

Rotate G toward Y

To make greens more yellow, the following values were used:
Center hue = 80˚
Hue region width = 60˚
Rotation amount = -15˚

Figure 26. Before (left) and after hue rotation of greens toward yellow (right).

Rotate Y toward R
To make yellows just a bit more “golden”, they were shifted toward red with the parameters below:
Center hue = 52˚
Hue region width = 50˚
Rotation amount = -14˚

Figure 27. Before (left) and after hue rotation of yellows toward red (right).

Boost “chroma” in Y

Boosting chroma in a specific hue region uses a similar tapering function as the hue rotation to again avoid harsh transitions. Values in the middle of the region get the full chroma scaling and the amount of scaling from the original value tapers off to zero at the full width of the cubic basis shaper.

To boost chroma in yellows, these parameters achieved the desired effect:
Center hue = 45˚
Hue region width = 40˚
Chroma scaling = 1.4

Figure 28. Before (left) and after chroma boost of yellow hue region (right).

Rotate C toward B

Center hue = 190˚
Hue region width = 40˚
Rotation amount = 30˚

Figure 29. Before (left) and after hue rotation of cyan toward blue (right).

Boost “chroma” in B

Center hue = 45˚
Hue region width = 40˚
Chroma scaling = 1.4

Figure 30. Before (left) and after chroma boost in blue hue region (right).

And finally, a comparison of the overall before and after:

Figure 31. ACES Output Transform (left) vs. LMT-Analytic 3 + ACES Output Transform (right).

And applied to a few other images:

Figure 32. ACES Output Transform (left) vs. LMT-Analytic 3 + ACES Output Transform (right).

Figure 33. ACES Output Transform (left) vs. LMT-Analytic 3 + ACES Output Transform (right).

More time could be spent matching this LMT more closely to a given PFE, but for general illustrative purposes, this is sufficient.

More complexity could also be added via other qualifying functions. For example, one could create a function to desaturate just shadows (i.e. values with a Y below a certain value). Or a function could further qualify hue regions with both Y and C tapers to only affect bright and saturated, yellows, for example. The possibilities are limited only by the complexity of the adjustment algorithms one can design.

Analytic LMT - Example 4 (complex)

While not as complex as the long chain of secondary corrections used in Example 3, this example recreates the functionality of a node graph with parallel color correction nodes that are mixed together using a given blend mode.

The look approximates that of a “bleach bypass” or “skip bleach”. A preview of the LMT’s effect can be seen in Figures 34 and 35.

Figure 34. ACES Output Transform (left) vs. LMT-Analytic 4 + ACES Output Transform (right)

Figure 35. ACES Output Transform (left) vs. LMT-Analytic 4 + ACES Output Transform (right)

Design Approach

The structural model for this LMT was taken from a simple but effective Youtube tutorial wherein creating an easy “bleach bypass” or “skip bleach” using blended node corrections is described.

The processing chain is illustrated in Figure 36. Two copies are made from the input ACES2065-1 data. The first copy is desaturated slightly and increased in exposure. The second copy is fully desaturated and boosted in contrast. The two modified copies are then mixed together (in ACEScct) using the “overlay” blending mode.

Figure 36. The processing chain used for example Analytic LMT 4.

Node A
Saturation (in linear)
Saturation is slightly reduced using a 3x3 saturation matrix with the saturation factor set to 0.9.

Exposure (in linear)
Adjusting exposure in linear is achieved by multiplying by a scale factor. In this example, a good scale factor was determined to be 2.0, which represents a doubling of exposure.

Figure 37. Original image (left) and result of processing through Node A (right).

Node B
Saturation (in linear)
Saturation is slightly reduced using a 3x3 saturation matrix with the saturation factor set to 0.0.

Contrast (in linear)
The math for adjusting contrast with a gamma adjustment in linear was reused from Analytic 1c. Contrast was increased around mid-gray with a gamma value of 1.2.

Figure 38. Original image (left) and result of processing through Node B (right).

Blend
The resulting images from Node A and Node B are compositing using the “overlay” blend mode. In the CTL example, this operation is performed in ACEScct, so there is a conversion from ACES2065-1 to ACEScct prior to the overlay blend and a conversion back to ACES2065-1 from ACEScct after the overlay blend.

Figure 39. ACES Output Transform (left) vs. LMT-Analytic 4 + ACES Output Transform (right)

And applied to another image:

Figure 40. ACES Output Transform (left) vs. LMT-Analytic 4 + ACES Output Transform (right)

Supporting CTL transforms

Below are download links for the CTL transforms used in the examples in this post.

  • Analytic LMT 3 : rough “PFE-like” look (high contrast / grungy) using sequenced hue-qualified adjustments
  • Analytic LMT 4 : “bleach bypass” look using overlay blend mode

Dependencies:

Notes on LMT implementation

The transforms used in all examples are provided as CTL code. LMTs do not need to be implemented in CTL. CTL was the easiest way to unambiguously express the operations used. If you are a “tinkerer” feel free to take a closer look at the formulas inside the CTL for the order of operations and other details on how these LMTs work.

The Common LUT Format (CLF), a core component of ACES 1.x, is designated as the official carrier for certain kinds of LMTs. CLF’s can encapsulate any number or ordering of process nodes that include 1D LUTs, 3D LUTS, matrices, and ASC CDL. However, CLF does not support math formulas, so CTL used for even simple shaper functions would need to be sampled to LUTs for implementation in CLF. This is potentially limiting, but extending CLF, or adding support for algorithmic description of LMTs, is under consideration for upcoming ACES enhancements and extensions.

Please note that CLF is a required item for LMT-supporting production and postproduction tools that wish to achieve the ACES Logo. We encourage all end-users to work with ACES Product Partners to provide you with this powerful capability.

Regarding LUTs, keep in mind that the quality of 3D-LUTs requires knowledge of input and output ranges as well as grid point spacing. ACES values are radiometrically linear and have a very wide floating point range. Spacing nodes directly in linear space does not always provide a good sampling of the useable range. CLF provides for forward and inverse ‘shaper LUT’ operations that (when wrapped around an appropriately constructed 3D-LUT) can effectively produce well-spaced LUTs with minimal interpolation errors. Implementers should review the ‘1D LUT’ section of the Common LUT Format specification [S-2014-006] for more information.

Where to go from here

With the “nitty-gritty” of what goes into and how to build LMTs described in Parts 2-4, one can see that both empirical and analytic LMTs are not always terribly difficult to make. Once constructed LMTs can be a valuable tool for those using ACES for their color pipelines.

LMTs are basically “preset” looks, and one may find it desirable to build up a library of pre-built filters that can be used to quickly decide a look that fits the project.

To hone in on the types of operations you might need to achieve a look, toying around with a color corrector’s operators can be useful. Of course each color corrector likely implements different math for its secondary operators and hue-specific adjustments, etc. but it can be helpful for look designers to at least determine what regions need to be targeted and in what direction they need to be adjusted.

Experiment on your own and try using LMTs to implement your own looks. Your ACESCentral friends would love to see what you come up with. Feel free to extend the provided functions or write additional ones. In this spirit, a few more creative looks will be posted in the coming weeks.

Questions and comments below!


(Zach Lewis) #2

Scott (and reviewers) – this series has really been extraordinary and enlightening. You have a gift for communicating this kind of stuff. Just sayin’.

Are these going to end up as part of an update to whichever technical bulletin (9?) that deals with LMTs? Or maybe as an update to the LMT CTL Readmes? This stuff is really too good and universally useful to not be part of the official documentation in some capacity…

This isn’t explicitly related to Part 4, but maybe falls under “Where to go from here”…

So – obviously, one of the best things about LMTs is that they’re inherently RRT-referred – which is to say, there’s a singular (or at least versioned) reference point that eliminates (much) of the ambiguity associated with, for instance, LUT exchange. That is, assuming everyone plays by the rules, and is, in fact, using the ACES-RRT.

But… despite the capabilities and abilities of LMTs under RRTs, it is a fact that many mostly-ACES shows are using an alternate Rendering Transform. Which means all “LMTs” are inherently alternate-Rendering-Transform-referred (and potentially less – or differently – capable, depending on the color volume treated by the rendering transform).

Similarly, imagine a world where there’s an official ACES parametric RRT – I imagine that parametric data would have to travel both forwards (for Target Conversion Transforms, as you’d have Parametric OCES’. POCES?) and backwards, to any LMT before it (PACES?).

In both cases… what’s the best way to communicate this information? ACESClip? In CLF, via the OutputDescriptor (iirc) tag? Both?

(I’m sorry if this flows outside the scope of this post; happy to repost elsewhere)


(Scott Dyer) #3

Thanks, Zach.

As you mention, a lot of information about LMTs already existed in in the LMT README that is part of the aces-dev repository and as part of TB-2014-010. I started with those documents as a foundation and wanted to “demystify” them a bit by showing specific examples of increasing complexity. I am glad you have found it useful and I hope others will as well. I imagine that for now it will exist here where it can collect commentary and feedback. Then, at a future point in time, we can determine how best to repurpose it for longer-term documentation.

Great questions. I’m not sure I have immediate answers, but these are concerns that should definitely be listed for consideration (perhaps a re-post of your last few paragraphs in “Feedback and Requests” category?) I’m interested to hear others thoughts on this as well.

One note:
Conceptually, I believe analytic LMTs should hold a “look” rather well across “parametric” output ranges (Not to get to into it, but I personally don’t foresee a fully “tunable” RRT. I mean “parametric” in the sense that the system tonescale adapts automatically to the output dynamic range of the viewing device. Thus, the adaptable tonescale is standardized and the rendering would thus be the same for everyone for a particular output dynamic range). Empirical LMTs will still be limited to the range/gamut of the output device from which the look was derived.
This is solely my hypothesis, however, and will need to be tested as part of any future ACES development. Now that output devices of many varying dynamic ranges are more readily available than when ACES 1.0 was developed, we should be able to more readily test these theories.


(Luis Barrancos) #4

Thank you for the informative post. I subscribe to Zach Lewis’s view, it would be great to have this series of posts on LMTs in some form as part of the official documentation.
Regarding the node B transformation near figure 38, don’t you mean full desaturation with saturation factor set to 0.0? Presently it’s the same as the node A.


(Scott Dyer) #5

Yes, indeed. Good catch!


(Nick Shaw) #6

This is a very important point. Since in CLF shaper functions currently need to be implemented as 1D LUTs, and operations like hue modifiers as 3D LUTs, that does not fit the goal of making analytic LMTs which are not limited to a particular range.

In reality, the range coded by ACEScct is probably adequate in most situations, so using that as a shaper might well not cause problems. I am certainly in favour of extending CLF to include mathematical formulae in the long term, but as a start it would be extremely useful to have some preset 1D transforms, such as linear_to_ACEScct and its inverse. I would vote for the inclusion of some other common camera log transfer functions as well, such as LogC, S-Log(1, 2 & 3) and REDlogFilm/Cineon, and ideally the inverse RRT and some inverse ODTs too. This would enable camera LUTs to be encapsulated in CLF LMTs more simply.


(Zach Lewis) #7

Thanks guys, lots of food for thought.


(Greg Cotten) #8

Scott and Nick - I do take some issues with splintering CLF into a catch-all color transformation descriptor - I think CTL does that pretty well already! I’m a big proponent of a LUT format being able to explicitly describe its operation without inference. Math equations are fine, but probably not named transforms. Does that make sense or am I crazy?


(Qusai Nofal) #9

Hi Scott,

Thank you for this insightful series. I’m using DaVinci Resolve 12.5 in iMac os Sierra. I downloaded the LMT files and added them inside the LUT folder, but for some reason they don’t appear inside DaVinci, what i’m doing wrong?

Thanks,


(Scott Dyer) #10

The files provided are CTL, not DCTL. CTL is the transform language used for the reference implementations of ACES transforms. DCTL is DaVinci CTL, which despite sounding similar in name, is a completely different language that Resolve uses to specify transforms. So that’s why they’re not showing up in DaVinci despite being placed into the LUT folder - they are not expressed in a file that Resolve will recognize.


(Qusai Nofal) #11

Thanks for the reply, in this case what’s the best solution to use these LMT files while working in DaVinci?


(Scott Dyer) #13

@Paul_Dore wow, this is a great effort! Converting these to DCTL is something I had intentions to do at some point but my focus has been elsewhere and so it’s great to see you’ve done all the heavy lifting. Thank you so much for sharing your work!


(Scott Dyer) #14

My answer to that would simply be “whatever options seem useful”.

I never intended for these example looks to be “final”. While I thought what I provided were reasonable examples, I never expected them to be used “as is”. I could have spent much more time constructing and tuning the parameters. In fact, I kept falling into that trap and had to force myself to focus more on communicating the concepts which was really the meat of this article series.

I wanted to illustrate the power of LMTs and show that an LMT can basically be anything that modifies the default look - whether it be a simple contrast adjustment, complex stack of secondary adjustments, or even a pre-set node overlay operation.

Nevertheless, if they’re useful to somebody, than all the better! :grinning:


(Scott Dyer) #16

Haven’t had a chance to test them yet but I will compare some values through CTL and through the Resolve processing and see how close they get.

My DCTL game is weak (from lack of use) and I have never made an OFX plugin so this could be a great learning opportunity for me as well. Maybe together we can make a good example LMT implementation via plug-in. Glad to have a partner in crime that can ask to help me through the tricky parts.


(Scott Dyer) #19

Ok, so total newb question here, but how do I “install” this to be able to try it out? From support forums it looks like I’m supposed to just put it in /Library/OFX/Plugins but:

(1) I don’t have that directory yet (I can just create it but that seems wrong, which leads me to…)
(2) Is there an OFX package or something that I need to install separately first?

Thanks.


(Scott Dyer) #21

Ok, thanks for confirming - it must be something else then. I can see all the ResolveFX plugins, but cannot see any user installed ones.

I checked everything as you said and that’s what I had done. I put the OFX/Plugins folder in the root Library (not user Library), with the LMT.ofx.bundle inside that.
I was not aware about the hardware configuration but have switched that to CUDA and restarted.

I’ll keep looking for help on the Blackmagic forum, even though you are a key participant in most of the conversations I’ve found that seem related to this…


(Paul Dore) #22

What are your specs with regards to operating system, graphics card, and version of Resolve?


(Scott Dyer) #23

I just don’t see it showing up anywhere where OpenFX is listed. It should just show up in the OpenFX library, right?

Have tried on both:
Macbook Pro (mid-2015), MacOS 10.13.3
AMD Radeon R9 M370X
Resolve Studio 14.3.0.014
and
Mac Pro (Late 2013), MacOS 10.13.3
AMD FirePro D700 6GB
Resolve Studio 14.3.0.014


(Dominik Belančić) #27

Firstly, thanks for doing the conversion to DCTL!

But, I don’t know if it’s supposed to be like that, but when I use the LMT Academy Analytic 3 the image becomes way too green and the only way to compensate is using an offset control going towards magenta.
It’s not a big deal, but I’m just wondering if I’m doing something wrong or does it really green up the image?


(Dominik Belančić) #29

Was using the one from the GitHub DCTLs, but I exported a DCTL from the OFX (totally forgot you could export) and now it’s all good