CLF Spec Review Period

[Thomas_Mansencal]

Thanks Scott and glad that the feedback was helpful!

[zachlewis]

VVV Thanks Doug! Please disregard what I said earlier – user error led me to draw incorrect conclusions. Nothing to see here. Keep on keeping on.

**edited to remove incorrect / misleading info.

[doug_walker]

@zachlewis, removing the LinearOffset does not have a meaningful effect on what curves may be represented. This is simply an internal variable in the equations that should not be exposed to the end user.

All the ARRI LogC curves up to about an ISO of 1400 may be represented using the proposed interface. Above about 1400 they use a Hermite spline to roll off the highlights and those curves would still need to be baked into LUTs. Exposing LinearOffset would not have any effect on representing ARRI curves. The only thing it would allow you to do is make discontinuous curves where the pieces do not meet at a common point. We don’t want to enable people to make broken curves.

Doug

[sdyer (Scott Dyer)]

@Thomas_Mansencal @doug_walker @zachlewis @Graeme_Nattress I believe I have addressed most of your points. Thanks again for your thorough reviews. I have attached a new copy but I’m not sure how helpful that is since it doesn’t redline the changes. There may be a few lingering items that I plan to catch as I read through it again (trying to address all your points simultaneously was very tedious). This is a very sizeable response and I’m positive that I will miss some things - but I want to get keep the conversation going…

Access live doc: https://www.overleaf.com/read/jdwpzdrdrnwv
Current (as-is) draft: CLF_3_0-2019-12-10.pdf (632.7 KB)

Some top issues that remain:

1. Matrix dimensioning
   A few back-and-forths on this above. @Graeme_Nattress, @Thomas_Mansencal and I believe at least one other has expressed confusion over why the matrix dimensions have a 3 at the end. I can see the argument on both sides but the more I look at it, the more I don’t really see it as terribly inconsistent. I could go either way on this but I don’t see it as a deal-breaker if left as-is.

2. Bit-depth
   @nick and @doug_walker have both voiced concerns that these sections are still not clear enough. There were many places in the old spec where things were casually mentioned (and usually not that clear). I think this version is better for trying to say it all in one place but in the editing process I may have lost the key points that others think are most important. (Inline notes to @doug_walker below)

3. Hard coded ranges
   There seems to be some debate over whether certain parameters should have reasonable limits imposed on them. Specifically the LogParams, ExponentParams, and ASC_CDL parameters. In some cases, limits are set so as to prevent errors (divide-by-zero errors, log of a negative number, etc.)
   Should values be left unrestricted whenever possible? Or is declaring practical range limits preferred?

4. File extension
   The AMF spec sets .amf as the file extension for those files. We do not specify a file extension for CLF files anywhere in the specification. Do we want to specify one? If so, is it .clf? Or should we leave the extension as .xml?

5. Log/Antilog
   @thomas has requested a generic Log/Antilog style that uses the basic LogParams structure. You can already use the LogToLin/LinToLog styles with all LogParams left at their nominal values (0 for offsets, 1 for slopes) and effectively get basic Log/Antilog behavior, so if I am understanding his request correctly, then this style would simply an alias to the LogToLin/LinToLog styles. We provide a base 10 and base 2 basic logarithm/antilogarithm.
   @thomas is this what you meant? Thoughts from others?

6. ASC_CDL versioning
   Per @Thomas_Mansencal, how do we maintain compatibility with a particular version of CDL? Should there not be a “version” attribute or element to protect us in case a new version of CDL is ever released?

7. Casing consistency
   The sub elements in the Range node are title cased, even though they are not anywhere else.

8. Examples
   If you’ve got practical examples using CLF, please send them my way and I’ll add them as supplements to the document. @doug_walker proposed an ACES-to-ACEScct conversion and I’m sure theres a few other simple yet practical examples we could use to augment the document.

I’m sure there are more that I forgot to summarize, but this message needs to get posted, so I will follow up if I realize there are items I missed…

(Some) major changes made in this version:

• IndexMap removed entirely from spec

• Created ACESInfo block with ACEStransformID and ACESusername as child elements (per @zachlewis) (@doug_walker, can you reflect this change on the diagram?)

• Restored CalibrationInfo block with its numerous, very specific child elements. I figured that since we’re going to allow ACES metadata, then there’s no harm in other optional metadata fields, as long as they’re contained in the Info element. I don’t think anyone will ever use these Calibration Info elements but they were already in v2, so it’s not like we’re adding anything…we’re just not taking it away either.

• Exponent

  • Renamed “style” params such that Fwd/Rev was always at the end of the name. (Per suggestions from @Zach & @Thomas_Mansencal)
  • Re-added the “monCurveMirror” functions (per @doug_walker) . @nick had been the only one to reply to my query about them, so I removed them since no one objected. @doug_walker has now asked for them to be included, so I re-added them.
  • Re-cased “moncurve” as “monCurve” and “Passthru” as “PassThru” (casing consistency, per @thomas)
  • Set separate valid ranges for “exponent” depending on “style” (per @doug_walker). I wonder if my revised note about exponent=1 is sufficient protection (similar issue to allowing 0 for offset). Can I just set range to (min,max] to imply exclusive on bottom end and inclusive on top end? I do a similar thing for “Power” in ASC_CDL node… (related to decision on #2 above)

• Implementation Notes

  • Removed sections formerly numbered 5.4, 5.5, 5.6 - Indexing Calculations, Floating-point Output of the ProcessList, and Half-float 1DLUTs (per @doug_walker - I agreed they weren’t adding anything)
  • Added to state that up to 3 LogParams elements can be used (in the case of defining separate operations for the three channels)

@doug_walker

4.4.7

• I’m thinking we should add the mirror option for moncurve too.
• The exponent attribute is always required.
• The valid exponent range you stated is only for the basic styles, the valid moncurve range is [1,10] (allowing it to go below 1 breaks the formulas).
• The Description element of the first example doesn’t match the params.
• Please remove the statement about “similar but different to 61966-2” (They are the same to within rounding error. People will think you are referring to the gamma 2.2 debacle, which is not what we want.).
• If you want to include values for all three common curves, the constants are – sRGB: (2.4, 0.055), L*: (3, 0.16), Rec.709: (1/0.45, 0.099).

• I re-added the “mirror” for moncurve style functions. @nick was the only one to have commented in favor or against their removal, so I removed them. New <Exponent> Process Node - #12 by sdyer
• Changed exponent attribute to required
• I have set separate valid ranges for “exponent” depending on the “style”. I added a note to warn about exponent=1 and wonder if this is sufficient protection against potential divide-by-zero issues (a similar issue to allowing 0 for offset). Could I just set the range to (min,max] to imply exclusive on bottom end and inclusive on top end? I already do a similar thing for “Power” in ASC_CDL node to avoid allowing that value to equal 0…
• Added more examples and renamed sRGB (that was changed in response to @nick who had pointing out that the example does not exactly match the spec - but it is what was intended)
  On the new examples, did I get the EOTF and OETF directions correct?

@doug_walker

4.4.8

• The CDL spec only places non-negativity bounds. Do we really want to place upper bounds?

Yes, technically you’re right, but I picked what I thought would still be substantially large end points. If it’s not a problem to specify it as “infinity” then I will, but I also didn’t want people trying to use Inf or -Inf values from 16-bit half.
The CDL spec says (for slope) “in practice, systems should probably limit at a substantially lower value”, and (for offset) “offset, can in theory range from -∞ to +∞ although the range -1.0 to 1.0 will fit most traditional use”.
I thought (based on values used for Exponent and such) that [0, 100] for Slope, [-100, 100] for Offset, and (0, 10] for Power was more than sufficient. Should I instead express these limits as in the CDL spec at their theoretical limits? (Related to #2 above)

@doug_walker

5.1.2 and 5.1.3

• During one of the calls I thought we had agreed to the bit-depth description I posted on May 1. The spec now says something quite different. The description as it stands now is not workable for implementors. Would you like me to propose new wording, or do you want to have a call to discuss, what is your preference on how to rework this?

There were a LOT of different snippets about this, although I did originally start from copy/pasting your post into the document. I guess when trying to remove redundancies and conflicting statements, I perhaps lost the original meaning of your words.
Yes, It would be of great help if you could suggest new wording that adequately addresses your main concerns regarding bit-depth handling. In the meantime, I will also attempt a revision.

@doug_walker

5.4, 5.5, and 5.6

• I find these confusing and am not clear what value they add above the actual interpolation directions in the appendix and other material already provided elsewhere. I would vote to remove them.

I am completely ok with removing these. I was trying not to cut too much out of the original spec. There were a lot of “floating statements” like these just thrown into the implementation notes. If you don’t think they add anything - then I’m all for cutting them. As of now, they’ve been removed.

@doug_walker

6

• We need better examples. If you want to include a complex example, you may want to consider an ACES2065-1 to ACEScct transform. Let me know if you’d like me to provide this or others.

Agree. I will welcome any examples - and some would be helpful as added to the spec. However, I also foresee many more helpful examples being generated to accompany the spec as a part of the implementation group. These are easy to drop in. If you have an ACES to ACEScct example ready to go, I’ll take it. Otherwise I’ll add one myself after the rest of the changes are decided.

@Thomas_Mansencal

It also goes beyond casing, for example, basicFwdMirror should really be basicMirrorFwd , see 32.1 .

You and others brought this up and I have fixed it. This is a new node so we can call it whatever we want. It does make more sense to have the direction consistently at the end of the style.

@Thomas_Mansencal

3 Specification (Page 7):

1. the sequence of transformations is described as a node-graph

I’ve modified this.

@Thomas_Mansencal

4.3 ProcessNode (Page 13):

1. ““32f”: 32-bit floating point (single precision)”
   No support for 64f double-precision? This contrasts with 5.3 recommending that “all calculations be done in at least 32-bit floating point.”

64f has never been supported as input/output bit-depths. I’m not sure why (considering we seem to support everything else). Anybody?

@Thomas_Mansencal

4.4 Substitutes for ProcessNode (Page 13):

1. Substitutes for ProcessNode : The first consumers for the specification will be software engineers, I would favour using terms they are used to and make sense in software development, e.g. Subclasses , Inheritors , Derived Nodes , etc… especially when it is stated that the attributes and elements are inherited .

I usually think in terms of classes, subclasses, inheritance, etc. but I thought that XML was the reason we used the terminology “substitution groups”. This wording is from the previous draft and I thought was the preferred terminology when speaking in the context of XML. I could be wrong. Though I’m not opposed to changing the terminology, I don’t think the meaning is currently lost by not using different words. But I’m open to suggestion if you think something could be said more clearly.

[doug_walker]

Hi Scott,

Thanks for carefully reviewing all the feedback, this draft is looking much better.

Here is my feedback on your points 1-8:

1. Matrix: Please leave it as is. It would cause huge interop problems with v2 files to make that change.
2. Bit-depth: I will propose some wording in a separate post.
3. Hard-coded ranges: The only thing I would change from your latest draft is to make ASC CDL ranges consistent with the actual ASC CDL spec. I would not want to deviate from that unless there is an extremely compelling reason. You also don’t need to specify all of these in the form [min,max], you could simply provide only the lower bound if that’s all we want to restrict (rather than using Inf for max).
4. File extension: Yes, excellent point! I agree is should be .clf.
5. Log base: As you say, the spec already allows an arbitrary base and I feel the Log2 and Log10 are a quick and easy short-hand for 99% of the pure-log use-cases. But I’m not opposed to adding it if Thomas really wants it.
6. ASC versioning: I had made a similar point to Thomas back during the v2 discussions and had proposed a style of “v1.2_Fwd” so that the style had the version. However, the group wanted to just call it “Fwd”. I don’t think any action is needed now but if a new version is introduced then a new style name will need to be added.
7. Casing: I don’t feel the need for it, but am not opposed if others want to change it.
8. Examples: Sure thing, I will provide some. But in the Exponent examples, swap the EOTF/EOTF usage you have currently (i.e., moncurveFwd is an EOTF).

Regarding the ACES elements, I think I interpreted what Zach was asking for differently than you. I thought he was asking that they be moved from the ProcessList level down into the Info block whereas you seem to think he was asking to create an ACESInfo sub-element under Info for them. I don’t see the purpose for the ACESInfo element but am not opposed to doing that if that’s what you and Zach prefer. Please confirm you want to keep it this way and I’ll update the diagram. However, I do think some more attention should be paid to them now given how important they will become to the success of AMF. Ideally, there would be an example with an AMF and a CLF linked in that way.

Regarding adding ‘64f’, this is not necessary. The key point is that the bit-depths only define the scaling of parameters in certain ProcessNodes, they do not affect the processing precision used by applications. In fact, ‘16f’ and ‘32f’ are already redundant since they imply the same scaling. We do not need to add ‘64f’ to allow applications to process at double precision. These are unrelated concepts and that is what the rewording of the bit-depth discussion should make clear.

Again, thanks for all the thorough work on this Scott.

I may start a separate post for for the bit-depth wording since this one is getting lengthy!

Doug

[Thomas_Mansencal]

1. Matrix dimensioning

@sdyer

I could go either way on this but I don’t see it as a deal-breaker if left as-is.

Your inner Matlab-fu knows it is not True This is my number one issue with the specification so far. As you know, I have sometimes (often?) a strong opinion on things and this is one of those cases

@doug_walker

• Matrix: Please leave it as is. It would cause huge interop problems with v2 files to make that change.

@doug_walker: Something quite bothers me in that statement and it is not the first time during the various conversations I took part in with respect to CLF 3 (or others with you, e.g. when we discussed about implementing the current ODTs with the new SSTS). I have felt a lot of resistance to change stuff from your end and I’m curious as to why. We are talking about CLF 3 here, not CLF 2.1 or are we? It is a major version of the specification thus backwards-incompatible changes should be perfectly valid and it is a unique opportunity, the only one, in fact if rigorous about development, to implement them. I don’t see where the interop problems would occur, do you have practical examples? Having the version of the specification in the file is to enable such changes and so that parsers can cope with them. My feeling, and correct me if I’m misinterpreting anything, is that you have been opposed to basically any change that breaks compatibility. To me, it defeats the entire purpose of a major version and ultimately, it prevents innovation. I surely would not expect Maya 7 to open a Maya 2019 file at all or at least without having a ton of stuff breaking.

3. Hard coded ranges

@sdyer

In some cases, limits are set so as to prevent errors (divide-by-zero errors, log of a negative number, etc.)

Which makes sense, but then why that particular number over another, and, why having them inconsistently specified between the various Process Nodes? Such limits should be delayed as much as possible in the software stack. Generally, I impose them at the UI level and even there, it is a soft limit. I would not mind having recommended ranges indicated in the specification though, but I don’t think it is the right place to have such normative hard constraints.

4. File extension

.clf sounds quite appropriate!

5. Log/Antilog

It is was more a nice to have than a strong request.

@sdyer

so if I am understanding his request correctly, then this style would simply an alias to the LogToLin/LinToLog styles.

Unless I missed something, the base default value is 10 though? What I’m asking for is the natural logarithm ln here.

@doug_walker

they do not affect the processing precision used by applications.

@doug_walker

These are unrelated concepts and that is what the rewording of the bit-depth discussion should make clear.

The explanation makes sense and adopted resolution sounds great, seems super important to make a good delineation here!

Cheers,

Thomas

PS: I haven’t re-read the updated specification yet, will be weekend duties

[nick (Nick Shaw)]

@Thomas_Mansencal

Unless I missed something, the base default value is 10 though? What I’m asking for is the natural logarithm ln here.

I empathise with the desire for completeness, but given that logarithms to different bases are convertible with a simple constant scale factor, there is an argument that the inclusion of base 2 and base 10 has a certain amount of redundancy for the sake of convenience. The linToLog and logToLin operators already include a base parameter, so a natural logarithm could be achieved by specifying e as the base (albeit to a fixed decimal precision) or by scaling logSideSlope.

Can you elaborate, @Thomas_Mansencal, on the reason for precise built-in ln() and exp() operators? As I have noted before, I don’t consider absolute precision to be a motivating factor, as LUTs are by their nature frequently imprecise.

[Graeme_Nattress]

Agreed with Thomas that the matrix dimensioning is my major concern too. It’s not about consistency but that the 3 a the end is redundant and doesn’t tell us anything useful. The definition merely needs to dimension the array, not combine dimensioning and “use” together. By the very dimension of the matrix it tells us it needs a vector3 input. Having the 3 at the end makes it look like the LUT definition of 3x 1D LUTs whereby each element of the matrix should be an RGB triple not a single number so yes, it’s inconsistent, but that’s not my primary reason for concern.

[doug_walker]

@Thomas_Mansencal, the priorities that inform my comments on any proposed changes are first, “how much value does it bring to end-users?”, and second, “how much complexity does it add to the implementations?”.

Looking at the proposal to change the way a Matrix is declared, if we start with the complexity issue, it is certainly making the code to support v3 and prior versions more complicated. Perhaps more significantly, it also adds new test cases that need to be added to the plug-fest/QA process and unit tests.

So making this change is definitely not ‘free’, but if there was a numerical issue that needed to be fixed or we wanted to add new capabilities, then it would very likely be worth doing.

And that bring us to my other consideration which is about how much value it brings to end-users. Hopefully we can agree that it actually brings no value to end-users of CLF.

You have argued it brings value to authors of CLF by making the syntax more like Matlab, etc. That is an aesthetic judgement and you may very well be correct, in any case I respect your opinion. On the other hand, it was not something I recall being mentioned during the v1 or v2 spec drafting meetings, so historically it has definitely not been a major concern.

Furthermore, CLFs will generally be authored via software rather than by hand, so the number of people actually encountering this syntax is very small. So to me, the cost/benefit trade-off is clearly not worth it.

Fundamentally however, I think changing the syntax from one version of the spec to another for reasons that are (in my opinion) hard to justify sets a bad precedent and could discourage vendors from wanting to implement these specs.

Doug

[Thomas_Mansencal]

Thanks @doug_walker, I understand better where you are coming from and a lot of your points make sense. Please be reassured that I also respect your opinion. That being said I don’t agree with the cost/benefit comparison because following this logic you end up with something that is really inconsistent, and that will be full of warts in the coming years. I could for example go on for hours about Maya MEL API on this particular topic.

You say that CLFs will generally be generated by software rather by hand but I cannot really say. I’m still generating LUTs by hands quite often or part of them and as I’m both author and end-user, I would tend to think that I’m not the only one.

To the how much complexity it adds to the implementation question, the answer is contextual. If you have implemented support for CLF 1 & 2, sure it will be a bit more work. If you have not implemented support for any of those, well you are lucky because you will have less branching to do, no more special case about the Matrix or testing.

How much software has support for CLF currently and how much will support it? I’m ready to bet that there will be more software supporting CLF 3 in the future than the number of software currently supporting 1 & 2. If the spec is riddled with issues and inconsistencies like that, it might as well not entice people to implement it. I’m looking at the future here.

Cheers,

Thomas

[zachlewis]

Scott, you are some kind of wonderful, I’ll have you know.

3. Hard Coded ranges

• I don’t feel too strongly about whether or not constraints should be imposed in the spec; if not, I do think it’s worth drawing attention to potential problematic parameters, and perhaps providing ‘suggested’ or ‘appropriate’ range values in the literature – especially for attributes whose default values would elicit computation errors for certain styles.
• Thanks for updating the language and rules pertaining to the quantity and quality of LogParam / ExponentParam elements permitted per node! That makes implementation much simpler. Very well articulated, as well. Nice work.

4. File extension
   I’m down with CLF yeah you know me.

5. Log/Antilog
   Interesting, I totally misread / misinterpreted what Thomas was suggesting.

• As far as convenient defaults go, I definitely wouldn’t mind “e” as the default base value; appropriate precision could be determined by the implementation. Manually specifying 2s and 10s is comparatively painless.
• So… just to put this out there… how would you guys feel about getting rid of the ‘style’ attribute entirely? Hear me out:
  • Most of the information conveyed by ‘style’ is implicit in the presence or absence of provided parameters.
    • a lone base attribute set to 2 or 10 implies log2/antiLog2 or log10/antiLog10, respectively (i.e., a linToLog / logToLin operation with default slope and offset values)
    • If a linSideBreak attribute is present, the node represents a piecewise camera style operation; if not… it’s one of the “affine” styles (linToLog or logToLin)
    • It seems to me that the only information uniquely conveyed by the style attribute is directionality
      • Which means style could be reduced to a choice of Log or antiLog… (this is what I thought Thomas was suggesting)
      • …or, preferably, replaced entirely with a “antiLog” boolean (antiLog = True means one of the “log-to-lin” algorithms).
  • On the other hand, removing styles means losing the convenience afforded by the log2/log10 styles – i.e., not having to provide LogParams. Personally speaking, as far as human-friendliness goes, I think the benefits of having a consistent place to look for parameter values outweighs the costs of having to explicitly provide LogParams in all cases. If you’re hand-writing CLF, you’ve already resigned yourself to a certain degree of inconvenience. Just my opinion. I won’t be heartbroken when you guys decide to veto my suggestion

6. ASC_CDL versioning
   I’m still wrapping my head around XML, so forgive me if I’m missing something, but… wouldn’t it make sense to reference and inherit (extend?) types from the existing ASC CDL schema definition itself? If the version1.2 (or 1.1) CDL XSD is explicitly referenced as part of the CLF XSD, wouldn’t that provide de facto constraints on CDL version compatibility? I’m sure this has been discussed elsewhere; feel free to ignore if this isn’t a productive comment.

7. Casing consistency
   I’d prefer casing consistency wherever possible.

That’s all the time I have for tonight… more tomorrow (or Friday).

Thanks Scott… thanks everyone else…

[sdyer (Scott Dyer)]

Re: Matrix dimensions

@doug_walker

first, “how much value does it bring to end-users?”, and second, “how much complexity does it add to the implementations?”

I didn’t realize this Array/Matrix issue would be the biggest sticking point. I sense both sides are pretty firmly rooted in their opinions

Personally I’ve never been that “bothered” by this, since the spec states what it means and what to expect. However, I’ve also not tried to implement CLF so never realized that it didn’t “make sense”. My comments here aren’t necessarily my opinion on what should happen to the spec, I’m just “thinking out loud” to explore this more.

So…playing as an advocate for @Thomas_Mansencal for a second…
For implementations… I’m no expert about how parsing (and testing) for XML usually works in implementations, so, I’m curious - do implementations really look at the values in the dim string to learn that it’s a Matrix? How is that 3rd value currently used? And why would it’s removal be so difficult for implementors to protect against?
It seems to me (if I were coding a CLF parser myself) that a robust implementation would examine the value assigned to a dim attribute string contained in an Array contained in a Matrix element to “validate” it. Currently, I would probably:

1. Is the Array in a Matrix element? If so, then my expectations are that it will have three values separated by white space. So…
2. Parse the string assigned to the dim attribute to see if there are 3 (and only 3) values separated by white space.
3. Check that those values are convertible to integers, and convert them.
4. Is the third value equal to 3?
5. Does the first value X second value equal the number of values that are separated by white space and contained within the Array element in the Matrix element?

Did I miss anything? If we agree these are reasonable steps to validate the node, then my question is, how much more complicated does it get if we remove step #4? If an Array is in a Matrix ProcessNode, we just do #5 (look at the first two digits). Any extra digits can be ignored. Or, if we want backward compatibility, we could add some complex if/then statements to also check the ‘compCLFversion’, etc. but even then, would simply ignoring the third dim value make a difference?

Then of course in the spec… we’d have to state that previous versions had a third value as part of the dim, but this styling is deprecated and implementations need only check the first two values in a dim string.

But then advocating for @doug_walker:
We could just leave it as is! Then:

• It’s never been a major concern in v1 and v2.
• It doesn’t really break anything.
• Current implementations wouldn’t need to change parsing logic.
• We’re just left with a syntax that is perhaps objectionable aesthetically and consistency (e.g. it’s just a quirk that for an Array contained in a Matrix, multiplying all the dimensions will not equal the number of elements contained in the Array).

[doug_walker]

Re: Matrix dimensions

Like Scott, I am also surprised that (after not coming up at all during the v1 and v2 spec writing) that this syntax issue has emerged as the main sticking point for v3.

I must say that I still believe that rewriting the syntax at every version sets a dangerous precedent and is unfriendly towards those that have invested time to help develop and implement the previous versions.

But I also believe that it is very important to somehow find consensus in these projects and often that requires making concessions.

If this would allow us to get Thomas and Graeme on board and move forward, I’m willing to withdraw my objection, as a gesture of good will.

Doug

[walter.arrighetti]

Hi all.
First of all congratulations for the massive work on this new CLF release. We’ve finally got it as a huge improvement over the previous versions.

I just have four comments – the former three of which I already brought forward during the first CLF VWG:

• XML namespace shall be used in all CLF-specific elements, and shall be distinct from the URI for AMF; for example, what about clf ? Even if namespace is relaxed as optional (effectively, a should), it shall still be mandatroy in case CLF is embedded into (i.e. “extends”) other XML documents (like AMF of course, or rather a FCPXML, or any other XMLs).
• The Info element may be harmonized with that of AMF; like re-use element names from AMF’s namspace. For example: ACEStransformID should be replaced by aces:transformID.
• There should be an optional UUID element (uuid) in the info section, as well as the explicit possibility for a digital signature over a CLF file in order to:
  • protect its integrity, i.e. avoid people from tweaking with a LUT with no one noticing;
  • protect its authenticity, i.e. a standardized way to check whether the LUT comes from a specific (and trusted) source.
• ASC_CDL elements should only be mentioned in the spec as plausible children of ProcessList and using their own (mandatory) namespace cdl. However, no CDL syntax should be described in the spec, but rather the original CDL technical reference document be quoted in its place.

For digitally signed CLF, just referencing the W3C XML-DSIG Specs is enough, because XML signing is already very well standardized: just look at how CPL or KDMs work in D-Cinema.

The reason why I insist on namespace is because XML is designed as an extensible language, where a smart (and mandatory) management of namespaces allows one to embed some XML into a completely different XML, as long as their elements have distinct and unambiguous namspaces – and pointing to different, correct URIs

The usecase-scenario, here, is embedding a CLF file within an AMF, rather than having an AMF referencing to an externally supported file (which may or may not be original). CLF may also be embedded in other XML-based files, like Autodesk Flame, FilmLight Baselight. Adobe Premiere Pro --even FinalCut X– as with any other current or future postproduction software, for as long as they speak XML as well.

[sdyer (Scott Dyer)]

@walter.arrighetti, there is existing precedent for different capitalization patterns in CLF (an evolution of two previous versions) vs AMF (which has the luxury of being written “from scratch”). So in general, across all your points, I’m not sure how to reconcile these differences. We’d need to change all of CLF to match the names and casing from AMF - that seems … unlikely to garner support.

@walter.arrighetti

XML namespace shall be used in all CLF-specific elements, and shall be distinct from the URI for AMF; for example, what about clf ? Even if namespace is relaxed as optional (effectively, a should ), it shall still be mandatroy in case CLF is embedded into (i.e. “extends”) other XML documents (like AMF of course, or rather a FCPXML, or any other XMLs).

There is currently a namespace defined:

<xs:schema targetNamespace="urn:NATAS:AMPAS:LUT:v2.0" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:lut="urn:NATAS:AMPAS:LUT:v2.0" 
    elementFormDefault="qualified"
    attributeFormDefault="unqualified">

Now there are a few things here that should probably change:

• Why is NATAS in the target namespace? Does anyone in the history of CLF know where the National Academy of Television Arts and Sciences has to do with any of this?
• The version number should be updated to 3.0.
• If that’s changing, then we can redefine the namespace prefix from lut to clf. I think clf makes sense as a prefix.

I propose something like:

<xs:schema targetNamespace="urn:AMPAS:CLF:v3.0" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:clf="urn:AMPAS:CLF:v3.0" 
    elementFormDefault="qualified"
    attributeFormDefault="unqualified">

@walter.arrighetti

The Info element may be harmonized with that of AMF; like re-use element names from AMF’s namspace. For example: ACEStransformID should be replaced by aces:transformID .

Theoretically, we could change ACEStransformID to transformID, but if we did, then, in our namespace wouldn’t ours be clf:transformID? (Also, if AMF is going to use amf: as you say, then wouldn’t it actually read amf:transformID? - that’s confusing. How do we know it’s an ACES Transform ID)
NOTE: I talked to Alex about this point and I believe he’s defined a separate aces urn to use for transformID.

Now, if you’re just talking about using the same names of metadata elements in the Info block, then that makes me think the following:

• The only element the two formats currently share is description.
• We don’t have author, uuid, datetime
• Likewise, AMF doesn’t have AppRelease, Copyright, Revision, CalibrationInfo, etc.
• Again, differencing in casing of words.
• Where do we stop with “reusing” metadata from one to the other?

@walter.arrighetti

There should be an optional UUID element ( uuid ) in the info section

This is probably the easiest change we could make almost immediately. @zachlewis
Is anyone opposed to adding a UUID under Info? It would require adding an import of dcml in the schema and adding uuid into the spec.
Of course then we have to ask what do we call it? Do we define it as name="UUID" to be consistent with our capitalization or name="uuid"?

@walter.arrighetti

as well as the explicit possibility for a digital signature over a CLF file in order to:

• protect its integrity , i.e. avoid people from tweaking with a LUT with no one noticing;
• protect its authenticity , i.e. a standardized way to check whether the LUT comes from a specific (and trusted) source.

I don’t know enough about XML to understand what this means. What would adding support for thomething like this involve?

@walter.arrighetti

ASC_CDL elements should only be mentioned in the spec as plausible children of ProcessList and using their own (mandatory) namespace cdl . However, no CDL syntax should be described in the spec, but rather the original CDL technical reference document be quoted in its place.

@zachlewis also asked about this earlier in the thread. Versioning of CDL was discussed earlier in the thread in feedback from @Thomas_Mansencal and @doug_walker weighed in. I’m not exactly sure why we redefined ASC CDL in our previous specifications rather than just referring to the distribution from the ASC CDL. Perhaps @jim_houston or @doug_walker know more of the history on how ASC CDL came to be re-defined by us rather than referring to the CDL definition…

[nick (Nick Shaw)]

One reason may be that the ASC CDL specification is not (AFAIK) published publicly anywhere. Is it still necessary to send a request to a special email address in order to be sent a copy of the spec?

[sdyer (Scott Dyer)]

All,

I have been given the task of locking the document for a Friday release so I would like any and all feedback resolved by Wednesday.
So if you have been holding back any feedback to add to any of the issues discussed above, please do so ASAP!

[zachlewis]

Sorry guys, I meant to reply before the end of last week. Better late than never.

• Namespace and Versioning

  • What are the implications of redefining the namespace prefix? Anything serious? I imagine after CLF matures from ‘Draft’ status, it’s the kind of thing that probably shouldn’t change, no?
  • (If it isn’t problematic, I do think ‘clf’ makes a better prefix than ‘lut’)
  • NATAS – legend has it, if you play a CLF backwards, you hear all… sorts… of things…
  • Speaking of changing stuff… on one hand, I share @doug_walker’s concerns re: the precedent set by changing something as fundamental as syntax each version. On the other hand, I also agree with @Thomas_Mansencal – that if ever there were a time to commit to compatibility-breaking changes, this is it.
  • Perhaps it’s worth formalizing a versioning policy for CLF releases? I don’t have The Answer, but here are a couple of ideas / conventions worth considering:
    • Semantic Versioning (a.k.a. SemVer) – basically, versioning follows a strict major.minor.patch convention, where:
      • major version updates indicate backwards-compatibility-breaking-changes;
      • minor version updates indicate new backwards-compatible features
      • patch version updates indicate backward-compatible fixes (no new features)
      • This is what Thomas is alluding to above – to paraphrase, since this is CLF 3.0 and not CLF 2.1, this is one of the few opportunities to introduce backwards-compatibility-breaking changes.
    • Contrasted with Calendar Versioning (a.k.a. CalVer) – where “versions get better with time”, and versioning codifies that passage of time, often with a scheme like YY.MM.DD.release or something similar. This is probably less appropriate for CLF.
    • Versioning CLF like software might not be the right approach, but given that the spec is versioned, and given that features and syntax have or may have changed, I think it provides a helpful framework for discussing features, changes, policies, and design decisions.

• Optional UUID element

  • Absolutely, UUIDs are great. Any reason not to make them required?

• Digital Signature

  • I also don’t know enough about XML to understand what this means.
  • @walter.arrighetti, is it a matter of updating the CLF schema definition, so that it’s possible for clients to support digitally signed CLFs? Or are you proposing support for digital signatures as a requirement for implementation?
  • If it’s the latter, it seems a little out-of-scope for CLF v3.0…

ACES / Info Element Stuff

• I apologize for my ambiguity earlier re: ACES-specific metadata. My feeling in general regarding metadata is that its presence and validity is motivated by practical, tangible implementations; and with ACEStransformID / ACESinfo, I’m really don’t see a pertinent use that isn’t duplicated or more appropriately implemented in AMF. Admittedly, I haven’t been keeping up with AMF as much, so I may have missed something that makes specific use of the transformID attribute’s existence in CLF.
• By the same token, I don’t see a tangible use for CalibrationInfo elements; but if it exists to support an external API, maybe the ACES Info elements could also be delegated to the same hierarchical level; which could serve as a convention for such information in future editions.

@nick – you’re right about the spec not being published anywhere. you know, I did actually send a request to that special email address, and got permission to share the spec and materials. It’s at work; I’ll see if I can dig it up tomorrow.

[jim_houston]

Remember that the Common LUT format had been a joint project of the Academy and ASC from it’s inception in 2006,

though version 2 had most of its changes in an Academy group call.

On the question of NATAS — in CLF version 1.0, the XML namespace identifier was NATAS:ASC

I believe this was the same as for the ASC CDL.

At one time, it was thought that it would be useful to have an XML schema at the

organization that was named in the namespace identifier but at the end of the

day, that was never done, so it has merely been a unique identifier string as the “urn”

There is not a reason to keep it the same.

However, since a large portion of the initial discussion and requirements came from the ASC,

I would like to keep a nod to that in the XML namespace.

i.e. replace NATAS with ASC.

in other words, <xs:schema targetNamespace=“urn:AMPAS:ASC:CLF:v3.0"

There was a participant in the early days from the TV Academy I think (in the ASC group)

Jim

[jim_houston]

In the definition of IndexMap it says
"The format of the two items in the list shall be newMinV alue@0 and newMaxV alue@1”
I think this is incorrect. Should be
value@LUTposition and value@LUT position
the numbers will be different if a 1023 1D LUT than a 4095 1D LUT. so ‘1’ is not a normative
requirement, and does not automatically scale all indexes. (if that is the intent, I do not see where
it is defined that a relative LUT index is defined over the range [0…1] for accessing an array.)

IndexMap was intended to map a value WITHIN the inside of an array if need be.
For example, for an SDI image you might have 64@4 and 940@1019. When creating the LUT
you would have to have 0’s in the 0,1,2,3 positions, etc…

if the example showed LUTposition=1 and LUTposition=0 then it would be OK, but this is unclear.
In either event, the normative line should be changed.

Jim