The behaviour when applied to the thumbnail makes sense. In ACES mode, Resolve applies source LUTs in linear AP0, after the IDT, but before the transform to e.g. ACEScct. So there would be values >1 which the LUT would clip.
But when applying it as an output LUT, I would expect it to be applied like a calibration LUT, downstream of the RRT and ODT, which is what you want. I will have to do some testing.