YUV to RGB conversion issues

Help & Support
1

Following the recent Optimize Playback thread, I seem to have opened Pandoras box when it comes to Color Model in Cinelerra.

I work mostly with 8 bit YUV footage from Panasonic and DJI cameras. Up until now, I have always set the color model to RGB-FLOAT for my projects, and this, combined with setting YUV color space to BT709 and YUV color range to MPEG has meant that what I see in the compositor looks the same as my final rendered video.

However, I have noticed that if I set the color model to YUV-8 Bit, then my footage has far more dynamic range (highlights are hard-clipped when set to RGB-FLOAT), and is more saturated. In fact, when set to YUV 8 Bit, the footage looks the same as it does in VLC player.

On the other hand, if I use YUV-8 Bit as the color model, the final rendered video does not look the same in VLC player as it does in the Cinelerra compositor - it is a bit de-saturated compared to the compositor.

What I would like to achieve is to retain the vibrant colour and highlights that I see in the compositor when the color range is set to YUV-8 Bit in my final rendered video. Can anyone point me in the right direction?

2

@glitterball3

In my little experience after rendering for 3 weeks to get the compositor color.
RGB-8 Bit > Colors equal to the original video.
YUV-8 Bit > Limited colors (saturated). different rendering than compositor.

JPEG > full color
MPEG > limited colors

You can try using Shotcut to do your color tests.

YUV-8 Bit + MPEG looks beautiful in the compositor but thats because the compositor uses X11 and X11 is RGB.

I could be wrong.

Using RGB + JPEG will have colors equal to the compositor.

The YUV of the compositor is not the same as the YUV of the render.

3

@sparkill Your observations are mostly the same as mine - having looked at it again, I would say that in YUV mode, the compositor looks more saturated than it does in VLC player.

However, RGB + JPEG, definitely does not show the same colours as the original video in the compositor for me - the image becomes quite washed-out looking. Also RGB+JPEG when rendered, does not look the same as it did compositor either - it looks similar to the results from RGB+MPEG (close to the original).

Is there any way to prevent the RGB-FLOAT conversion from clipping the highlights?

4

In theory RGB-FLOAT should be the only mode that does not create clipping, so Im more and more confused…
Anyway, if you can find some certainty about CinGGs behavior with colors, let us know and well put it in the manual.

EDIT:

The only note to make is to use RGB-FLOAT with BT2020 and not RC709.

What we see in the Compositor depends also on the color space of the monitor.

5

If its of any help, what I have found is;

If shooting restricted range video as most camcorders do, the colour range needs to be set to MPEG. If shooting full range video as many mirrorless cameras do, it needs to be set to JPEG. Analogy: Putting restricted range video into JPEG colour range is like trying to fill a bucket with a single glass full of water and trying to put full range into MPEG is like trying to get the bucket full into the glass. JPEG into MPEG clips at both ends, losing detail, MPEG into JPEG needs stretching, causing distortion. Assuming, of course, the video scopes are being used as a guide.

I always use RGBA-FLOAT whether shooting 10-bit full range or 8-bit restricted range. I get the impression (correct me if I am wrong!) that the plugins work in RGB, so it is better to set the timeline to RGB at the start to avoid converting YUV to RGB for processing and then back to YUV again.

I use BT709 colour space, even for UHD video, as I output 1080p which will become BT709 on export anyway.

In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead.

6

The easiest way to see the clipping that I am talking about, is to load a video file with some over-exposed highlights, and add a blue banana effect.

Then, switch between RGB-FLOAT and YUV-8 Bit. The mini histogram in the value slider of Blue Banana will show the out-of-range values when in YUV mode, but these will be clipped when in RGB-FLOAT.

Be careful when switching between MPEG and JPEG in the Preferences dialog, as this may change YUV to RGB in the background (if the format dialog window is also open, it will not show the updated value until you close and re-open it).

In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead.
I was indeed rendering in ProRes, I will try rendering in DNXHR instead.

Edit: I think that the JPEG vs MPEG YUV color range may well be a factor here - in my camera, I have the range set to 0-255 (the other option is 16-255). However, if I set the YUV color range to JPEG in Cinelerra, the final render does not match what I see in the compositor.

Edit 2: If YUV color range is set to JPEG, then the exported video will be much darker than in the compositor. If I import the rendered video back into cinelerra, then it looks exactly the same in the compositor as the original video. However, the timeline preview will be much darker (just like in VLC)!
So, theres likely a clue there as to why the rendered video looks darker than the original.

Is it possible that MPEG is assumed as the color range in many areas of the software as well as in external video players and tools? If this is the case, is there a way to squeeze that extra range back into the MPEG range in Cinelerra without just discarding it via hard clipping?

7

I think that Im getting closer to understanding the issue - if anyone knows better, then please correct me.

  1. My videos are recorded using the 0-255 range.

  2. 0-255 corresponds to JPEG in the YUV color range: setting, and with this set, I can see the full range of black to white in the compositor.

  3. When I render a video with YUV color range: set to JPEG, the outputed file is usually interpreted as 16-235 and blacks appear crushed and highlights are clipped - both players such as VLC and even other video editors such as Kdenlive interpret the rendered video as being 16-235. However, if I import the rendered file back into cinelerra it is interpreted as being 0-255 without the crushed blacks and clipped highlights.

  4. When I set YUV color range: to MPEG, what I see in the compositor is exactly the same as how the rendered video is interpreted by other players, however any values outside 16-235 are clipped by Cinelerra and cannot be recovered using any of the color correction effects.

At present, I can either:

a. Set YUV color range: to MPEG and the final output will match what I see in the compositor, with the caveat that I may lose some shadows and highlight information from the source media.

or

b. Set YUV color range: to JPEG and accept that the final output will be misinterpreted and that my shadows and highlights will be crushed and clipped.

Another question that I have is about the pipeline in Cinelerra:

Is the YUV color range: setting in Appearance a project-wide setting, something that affects the final rendered output, or just the compositor?

It seems to be a setting that affects how all media is interpreted - so what happens if there are mixed sources?

I think that a better way of working would be to allow the values outside of 16-235 to continue to exist (rather than being hard-clipped) in a RGB-FLOAT project with color range set to MPEG. These values would be outside the legal range in the scopes, but at least we could pull them back into range and recover highlights/shadows during colour correction

8

@glitterball3

Another question that I have is about the pipeline in Cinelerra:

Is the YUV color range: setting in Appearance a project-wide setting, something that affects the final rendered output, or just the compositor?

It seems to be a setting that affects how all media is interpreted - so what happens if there are mixed sources?
As far as I know, the Project Setting is used for everything. So if there are mixed sources, it uses the same current Project setting for all sources. So for advanced usage, it is always best to set up your project settings first.

https://cinelerra-gg.org/download/CinelerraGG_Manual/Automatic_Best_Model_Media_.html

9

@glitterball3
To see the rendered video in a player without color problems (I cant tell in other NLEs), it is advisable to write in the wrench options the 3 information about the colors you are using as metadata. They are as follows:
colorspace= … (color matrix: RGB, YUV,…)
color_trc= … (transfer characteristic or gamma: sRGB, Rec709, …)
color_primaries= … (gamut: sRGB, Rec709, BT2020, …)
These parameters do not affect the rendering you do in CinGG, but are read as metadata by players (Vlc, mpv, …) that take them into account to play the video in an optimal way.
https://cinelerra-gg.org/download/CinelerraGG_Manual/Extra_acin_a_Options_Render.html

In case of different and non-uniform assets it is advisable to standardize all sources to the same model color, gamma and gamut (and that coincide with those of the project). So render/Transcode inside CinGG or, even better, externally from terminal with ffmpeg, etc. In any case, as Phyllis said, the rendering is only affected by the Project settings (and wrench parameters).

10

@phylsmith2004 & @andreapaz

Thanks for all of the suggestions, however after trying even more things out, it seems that the main problem that I have is that what I see in the compositor is simply not the same as I see in other editors or players unless MPEG is selected as the YUV color range. This comes with the caveat that anything outside the 16-235 range is hard clipped.

Is there any way that this functionality could be changed, or an option added to prevent everything being hard-clipped when using MPEG as the YUV color range? Perhaps only hard-clip the range closer to the output after video effects have been applied?

11

I have to say that things are easier for me now I have changed my main camera.

I was shooting 10-bit 420 full range (full range cannot be changed), 2160p at various compression settings i.e. long-GOP or all-i at anything up to 400mbps, sometimes in F-log, sometimes in Eterna profile, with my X-T3. Now I am shooting 10-bit 422 MPEG long-GOP at 50mbps, using Cine D profile or occasionally F1, with my HC-X1500 and I prefer the image quality slightly, this has made most of my Cin troubles disappear. I am now importing 16-235 rather than the 0-255 I was, I set it to MPEG BT709 and use RGBA-FLOAT, now everything looks good from first import into Cin to the final render. Im damned if I can see any difference in the outputs between full range in and restricted range in and in any case, output will be restricted range BT709 so no I longer see any logic in importing full range by choice.

12

@dejay Your settings do make sense for 10-bit, though Im still recording in 8-bit, and that extra range does make a difference in 8-bit.

13

@glitterball3

Is there any way that this functionality could be changed, or an option added to prevent everything being hard-clipped when using MPEG as the YUV color range? Perhaps only hard-clip the range closer to the output after video effects have been applied?

Not that this will help but I logged a BugTracker ticket #605 because I do not want this to get lost if someone comes along to address the issue.

14

@dejay

In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead.

I also finally logged a BugTracker ticket #606 so it can be fixed when a pro (pun intended) comes along.

15

@glitterball3

Could you do a little test with the settings below to see if the rendering viewed in VLC match with the Compositor, please?

Because it creates a huge file it is better highlight 5-6 seconds in the timeline and using Selection checkbox in the Render window.

  • Preferences: - Playback A TAB - Video Out section Play everyframe = unchecked Video Driver: X11 Use direct x11 render if possible = checked - Appearance TAB - Color section YUV color space: BT709 YUV color range: JPEG- Format: - Video section Frame rate: — Width: — Height: — Color model: RGBA-8bit (also RGBA-FLOAT)- Render: File Format: FFMPEG | qt Video Preset Compression: png.qt Bitrate: 0 Quality: -1 Pixels: rgb24 (also rgba64be)
16

@igorbeg

Could you do a little test with the settings below to see if the rendering viewed in VLC match with the Compositor, please?

Because it creates a huge file it is better highlight 5-6 seconds in the timeline and using Selection checkbox in the Render window.

I did the tests with the following combinations of format settings (RGB-8Bit, RGBA-8Bit and RGBA-FLOAT) and render settings (rgb24 and rgba64be), with the following results comparing the compositor to VLC player:

RGBA-8bit + rgba64be is very different to compositor - very dark
RGB-8bit + rgba64be is slightly different (colour hue)
RGBA-FLOAT + rgba64be is very different to compositor - very dark
RGBA-FLOAT + rgb24 looks identical to me
RGBA-8bit + rgb24 looks identical to me

BTW, there is a bug in .css of this forum that causes blockquotes not to be formatted properly when it is a wpforo-qa-comments.

17

@glitterball3

Mmh, in my case it works also for rgba64be.

Could you convert your RGBA-8bit+rgb24 output (.qt) to .mp4 with ffmpeg and see if it still match VLC vs Compositor? I am not an expert on ffmpeg, but in this test lets see how ffmpeg convert by default. I used the command below and in my case it matched.

ffmpeg -i input.qt output.mp4Could you post the outputs of the ffprobe info about your source.xx, RGBA-8bit+rgb24.qt, RGBA-8bit+rgb24_byFFmpeg.mp4 files? I use this command:

ffprobe -hide_banner -i input.qtMy tests are on a .mov file source: mjpeg (720p yuvj422).

I hope dont waste your time.

18

@igorbeg The result of that conversion looks exactly the same in VLC player (the same as in the compositor).

Output of ffprobe:

RGBA-8bit+rgb24.qt

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from TestPngQt:Metadata:major_brand : qtminor_version : 512compatible_brands: qtencoder : Lavf58.76.100Duration: 00:00:08.60, start: 0.000000, bitrate: 2007414 kb/sStream #0:0: Video: png (png / 0x20676E70), rgb24(pc), 3840x2160 [SAR 1:1 DAR 16:9], 2007280 kb/s, 25 fps, 25 tbr, 12800 tbn, 12800 tbc (default)Metadata:handler_name : VideoHandlervendor_id : FFMPStream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)Metadata:handler_name : SoundHandlervendor_id : [0][0][0][0]FFMPEG Output.mp4 file:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from output.mp4: Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.76.100 Duration: 00:00:08.62, start: 0.000000, bitrate: 33723 kb/s Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 3840x2160 [SAR 1:1 DAR 16:9], 33662 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0]

19

@glitterball3 And the ffprobe of your source file, please?

Your outputs are as mine. Good but, until now, I can not reproduce this output inside Cinelerra-GG and rendering in h264-yuv444p: it always render to yuvj444p and it looks a bit more dark. (And I write color_range, color_primaries, color_space, movflags=+write_colr in the Render codec)

20

Heres the output of ffprobe for the original video:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55673f3d3ac0] st: 0 edit list: 1 Missing key frame while searching for timestamp: 3600[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55673f3d3ac0] st: 0 edit list 1 Cannot find an index entry before timestamp: 3600.Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 18-surfer-wide.MP4: Metadata: major_brand : mp42 minor_version : 1 compatible_brands: mp42avc1 creation_time : 2021-12-23T17:01:38.000000Z Duration: 00:00:22.56, start: 0.000000, bitrate: 92715 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 92496 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default) Metadata: creation_time : 2021-12-23T17:01:38.000000Z vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 125 kb/s (default) Metadata: creation_time : 2021-12-23T17:01:38.000000Z vendor_id : [0][0][0][0]