Loss of highlight detail, this time during rendering. My preferred intermediate codec is ProRes and this loss of detail only happens when rendering from Cin, its OK from other NLEs. It is nothing to do with the latest Cin update, which I waited for before reporting this. It is not just ProRes, it is easier to say the only output options to give full detail are mkv, specifically HuffYUV, rather than list those that dont. I have tested all options that offer a visually lossless output.
I have tried all variations of ProRes and ProRes_ks, I have also purged the .bcast 5 folder and started again from scratch. I also tried from Kubuntu to eliminate it being a Manjaro thing.
First of all, let me just clarify that Loss of highlight detail (part 1) is still solved and there is no relation between the problem in part 1 and this part 2 ? I am just asking because you labeled it part 2 and I want to make sure I did not misunderstand.
I have tested all options that offer a visually lossless output. So, you tested rendering using ffmpeg, type m2ts with lossless.m2ts or visually_lossless.m2ts?
There are so many options that could be added/changed in the ProRes rendering that most likely we have not added what other NLEs use and know about (probably because there is some audio/video expert there who knows about that stuff! as opposed to us computer people). Is there a direct ffmpeg command line that you have used to get good results with ProRes ? If so, we could possibly use that to come up with a good ProRes option file with a variation that solves the loss of detail. I will try rendering with ProRes and see if I can easily see this loss, but usually my eyes are not good enough to see what you see.
Although the results of my ProRes rendering are not 100% the same as the original. I have a video showing no discernible difference, but I have to explain what you see in the video.
https://streamable.com/uvoozj
#1 ffplay of the very short original video that is 1920x1080.
#2 ffplay of the Cinelerra rendered using ProRes, profile 5.
#3 I run ydiff (explained in the manual) which is a subtraction of the 3 rgb channels between the 2 video files.
(a) you will see a grey colored screen with me wiggling the cursor around and note how you see no lines showing real differences/loss. If there was a lot of detail losses, you would see them as little lines. The manual in section C.1 provides a little more information.
(b) next you see the numerical value differences in columns 2, 3, and 4. If the files are exact, then these will all be 0. You can see that they are not. I believe column 5 is the frame number.
We could provide a ydiff compiled program for you to test on your video for Arch (probably that would run on Manjaro too) if you would like to try this. However, it may be necessary to have some specific library installed and I do not know what that might be.
Thanks Phyllis. I wasnt sure how to connect the two, because the effect is the same, while at the same time differentiate between them because the cause is different. Yes, part 1 was and still is, solved.
I did try visually_lossless. m2ts. However, since posting I fiddled a bit more and found that DNxHD, which wont work because it doesnt handle UHD, leads into DNxHR which does (a mite confusing, but Shotcut does the same thing). Rendering as DNxHR HQX also keeps highlight detail. Hopefully that information may help.
While DNxHR is an improvement on HuffYUV (smaller file sizes and motion does not halt as it sometimes does with Huffy) and I can work with it, it is less useful to me than ProRes.
I dont use any ffmpeg commands, GUI only.
Thanks for the demo, but it doesnt really address my problem, the way to see what I am trying to explain is to look at the previous thread and compare the YUVA 8-bit pngs with the RGBA-FLOAT ones, because what is happening is almost identical, the only difference is that now the detail is present in the compositor window and missing in the rendered version.
We would like to include a potential fix in the July 31 new builds but unfortunately we have been struggling with this over a few days now and are no closer to finding a test case that shows the problem.
Could you send us a short video (like of the sky with clouds and trees and a building, like the grey skies png you sent previously) that was created on the same camera:
Camera details: Fuji X-T3 shooting in F-Log at 1 stop over exposed (1 to 2 stops recommended), 10-bit interframe HEVC at 200mbps.
that definitely will show us when we render it to Prores, the loss of detail. Thank you. You can send here or privately to: phylsmith2017@gmail.com
Theyre 2160p @ 200Mbps, I need to think of a way to reduce the file size without losing the detail. Ill try over the next few days, any fix would be unlikely to be in the new release now in any case would it. In the meantime any ideas as to how or what format would be welcome.
It seems to be an impossible task. Due to the fact that ProRes is not the only format to have this problem, I cannot produce a suitable clip that does not have the distortion, because when I do it is either too big, or not a permissible format (e. g. DNxHR). One or two attempts that looked good here, when posted and played back exhibited the problem, so were useless. Attempting to do it via Shotcut, when imported into Cin and rendered ProRes, the problem is not there.
If it helps, when importing into Shotcut the waveform scope shows way over 100 (presumably IRE) and the highlights are blown out, but applying the BT709 LUT brings the scope down to 100 and the highlight detail appears. I suppose much the same as when I tried YUVA 8-bit before the first fix.
I have no idea how big a file I can attach to an email although I will try later, so I wonder if it would work if I post a couple of clips on Dailymotion for you to download?
Edit: For the interested, I managed to get some clips to Phyllis by email.
Personally I often use pCloud which allows to share through the cloud up to 10G with a free account and it offers an automated synchronization application for linux.
I know that there are even other cloud sharing services with even greater transfer capabilities through free accounts.
Thunderbird offered me Filelink, which I declined. Having just spent a couple of days trying to find a format that would do what is required, I wasnt going to spend (even more) time on setting it up just for one file, but Ill know how to send big files in future if I ever have the same need again.
As I also want to use ProRes as an intermediate visual lossless codec to preserve legacy SD and HDV video, I am just curious how you encoded ProRes? That is, which tool (ffmpeg)?
Its dead easy in Cin_GG. Open the render window, set File Format to FFMPEG and select pro from the second box dropdown list. Click the Video spanner, select whichever ProRes codec you prefer from the dropdown list and adjust to your preference.
Just a thougt from me. If I understood you right, rendering to ProRes was ok from other NLE. Just try to read in one such ok ProRes file type in CinGG to see if the loss of hightlight detail appears. Try to render it to ProRes again from CinGG. If the issue appears, it is not specific for your source file type, I think.
Thanks. I spent a lot of time over a couple of days trying to produce a file that was suitable to send to Phyllis, trying different combinations from several NLEs as well as many formats from Cin, each of them were then imported into Cin for test ProRes renders. What became obvious was that when passed through another NLE first, the distortion did not occur in a Cin Prores render. What I did notice was that in Shotcut, before applying the BT709 LUT the WFM display was well over 100 (percent or IRE I dont know) and the distortion was there, but applying the LUT corrected both, although it is not like that in Cin. In Cin only some output formats do not exhibit the problem, but many do.
Unfortunately, we still can not see any problem after 2 more days of testing and even using the Highlights…mov file you provided. All we see is that there is no loss of detail and no banding. The attached 2 png-s show (the 2nd png is in the next note):
inside of Cinelerra we did a subtract in the patchbay of the original file and the rendered prores file and the difference is shown on the first track in the Compositor. It is all grey with no obvious banding or pixel differences.
blowing up the last frame of each of the before and after videos and looking at the detailed pixels, they really look the same. GG also rewrote the ydiff program to handle more than 8 bits and when we compare the 2 files, there is no perceivable differences.
It looks like you will have to continue to use a workaround until something changes so that we can detect the difference.
Looking at the second PNG, is that how the file arrived in your inbox? If so it is showing the problem i. e. burnt out highlights in the middle of the cloud formation, which means I didnt manage to send you a good original to work from after all, even though it looked OK here before I sent it. Several times I thought I had it just right and when I posted here the distortion was back, so I deleted the posts and started again.