How to Optimize Playback?

Help & Support
1

Hello, I recently gave Cinelerra-GG a real shot, and was very impressed with it on multiple fronts. It has a lot of great features that other FOSS NLEs dont have, chief among them being hardware decoding. It has the smoothest playback between Kdenlive, Blender, Olive, and Pitivi, and has no lag whatsoever for 4K video. Even for codecs like Quicktime which NVIDIA doesnt support hardware decoding for. It has the best documentation by miles compared to the other FOSS NLEs. Amazing job on that, by the way; its unfortunate that most projects dont have documentation this thorough and in-depth. I learned a lot about video in general from reading some of it, even.

However, when I have two video tracks playing at once, the framerate is cut in half. When I add an effect like Color 3 Way to a clip, it cuts the framerate in half again. Ive tested this by enabling/disabling the effects, and enabling/disabling Play Track in the patchbay and looking at Preferences for Playback A. Im not sure if this is typical behavior, which is why Im mentioning it. Im not sure if this has something to do with driver support for GNU/Linux. I get framerates of 3-5fps or 11fps instead of 25fps.

So, here are some things Ive tried to optimize media, which work, but none of which provided a complete solution for me:

  • Turning on background rendering—this worked very well, but it changes the color space when rendering the JPGs so I dont get an accurate impression of the color. It also takes quite a bit of time to work through the entire track. The only way to turn it off is to toggle background rendering again, which starts all over from the top, despite there being thousands of frames in my /tmp folder generated already.

  • Tried proxy clips with mpeg codec. This actually resulted in worse framerates without the scale factor. Additionally, mpeg does not include an alpha layer, which is necessary for some motion graphics I have added to the timeline. Otherwise I dont get a clear picture of the playback I attempted to change the proxy container to webm and used yuva420p pixels, but this didnt produce transparency either. I then read that VP9 does not account for transparency and to use VP8 with yuva420p pixels, but when I tried this, I got an error. I tried mov as well, the only other codec I know supports alpha, and it gave me a very similar error:int FFMPEG::open_encoder(const char*, const char*): open failed libvpx:/path/to/file.proxy-MP4.webmint ProxyRender::create_needed_proxies(int): Error making proxyIm not sure if this just means its not possible to create proxies with transparency.

  • Rendering out the motion graphics files to a .tiff sequence with LZW compression and importing that—this produced worse results than the .mov file. By the way, the JPEGLIST example in the documentation didnt work without tweaking it to use absolute paths rather than relative paths to the images in the sequence; Cinelerra-GG couldnt find the files.Additionally, its strange, but whether vdpau is on or off doesnt seem to make any significant difference to the playback speed, even for H.264 media.

Is there anything else I can try? Or is there something that I screwed up when trying these things?

I increased Cache size from 256MB to 8192MB, but it didnt seem to make any difference, even on the system monitor. It only uses as much as 6GB of RAM at any time.

By the way, these are my system specs:

Arch Linux, X11, GNOME 41.3, NVIDIA RTX 2060 Super, AMD Ryzen 5 3600x 6-core, 32GB of RAM, using Linux-Zen with proprietary NVIDIA drivers (nvidia-dkms)Using 3840x2160, RGBA 8-bit settings for project, YUV color space in Preferences is set to BT709 instead of the default BT601_NTSCMy system monitor reports that normal playback uses about 65% of my CPU, while Nvidia X Server Settings reports about ~20% of my 8GB of my dedicated memory, at most 30% of GPU Utilization, and sometimes 50% of Video Engine Utilization.

Thank you for any help.

EDIT: Also, I built Cinelerra-GG from the cinelerra-gg AUR package instead of using the AppImage. Ive used both with identical experiences but just found the AUR an easier way to manage updates.

2

@prolinuxnie

Thank you for the good feedback. Hopefully other users will have some hints that might prove useful, although it seems you have already tried most of them. The forum is sometimes late being propagated so I have added a couple of names below so that they get quicker email notices.

You can switch to not Play every frame but that is probably not what you want to see because obviously it skips frames. Plus, it generally is most useful in keeping the video more in sync with the audio.

@Andrea_Paz (builds and uses Arch and probably has read most of the manual).

@Igor_Beg (he is quite versatile in using Proxy with Alpha so may have hints).

Credit for the Manual goes to Terje - who started it all; Andrea for keeping it updated and adding video information; MatN and Andrew for reviewing for corrections; and especically both Andrey who created the LaTeX version used in the pdf version (before that it was just in LibreOffice format and very difficult to maintain) and Georgy who did the hardest programming work to get the html version going and incorporating it with Alt-h into the source base.

3

Hi,
a first tip is to set the video driver to X11 and uncheck play every frame in Preferences → Playback A (The latter only for editing; when using plugins it is better to reactivate it).

I have problems with the mkv container, which causes me seek problems with consequent drops in fps.

Another advice is the transcoding of the media in formats suitable for editing (ProRes, HDxHR, cineformHD and ffv1; or anyway with All-Intra compression). Even, if the Compositing and Color Correction phase is important, it can be useful to use image sequences (OpenEXR, for example).

More tips can be found in the manual:

https://cinelerra-gg.org/download/CinelerraGG_Manual/Tips_Improving_Smaller_Comp.html

However, it is true that there is something not optimized in the playback: I moving from a good Intel i7 Sandy Bridge (11 years old) to an AMD Ryzen 3700X, I noticed a marked increase in power in rendering, but not in timeline playback, in which I record the same freezes and drops in fps that I had before.

1- Yes, every time you intervene on the timeline, the bacgkround rendering starts all over again, unfortunately. For the color: maybe its due to the color space and color range discrepancy between the original video and the jpegs. See:

https://cinelerra-gg.org/download/CinelerraGG_Manual/Color_Space_Color_Range_Aff.html

2- Proxies can give instability problems. Mpeg was made the default because it was lightweight and compatible, but it carries with it old age and compression (including color) problems. Try the proxy versions of Prores or DnxHR.

3- For image sequences, the problem of absolute paths is debated these days. I also need to edit the path. Try using OpenEXR which takes up a lot of space but is an efficient and high quality format.

4- I have AMD and with vaapi I find a small improvement, but not much.

5- I too find no difference by increasing the cache.

In short, CinGG has been carrying these problems for a long time and I dont think they can be solved, but only try to minimize them. Think that there are those who can run CinGG on Android tablets…

6- No problem using AUR, its always up to date as well as appimage. I, for testing, compile from git.

Major credit for the manual goes to Phyllis (the creator); and mention should also be made of Andrey who created the LaTeX version (and thus the pdf as well).

4

Additionally, its strange, but whether vdpau is on or off doesnt seem to make any significant difference to the playback speed, even for H.264 media.
In my case (AMD APU), I experienced a similar issue with VAAPI using the AppImage - first check that vdpau is actually working at all by launching Cinelerra from the terminal. GPU acceleration falls back to CPU in the case of an error.
There were various errors for me, which I was able to fix by extracting the appimage, creating symlinks and overwriting some of the files. However, getting VAAPI working actually provided worse playback for me than the default none.

However, it is true that there is something not optimized in the playback: I moving from a good Intel i7 Sandy Bridge (11 years old) to an AMD Ryzen 3700X, I noticed a marked increase in power in rendering, but not in timeline playback, in which I record the same freezes and drops in fps that I had before.
Very similar experience for me: I changed from a quad core Ivy Bridge i7 to a Ryzen 7 5700G, and it made hardly any difference to 4K playback.

On a separate note regarding colour accuracy, I have never understood the default options for YUV color space and range; surely these should be set to BT709 and MPEG? With the defaults, the final output always looks different - at least for my workflow.

5

@glitterball3
Can you describe the changes you make to make vaapi/vdpau work? We could put them in the manual.

On color accuracy:
CinGG lacks true color management, so its easy to have inconsistent behavior. Some tips:
https://cinelerra-gg.org/download/CinelerraGG_Manual/Video_attributes.html#sec:video_attributes
https://cinelerra-gg.org/download/CinelerraGG_Manual/Color_Space_Color_Range_Aff.html
https://cinelerra-gg.org/download/CinelerraGG_Manual/Automatic_Best_Model_Media_.html#sec:conform_the_project
I enclose an in-depth attempt I made; but it is incomplete and probably full of errors.

CMS.txt

6

@andreapaz Thanks for responding so quickly!

When I find myself editing video again, Ill try these tips and see how it goes. Two things I found out since writing the initial post:

  1. Background rendering is still very handy for seeing a particular effect in realtime; if you only need to render 5~10 seconds or so, it works well. I was using it wrong by trying to cache the entire timeline when I should have just been trying to cache a small area briefly.

  2. There was a big bottleneck when accessing the media from a network drive (surprised I didnt consider this earlier). I didnt experience this with DaVinci Resolve on the same machine with Windows, presumably because it downloads/caches the media locally shortly after playing it for the first time. After transferring the media to a local drive, the application is significantly more responsive overall, though there was only about a 2-3fps increase in playback.

I think the next bottleneck to address would be using faster drives; Im using a 10TB mechanical drive so the speeds arent as high as they would be with an SSD. If I got a CPU with 16+ cores, I feel like this would be enough to play it back in almost realtime, as my (12; 6 physical, 6 virtual) CPU cores currently hit around 70~80% usage. Although I dont know how much difference it would actually make considering yours and @glitterball3s experiences.

I will try one of the intermediate formats with my next project. And Ill give the OpenEXR image sequences a go too. If this works, I wont need to deal with not being able to have proxies with an alpha layer.

Im working with the .mp4 container for media, by the way.

I also noticed a loss in fps in some areas with vdpau enabled vs not enabled. But it wasnt consistent. I ensured that it was enabled during my testing before by checking the console responses from the application, which would throw an error as soon as it hit a .mov file, which vdpau cant decode.

I had play every frame unchecked already and by default in both the AppImage and AUR package. I found that the audio got out of sync quickly with it on, but can see how it might be useful.

7

@andreapaz Thanks very much for your document and those links - it sent me down a bit of a rabbit hole with some surprising results:

I have always set my projects to RGBA-FLOAT and not thought any more about it, however having just played with all of the options, I found the following:

  1. RGB-8 Bit results in significantly better playback: I can get 4K 25fps playback with a Blue Banana effect inserted (though it still pauses whenever the source video file changes).

  2. Using YUV-8 bit, I can get about 20fps with 4K and a Blue Banana effect inserted (basic colour correction). However, using YUV, I was able to recover highlights that were completely clipped when using RGB Float - a very useful thing to know! Footage does seem to suffer more from banding though when using this, so it is probably more useful to use as a way of rescuing some over-exposed footage by transcoding. Using this mode, the overall image is brighter and appears more saturated, however using RGB looks closer to how the clips appear when played in VLC Player, so rightly or wrongly, I prefer the consistency of that.

Armed with this info, I will probably edit using RGB-8 Bit, render and do final colour corrections with RGBA FLOAT, and use YUV if I need to rescue an over-exposed clip.

Can you describe the changes you make to make vaapi/vdpau work? We could put them in the manual.

There is a stage missing from this as I cant remember which symlinks I created, however the terminal error messages should help if someone needs to do this.

What I did:

  1. Extract the files from the AppImage:

/path-to-appimage/CinGG-20211231-x86_64.AppImage --appimage-extract2. Run CIN

/path-to-appimage/squashfs-root/usr/bin/cin3. Under performance tab in Preferences, set Use HW Device to vaapi. Click Apply.

  1. Quit and restart CIN

/path-to-appimage/squashfs-root/usr/bin/cin 5. At this stage, there may be some errors in the terminal. In my case (using Kubuntu 21.10), the va api library installed on my system was newer than the one requested by Cinelerra, and this was returning some errors. I was able to add some symlinks to the newer va api files - but I cant remember where I added them.

  1. Finally I simply overwrote the extracted /path-to-appimage/squashfs-root/usr/lib/libva.so.2 file with /usr/lib/x86_64-linux-gnu/libva.so.2.1200.0 (the one installed on my system).

  2. Restart CIN, and va api now works, though for me, performance was worse than the default HW Device none.

8

Concerned about errors and problems generated so tried to reproduce.
Test file used: https://gemtvusa.com/life-untouched-hdr10-uhd-4k-demo-60-fps/
with only 1 video and 2 audio tracks.

  1. Proxy

use VP8 with yuva420p pixels, but when I tried this, I got an error. I tried mov as well,

int FFMPEG::open_encoder(const char*, const char*):
open failed libvpx:/path/to/file.proxy-MP4.webm
int ProxyRender
Error making proxy
Proxy is EXTREMELY worthwhile for 4K files and my test case got 60 Framerate Achieved on this 59.9401 frame rate file. Without proxy (and it is a trivial one-click to switch back and forth) it averaged about 24 Framerate Achieved. I could not reproduce Error making proxy - I tried both of the following:

1/2 scale; File Format ffmpeg mov (video wrench not changed at all from default)
1/2 scale; File Format ffmpeg webm (video wrench switched to vp8.webm)

  1. Documentation Example

By the way, the JPEGLIST example in the documentation didnt work without tweaking it to use absolute paths rather than relative paths to the images in the sequence; Cinelerra-GG couldnt find the files.
I thought I fixed this on Dec. 15, 2021? I will look at it again.

  1. Vdpau

but whether vdpau is on or off doesnt seem to make any significant difference to the playback speed, even for H.264 media
It is my understanding that if you are not using all of the CPU already for CinGG playback and simultaneously running multiple tasks, you probably will not see any benefit during playback. In your case you report CPU at only 65%.

  1. Playback Optimization

is true that there is something not optimized in the playback
This could probably be improved a little if done very meticulously, but my understanding from the past is that no matter how many CPUs your computer has, many times the code is written to actually run single-threaded or you end up with frame 30 being displayed before frame 28.

  1. Vaapi

In my case (AMD APU), I experienced a similar issue with VAAPI using the AppImage - first check that vdpau is actually working at all by launching Cinelerra from the terminal. GPU acceleration falls back to CPU in the case of an error.
There were various errors for me, which I was able to fix by extracting the appimage, creating symlinks and overwriting some of the files. However, getting VAAPI working actually provided worse playback for me than the default none
I do not understand what fix could be needed and am concerned about AppImage not working with vaapi on other distros/computers. So wondering if the problem is due to where I create the AppImage, I tested there and could see it was working. Then I moved the AppImage to another computer and could see it was working there also. I could not reproduce a similiar problem – the first computer was an AMD running Fedora version 32 and the second computer was an HP running Fedora version 31 with obviously different graphics boards.

  1. Different Computers definitely provide much different experiences

However, it is true that there is something not optimized in the playback: In moving from a good Intel i7 Sandy Bridge (11 years old) to an AMD Ryzen 3700X, I noticed a marked increase in power in rendering, but not in timeline playback, in which I record the same freezes and drops in fps that I had before.

Very similar experience for me: I changed from a quad core Ivy Bridge i7 to a Ryzen 7 5700G, and it made hardly any difference to 4K playback..
Repeating myself – sometimes the program just has to run single-threaded.

Statistics from tests for me to get a better understanding:
#1 laptop - AMD Ryzen 7/1700; 16 processors; cpu MHZ=1374; bogomips=5988
#2 laptop - HP Intel I7-5500u; 4 processors; cpu MHZ=2394; bogomips=4788
#3 desktop- AMD Epyc (big) ; 128 processors; cpu MHZ=1196; bogomips=3999

#1 Framerate Achieved varied from about 22-37 (average about 30) with 29% cpu usage.
Note this is a Ryzen 7 similar to glitterball3 and played back smoothly.
Also, note that when I switched to 1/2 Proxy, it played back at full 60 fps.
#2 Framerate Achieved varied from about 22-46 (average about 27) with 75% cpu usage.
This computer is quite old.
#3 Framerate Achieved varied from about 3-60 (average about 42) with 3.6 %cpu usage.
This playback was so uneven and jerky on the 128 CPU computer that it was hard to watch.

9

I can not speak about performance because my computers are very old.
So I use Proxy feature.
My Proxy settings for a Project Format 1920x1080 @29.97fps RGBA-8bit 16/9:

Settings-> Proxy settings… Scale factor: 1/3 or 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | mov Video Preset–> Compression: mov.mov Bitrate: 1800000 Quality: -1 Pixels: yuv420pUsually I use mpeg coded but mov codec is more friendly for many frame rate, in my cases.

I dont have any problem with Alpha channel and Proxy.
Could you test it with a video file (Snow_overlay.mp4) I used for Merry Christmas and Happy New Year 2022 – Look at Me – The artist inside you – Artists Forum for Cinelerra GG Community , please?

You can see the video on Youtube https://www.youtube.com/watch?v=dbcIbygj5F4 and download it from Snow overlay
Mediainfo program and ffprobe indicate this video as yuv420p, not yuva420p.

For testing create 2 video tracks: on top video track put the Snow_verlay.mp4 video, on the bottom track any your video.

10

@igorbeg

I downloaded the file from mediafire and added it to the timeline above another video, but it just seems to be black without transparency. I also converted it to a .mov proxy with the same results.

I also just tried converting a .mov file of my own to a .mov proxy with your settings, but it got rid of the transparency and replaced it with a solid layer of black.

Project settings are 1920x1080 29.97fps, RGBA-8bit 16/9.

For good measure, I tried and was able to duplicate results with the AppImage.

11

@phylsmith2004

I do not understand what fix could be needed and am concerned about AppImage not working with vaapi on other distros/computers. So wondering if the problem is due to where I create the AppImage, I tested there and could see it was working. Then I moved the AppImage to another computer and could see it was working there also. I could not reproduce a similiar problem – the first computer was an AMD running Fedora version 32 and the second computer was an HP running Fedora version 31 with obviously different graphics boards.
Im using Kubuntu 21.10 - the next time I install from scratch, I will go through the steps again so that i can send through the errors - I did have to setup symlinks as Cinelerra was trying to find va api system files that were not in the expected location.

#1 Framerate Achieved varied from about 22-37 (average about 30) with 29% cpu usage.
Note this is a Ryzen 7 similar to glitterball3 and played back smoothly.
Also, note that when I switched to 1/2 Proxy, it played back at full 60 fps.
I should point out that I can play back 25fps 4k h264 footage in RGBA Float mode smoothly (until the source video file changes) as long as no effects are applied. As soon as I apply even the most basic colour corrections using, my system can no longer manage 25fps. Even my old i7 3632QM laptop can manage to play the 4k footage at 25fps until effects are applied.

12

@prolinuxnle

Sorry, I am a stupid human.
I had used a different Overlay Mode in Patchbay for the video track with Snow_overlay.mp4: Arithmetic…-> Addition.

The settings below work with Alpha channel for me. Could you try them, please?

Proxy #1 Scale factor: 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | qt Video Preset–> Compression: png.qt Bitrate: 0 Quality: -1 Pixels: rgba (or rgba64be)Proxy #2 Scale factor: 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | qt Video Preset–> Compression: magicyuv.qt Bitrate: 0 Quality: -1 Pixels: yuva444pProxy #3 Scale factor: 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | qt Video Preset–> Compression: openjpeg.qt Bitrate: 0 Quality: -1 Pixels: yuva420pProxy#1 seems to work fine on my old Notebook.

I used a video with Alpha channel downloaded from https://www.videezy.com/abstract/56411-explosion-fire-with-alpha-channel. It is a quicktime ProRes_yuva4444p 3840x2160 @24fps.

I hope it works also for you.

13

@igorbeg That works wonderfully, thank you! Sorry for the late response. 1/2 proxies work great for me (and now I can have transparent proxies :))

@glitterball3 @phylsmith2004

I should point out that I can play back 25fps 4k h264 footage in RGBA Float mode smoothly (until the source video file changes) as long as no effects are applied. As soon as I apply even the most basic colour corrections using, my system can no longer manage 25fps.
I have the exact same experience. Although, in my case, as soon as I have two video tracks playing at once, that halves the framerate.

I thought I fixed this on Dec. 15, 2021? I will look at it again.
I downloaded the PDF version of the manual about a week ago and had to modify the script to use relative paths. I assume the HTML version would be more up-to-date.

I could not reproduce Error making proxy
I was able to reproduce it across two Arch Linux systems running both the AppImage and version compiled using the AUR PKGBUILD. Im not entirely sure whether it relates to the format I chose to proxy, or me choosing a format, generating the proxies, deleting the proxies in my file manager, and then choosing the same file format and generating the proxies again. The error suggests that its looking for filenames for proxy files that should exist, but dont.

I say this because I generated VP9 proxies, then tried a VP8 proxy, but I wonder if Cin-GG detected those proxies as being the same format despite having a different codec because they share the .webm container. Also, Im choosing yuva420p when trying to generate the VP9 proxy, not yuv420p.

At any rate, Im still not able to create VP9 proxies with the same error, but am able to create .mov proxies on both systems now (though it doesnt have transparency, and the only option for the base .mov codec is yuv420p).

Thank you for all the other information and especially the benchmarking with different CPU types. Gives me a better understanding of the importance of single-threaded performance—although interesting that the 128-core CPU was able to hit a higher upper limit of FPS even with half the single-threaded performance of #2.

14

About Proxy.
I am glad it works for you (@prolinuxnle) too.
There are other presets for proxy with alpha channel could be useful.
Below some of those:

Proxy #4 Scale factor: 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | pro Video Preset–> Compression: prores_4444.pro prores_4444xq.pro Bitrate: 0 Quality: -1 Pixels: yuva444p10leProxy #5 Scale factor: 1/4 Rescaled to project size (FFMEG only) = unchecked File Format: FFMPEG | mkv Video Preset–> Compression: user_ffvhuff.mkv Bitrate: 0 Quality: -1 Pixels: yuva4–p yuva4–p–le@phylsmith2004 @andreapaz
Would it be possible to add a note in the Manual/Proxy for Proxy with alpha channel?

15

@prolinuxnle

I assume the HTML version would be more up-to-date.
The basis of both the html and pdf versions of the manual is the source code in Latex. So this is the most up-to-date version of the manual. Every end of month html and pdf are updated.

@IgorBeg

Would it be possible to add a note in the Manual/Proxy for Proxy with alpha channel?
Good idea.

EDIT

@phylsmith2004
I append Advanced.tex

Advanced.tar.gz

16

Off Topic:

I have noticed that my previous post About Proxy… (20/01/2022 10:18 am) is gone on the top of the posts. But I didnt touch any button. Am I wrong something?

17

@igorbeg, I saw you post something but never got to read it.

As long as were talking about getting banned, my home IP address (static) appears to have somehow gotten the first account I tried to register banned through WordFence; when clicking the activation link, I was told something along the lines of being blocked. I had to connect via a VPN to register this account, and am now able to connect/post from my home IP address. Not sure what that was about.

18

Off Topic:
Ooh, I just understood why my post went to the top. There is a Pull Down menu, with three options, on the bottom-right corner of the topic starter: Most Voted, Oldest, Newest.
The default is Most Voted. And I see the number 1 on the left side of that my post that, I think, it had been voted. Choosing Oldest option from Pull Down menu I can see the posts in chronological order.
I am stupid^2. Sorry,… again.

Off Topic 2:
Reading Proxy chapter on the Manual_2021-12-31, I think there may be a misunderstanding.
Quote from Manual:
When rendering, the original setting will be used to avoid loss of image quality.

It is important to note that when Proxy is enabled the Rendering will use proxy files, not the original files.
You can use that for testing because the time of Rendering will be faster, in general.
So, you have remember to disable Proxy (using the good** Ctrl+R** shortcut, if you want) when you want to create the final Rendering.

19

@igorbeg

Off Topic too ! Thanks.

Thanks for explaining why it was out of order because I had a hard time finding your note the other day.

Andrea has incorporated your Proxy examples for Alpha inclusion into the Manual as I am sure this will come up again. Also, he has corrected your latest suggestion when rendering to be sure to go back to highest quality.

20

@phylsmith2004 @andreapaz : Thank you!

@prolinuxnle : Sorry for all my OFF Topic in your Request.