AppImage excessive RAM useage

Help & Support
21

So far, I cannot duplicate your problem. On Mint 20.1 using the AppImage 2021-02, I have rendered a H.265 300 MB file into DNxHR_HQX format, which became a 6.6 G file. Loading this in the AppImage made not much difference compared to the 2020-10 static release. Loading and playing the DNxHR video in the AppImage, ps_mem reported 1.7 G used, the same video in the 2020-10 static took 1.5 G.The same sizes were reported for the original H.265 file.

For this I kept the composer and viewer windows closed (composer open added 0.1 G), and hw device set to none .

The DNxHR file specs are: format 2016p23.976 , bitrate 700 Mb, pixel format yuvp10le .

I also tried a smaller DNxHR file, 1080p29.97 created by Adobe, same. Not much increase in size .

Any suggestions on how to duplicate it? How big were the files you were importing, and what is the pixel format, color space and RGBA settings?

22

Mat, sorry for the delay, but I needed time to do a few more tests. Im going to try to put all the information in this post.

DHxHR_HQX is available in the 20201030 release. In the render window File Format FFMPEG, in the small box select qt from the dropdown list. Click the video spanner icon and in the compression window dropdown list select dnxhd qt, click Video Options View, highlight profile, click the profile button and select dnxhr_hqx.

The Use HW Device window is blank, I have never needed to change it. I set it to vdpau to test, but it made no difference. It is the same in both versions.

I have rendered DNxHR_HQX with both versions, I have been doing so for a while in 20201031.

I am importing 25fps 4:2:0 full range 10-bit F-log, all intraframe, HEVC at 400mbps, which I then transcode to DNxHR_HQX at 720mbps. I was trying transcoding smaller sections of movie to DNxHR444 at around 1700mbps which worked well, but when I tried importing the 92 transcoded clips the Appimage stopped well short and froze. I have a widget on the KDE panel that shows RAM useage and it was showing 55.71Gb. I then re-transcoded to DNxHR_HQX and tried again, it loaded them all but still filled the RAM and froze the system when generating proxies, the only escape is to press the power button and switch off. I temporarily set up a swap partition. It loaded the transcodes, but froze when generating proxies because it filled that as well, nominally 128Gb of memory!

Both times I transcoded in 3 sections, but it is necessary for all the clips to be on the timeline for editing. When I import the 92 transcodes to single user 20201030, the widget indicates 28.03Gb and there is no problem generating proxies.

DNxHR_HQX is 10-bit and yes, I am using RGBA-Float. I have tried various loading strategies but all show roughly the same RAM useage.

Edit: I meant to add that the 20201031 Appimage does not have the problem, but the unnumbered For Newer Distros and the 20210228 both do.

23

In reconstructing GIT versions, I have found that around 12/01 after the interlace and aspect modifications were added, Memoryusage increased by 15%. Sometimes when new constructs are created, it requires to de-construct them also so that the memory is returned. Not sure if this is the case here though but if it is, then maybe it can be changed to give up that increased memory (will ask Andrew when he gets back).

But this only accounts for part of the increased Memory usage – there is another 10% increase by the time we get to February 28 so I will continue to track that done and see if it can be changed.

24

As a final comment, for now at least, I am now using the 20201031 Appimage and it has no sign of the problem. If I import all 92 DNxHR_HQX transcoded files it shows 28.03 Gb RAM used, which rises to 32.54 while generating proxies.

25

@DeJay, I have not forgotten but was fully occupied. Should have more time after Tuesday. Testing so far however on Mint 20.1 between the 2020-10 static version and the 2021-02 AppImage I see about 20% more memory usage, so far less than what you are seeing.

26

@DeJay, I have not forgotten but was fully occupied.

I wasnt suggesting you have, I am just clarifying how things are currently, especially as it seems in hindsight that the thread title is inaccurate and it does not seem to be Appimage that has made the difference.

Before I retired, in a different field, I was the fixer and I know how important every little piece of information can be, as long as it is reasonably accurate.

27

Although I also have been looking at this in the last 3 days, I still have not come any closer to an answer. The strange thing is as I compile from GIT, the memory usage goes up even on 2020-11-26 when the only changes were the addition of more format files for rendering choices and additional aspect ratio choices. Neither of these should affect memory usage.

28

@DeJay

@MatN

I think I found the problem and so now there is a test image at:

https://cinelerra-gg.org/download/images/cin-x86_64-TestOnly.AppImage

which contains all of the current mods except what seems to be the line causing the increased memory usage. The line commented out is in fileffmpeg.C and is: ff->video_probe(1); and its usage appears to be only for interlace reporting so should not affect usage.

When/if you have time, can you verify that this solves the problem for the Dnx stuff. If it does, then we can go from there.

29

Hi Phyllis, yes you have found the problem. The test image loaded all 92 transcoded clips, DNxHR_HQX 2160p @ 728Mbps, showing A RAM useage of 27.78Gb, nearly the same as 20201031. When generating proxies it went up to 32.34. Again, nearly the same.

Thanks.