Multiple GPUs / HW device selection / FFStream decode debugging / nvidea eGPU / VDPAU

General
1

Dear cinelerra community,

I came across this great video editor and believe it will be the only one currently supporting my uncommon setup. I use a Lenovo x230 + an external Nvidia GTX970 (so its an eGPU via the expresscard slot). On my current Ubuntu 18.04.4 LTS, the Nvidia-driver-435 proprietary drivers are installed, and work in my opinion properly (e.g. playing some games on Steam fully utilizes the eGPU). Furthermore, playing some mp4 footage from my GoPro Hero 7 (h264 codec) via vlc and mpv works flawless, since they both use VDPAU for hardware acceleration of the GTX970. Heres the output of mpv player:

ig0r@ig0r-ThinkPad-X230:~$ mpv --hwdec=vdpau testvideo.MP4 *
Playing: testvideo.MP4
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: Using non-standard frame rate 59/1
(+) Video --vid=1 () (h264 2704x1520 59.940fps)
(+) Audio --aid=1 --alang=eng () (aac 2ch 48000Hz)*
Using hardware decoding (vdpau).
VO: 2704x1520 vdpau[yuv420p]
AO: 48000Hz stereo 2ch float

The GPU is fully utilized, the video playback is stunningly flawless, exactly as expected.

As far as I can tell, everything is set up properly. In cinelerra in settings/preferences/performance I use VDPAU as a HW device. However, it seems that cinelerra has some troubles decoding the video using this dedicated HW device:

ig0r@ig0r-ThinkPad-X230:~$ cin
Cinelerra Infinity - built: Mar 31 2020 12:53:11
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

RenderFarmClient::main_loop: client started
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: Retry limit

This test, of course, is performed with the eGPU connected. Interestingly, when I now change the performance setting to VAAPI (while still the eGPU is connected), no complaints in the console, and the video is played properly in the viewer. It seems to me that my cinelerra does only see the internal Intel onboard GPU, but not the external one.

Now, of course I could just settle with this, but this would just waste a fairly powerful GPU on my setup. Furthermore, I think many people in the future would benefit of this interesting setup.

I documented a couple of answers of interesting questions within a similar OpenShot forum. However, they dont seem to care much. Feel free to check it out here, since it might already answer some debugging questions on my setup:

I would be very grateful if somebody can help me debugging this issue, since I really look forward in utilizing the eGPU while video editing. Im happy to follow up on any questions or document this issue further. It might be as simple as changing something like the GPU number inside some settings, or I just miss the obvious here, but searching the forum and google did not help so far.

I wish you all the best in those difficult times! Cheers!

2

@Ig0r

I discussed this with the programmer and there is 1 possibility he saw looking at the code but we have no way to test this with any of our setups. It would certainly be a shame to miss out on using your external GTX970.

Do you have the capability of compiling Cinelerra from source code? You do not have to be a rocket scientist to do this but just fairly computer knowledgable. If so, there is a patch attached that was just generated that may provide the only possible solution and it will take a lot of time and some back and forth work for you to even test this. How to compile from source code is documented in the manual which is on the cinlerra-gg.org website under documentation.

I will provide a little more information in a subsequent response but I have to look some things up first. And yes, we would definitely be interested in your results if you can get this to work so we can pass it along to other users.

3

@Ig0r

Well the patch did not attach so I am listing it here but the formatting will probably mess it up.

— ffmpeg.C.orig 2020-03-05 12:19:44.000360071 -0700
+++ ffmpeg.C 2020-04-12 12:34:25.825652597 -0600
@@ -1099,7 +1099,8 @@
if( hw_pix_fmt >= 0 ) {
hw_pixfmt = hw_pix_fmt;
avctx->get_format = get_hw_format;

  • ret = av_hwdevice_ctx_create(&hw_device_ctx, type, 0, 0, 0);
  • const char *device = getenv(CIN_HW_DEVICE);
  • ret = av_hwdevice_ctx_create(&hw_device_ctx, type, device, 0, 0);
    if( ret >= 0 ) {
    avctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);
    ret = 1;
4

Could you disable HW_device in Settings, that is in Settings->Preferences, Performance tab change Use HW Device to none. Quit out of Cinelerra, restart, and try to load your testvideo.MP4? And then check for terminal window messages. The reason I am asking is because of the following errors you received are odd as commented on next.

FFStream::decode: failed OK message and expected.
HW device init failed, using SW decode. OK even though not what you want.
file:/home/ig0r/testvideo.MP4 This is the file under question.
err: Invalid data found when processing input WHY does the software report error?

Of concern is this last message. If you switched to SW (software) decode, then why would there be an error? Also, it might help if you could put your test video somewhere
for me to download because of the Invalid data found when processing input (or email
to me privately if you do not want it seen elsewhere to phylsmith2017@gmail.com). Perhaps it is just a problem caused by the previous attempt to process via hardware but it would be good to know for sure.

I have reviewed some of the information in your other forum as you suggested. The output of your vdpauinfo looks normal as far as I know. After talking to GG, the first output line may be more important than we currently realize:

ig0r@ig0r-ThinkPad-X230:~$ vdpauinfo
display: :0 screen: 0

As you noted in the other forum commentary it seems to be an issue with ffmpeg this
is consistent with what gg noted when he studied ffmpeg.c this morning. Currently only
the default device is used. The patch previously attached allows for setting a
specific device instead to pass to ffmpeg. It looks like CIN_HW_DEVICE environment
variable will have to be set after the patch is applied before starting up Cinelerra.
But I do not know what GG says it has to be set to in your case (but will have to ask
if you even get this far). When he looked at the documentation for ffmpeg, there was
information about changing the X11 display device so it really could be something like:

It might be as simple as changing something like the GPU number inside some settings,

So that instead of display: :0 screen: 0 it would be display: :1 screen: 0. Meanwhile
I will try doing some experiments on intel hardware that might have 2 graphics cards.

5

@PhyllisSmith, thank you so much for coming back to this so quickly, I really appreciate your efforts!

Concerning the self build: Ill give it a try ASAP, never did that before but I guess Ill manage. Not quite sure where exactly the provided patch (thx!! awesome!!) comes into play while building, but I didnt go through the whole build docu yet. Ill keep you posted on that one of course.

Im happy to provide the testvideo - Ill share it via a google drive to your email, thanks for that! Also, I put various debugging screenshots there (since Im not sure how to place them here).

I did the test on setting the HW devide to none with the following output:

ig0r@ig0r-ThinkPad-X230:~$ cin
Cinelerra Infinity - built: Mar 31 2020 12:53:11
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

RenderFarmClient::main_loop: client started
FFStream::decode: Retry limit

So there is no complaint, but the playback is rather stuttery (compared to the VAAPI setting where at least the Intel GPU is being used) and the CPU is fully loaded too (so indeed SW decode mode).

I did some quick tests on changing the video driver in settings/preferences/*playback A from X11 to X11-XV to X11-OpenGL and also playing around with the default display (from to 0 to 1)

The last line *FFStream::decode: Retry limit *btw I see always, no matter what HW device I use.

Correct me if Im wrong, but I think mpv player also has FFMPEG as engine. If there would be an inherent issue with FFMPEG, it would also show up in using the mpv player I guess, which apparently doesnt generate any issues.

In any case, thanks so much for helping me out on that one. And great if you could somehow test this on some hardware in parallel. But as I said Im happy to test this with my setup as good as possible too.

All the best from Austria!

6

@PhyllisSmith, I just finished with my single-user build. I did the following, hopefully it was correct.

  • installed git and cloned the repository to a local drive (all via root)
  • The provided patch implementation was as simple as finding the ffmpeg.C inside the cinelerra subfolder of the cloned repository, making a copy just to be save (cp ffmpeg.C ffmpeg.C.orig) and deleting the lines starting with - and adding the lines starting with + in your patch code from above. The specific code section inside the ffmpeg.C now looks like this:

.
.
if( hw_pix_fmt >= 0 ) {
hw_pixfmt = hw_pix_fmt;
avctx->get_format = get_hw_format;
*const char device = getenv(CIN_HW_DEVICE);
ret = av_hwdevice_ctx_create(&hw_device_ctx, type, device, 0, 0);
if( ret >= 0 ) {
avctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);
ret = 1;
.
.

  • Since this was my first build of cinelerra, I followed the commands in the documentation:

./blds/bld_prepare.sh ubuntu
./autogen.sh
./configure --with-single-user
make 2>&1 | tee log
make install

  • The *make2>&1 | tee log *took quite a while (approx 15 Min), as far as I can tell everything went smooth, I didnt find any errors in the last section of the output log.
  • Start the build cinelerra via ./cin in the bin subdirectory → and wow it really starts! :slight_smile: So I think the build was successful with the provided patch code.However, the output log after loading the same testvide.MP4 gives similar results. First, I tried just via changing the performance settings (from blank to VDPAU to VAAPI) and again we get the HW device init failed output when I try to use VDPAU, and everything seems fine with VAAPI.

I also manually set the CIN_HW_DEV environment variable to vdpau, but with the same scenario:

root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEV=vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEV
vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# ./cin
Cinelerra Infinity - built: Apr 13 2020 11:28:12
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: Retry limit
FFStream::decode: Retry limit

I noticed that apparently cinelerra tries to init the HW device more often now, maybe a sign that the patch is indeed implemented in this build?

In any case, I think we did a step forward, even though the results are not quite there yet. At least I gained some know-how and can go forward with further patch implementations and single-user builds to debug this on my specific setup.

Let me know if you have some further ideas and how we should proceed best - looking forward to it!

All the best! Cheers!

7

EXCELLENT assist! Thanks for sending the test video becaise with software only we also get a retry limit but it loads so gg can look at that.

OK, the CIN_HW_DEVICE environment variable must be set to your actual eGPU display hardware device.

If you are using the Display variable to get to your device:

for example we would use: export DISPLAY=:0.0

for the primary display. You should: export CIN_HW_DEV=:1.0

for an alternate display on display 1, screen 0.

8

@Ig0r

Clarification (sorry). The environment variable should be CIN_HW_DEVICE, not CIN_HW_DEV. So as not to conflict with the original. The Settings->Preferences should still be set to vdpau.

9

@phyllissmith, thanks for the super fast follow up. I tried a couple of things now, all having the VDPAU in the HW device performance setting, but these two are the most representative I think:

root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $DISPLAY
:0.0
*root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEV=vdpau *
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEV
vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEVICE=:1.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEVICE
:1.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# ./cin
Cinelerra Infinity - built: Apr 13 2020 11:28:12
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

Failed HW device create.
dev:vdpau
err: Unknown error occurred
virtual int FFStream::decode_activate():
open decoder failed
virtual int FFStream::decode_activate():
cant open input file: /home/ig0r/testvideo.MP4
Failed HW device create.
dev:vdpau
err: Unknown error occurred
virtual int FFStream::decode_activate():
open decoder failed
virtual int FFStream::decode_activate():
cant open input file: /home/ig0r/testvideo.MP4
Failed HW device create.
dev:vdpau
err: Unknown error occurred
virtual int FFStream::decode_activate():
open decoder failed
virtual int FFStream::decode_activate():
cant open input file: /home/ig0r/testvideo.MP4
FFStream::decode: Retry limit

It seems to me that we try to point to display 1, even apparently we use display 0. This fits to the output (as you pointed out already) of vdpauinfo:

*ig0r@ig0r-ThinkPad-X230:~/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/cinelerra$ vdpauinfo *
display: :0 screen: 0

Also the nvidia driver panel seems to state display 0 and only one screen, so I guess 0. I changed the numbers of CIN_HW_DEVICE=: to 1.0 / 0.1 / 1.1 but thats no good either.

So for me the straight forward things to do would be:

root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $DISPLAY
:0.0
*root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEV=vdpau *
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEV
vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEVICE=:0.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEVICE
:0.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# ./cin
Cinelerra Infinity - built: Apr 13 2020 11:28:12
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: Retry limit

But I think it seems to be exactly what has been used as default, since the error messages are again the same.

Looking forward to continue this topic whenever you have some new ideas, Im happy to try.
All the best meanwhile!

10

I did not mean to imply that there was anything wrong with ffmpeg concerning your statement If there would be an inherent issue with FFMPEG, it would also show up in using the mpv player. It is just that Cinelerra interface to it may be a problem. (Although there are currently 9 patches that gg had to make to ffmpeg to get it to communicate to Cinelerra so there are problems).

Have not yet had time to do any more tests here but that is planned for today. I am very happy to hear that following the Manual directions worked without any problems as things change fast and a bunch of users and me try to keep it up to date.

11

@phyllissmith, no problem if there is some delay. Im anyway really stunned on the response time :slight_smile:

BTW let me know if theres a way to donate or buy you a beer, since Im a big fan of this community based concept and I appreciate the time and efforts you guys but into this things.

12

@Ig0r

4 things:

#1 your test video if yuvj420p – note the j (also frame rate 59 is unusual).You can see this by running:

$ ffprobe testvideo.MP4

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x4da5880] Using non-standard frame rate 59/1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from testvideo.MP4:
Metadata:
major_brand : mp41
minor_version : 538120216
compatible_brands: mp41
creation_time : 2020-02-29T13:29:01.000000Z
firmware : HD7.01.01.90.00
Duration: 00:00:11.03, start: 0.000000, bitrate: 60417 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p

#2 our testing on an intel device with 2 graphics board shows below Successfully created a VDPAU device AND Unsupported sw format: yuvj420p so that is why vdpau does not work here and I am guessing for you also.

in libva info: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7f0de4026e80] Successfully created a VDPAU device (OpenGL/VAAPI backend for VDPAU) on X11 display :0
[AVHWFramesContext @ 0x7f0de7388900] Unsupported sw format: yuvj420p
[h264 @ 0x7f0de73a92c0] Failed setup for format vdpau: hwaccel initialisation returned error.
[h264 @ 0x7f0de73a92c0] Invalid return from get_format(): vdpau not in possible list.
[h264 @ 0x7f0de73a92c0] decode_slice_header error
[h264 @ 0x7f0de73a92c0] no frame!
[AVHWFramesContext @ 0x7f0de73bdfc0] Unsupported sw format: yuvj420p
[h264 @ 0x7f0de73c9300] Failed setup for format vdpau: hwaccel initialisation returned error.
[h264 @ 0x7f0de73c9300] Invalid return from get_format(): vdpau not in possible list.
[h264 @ 0x7f0de73c9300] decode_slice_header error
[h264 @ 0x7f0de73c9300] no frame!
[AVHWFramesContext @ 0x7f0de73b3800] Unsupported sw format: yuvj420p
[h264 @ 0x7f0de73b8d40] Failed setup for format vdpau: hwaccel initialisation returned error.
[h264 @ 0x7f0de73b8d40] Invalid return from get_format(): vdpau not in possible list.
[h264 @ 0x7f0de73b8d40] decode_slice_header error
[h264 @ 0x7f0de73b8d40] no frame!
[AVHWFramesContext @ 0x7f0de73b3800] Unsupported sw format: yuvj420p
[h264 @ 0x7f0de7426dc0] Failed setup for format vdpau: hwaccel initialisation returned error.
[h264 @ 0x7f0de7426dc0] Invalid return from get_format(): vdpau not in possible list.
[h264 @ 0x7f0de7426dc0] decode_slice_header error
[h264 @ 0x7f0de7426dc0] no frame!
FFStream::decode: avcodec_send_packet failed.
file:/tmp/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/tmp/testvideo.MP4

#3 I forgot to relay this. In your Cinelerra directory under the subdirectory ffmpeg, there is a file called decode.opts which is used during the load video operation. Change the line loglevel=fatal to loglevel=verbose. Restart Cinelerra from the command line and then load testvideo.MP4 and you should see messages similar to the above. Do you see Successfully created a VDPAU device with X display 0 and unscupported format yuvj420p? If so, then whichever graphics card is in use (Display 0, I guess from you vdpauinfo output) does not support yuvj420p.

#4 We are now building Cin on a Fedora computer that has a GTX970 and will see if it does support yuvj420p – it is not clear from the vdpauinfo output (yours and ours on an intel computer) if that is supported.

13

@Ig0r

Sam (@Sam) maintains the website in Europe and it is not set up to take donations but perhaps he could use a virtual beer – you would have to ask him. GG and I are not interested in donations of any kind because we just do this for fun and do not want to feel obligated to work on any specific thing.

14

@Ig0r

We do not work for profit, but on a voluntary basis for the common good, and so do I. Nevertheless we want to offer our community the possibility to spend a beer or a pizza for the server costs and infrastructure. We are working on making this possible in the future as well.

15

One more thing. Our intel computer vainfo reports the following (see below) supported devices and you can see on the last line that JPEGBaseline (which we are assuming includes yuvj420p) is supported. Your vainfo does not include this. So vaapi works on our computer. I can not understand why vaapi works on your computer but am guessing that it is using the other graphics board?

$ vainfo

Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264MultiviewHigh : VAEntrypointVLD
VAProfileH264MultiviewHigh : VAEntrypointEncSlice
VAProfileH264StereoHigh : VAEntrypointVLD
VAProfileH264StereoHigh : VAEntrypointEncSlice
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
VAProfileJPEGBaseline : VAEntrypointVLD

When we booted a Fedora computer with only the one GTX970 board, it also reports that yuvj420p is not supported. Vaapi does not work to load your test video on that single board computer.

I have not studied your last note yet but will do so.

MY CONCLUSION - the only problem is that yuvj420p is not supported by your Graphics card. And again am guessing that it has been using your GTX970 board all along (based on your previous note that I glanced through, where Display 0 really is your eGPU.

16

@Ig0r

And just one last thing. You inclused the following information on mpv play output as follows*:*

Using hardware decoding (vdpau).
VO: 2704x1520 vdpau[yuv420p]
AO: 48000Hz stereo 2ch float

Note that it reports that it is hardware decoding using vdpau, but the VO reports opengl and yuv420p not yuvj420p so it is possible that the use of the graphics board is really via OpenGL.

I have tried to find any reference on the internet that supports my conclusion that the GTX970 simply does not support yuvj420p but could not. GG agrees that that is as far as it goes and believe we have found the only source of the problem and the problem is solved (the patch was not even necessary).

17

About MPV/mplayer using your GTX970 board and vdpau. GG had a chance to look at the source code and although he did not get into great detail, it looks like the code does not go through FFmpeg but rather uses OpenGL/shader via an API. Therefore the eGPU does not get a direct chance to reject the yuvj420p format. This was probably a major task – the MPV github is very active with more than just 1 developer (and ongoing changes are being made almost daily with wm4@nowhere a funny guy who comments are so funny).

18

@sam & @phyllissmith, I admire your philosophy! Nevertheless, keep me posted on any news on how to sponsor a beer/pizza. I guess the best way to pay back the community is to come up with an article on reddit and describe my fight + (hopefully) solution on such an eGPU setup for video editing on linux (since in the end I spend quite some time on this now, people may benefit from some learning).

@phyllissmith, thanks for providing so much info on your testing! Ill go through point for point now.

#1: Yes, I can confirm the same output, showing me the yuvj420p stream info including the non-standard frame rate. A quick note: I guess many action cameras nowadays use such a frame rate, no clue when such high frame rate become standard.

#2: While I do can execute va_openDriver(), so its kind of waiting for a command, I can not get either the Successfully created a VDPAU device OR Unsupported sw format: yuvj420p anywhere yet. I guess the va_openDriver() works since there is the intel GPU running in parallel.

#3: This is related to #2, so no I do not see any successful VDPAU device creation after changing to verbose logging. The error messages rather stay exactly the same:

root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $DISPLAY
:0.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEV=vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEV
vdpau
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# export CIN_HW_DEVICE=:0.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# echo $CIN_HW_DEVICE
:0.0
root@ig0r-ThinkPad-X230:/home/ig0r/cinelerra_singleuser_1/cinelerra5/cinelerra-5.1/bin# ./cin
Cinelerra Infinity - built: Apr 13 2020 11:28:12
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: avcodec_send_packet failed.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: failed
HW device init failed, using SW decode.
file:/home/ig0r/testvideo.MP4
err: Invalid data found when processing input
FFStream::decode: Retry limit

#4 & subsequent post: Yes, my vainfo does not include the VAProfileJPEGBaseline : VAEntrypointVLD, but interestingly Im also able to open and process the testvideo.MP4 when I disconnect my eGPU, and setting everything to vaapi (so it at least uses hardware acceleration of the onboard intel gpu). There is a screenshot attached, you also see that the CPU loading is rather low (compared to SW decode), which is good!

Related to your last two posts: Oh boy, what a bummer. I was about to mention the superb performance of mpv using this test video. Your explanation makes a lot of sense, thank you very much for digging into all this!

mpv player was keeping my hopes up high, but it really does look like that the GTX970 does not support this yuvj420p out of the box. Could do do me a favor and provide me a test video which uses a format which is supported by this GPU, please? (Best via the already shared google drive).

If this does work, I would need to change the standard format of GoPro (haha, yeah right..) or use ffmpeg to do some kind of conversion (thats really the worst case scenario for me, since it requires a lot of time and disk space and especially proper settings of the ffmpeg conversion).

Additionally, if this test is successful we can go ahead and close this thread.

Thanks so much for all your work you have done here!
Cheers!

cinelerra_vaapi_NOeGPU.png

19

I uploaded tutorial.mp4 to the shared drive and it is also at:

https://streamable.com/0c9mrv

Just about any .mp4 file will work. There is a Settings choice of Transcode in Cinelerra to convert a file to something besides your yuvj420p format. This is for convenience sake as doing this from the command line just using ffmpeg is really faster.

There is no problem using the non-standard frame rate; I just noted it out of curiosity and was amazed that they did not at least just use 60 instead of 59.

The verbose loglevel was put in the wrong file because that I was I suggested. That is the source file but not the actual running file in /bin subdirectory. Correct spot is in {cinelerra_path}/bin/ffmpeg/ decode.opts (sorry about that). It is worth doing as you load tutorial.mp4. You should see:

[AVHWDeviceContext @ 0x7fff1c0ba540] Successfully created a VDPAU device (NVIDIA VDPAU Driver Shared Library 440.64 Fri Feb 21 00:41:34 UTC 2020) on X11 display :0
[h264 @ 0x7fff1c5b2a00] Reinit context to 1440x1088, pix_fmt: vdpau
[h264 @ 0x7fff74038cc0] Reinit context to 1440x1088, pix_fmt: vdpau …

(and no switching to SW decode)

I

20

@phyllissmith, wow THANKS SO MUCH for solving this puzzle!!

Indeed, when using your test video (which is .MP4 but yuv420p instead of yuvj420p) the verbose output says

[AVHWDeviceContext @ 0x7f8f5c050640] Successfully created a VDPAU device (NVIDIA VDPAU Driver Shared Library 435.21 Sun Aug 25 08:06:02 CDT 2019) on X11 display :0.0
[h264 @ 0x7f8f5c588480] Reinit context to 1440x1088, pix_fmt: vdpau
[h264 @ 0x7f8f7c6e8f40] Reinit context to 1440x1088, pix_fmt: yuv420p

Cinelerra is fully supporting the VDPAU HW device, so the CPU utilization is fairly low at ~30%, compared to 100% when software decoding.

However, when shutting down the eGPU and switching to vaapi with the intel onboard gpu cpu utilization is even lower!! At ~15%!! At least thats the case for the provided testvideo (yuv420p), and unfortunately this is not true for the GoPro footage (which is yuvj420p of even h.265). I suppose this is due to the extra GPU traffic on the express card slot device?

I need to point out here how amazingly good cinelerra performs compared to Olive, KDENLIVE and OpenShot (lets dont talk about DaVinci resolve, since they dont even support mp4 files in linux (but in windows!?)). All of them utilize the CPU at least 50-60% while just previewing the freshly loaded yuv420p testpiece on my machine, compared to 15% when previewing in cinelerra - amazing guys, chapeau!

Anyhow, what a disappointment, that apparently the GoPro Hero 7 Black (when used in compatibility mode = h.264 + HEVC) saves the videos in yuvj420p, which (of course) can not be changed and apparently is not supported by my nvidia GTX970 to actually do HW decoding (although I could not really find any details on that either, yet..).

I was aware of the fact that the GTX970 will not support h.265 (check the picture attached from the nvidia page)(btw, the GTX960 does seem to support h.265), but I was not prepared to run into these kind of issues once using h.264 in the first place. One letter in the used color space made all the difference, and my GPU practically useless, unless I do some extra conversion via ffmpeg (did I get it right?). In case of people having some additional input here Id be glad.

Well, it was indeed a big learning for me (and hopefully others) and my solution might as well be to get a newer Nvidia card as my eGPU (it seems Ill go for something like the 1050, since it is fully supporting h.265 which Ill set in the GoPro as well). In the end I just wanted to edit my GoPro footage - didnt expect it to be that difficult.

In any case, once Ill have the new setup running Ill stick to cinelerra, since the tests above already showed by far superior GPU utilization compared to other free video editors out there.

Id like to thank you, @phyllissmith, again for all your patience and amazingly fast and competent support.

All the best!

nvidia_sdk_1.png