Rotation meta data usage in viewer/compositor

General
1

@cinadmin Unfortunately I cant post my question directly here, as the wordpress spam protection thinks my post is suspicious.

Please find the file attached, and if possible post this for me, since I cant do it.

Thanks and best regards,
Christopher

(Edited by PYSmith BUT it would not let me put it in either so you will have to read the cinel.txt file - it does not like the technical data out of ffprobe)

Dear Cinelerra-gg community,

I recently came across a really good setup for the GoPro mount while downhill mountain biking: A chest strap. I can highly recommend this, especially with any wide angle camera since it is an awesome perspective.

In this setup, the GoPro (in my case) is mounted upside down, and it stores the orientation information (if recognized correctly) in its meta data, check this after

ffprobe GH011430.MP4 (check the attached file for results of the ffprobe)

…

Metadata:
creation_time : 2020-07-05T10:30:19.000000Z
handler_name : GoPro SOS
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 100359 for input stream 3
Unsupported codec with id 0 for input stream 4

The mpv player in my ubuntu 20.04 linux machine uses this information to flip the image correctly (while still using full hardware decoding, so perfectly smooth image), while cinelerra does not, apparently?

Could you guys please give me a guide whats the best approach here? In the viewer you cant apply any filters, and even later on it is from a resource point of view a waste to use those filters, I guess?

By the way, sometimes the camera gets mixed up and does not store this meta data information correctly, so an easy workaround for mpv player to play it smooth and with the correct rotation is to just ffmpeg copy to a new file with new meta data information, so like this:

ffmpeg -i input.MP4 -c copy -metadata:s:v:0 rotate=180 output_rotate.MP4

This way you dont loose any information and it is super fast; the old file you may just delete.

Thanks a lot! Cheers!

cinel.txt

2

@Ig0r

Super great idea! But dont you still get a lot of motion even with a chest strap?

Could you guys please give me a guide whats the best approach here? In the viewer you cant apply any filters, and even later on it is from a resource point of view a waste to use those filters, I guess?
Probably someone has a better idea then we do or to just use the Rotate plugin and watch in the Compositor since the Viewer has no real capabilities. What gg says is to just run the ffmpeg command line first - it works and its fast and then you will not have to use a Rotate plugin which takes cpu cycles.

Hey, how about sending us one of these as an example so we can see how well it worked and also see how it is upside down in the Viewer in case gg gets an idea? Email or dropbox or share it somehow (phylsmith2017@gmail.com). And of course we want to see how hard the mountain biking trail is (we just bike around the lake here!).

3

@phyllissmith

Hey!

Im a bit confused - so gg said I shall run the ffmpeg command line first. But thats exactly what I did, since thats the straight forward solution, and I just checked again, it does not work apparently. :face_with_raised_eyebrow:

I uploaded an original raw video where I added the rotation via this command. So in mpv player it does play in the correct orientation, but in cinelerra it does not (neither in viewer, nor in compositor). I send you the link to the folder via email. Its the smallest file I could find, and it does not contain any good trails unfortunately, just some lame forest roads.

Concerning the GoPro mounting: From generation 7 upward it has a stabilization build in (I guess via Gyro-Sensor + Softwarecompensation) which works reasonable well for those kind of trails.

Lets follow up on this a bit more.
Cheers!

4

@Ig0r

Im a bit confused – well, I confused you. I misunderstood that when you ran the ffmpeg command, it actually rotated the frames but instead it looks like it just changes the metadata and that is why it is so fast. The problem with using metadata in Cinelerra, is that it will be slower because it would use the metadata to rotate every single frame each time rather than rotate the whole thing and be done with it.

Currently CinelerraGG does not handle metadata or sidedata. GG is looking at that code now but it looks like a can of worms and obliviates the raw data.

So the best real solution we came up with is just to load the original file into Cinelerra and add the Rotate plugin set to 180 degrees. Then if you want to see it in the Viewer so it is not upside down, just make a Clip of the whole thing and drag the clip to the viewer. Not the most eloquent of solutions if you have a lot of media and your workflow is to load it into Resources only, since you have to put it on the timeline to add the Rotate plugin, but seems to be a reliable solution.

I did try using -vf rotate filter with ffmpeg command line but it was slow, came out badly, and actually aborted before finished.

5

@ig0r
I was supposed to mention that as shown in your ffprobe output the side
data reports displaymatrix: rotation of -180.00 degrees. Which gg
says means that the data is not actually rotated but is just displayed
as rotated.

6

** Posted by: @phyllissmith **@ig0r
I was supposed to mention that as shown in your ffprobe output the side
data reports displaymatrix: rotation of -180.00 degrees. Which gg
says means that the data is not actually rotated but is just displayed
as rotated.
@phyllissmith,

yeah, I think thats the whole point in having the metadata. I was just wondering if cinelerra can handle this stuff as mpv or vlc do it, since thats quite awesome; but apparently this is not the case. Maybe a possible feature in the future?

I think neither of the workflows you suggested suit me, currently. As you know, there is the ongoing discussion about fully utilizing my GTX1050 (Page 4 | Multiple GPUs / HW device selection / FFStream deco...) so doing all this rotation stuff via cinelerra is not the fasted for sure, since somehow the performance is not there were it should be, yet.

Ill rather look for a lossless ffmpeg approach (which perfectly uses my nvidia gpu) to rotate all videos instead, so via a fully hardware supported ffmpeg transcode command.

If successful Ill share this in a follow up post here, since people might run into similar issues.

All the best!

7

@Ig0r

Just a short followup concerning metadata. I was just wondering if cinelerra can handle this stuff as mpv or vlc do it, since thats quite awesome; but apparently this is not the case. Maybe a possible feature in the future?

Possibly GG will implement this in the future. He looked at how ffplay does it and it goes out of its way to special case this, as Im sure mpv/vlc do. He found out that his new smart phone also has the rotate problem so is more motivated to make a Cinelerra preference to check the metadata. But I do not know when this might be available so your pursuing other avenues is best.

We have made no progress on understanding the eGPU gtx1050 usage failure with Cin.

8

@Ig0r

metadata. I was just wondering if cinelerra can handle this stuff as mpv or vlc do it, since thats quite awesome; but apparently this is not the case. Maybe a possible feature in the future?

Auto rotate ffmpeg media has been added to the GIT for CinelerraGG and will be in the July 31 builds. It is rotated automatically in the display (timeline, compositor, viewer) and also in the render. If you do not want it to be rotated by default, you will have to disable the Flag in the Settings-> Preferences. This metadata also shows in the Resource window for the media in Info->Detail. You will see:

1 video stream
vid0 (0), id 0x00001b:
video1 h264 2704x1520 59.94 pix yuvj420p
color space:bt709/ range:pc
3824+0 frms 63.80 secs 0:01:03.80
rotation angle: 180.0

9

Auto Rotate → OK
Notes on Auto Rotate:
1- You have to restart CinGG to affect the checked/unchecked flag. (make the restart automatic?)
2- Some of my videos flipped 90°, I had rendered them after straightening them with Rotate plugin. These videos, I think, keep the original metadata 90°, because, with Auto Rotate active, they are now flipped. So for old projects where the rotation metadata was present, and which have been straightened and rendered, they must now be opened with Auto Rotate disabled. (no problem, Im just pointing it out)

10

@andreapaz

Thanks for pointing this out and testing as I did not realize this so the extra warning is needed. Also, this needs to be added to the manual (hint, hint).

11

@phyllissmith @andreapaz

Unfortunately I cannot reproduce the success of this new feature. I just downloaded the July static single user build.

I found the option in appearance in the lower left (see screenshot attached) and tested it on yet another MTB footage of mine where I had to change the meta data as explained above for mpv player to display it correctly. However, neither setting (on and of) nor vaapi via internal intel or vdpau via external GTX1050 worked in cinelerra for me.

In the screenshot youll also see loads of

.
.
[in @ 0x55906b182c00] Changing video frame properties on the fly is not supported by all filters.
[in @ 0x55906b182c00] filter context - w: 2704 h: 1520 fmt: 12, incoming frame - w: 2704 h: 1520 fmt: 46 pts_time: 144.578
.
.

messages (decode ffmpeg from fatal to verbose), which do not show up if I play the footage without any meta data (which I still have).

Any thoughts on that?
Let me know if I can support further, unfortunately this new feature is not yet usable for me.

Take care! Cheers!

auto_rotate_debug_1.png

12

@Ig0r

tested it on yet another MTB footage of mine where I had to change the meta data as explained
That does not make any sense to me. You had sent me a test file which showed ffprobe of metadata rotation of -180 degrees (mtb_rotation_manual_1.MP4) and

after gg made the program code change, it correctly and automatically rotated the image. You can see this in the attached png. I have no explanation yet and I tried both X11 and X11-OpenGL as the driver. We both tested again just now. Something must be different but obviously you have the latest code installed or you would not be able to see the flag. I will experiment some more. Is there possibly something different about the metadata rotation value that our code does not check for?

rotation.png

13

04/08/2020 12:34 am@Ig0r(This message was put in the wrong place so I just moved it).OK, I see the problem. I was not using vaapi, but rather none and I also see no rotation with using hardware devices. I will ask GG if there is anyway to fix it in that case. Bummer.The message you see is a warning and not much help.

14

@phyllissmith

As I mentioned in a different post here I tested the current version of cinelerra on my fresh ubuntu 20 install.

Im very happy that, as far as I can tell, the auto-rotate is now working properly also for vaapi and vdpau accelerated hardware settings.

I think we can close this topic here, since you guys did a great job fixing this issue. Big thanks to you!

Cheers and all the best!

15

OK, thanks for the feedback. Consider it closed because I have not been able to figure out how to actually close it!