fbpx
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC: Blending higher frame rate footage

Blending higher frame rate footage 10 May 2020 19:19 #107084

  • Itstonybabyvlog
  • Itstonybabyvlog's Avatar Topic Author
  • Offline
  • Junior Boarder
  • Junior Boarder
  • Posts: 29
  • Thank you received: 0
So I’ve been struggling with something that I haven’t found a clear answer to. I want to edit/export at 24fps, but I can’t slow footage shot at 24fps for slow mo. I can shoot at 60fps, and drop in a 24fps timeline, but then FCPX drops every 1.5 frames (1 then 2).

So all the advice I get is “shoot your footage at 24fps and shoot your slow mo at 60fps” which is all fine and dandy, but when your in the field, you don’t always know what’s gonna be slow mo or not (its a travel vlog, not a movie where every shot is planned).

Then it hit me... a 24fps video has more motion blur because each frame captures more movement, ergo each frame will be slightly blurred as a result. 60fps has less motion blur because each frame has captured a shorter period of movement. And dropping a 60fps video into a 24fps timeline will result in a very sharp, but choppy image.

We have the optical flow button, which when used on a 24fps piece of footage, will take a frame from each side and combine it to create the illusion of slow motion, adding frames to fill in the gaps. So why can’t I do the same thing with 60fps footage but in reverse? Take two or three frames, combine them into one frame, and put a 60fps video and make it a 24fps video? Each frame of the 24fps would consist of either 2 or 3 of the original 60fps footage, and therefor have a slight blurring, just as if it had been shot at 24fps. Instead of dropping frames to conform to the 24fps timeline, combine all the frames with something like optical flow to fit the 60fps video into the 24fps.

This eliminates all my problems, I can shoot at 60fps, slow down what I want to slo mo, and for regular footage, combine the 60fps footage to get that natural motion that you get with shooting at 24fps.

Except... how do I do this? Is it even possible?

Please Log in or Create an account to join the conversation.

Blending higher frame rate footage 11 May 2020 04:22 #107090

  • FCPX.guru
  • FCPX.guru's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • bbalser.com
  • Posts: 3591
  • Karma: 34
  • Thank you received: 497
1- Shoot everything at 60fps.
2- Edit everything at 24fps.
https://support.apple.com/guide/final-cut-pro/change-clip-speed-ver40b00150/mac

That's what I do, no problems. I also export 59.94 for broadcast, MPEG-2, no issues.

Please Log in or Create an account to join the conversation.

Blending higher frame rate footage 11 May 2020 06:02 #107096

  • Itstonybabyvlog
  • Itstonybabyvlog's Avatar Topic Author
  • Offline
  • Junior Boarder
  • Junior Boarder
  • Posts: 29
  • Thank you received: 0
Well that doesn’t fix the problem. Your dropping 60% of the frames that were shot at 60fps... and 60fps doesn’t have any motion blur. So you end up with sharp, jittery footage.

Please Log in or Create an account to join the conversation.

Blending higher frame rate footage 11 May 2020 12:59 #107104

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1554
  • Karma: 27
  • Thank you received: 332

Itstonybabyvlog wrote: Well that doesn’t fix the problem. Your dropping 60% of the frames that were shot at 60fps... and 60fps doesn’t have any motion blur. So you end up with sharp, jittery footage.


No timeline with non-multiple frame rates can be perfectly rate-conformed. In some cases it's good enough that most people won't notice it, but it won't be like shooting 24/48/72 fps. For that reason the ability of some cameras like the Panasonic S1H to shoot at 48 fps for 24 fps distribution is highly prized.

Rate conforming 60 to 24 usually works better than 30 to 24, but it's not perfect. The best stop-gap solution is use FCPX optical flow rate conforming -- which is *different* than optical flow retiming.

To use optical flow rate conforming put your 60 fps material in a 24 fps timeline, then in the FCPX video inspector, scroll to bottom and under Frame Sampling select Optical Flow. Then render that clip in the timeline and examine the playback smoothness.

This normally greatly improves it at the cost of significant CPU time, In some cases it can cause morphing artifacts so you must check for those.

As FCPX.guru said, you can often get by without doing this. The issue is not what it looks like for *your* local playback but what it looks like at the end of the distribution chain - after uploading, transcoding, streaming, and playback on the client's device or TV. Is it something only *you* would notice because you're lookiing for it, or is it something the viewer would notice, unprompted? A lot of broadcast TV right now uses 3:2 pulldown which is obvious when going frame-by-frame on a DVR. It's not as smooth as true 24 fps playback, but almost nobody notices it.

Please Log in or Create an account to join the conversation.

Last edit: by joema.

Blending higher frame rate footage 11 May 2020 15:36 #107107

  • Itstonybabyvlog
  • Itstonybabyvlog's Avatar Topic Author
  • Offline
  • Junior Boarder
  • Junior Boarder
  • Posts: 29
  • Thank you received: 0
woah! Joema... did you just tell me that what I thought didn't exist.... actually exists??

my previous post:
So why can’t I do the same thing with 60fps footage but in reverse? Take two or three frames, combine them into one frame, and put a 60fps video and make it a 24fps video? Each frame of the 24fps would consist of either 2 or 3 of the original 60fps footage, and therefor have a slight blurring, just as if it had been shot at 24fps. Instead of dropping frames to conform to the 24fps timeline, combine all the frames with something like optical flow to fit the 60fps video into the 24fps.

What is this "Optical flow rate conforming" you speak of? Is it voodoo magic? Is it alien technology? Does it warp the fabric of the cosmos?

Or does it do exactly what I suggested above? See, the problem I have with dropping 60fps in a 24fps timeline is the dropping of the frames, and the loss of motion blur. And the first advice I get is to drop my 60fps footage in a 24fps timeline :S :S :S :S :S ergggghh!! (bang your head)

But if this "Optical flow rate conforming" you mention does what I hope it does, combining frames in a 60fps clip together to create 24 frames that are made up of combined frames from the 60 frames of the 60fps... then thats exactly what I need!! I imagine the combined clips would be a bit blurry (motion blur as if shot at 24fps), and no jittering because technically, all the frames are used.

I know a lot of you said nothing is perfect... and yes, Ideally I need to shoot my 24fps footage at 24fps natively, and shoot my slo mo at 60fps so I can slow it down in post. But it just aint gonna happen that way! Travel vlogs are not scripted. So I need to be able to have the flexibility in post to say "Hey! that footage would look great in slo mo!" and not "carp! I can't slo mo it, I shot it in 24fps).

I'm going to do some research on this "optical flow rate conforming", but if you know any details or if anyone else knows any details on specifics of how it works, or how it changes the clip to fit the 24fps timeline, I would love to hear about it.

I hope... it doesn't simply drop frames to make it fit the timeline.

Please Log in or Create an account to join the conversation.

Last edit: by Itstonybabyvlog.

Blending higher frame rate footage 13 May 2020 18:45 #107145

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1554
  • Karma: 27
  • Thank you received: 332

Itstonybabyvlog wrote: ...Ideally I need to shoot my 24fps footage at 24fps natively, and shoot my slo mo at 60fps so I can slow it down in post. But it just aint gonna happen that way! Travel vlogs are not scripted. So I need to be able to have the flexibility in post to say "Hey! that footage would look great in slo mo!" and not "carp! I can't slo mo it, I shot it in 24fps)....


We frequently debate this issue in my documentary team. The dilemma is lots of people want (or THINK they want) 24 fps for a final product but they also want smooth slow motion, yet most cameras cannot shoot 48 fps and 60>24 rate conforming isn't perfect.

IMO the solution is discard the obsession with 24 fps and just use 30 (ie 29.97). That way you have perfect 1/2 speed slow motion by using common 60 fps (59.94) cameras. It avoids compute-intensive optical flow conforming and the constant scrutiny for optical flow artifacts.

There is nothing wrong with 29.97 - much broadcast TV in the US uses 29.97 acquisition, except for ABC, Fox and ESPN which use 720p/60 for live broadcast.

There is constant discussion about the alleged cinematic look of 24 fps. I personally like it and most of our products use 24 fps final encoding. But -- I don't think most viewers - unprompted - will notice the difference between 24 and 30 fps. However almost anybody will notice optical flow artifacts or jerky rate conforming, which is often caused by 30>24 or 60>24 rate conforming.

If we could go to mostly Panasonic Varicam or similar cameras, we could shoot at 48 and deliver at 24. However even this has problems because the new ProRes RAW upgrade on the S1H does not work at 48, only 24, 30 and 60.

Please Log in or Create an account to join the conversation.

Last edit: by joema.

Blending higher frame rate footage 13 May 2020 19:08 #107146

  • Itstonybabyvlog
  • Itstonybabyvlog's Avatar Topic Author
  • Offline
  • Junior Boarder
  • Junior Boarder
  • Posts: 29
  • Thank you received: 0
I have considered this, shooting at 30 or 60fps, and your right... I question the need for 24fps. But even if I decide to have a 30fps timeline, am I not running into the same problem whereas when I drop a 60fps clip into the timeline, it will drop every other frame? Isn’t that going to result in choppy video?

Please Log in or Create an account to join the conversation.

Blending higher frame rate footage 13 May 2020 19:28 #107147

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1554
  • Karma: 27
  • Thank you received: 332
The root problem is 60>24 or 30>24 are not evenly divisible. The final product must have 24 frames every second and neither source nor destination frames can be stretched or bent without changing the playback rate. Frames must be discarded by some algorithm. Now that we have powerful computers, optical flow can synthesize in-between frames to paper over the differences, but it doesn't work perfectly.

With 60>30, FCPX will discard every other frame, but it's still 30 fps so it's not really jerky. Advantages: (1) It's a continuous, even process so there is no motion cadence problem (2) It gives perfectly smooth 1/2 speed slow motion - always available without thinking in advance to put the camera in a special mode.

However the effective shutter speed (assuming the 180 deg. rule) will be faster so there will be less motion blurring than 30 fps acquisition using the same 180 deg. shutter angle. For 60 fps normally your shutter speed is 1/120 and for 30, 1/60. This is why low light performance suffers a bit at 60 fps - the shutter is only open 1/2 the temporal duration for each frame vs 30 fps. The data rate and volume is also bigger at 60, so that's another cost.

In theory you could use a slower shutter speed at 60 fps (say 1/60 instead of 1/120). That would restore some frame blurring and also give better low light performance. I tested this once but can't remember the results. The results might vary based on camera and sensor scan method.

Any such changes, no matter how trivial, should be thoroughly tested throughout the post production pipeline, and across all cameras in the fleet.

Please Log in or Create an account to join the conversation.

Last edit: by joema.

Blending higher frame rate footage 14 May 2020 07:28 #107155

  • Itstonybabyvlog
  • Itstonybabyvlog's Avatar Topic Author
  • Offline
  • Junior Boarder
  • Junior Boarder
  • Posts: 29
  • Thank you received: 0
Excellent reply! And you got me thinking about the shutter speed. I was thinking, what’s the difference between shooting video at 30fps at a 1/60 shutter speed vs shooting 60fps at a 1/60 shutter speed? It’s the same one second of video in each one, passing by your eyes at the same speed.

And then I had another idea! Why don’t I just shoot everything at 60fps, at 1/60 shutter speed (I have an ND filter), and drop everything into a 60fps timeline?

Shoot at 60 frames per second, with a shutter speed of 1/60
Then drop that into a 60fps timeline in FCPX
drop your footage to 40% slo mo
Do one clip using standard slo mo, and one clip using optical flow (see which looks better)

iIf you were to take 60 FPS footage and drop it in a 24 FPS timeline, you would get your whole 24 frames of the 60 by dropping to 40% slo mo. That’s what I currently do.

But...
By dropping 60 FPS footage in a 60 FPS timeline, then reducing that to 40%, you are losing 60% of 60 frames, which is... 24 FPS!

In theory, 60 FPS slowed to 24 FPS in a 60 FPS timeline, should look the same as 60 FPS slowed to 24 FPS in a 24 FPS timeline.

Am I on to something?

Please Log in or Create an account to join the conversation.

Blending higher frame rate footage 14 May 2020 09:33 #107157

  • Redifer
  • Redifer's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 857
  • Thank you received: 79
Shoot at 60 or 120 and edit and deliver in 30. 24fps is neat but overrated. Most people shoot in 24fps just because they think it makes them a fancy movie director or what have you. 30fps achieves a very similar effect and is less of a headache. Assuming you're not planning on a theatrical distribution, of course. If you want motion blur you can go into motion and create a FCP Title with a a 180º shutter and 32 samples (at least) and then drop that on top of all of your footage that you think needs to have more temporal blur. I forget how to create the Motion Blur effect in Motion though... that's the thing with Motion.... it is super easy to forget how everything works. But I digress.

Please Log in or Create an account to join the conversation.

Last edit: by Redifer.

Blending higher frame rate footage 14 May 2020 12:09 #107161

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1554
  • Karma: 27
  • Thank you received: 332

Itstonybabyvlog wrote: ...By dropping 60 FPS footage in a 60 FPS timeline, then reducing that to 40%, you are losing 60% of 60 frames, which is... 24 FPS!

In theory, 60 FPS slowed to 24 FPS in a 60 FPS timeline, should look the same as 60 FPS slowed to 24 FPS in a 24 FPS timeline....Am I on to something?


The issue is not simply discarding frames to mathematically match an overall frame count. Rather, due to motion of the subject, the discarded frames (1) Must be done evenly in each small unit of time, and (2) The ending frame edges in that time unit must synchronize with the underlying 24 fps timeline.

This is just not possible for down conversion of non-multiple frame rates. Imagine if you had a mechanical sprocket with 60 teeth per foot, and a mating chain with only 30 links per foot. You could cut off every other tooth on the sprocket and it would work. The applied force at the tooth/link surface would be consistent as the sprocket rotates.

Now if you had a sprocket with 60 teeth per foot and a chain with 24 links per foot, that is not possible. To avoid clashing, some uneven number of sprocket teeth must be removed. That in turn produces uneven force at the mating surface because the tooth spacing is uneven. In video terms it would produce a disturbance in motion cadence.

There is no simple solution for this -- the underlying issue is fundamental and unyielding. What Redifer said is correct -- just shoot at 60 or 120 and deliver at 30 (obviously I mean 59.94 and 29.97).

The only issue to resolve is investigating subtle aesthetic issues caused by the higher shutter speed, and whether lowering shutter speed to 1/60th at 60 fps works OK on your camera fleet.

However 60 fps can't be that bad or different than 30 because nobody ever complains that ABC, Fox and ESPN live broadcasts look different or have a staccato look to the motion, and those are all 720p/60, whereas everyone else in the US uses 1080i/30.

There is an obsession with 24 fps which ends up hurting final quality because of the mis-match between that frame rate and common cameras that shoot 30 or 60 fps. I've seen final products shown for a mass audience which had major motion cadence artifacts because some cameras were at 30 fps, the final timeline was 24 fps, and the editor did not properly conform the frame rates. This is typically caused by a decree that the product is 24 fps and not coordinating that all cameras throughout the shooting schedule are at 24. It is easier to achieve on a scripted narrative but almost impossible on a documentary with contributed or archival footage from a wide selection of cameras.

Please Log in or Create an account to join the conversation.

Last edit: by joema.
  • Page:
  • 1