Welcome, Guest
Username: Password: Remember me

TOPIC: Interlaced MXFs messed up in 10.4

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:20 #93341

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
joema wrote:
We need to study a clip of the original camera material -- not just look at it, but examine the video file header and encoding details. This need not be sensitive stuff. All work has out takes where the camera is pointed at the floor, record button accidentally pressed for a few sec, etc. As an editor I spend lots of time discarding that stuff.

There is nothing odd about this clip, it is a simple Compressor transcode of an h264. This test clip gives different results when exported from 10.4 and 10.3.2. That alone indicates that something happened regarding MXFs in 10.4. The output is wrong in 10.4, but right in 10.3.2, so there's a bug. If something is wrong with the file, why is output correct in earlier versions of FCP?

Furthermore, I first noticed this behavior on broadcast video from different clients. This material was
  1. made with different encoders (Adobe Media Encoder and Rhozet)
  2. historically always solid, functioning and compliant
  3. working just fine in FCP 10.3.2
All this indicates that there is nothing wrong with the files, that the problem lies in 10.4. Likewise, I am quite sure that the test clip is structurally sound, since it behaves in the exact same way: works as expected in FCP 10.3/QT7/VLC; interlacing messed up in 10.4.

Since I can reproduce it on my two machines and Tom Wolsky reports the same behavior with the test clip (a clip I feel confident is not faulty, seeing as it works everywhere I've tried it except 10.4), I have my answer; this is a reproducable bug. I do not know if it affect other codecs than the ones I've tried, or other frame rates besides 25 fps, but it is bad enough to keep me from using 10.4.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:29 #93342

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
There is a problem with the media. I tested 25p media in a 25i project in both .4 and .3.4, and they both show interlacing correctly, which your media does not in those applications. If it's interlaced it will show interlacing on motion which doesn't appear in the media you sent except in .4, which seems to be the only application reading the interlacing correctly.

I am not using 25p source material, the bug I am encountering affects interlaced source material.
But, are you saying that you ran the test clip in 10.3.4 and got the same results as in 10.4? That would be interesting, perhaps the bug was introduced prior to 10.4? I am running 10.3.2 due to other bugs in later versions. Also, what does the test clip look like in QT7 on your machine?
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:30 #93343

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4105
  • Thank you received: 660
  • Karma: 106
What I don't understand is why something that's supposed to be interlaced doesn't show any interacting. That makes no sense to me. If it's interlaced it absolutely should be visible on motion when viewed at 100%.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:42 #93344

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4105
  • Thank you received: 660
  • Karma: 106
The test clip that's supposed to be interlaced, that is interlaced according to Media Info, does not look interlaced QT7. It does not have interlacing and looks progressive.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:44 #93345

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
What I don't understand is why something that's supposed to be interlaced doesn't show any interacting. That makes no sense to me. If it's interlaced it absolutely should be visible on motion when viewed at 100%.

Because the interlaced clip was made from a progressive source. This means that there is no movement between the top and bottom fields of a single frame. The fields are still there, they just make up a single frame. All movement occurs every other field (compare it to a PAL DVD of a Hollywood film, for instance). The correct behavior is that every frame should look progressive, the interlacing should be invisible, which makes this incorrect behavior easy to spot.

My broadcast material is not made from a progressive source and shows the same bug. It is a lot less apparent in a still frame that everything is shifted a half frame, but the problem is still there.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:49 #93346

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
The test clip that's supposed to be interlaced, that is interlaced according to Media Info, does not look interlaced QT7. It does not have interlacing and looks progressive.

Great, that is the way it's supposed to be. It doesn't look interlaced, because there is no movement between the top and bottom fields of a single frame. When run in 10.4 it looks interlaced. That is the (show stopping) bug.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 00:55 #93347

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4105
  • Thank you received: 660
  • Karma: 106
no movement between the top and bottom fields of a single frame

The bird flaps its wings. There's movement within the frame that changes in 1/50 of a second. The fields should be different otherwise it's not interlaced.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 01:04 #93348

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4105
  • Thank you received: 660
  • Karma: 106
If I output progressive media as interlaced, it will look interlaced even if the original is made up of single frames with two identical fields.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 03:27 #93349

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 993
  • Thank you received: 209
  • Karma: 17
joema wrote:
We need to study a clip of the original camera material -- not just look at it, but examine the video file header and encoding details...
fb_527411383 wrote:
There is nothing odd about this clip, it is a simple Compressor transcode of an h264. This test clip gives different results when exported from 10.4 and 10.3.2....The output is wrong in 10.4, but right in 10.3.2, so there's a bug. If something is wrong with the file, why is output correct in earlier versions of FCP?...

All we really know is still frames of your output look different. I realize you characterize the 10.3.2 behavior as "right", but it's conceivable that 10.3.4 is in error and they finally fixed it on 10.4.

Issues involving interlacing can be confusing since various playback methods handle them differently. Quicktime 10 automatically deinterlaces under some conditions based on the video header. VLC by default does not deinterlace but has various optional deinterlace modes. QT7 only deinterlaces if enabled in Window>Movie Properties>Video Track>Visual Settings.

In general interlaced material should look interlaced on output. The playback chain is responsible for deinterlacing, so if viewed by a tool without deinterlacing, it should look interlaced. If you don't positively understand the playback behavior, you can't be certain of what you're seeing.

If your source material is progressive and is encoded as interlaced or PsF, then that's another possible complication.

The metadata atom called "fiel" in the video header is crucial to apps and encoders handling cases like this correctly, yet it is often mishandled by software. If you examine the definition on table 4-2 of this page you'll see why: developer.apple.com/library/content/docu...00939-CH205-BBCBACAB

We can't replicate this because we don't have a camera test clip. I spent 30 minutes looking for some 1080i 25 fps 10-bit 4:2:0 AVC Intra test material but couldn't find any. If you can provide one I'll be happy to investigate further.

Have you installed the latest Apple Pro Video Formats on your machine? I think it was updated for 10.4. support.apple.com/kb/DL1947?locale=en_US
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 07:22 #93351

  • ronny courtens
  • ronny courtens's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4156
  • Thank you received: 881
  • Karma: 195
The only way to be sure of any of this would be to get a sample of the real footage you work with. Not some clips that have been converted and transcoded. If you send us a sample of your original and unprocessed media, we can have a look at it. We deliver interlaced MXF for broadcast on a daily basis, never had an issue with FCP X.

- Ronny
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 09:36 #93355

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
This thread has really taken a detour, which is fine seeing as Tom Wolsky has exactly reproduced the bug, which means that my recourse is to go back to a previous version of FCPX until this is fixed.

It seems like the concept of interlaced footage that looks progressive confused the subject, for some reason. I'll address the posts below, but to try to get the issue from the test clip itself, here is a different clip, from a completely external source.
Original frame (2 sec into the clip):
OriginalinQT7.jpg


Same frame viewed in 10.4:
InFCPX10.4.jpg


In 10.4, zoomed in:
Zoomedin.jpg


The frame's burned-in timecode is 00:53:52:15, but in 10.4 that frame contains one field from that frame and one field from the next. If anyone wants to try for themselves, the clip is the MXF named PAL 1080i MPEG XDCAM-HD and found here.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 09:43 #93357

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
no movement between the top and bottom fields of a single frame

The bird flaps its wings. There's movement within the frame that changes in 1/50 of a second. The fields should be different otherwise it's not interlaced.

Had this been a real bird, shot with a camera that recorded interlaced video, then yes. This animation, however, is made up of still frames. There is no rendering of the bird between the two frames, the movement only happens every other field, every 1/25th of a second. The incorrect interlacing seen in 10.4 is one field from one still frame and anther field from the next still frame.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 10:01 #93359

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
The fields should be different otherwise it's not interlaced.

That a file is interlaced is a technical property of the format itself, basically metadata. It dictates how the file should be played back, and is not based on what the actual fields look like. A still image rendered into an interlaced video is still interlaced video, even though there is no movement at all.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 10:08 #93360

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
Tom Wolsky wrote:
If I output progressive media as interlaced, it will look interlaced even if the original is made up of single frames with two identical fields.

This is absolutely NOT the case when going from 25p to 25i. That would require calculating fields that are not present in the original and throwing away fields that are. That would be horrible, and thankfully that's not how it's done.

When converting between 24 fps and NTSC frame rates, the interlacing IS very visible, but that is a completely different process, and not at all relevant to this topic.

BTW, it seems like the bug only affects PAL frame rates, after some further testing.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 11:23 #93365

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
joema wrote:
All we really know is still frames of your output look different. I realize you characterize the 10.3.2 behavior as "right", but it's conceivable that 10.3.4 is in error and they finally fixed it on 10.4.

What is your basis for this hypothesis? Are previous versions of FCPX known to mess upp interlaced MXF files?

Every file I've worked, from various sources have worked just fine previously. The output has worked fine on delivery. In 10.4 this is no longer the case. Claiming that everything else was/is broken (the files themselves, FCPX 10.3, QT7, the broadcasters' QC, etc), and that FCPX 10.4 is working as it should is a stretch, to say the least.

joema wrote:
In general interlaced material should look interlaced on output. The playback chain is responsible for deinterlacing, so if viewed by a tool without deinterlacing, it should look interlaced. If you don't positively understand the playback behavior, you can't be certain of what you're seeing.

If your source material is progressive and is encoded as interlaced or PsF, then that's another possible complication.

To me there is nothing complicated or confusing about the way the test clip looks. The file format is interlaced, and it's made from an original source that is progressive. There should be no interlacing artefacts when viewed, as there is no movement between the top and bottom fields. The fields still exist, though. This is not unusual or confusing if you're used to working in PAL frame rates. I used this very source material so I would know exactly what the output should be, what every frame should look like. There is no confusion or uncertainty to what I'm seeing. The clip looks different in 10.4 to how it looks everywhere else, and it looks wrong in 10.4.

joema wrote:
We can't replicate this because we don't have a camera test clip. I spent 30 minutes looking for some 1080i 25 fps 10-bit 4:2:0 AVC Intra test material but couldn't find any. If you can provide one I'll be happy to investigate further.

Well, that's not true; Tom Wolsky exactly replicated the bug with the test clip given. He reported that the file looks different when viewed in QT7 and in 10.4. That's enough for me to confirm that the bug affects more than just my machines. If you want to try a different clip, get the PAL 1080i MPEG XDCAM-HD MXF from here. (The same link as I posted above).

As I stated above the video I've seen this with does not come from cameras, it comes from various transcoders (Adobe, Rhozet, Compressor), and I have absolutely no reason to doubt that it is up to spec. Again, here is the chain of events:
  1. Everything works fine in 10.3
  2. Two of my machines are updated to 10.4
  3. This bug turns up and is consistently reproducible
  4. Tom Wolsky reproduces the same behavior in 10.4
  5. Going back to 10.3 fixes the problem
To me, it is abundantly clear that the issue lies in 10.4. If anyone can show that these test clips behave differently on their installation of 10.4, that would be very interesting and give a clue to where the bug lies, but so far all I've seen is confirmation that the bug exists (from Tom Wolsky).
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 11:33 #93366

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • OFFLINE
  • Moderator
  • Posts: 4105
  • Thank you received: 660
  • Karma: 106
There's also something wrong with MXF 25p Compressor generated files. They look progressive in QT7 and are set as progressive in FCP settings, but they show interlacing in both .3.4 snd .4.
Last Edit: 16 Jan 2018 11:38 by Tom Wolsky.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 11:41 #93369

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
ronny courtens wrote:
The only way to be sure of any of this would be to get a sample of the real footage you work with. Not some clips that have been converted and transcoded. If you send us a sample of your original and unprocessed media, we can have a look at it. We deliver interlaced MXF for broadcast on a daily basis, never had an issue with FCP X.

- Ronny

I'm not following you, why would (verifiable valid) transcoded media be unreliable in FCPX? As stated above the media I work with in this context all comes from (different) transcoders, not from cameras.

If there is something wrong with the clips linked in this thread, it would be great to find out:
  1. Exactly what the technical issue is, and how this is determined?
  2. Why does it play back differently in 10.3 and 10.4 (and only look incorrect in 10.4)?
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 14:05 #93376

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 993
  • Thank you received: 209
  • Karma: 17
fb_527411383 wrote:
...If there is something wrong with the clips linked in this thread, it would be great to find out: Exactly what the technical issue is, and how this is determined?
Normally we'd look at an original camera file, inspect the internal video header and encoding characteristics and test it various ways on various software. Since all you've provided is a Compressor-generated file, that is a "tainted" copy.

This doesn't mean there's no bug, it just needlessly introduces uncertainty, causes everybody a lot of extra work and protracts problem resolution. I agree there could be a bug in Compressor 4.4 or FCPX 10.4 when handling a specific type of 1080i 25 fps material. I'm looking at that now.

If this is a bug we'll nonetheless need a portable replication scenario to submit to Apple. They will expect an original camera file, not an encoded file from Compressor or FCPX.

fb_527411383 wrote:
...Why does it play back differently in 10.3 and 10.4 (and only look incorrect in 10.4)?...

I don't see your test clip playing back any differently IN 10.3.4 vs 10.4. I do see the rendered, encoded OUTPUT playing differently in a variety of players. However this seems to vary based on the output encoding method. I'm examining it further now.
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 14:28 #93378

  • fb_527411383
  • fb_527411383's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 84
  • Thank you received: 7
  • Karma: 2
joema wrote:
Normally we'd look at an original camera file, inspect the internal video header and encoding characteristics and test it various ways on various software. Since all you've provided is a Compressor-generated file, that is a "tainted" copy.

I still can't make any sense of this. If the clip is faulty, that has to be determinable by looking att the clip itself; not by looking at a completely different clip. In this case, with Big Buck Bunny, there are no camera files, only rendered video. This clip behaves oddly, and there are no camera files to inspect. The rendered files from Compressor (or any other transcoder) are the actual output that is being used. If they are behaving oddly, they are the files that need to be inspected.
joema wrote:
fb_527411383 wrote:
...Why does it play back differently in 10.3 and 10.4 (and only look incorrect in 10.4)?...

I don't see your test clip playing back any differently IN 10.3.4 vs 10.4. I do see the rendered, encoded OUTPUT playing differently in a variety of players. However this seems to vary based on the output encoding method. I'm examining it further now.

This is interesting. You're saying that with Show both fields checked in the viewer, it looks the same in 10.3.4 and 10.4? Is the interlacing messed up?
The administrator has disabled public write access.

Interlaced MXFs messed up in 10.4 16 Jan 2018 14:46 #93379

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 993
  • Thank you received: 209
  • Karma: 17
fb_527411383 wrote:
...You're saying that with Show both fields checked in the viewer, it looks the same in 10.3.4 and 10.4? Is the interlacing messed up?

You're right if View>Show in Viewer>Both Fields is selected on 10.4, I see it during FCPX playback. Still investigating...
The administrator has disabled public write access.