Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: New Proxy Workflow in 10.4?

New Proxy Workflow in 10.4? 15 Dec 2017 00:33 #92344

  • Gudgefilm
  • Gudgefilm's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 18
  • Thank you received: 1
  • Karma: 0
Anyone notice the 10.4 release notes and the section on proxies?

Under the FCPX 10.4 release notes "issues" section, it says "Third-party asset management systems can now send low-resolution media to Final Cut Pro through XML import and later replace those low resolution files with full resolution files through another XML import". Has anyone tested this? Does this open up the ability to create 3rd party proxies and then ingest and FCPX will properly relink? Something else?

Their release notes here: support.apple.com/en-us/HT201237

(This question originally asked by Chris SInclair on FB but no answers yet. He and I are hopeful this means good things!)
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 15 Dec 2017 03:36 #92345

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2285
  • Thank you received: 302
  • Karma: 33
Basically, it allows MAM systems to do that via the XML Object functioning. But it's not something opened up to the casual end user. We will have to see if someone makes an app that can manage to use this getting FCPX to use externally generated proxies.

But some really nice stuff in those release notes, like warning you if you're exporting while set to work with proxies.
- Contextual menu option to create a new project from a selection of one or more clips in the Browser
- The Toggle Fade In and Toggle Fade Out commands now appear under the Adjust Volume menu
- Press Ctrl-Tab to cycle through tabs in the Inspector when the Inspector is selected
Just to name a few...
Last Edit: 15 Dec 2017 03:39 by FCPX.guru.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 01:01 #92370

  • csinclair
  • csinclair's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 8
  • Karma: 0
I was looking into KeyflowPro the other day, and they were describing how you can choose your proxy flavor (resolution, etc), and then drag and drop projects from their MAM system directly into FCPX. Is that an example of a tool that utilizes this? I'm not too familiar with Keyflow Pro, I just watched some of the tutorials online.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 04:51 #92371

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
I was hoping for a selection of how compressed I wanted my proxy files to be, I love the way Filipe Baez created his proxy workflow, using compressor and Automator and the cloud to send low rez versions of proxy from 1 machine to another...

It makes so much sense to do it, how could Apple not include it as a standard wrokflow method, I mean, why not have 5 or 6 levels of compression? They can compress once to create the proxy file, why not add a few levels and give the users the choice???

That has to be a no brainer for an update??
Last Edit: 16 Dec 2017 04:51 by MsJustine.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 07:17 #92372

  • Axel
  • Axel's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 429
  • Thank you received: 53
  • Karma: 9
MsJustine wrote:
I mean, why not have 5 or 6 levels of compression? They can compress once to create the proxy file, why not add a few levels and give the users the choice???

That has to be a no brainer for an update??


1. FCP should have the functionality of the said 3d party MAMs which give you the choice.

2. Why a WARNING? This looks to me like the cheapest of excuses for a poorly thought-out proxy workflow. FCP should just NEVER refer to proxies for final export, period.

3. If I chose "Proxy" in the viewer, FCP should play back the proxy versions
a) if
b) as soon as
they are available, allowing me to start editing without having to wait (just like the Optimized workflow).
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 08:37 #92375

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
I wish I could ask the developers what was the reason they decided from the outset to limit the proxy to 1 level and never develop that workflow further, yet they are hell bent on VR, yet other aspects are left under-developed.

I wonder if FCPX is heading down a rabbit hole path of development that will see FCPX only really become viable on the top end iMac Pro and Mac Pro and laptops and mac mini's will no longer be able to be used...

Not everyone has a MAM, a Jellyfish, $1m for iMac Pro with 18 cores and 128GB RAM, nor do they need it.

This proxy workflow is indicative of the way Apple is thinking, in that even proxy workflow at 8K is huge files, so a decent laptop will struggle even to convert to something tending standard definition if you try and convert using compressor.

It has not been addressed, updated or discussed at any great length, does Apple even know what a huge file a proxy file is when it is the product of 10K or 8K RED??
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 12:30 #92377

  • Scrubelicious
  • Scrubelicious's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1207
  • Thank you received: 108
  • Karma: 46
MsJustine wrote:
I wish I could ask the developers what was the reason they decided from the outset to limit the proxy to 1 level and never develop that workflow further, yet they are hell bent on VR, yet other aspects are left under-developed.

I wonder if FCPX is heading down a rabbit hole path of development that will see FCPX only really become viable on the top end iMac Pro and Mac Pro and laptops and mac mini's will no longer be able to be used...

Not everyone has a MAM, a Jellyfish, $1m for iMac Pro with 18 cores and 128GB RAM, nor do they need it.

This proxy workflow is indicative of the way Apple is thinking, in that even proxy workflow at 8K is huge files, so a decent laptop will struggle even to convert to something tending standard definition if you try and convert using compressor.

It has not been addressed, updated or discussed at any great length, does Apple even know what a huge file a proxy file is when it is the product of 10K or 8K RED??

How many levels do you need? Also don't forget about performance and space ratio.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 12:38 #92378

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
It is not about how many levels, that would be daft, back in the olden days of yore, when computers where big and beige, and interfaces small and chunky, it was possible to move a slider to determine the quality of the compression.

Like Filipe Baez demonstrated, this workflow can be very effective if you need to send video over a bandwidth limited network, say a mobile network.

I cannot see how Apple could not have included this workflow in the update, instead we have to use a number of steps, delete and rename files, and re-import, and so on...

Apple focused too much on VR, when VR is not yet ready for mainstream, look I see the attraction of VR, but it is not yet mainstream, editing is, we have mac book airs, underpowered macbook pro's, old imacs, and the like...

All I would have loved is a way to save the RED files here and have a very compressed file to edit, and when I have finished editing, delete the proxy file, and share, The proxy file should be scaled to my choice, not some muppet's idea of what is best for me...
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 15:34 #92379

  • Scrubelicious
  • Scrubelicious's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1207
  • Thank you received: 108
  • Karma: 46
IN that case you might find this interesting:

The relinked files can have a different resolution and codec than the original files, and they can be trimmed versions of the original files, but the relinked files must include all the media in the referenced clips. Otherwise, the relinked files must share the following qualities with the original files:
  • Same media type (for example, video or audio)
  • Same frame rate
  • Similar audio channels

At the moment I am myself working on a Documentary with approx has 13TB of media... which we brought down to 1,5TB with Proxy. Not bad for my taste ;)
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 15:45 #92380

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
The point I thought of proxy is to change the resolution, so that it requires less on board computer resources to edit, so changing the duration makes no sense...

I expressed this thought a long time ago, when it comes to importing material, the old solid state import window of FCP7 was a great method, you could select ranges and only import those ranges, so that in the project you have only the material you think you will need.

Look Apple kind of provided a work around this, with the hide rejected in the view browser as an option, it is basically the same concept, where as my suggestion was active, Apple's implementation is passive, it does not destroy or break the relationship of the range to the clip. Apple's method is destructive to the relationship, but in the end does that matter??

Proxy files today are and will become resource hogs, at 25% resolution, an 8K RED is 2K, and that is a huge file, to got to 1K or 1/2K is better, and this is what Filipe Baez showed and yes it is a great workaround.

It should not have to be, why Apple has not updated this feature to accommodate 8K or 10K files, and now we have VR, 10K and higher will become standard, so soon 25% of 12K will be 3K and wow...huge..

That is what I am saying, has Apple lost focus of the size of the proxy generated. Sure I can do the Baez method, time is precious, to select in preference the proxy file required....Why Not???
Last Edit: 16 Dec 2017 15:47 by MsJustine.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 16 Dec 2017 15:49 #92381

  • Scrubelicious
  • Scrubelicious's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1207
  • Thank you received: 108
  • Karma: 46
Maybe I miss understood you... but the Proxy I am working with are 1x1 in the length just resolution is different. and again you can create your own Proxy as long the above requirements are checked. I have done this with ProRes Proxy....

Using h264, I think defeats the purpose when stress on hardware is the issue.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 18 Dec 2017 15:15 #92450

  • Gudgefilm
  • Gudgefilm's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 18
  • Thank you received: 1
  • Karma: 0
Helpful. Thanks.
Hopeful for how a 3rd party may use this functionality!
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 18 Dec 2017 15:19 #92451

  • csinclair
  • csinclair's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 8
  • Karma: 0
Keyflowpro users, does its proxy generation utilize this feature? The latest FCP.co post says this proxy generation ability might be FCPX's biggest new feature. My intrigue with Keyflowpro began with a post about their features on this site. Would love to know.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 11:25 #92490

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
KeyflowPro looks interesting, expensive for a single user, I do see how it could work in a large multi-user shop.

Why is it so hard for Apple to create a number of proxy file codecs that you the user can self select to encode to? Set this in Preferences and know that when you select Proxy in the view, it is the codec you select?

A suggestion maybe would have this on the import window, it would help so much with media management? I have no idea how this would affect say VR/360 I am sure it would be terrible to edit with, but for VR/360 you need monster systems anyway, so proxy would be useless...

Just when it comes to on the road editing, having a small quick to encode, proxy that is more than 25% of the original, 50% or 75% would be so handy, right on the ingest...Am I wrong???
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 12:29 #92494

  • Axel
  • Axel's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 429
  • Thank you received: 53
  • Karma: 9
MsJustine wrote:
A suggestion maybe would have this on the import window, it would help so much with media management?

Yes. You are accepting that media management requires the media to be ingested, imported first or simultaneously. As if that was not an arbitrary limitation. I no longer like this concept. FCP X should have been able to browse all video, audio and stills on your system as an advanced player since 10.1. Reading all metadata, adding new (renaming, tagging, subclipping) and exporting everything to predefined locations with custom settings.

Kyno can do exactly that. If you right-click to >convert >edit >ProResProxy 422 you can further go to >video >settings >frame size and there down to SD 480p. You can save that preset as MsJustines Proxy.

Then, if you quit Kyno, nothing happens. Nothing is saved, and why would it? Nothing ever was imported. There is a folder with your proxies, and if you open it with Kyno, you can send the XML to FCP with cmd+shift+f, by which the FCP import dialog opens and asks you if you want to create a new event or use the previously used one and whether or not you want to leave files in place. All metadata included.

That's the way it should work, and maybe we'll see this functionality with 10.5, as it can't be too hard to do.

I don't need proxy, and I haven't tested the new externally encoded files workflow. I just wonder if there wouldn't be conflicts with different frame sizes if you use transform tools asf. As I understood, ProRes Proxy nominally has the same frame dimensions as the source material but is actually half (?) or even quarter (?) resolution.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 12:45 #92497

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
OK, I get what you are trying to say with using Kyno, that is exactly what if Apple had continued development of Final Cut Studio would have had.

The point is, at some point ingesting into FCPX has to occur, so having a "front end" pre ingest app be it Kyno, or Apple based, matters not, the point is, converting the high rez RED into something tiny and GPU friendly is mandatory, why Apple has not bought Kyno when they have just bought Shazam, not sure why not...

All I am asking Apple and I have made a number of requests about this, is a way to use proxy in FCPX that is less GPU intensive than it is now, that is all, and having a front end app like kyno for small users is maybe a bit too much.

Nothing wrong with kyno per se, fcpx could use a number of codecs dedicated to low rez on the road editing, files are getting bigger, frame sizes are heading into giant size...

For me kyno is too much, and too expensive, the Baez method so far is the best option, a tad time consuming, but it gets around the limits of the codec and gpu clashes...

Thanks for the advice on kyno, it looks great..
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 16:11 #92514

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1023
  • Thank you received: 212
  • Karma: 17
Axel wrote:
...You are accepting that media management requires the media to be ingested, imported first or simultaneously. As if that was not an arbitrary limitation. I no longer like this concept. FCP X should have been able to browse all video, audio and stills on your system as an advanced player since 10.1. Reading all metadata, adding new (renaming, tagging, subclipping) and exporting everything to predefined locations with custom settings....Kyno can do exactly that....

One of the greatest accomplishments of FCPX is the hyper-fast, fluid skimmer. Whatever coding tricks Apple used, nobody else has been able to duplicate that performance, not even Kyno. For the "leave files in place" case, FCPX can rapidly ingest material and skim it -- and on a fast machine it can do this on 4k H264 without even creating proxies.

For the "in place" case, FCPX is obviously not doing much to import that or ingest wouldn't be so fast. This could be further accelerated by not building waveforms. They could theoretically extend this to some kind of advanced player, but it would still require some ingest process, however brief, and that data must be stored somewhere. But it is obviously doable since FCPX is doing it now.

FCPX is so fast and all external tools are so much slower that I essentially use it as a player. I ingest virtually all media assets from a shoot, mostly without even looking at them. Using "leave files in place" this is extremely fast and doesn't take much disk space unless proxies are created. Then I can skim, rate, reject and keyword material faster than any other 3rd party tool.

Apple could make an advanced "skimmer like" player, and this was the original concept for what became FCPX -- called First Cut, it would skim and tag material which would then be handed off to Final Cut. However I'm not sure of the benefit since either way that content must go through some ingest. But with ever increasing media volumes maybe some kind of advanced skimmer-like player would be worthwhile.

The problem is for skimmer-like performance some ingest is required. If there was a faster way (except maybe deferring waveform generation or optimizing thumbnail generation), Apple would already use this on FCPX. But they have the code and basic technique, which nobody else has. A skimmer-like tool with a lightweight, expedited ingest could help editors wade through today's vast oceans of media. Like the original "First Cut" concept, it could then hand off selected material to FCPX, hopefully using the same ingest data without regenerating it.

OTOH that feature mostly already exists in FCPX, if you simply accept the "import everything" workflow using leave files in place. Putting this outside FCPX might give you a different mental perception, but something similar would be happening under the covers. Today using FCPX you can even ingest everything to a junk library, rate/keyword the content, then copy that to your final library and discard the junk library -- not too different from how an advanced player or "First Cut" utility would work.

The advanced player could be made somewhat faster by deferring waveforms and maybe optimizing thumbnail generation. But those enhancements might also benefit FCPX, and if added then FCPX becomes as fast and lightweight as the player, thus reducing the benefit of an external player.
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 16:24 #92516

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 970
  • Thank you received: 46
  • Karma: -78
What I got from watching both the kyno video and the webinar by Larry Jordan, is that kyno is sort of a new updated version of the same system in Final Cut Studio if you used a solid state/P2 card, in which you selected a range and only imported that.

You can do this with kyno, but as far as I know and just checked, you cannot import a range or series of ranges into Final Cut Pro X, only the full clip. Maybe there is a circumstance in which you can, but if you have a .mov file that is not on an SD card, whole clips to be ingested.

This is where kyno or something like kyno would be great, to skim and ingest only the range/ranges you need, my question is what is stopping Apple from buying kyno for the technology, like they did for the old Color app in old final cut studio, that was bought, reworked/rebranded into final cut studio, a great app mind you...

We will all at some point love a low rez simple to use, multi "level" proxy codec, that can be set and forget, and just does what it says on the tin, a simple range of compression values, I saw this on the very old premiere 5.0, the one with the white horse on the box-art, and Avid back in the day had this, we used it to ingest for playback, given the tiny hard drive and it was for local broadcast, so who caed if it was VHS!!!!

Please dear Apple pro app dev..can we please have a range of codec for proxy workflow????
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 16:41 #92517

I bet the "internal" Proxy workflow will remain the same to keep things simple. (half size, ProRes Proxy) if the user is fine with it ... nice

the point is an advanced external Proxy source coming now to open optional possibilities for more sophisticaded workflow...

so where is the problem ?
The administrator has disabled public write access.

New Proxy Workflow in 10.4? 19 Dec 2017 17:18 #92519

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1023
  • Thank you received: 212
  • Karma: 17
MsJustine wrote:
...Why is it so hard for Apple to create a number of proxy file codecs that you the user can self select to encode to? Set this in Preferences and know that when you select Proxy in the view, it is the codec you select?...A suggestion maybe would have this on the import window...Just when it comes to on the road editing, having a small quick to encode, proxy that is more than 25% of the original, 50% or 75% would be so handy, right on the ingest...Am I wrong???

MsJustine you've asked this question several times, and I think it's a good one. Some software such as Premiere Pro CC allow multiple proxy resolutions and different proxy codecs. I'll try to provide a complete answer.

The hard part is not the UI aspect, but every new codec requires new code paths to implement and ongoing test and validation. That is one reason why video software is so prone to crash -- it must deal with many encoding formats in a time-critical fashion. This implementation cost and risk plus support cost must be weighed against the actual (not perceived) benefit.

The data often cannot be validated by the software since the container formats usually were not designed for this. The standards did not implement (say) CRC checksums in the header to validate each data block in the file, nor would there be execution time for that. The header is full of metadata "atoms" which may be somewhat poorly standardized and ambiguously defined. A programmer might have to read multiple white papers, talk to colleagues, review and test multiple products from different companies to decipher what the common practice is for handling these items. A good example of this is the meta atom called "fiel" which you can see described on table 4-2 of this page: developer.apple.com/library/content/docu...00939-CH205-BBCBACAB

In the real world when programmers, test engineers and support engineers have to implement (and support) things, they are keenly aware of issues with risk and reliability. In a well-run development group, this often leads to conservatism when implementing new features. In other groups they will throw in more features but the users ultimately pay the penalty.

You can obviously add lots of different proxy codecs and sizes -- Adobe did that with Premiere, but that software has a reputation for being quite crash-prone.

An important issue is the cost/benefit tradeoff. The question is would smaller proxies or those using more compressed codecs help from the standpoint of ingest or editing performance. Almost certainly no for compressed codecs but even for ProRes, smaller proxies may help less than you think.

E.g, the entire full resolution camera native file (often H264) must be read to generate the proxy -- no matter what proxy size or codec. It is reading and decoding that file that's slow -- not generating a ProRes proxy. This can be seen by my 2017 iMac being 2x faster than a 12-core D700 Mac Pro at ingesting 4k H264 and generating 1080p ProRes proxies. The burden is not on the output side (proxy generation) it is on the input side -- decoding all the 4k H264 material. The Mac Pro (which does not have Quick Sync) is much slower.

So for the ProRes proxy case, whether the output size was 1080, 960, 540, or whatever -- most of the work is on the input side of decoding the full-res camera file. So a smaller proxy will take about as much time to generate as a larger one.

The other important item is how much a smaller proxy would help editing performance. The current FCPX proxies are fixed at 1/2 the linear resolution or 1/4 the pixel resolution, and always ProRes. For 4k H264 this provides good editing performance on a modest machine -- my 2013 MacBook Air edits 4k pretty well using the current 1/4 size proxies. But that's a few simple edits, not applying effects.

What about 8k material? The current FCPX system produces 4k ProRes proxies for 8k originals. Why not add 2k (or 1/16 the pixel resolution)? Two main reasons:

(1) 4k ProRes proxies are still pretty fast on most machines. Anybody who doubts this can encode a file to 4k ProRes and edit that. It runs fairly well on somewhat older machines (for basic edits).

(2) No matter what the proxy size, you cannot totally escape the original media performance cost. E.g, when you render a clip in the timeline (as is often required to smoothly evaluate things like stabilization, noise reduction, retiming, etc) it sometimes must read and decode the original media -- even in proxy mode. Anyone can prove this by applying optical flow or Neat Video to a 4k clip in proxy mode, render that clip in the timeline, then repeat that on a fresh library using original media. In both cases it takes about the same time to render the clip in the timeline.

Some FCPX built-in effects such as sharpening will be faster if rendered in the timeline in proxy mode than for original media.

If you don't explicitly render the clip, effects will generally auto-render a low fidelity version which allows you to visually estimate the look. This action is often faster on proxies, so they help in that case.

But you can improve performance on 4k and above right now by editing on a 1080p timeline. This limits the render file resolution to 1080p so timeline rendering happens faster. If 4k export is needed, the timeline can be cut/pasted into a 4k project.

So for people who will be editing 8k, they will need a powerful machine, no matter what the proxy size. For people editing 4k H264, you still need a relatively fast machine once you progress beyond some basic edits -- even using the current 1/4 pixel res proxies. Providing 1/8th or 1/16th res proxies would help some, but they would not transcode any faster.

Small cloud-based proxies to facilitate geographically distributed collaborative editing is another issue, but it's much more complex than the proxies. The place where FCPX's proxies need serious work is in the implementation and management of externally-located proxies (ie outside the library) to facilitate a proxy-only workflow. That system does not work well and relink is broken for that case. It badly need improvement.
Last Edit: 19 Dec 2017 17:21 by joema.
The administrator has disabled public write access.
  • Page:
  • 1
  • 2