I'm having an issue with the render function. After adding effects etc I used to be able to select and pre render chosen clips so I could play them back smoothly on the time line. This is not working. I can render the whole time line and this has no effect. Some clips will always take 2 extra seconds to display regardless if the effects are on or off. Have you got any suggestions on how to fix this. I have re installed fcpx and I have the latest version now.
tomasz.derlacki wrote: ....After adding effects etc I used to be able to select and pre render chosen clips so I could play them back smoothly on the time line. This is not working...
In general it should work, but there have long been problems with FCPX properly using render files. In some cases it produces render files then won't use them. In other cases it loses render files when no edits are made or when simply toggling between proxy and orig. media mode (when both have valid render files) or when re-launching FCPX. In certain cases you can render the timeline, it won't use the render files, then you re-launch FCPX, render timeline again and it will use them. In yet other cases you render the timeline and the render dots stay present. This seems partially related to which effects are used and the order in the effects stack. There is a group of "poorly behaved" built-in effects which apparently trigger this, but the underlying problem is likely broader.
The problem is severe when using a compute-intensive effect such as Neat Video and a poorly-behaved built-in effect in the stack prevents re-use of those render files for export. In those cases the export time may suddenly increase by a factor of 50, despite the timeline being fully rendered.
There is yet another case for "fast path" export whereby if the timeline, render format and export format are all ProRes 422, it's supposed to simply grab the render files and write to output without any encoding. For unknown reasons it sometimes fails to do this.
So in general there are two broad cases: (1) Do existing render files speed up timeline operations, and (2) Do existing render files speed up export. In some cases the render files will speed up timeline operations but FCPX fails to use them for export. Within those broad categories there are many sub-cases involving exact circumstances of render file durability, and whether they are used.
I've had multiple long conversations with Apple Pro Apps escalation support, both verbally and in writing on this. For some of those Apple Engineering was being consulted during the call. They refuse to admit it's a problem, so I basically gave up for now. I provided them in writing a very easy, concise replication scenario, so there is no question of misunderstanding or ambiguity. This situation apparently requires escalation to the product management level.
It's possible there is yet a new issue on 10.4.10, but so far I haven't seen it. I just noticed if a built-in "Basic Lower Third" title is placed over a clip with Neat Video, even with no other 3rd party or built-in effects, the render files will be used for timeline operations but not for export. IOW during export FCPX will re-compute all clips with Neat Video, even if valid render files exist.
I have an open case with Apple Support now to address this very issue. Mine originated with the fact that the timeline always shows a need to re-render each time you re-open the project after a close even though the project was completely rendered before closing. This was demonstrated to Apple Engineering with only standard Apple titles, generators and SFX. It was easily recreatable.
I don't really pay attention to the time it takes to do the final render but the more to the performance of the timeline. I want to watch the footage with added effects and transitions to gauge the flow but then hit some snags on certain clips and transitions. Pre rendering them makes no difference at all to how they perform on the timeline. I have tried deleting rendered files and re rendering with no effect. Re installed fcp but that did not help.
Tom Wolsky wrote: Could you give the steps to reproduce this? Thanks.
There are various manifestations of this but one of the simplest scenarios is below. I discussed this at length with Pro Apps Escalation Support, who after discussing it with Apple Engineering said any alleged use of render files to accelerate export was undocumented and a bug could not be filed.
They said this doesn't matter, there is no documented behavior that render files accelerate export, therefor there is nothing to fix or improve.
- Import a 60 sec 4k ProRes 422 clip
- Add Color Wheel effect and make any adjustment. Render dots appear.
- Do timed export to 4k ProRes 422 on a fast SSD (at least 900 MB/sec). On my 10-core Vega 64 iMac Pro, this takes about 22 sec
- Render timeline using CTRL+R. Render dots vanish.
- Repeat above export. On my machine the export time diminishes from 22 sec to 6.9 sec, or about 3x faster. This is 100% reproducible. The more compute-intensive the effect, the greater the *export* performance benefit of pre-rendering the timeline.
Tom Wolsky wrote: Thanks joema. And this is now broken in 4.10? I can’t test it with 4.8 unfortunately.
No, it's been this way quite a while. I tested each version since 10.4.6 and it happens on all of them. I don't know how far back the behavior goes.
To be clear -- the specific scenario I posted above is not a bug -- it simply illustrates the "fast path" optimization for exporting a fully-rendered ProRes 422 timeline to ProRes 422 output, vs a non-rendered timeline. The significance is Apple strongly claimed there was no such optimization and even if it existed it's undocumented, therefore no related behavior is subject to an improvement request. IOW if sometimes an export is 5x slower due to failure to use render files, Apple doesn't accept that as a bug.
I simply used this scenario because it's the simplest I could think of. Below is a slightly more complex one that is a true bug, and happens on all versions I tested since 10.4.6. Maybe it existed before then. For testing purposes, background render is disabled in all these cases.
- Import a ProRes 422 test file
- Adjust color with color wheels
- Add the built-in Aged Paper effect
- Do a timed export to ProRes 422
- Render timeline with CTRL+R; note render dots vanish
- Do timed export to ProRes 422 -- it is no faster; fails to use render files for export.
- Re-launch FCPX. Note render dots are back -- it lost the files. That is 100% reproducible on the first re-launch.
- Render timeline again with CTRL+R; render dots vanish
- Do timed export to ProRes 422 -- it is now much faster. THIS time it uses the render files.
Above is one small example. There are many variations of this. In a severe situation it can unpredictably cause render or export performance to become 50x slower.
Thanks Stooovie for reporting that. However this may be an architectural issue regarding how render cache is used. It is very unlikely to be fixed in a point release.
In general the FCP render cache works fairly well, but in some cases has significant issues. Unfortunately there is no reliable solution. Disabling background rendering and manually rendering as needed by selecting clips and doing CTRL+R can help avoid build-up of unused render files. But this doesn't address the root problem.
There are several variables and manifestations, often related to the specific effects or plugins used. These may included invalidated render files from simply switching between proxy and orig. media modes (when both modes are fully rendered), or invalidating render files after re-launching FCP, or clips which never render, or clips which show as rendered but the render cache is not used. It is possible (but not certain) that over the next year as plugins are ported to the FxPlug 4 framework (required for Apple Silicon), it might work better. However it's not isolated to 3rd-party effects built with FxPlug 3. Some of the current behaviors involve built-in effects.
I need to re-run all my tests and make another attempt to get FCP development to act on this, but that is a week of work, and the last time FCP escalation support was not receptive. Their viewpoint is for non-user-facing performance issues involving render cache, this is not documented therefore any observed behavior is not a bug. Details below.
Stooovie wrote: ...my issue is not even using caches for exporting, as the renders are.fairly quick. The issue is i can't sometimes play back in timeline in full framerate. Thats an extreme bug ij my book.
That is one of the symptoms. If the timeline is fully rendered, the scrubbing performance should be roughly equal to that timeline exported to ProRes 422, re-imported and edited. The cache is composed of fully-rendered ProRes segments -- no Fx computation should be required for scrubbing. It should be "baked in" to the render files, exactly like Fx are baked into the export. Yet there are cases with certain effects where playhead or skimmer actions remain laggy and slow. No render dots show, but it's either not using the cache properly or unnecessary Fx computation is happening when simply scrubbing.
Even though export performance seems "fairly quick" in your case, this can vary greatly based on specifics. E.g, if editing a 4k timeline in a 1080p project for 1080p H264 export, in general it must re-read, decode and render the original source files, plus encode the export to H264. This is because you might have scaled or zoomed into the frame, and using the 1080p render files would cause loss of resolution. The additional work is normal and can hide some moderate render cache issues. But if you used Neat Video or another compute-intensive effect, then it failed to use render cache, you'd know it. The export performance would be vastly slower.
But for a 4k fully-rendered timeline and 4k ProRes 422 export, it will not generally re-read the source files, but simply grab the render segments and concatenate them to an output file. It is extremely fast -- except when it doesn't work properly.
So there are various scenarios, each with differing impact and behavior, but they all have in common non-use of render cache.
It is conceivable there is some semi-justifiable explanation based on current design. E.g, certain Fx might require the Motion engine to re-interpret the stored edit directives, even though the clip is rendered and cached and no changes were made.
In either CPU or program caches, there are tag fields assigned to each cached item. Those store the valid/dirty state of the cached item plus facilitate lookup. From a developer standpoint, you're frequently concerned about inadvertently using old or "stale" cache data, which is called failed cache coherency. That produces incorrect results so it's sometimes easier to order a re-render or re-fetch, "just in case". Something like that may be happening here.
Interesting that you noted 4K media on a 1080 timeline which is often part of my workflow. Up until the 10.5 update I never experienced any real negative render impacts since I transcode everything and assumed that background renders were using the ProRes media for editing. I keep background rendering turned off and found on a recent project that doing a manual timeline render (shift+Ctrl+r) takes twice the time of the actual project length. I also watched the iStats Menu results for R/W operations on the SSD drives and was surprised to see typical speeds in the 40-60mbs range even though my drive are capable of 400-600.
Also found that sending an export to Compressor took 16mins for an 8 min project which I found unusually slow, given that I am on a 2017 iMacPro with 32G Ram and a Radeon ProVega64 with 16G ram.
john.kuehne wrote: Interesting that you noted 4K media on a 1080 timeline which is often part of my workflow....
I didn't mean to imply there was something fundamentally slower about 4k media in a 1080 timeline. In general it helps a bit because the render files are smaller and generate faster. However in certain cases (I don't know how many) it cannot properly use the 1080p render cache for export and must fetch and decode the 4k media. One obvious case is a scale operation. You can zoom 200% in a 1080p timeline based on 4k media, export to 1080p and the original 4k resolution will be used for the zoom. None of that is a bug, just good design.
If you use effects with a low compute burden, or do some operation (like a scale on 4k in a 1080p timeline) which *appropriately* prevents use of render cache on export, you may never notice if the render cache system malfunctions. You are already paying a modest perf. cost for the scale, and your other Fx don't have that much cost even if FCP does erroneous re-rendering.
OTOH if you use Neat Video, Flicker Free, Imagenomic Portraiture, etc, and pre-render the timeline, and then the render cache malfunctions, you definitely know it. An export which formerly took minutes could take an hour.
Or in a fully-rendered 4k timeline if you export to 4k ProRes 422, that should be lightning fast -- it's not even encoding, just copying cache data to the output. But if the render system malfunctions, the amount of perceived slowdown depends on the render cost of the Fx in the timeline relative to your normal export performance.
@joema - thanks for the continued perspectives. My experience has absolutely been a slowing of export/render times when using 4K media on a 1080 timeline.
Just curious if there is more efficient (from the render perspective) workflow than the way I am doing it. Would appreciate any recommendations.
1. Import 4K green screen media with transcode.
2. Import audio.
3. Create a synchronized clip
4. Create a compound clip from the synchronized clip.
5. Key the green screen footage and crop.
6. Size the media to 150-175%
7. Add the sized logo bug to the compound clip
8. Add the background image to the compound clip.
9. Edit the clip for content.
10. Manually render the compound clip in advance of putting it on the timeline.
11. Add the compound clip to the main timeline.
Noting that adding the compound clip to the main timeline indicates a need to render the timeline. Not doing a manual render (shift+ctrl+r) impacts visual quality and timing related response times of any title plugins so I typically with perform the manual render. This manual render recently (after the 10.5 update) now takes almost double the time of the original media.
Thanks for any perspectives as we continue to deal with having to re-render these timelines when re-opening them on a new FCP start.