Importing FX9 (Leave Files In Place) With or WITHOUT XML?

I've had a few HFS+ filesystem crashes due to software bugs in MacOS code, confirmed by Lloyd Chambers at Mac Performance Guide. This happened when doing a force quit on his I/O stress app DiskTestR. The filesystem could not be recovered by Disk Utility, Disk Warrior or anything else. Lloyd said it was a known HFS+ bug which Apple never fixed. I haven't had that in about three years, so hopefully it's fixed now.

I previously tested several Pegasus R4 arrays for Promise, Inc, and those had a problem whereby RAID-5 failover worked but the rebuild phase took several days if using 128k stripe size, during which performance was almost unusable. Here are some graphs of those tests: 

photos.smugmug.com/photos/i-pdz6p2j/0/632a78ee/O/i-pdz6p2j.jpg

I have nine OWC Thunderbay arrays and my experience with those has been fairly good, but I'm working on a problem whereby one of them will intermittently cause a kernel panic during MacOS reboot. Fortunately it hasn't corrupted any data yet but for safety, every time that happens I must run First Aid on all attached arrays which takes a long time. Whether those are RAID-0, RAID-5, etc. doesn't make any difference in cases like that.

Re database crashes, I worked in escalation support for SQL Server for 10 years and probably patched a hundred corrupt servers at various Fortune 500 companies -- many of which were on RAID-5. For a history of that product see this book (available on line):  flylib.com/books/en/3.172.1/

Since those days the software is much more reliable, and the SQLite database used by FCP is quite good. However we still have periodic reports of damaged FCP projects and libraries, which are essentially application-layer database corruption.

In 2019 my team had one of our QNAP servers stolen from a car, despite being parked in a "safe" place. That one used RAID-6 but it obviously didn't help that situation. Fortunately we had two redundant servers in separate locations.

My heart stopped when I read this as I was looking at an import workflow for the 12Tb of footage I have from my Sony A6600 to import and ran into this same issue, on version 10.5.2 you cannot select "Leave files in place" when you run the CMD-I import screen. Other footage, no problem.

However, if you use finder to drag and drop the footage into 10.5.2, then it works fine, it leaves the files in place, imports them and the Library file size doesn't increase with that of your footage file sizes. The footage I have is 4K, 100 Highest Quality Setting at 25p.

My Import Preferences are set to 'Leave files in place' and 'Optimise Media' is turned off etc etc
Hopefully do it this way its not going to bite me in the back later on down the track....

joema wrote:

However, a new problem is the Catalyst gyro data is only present in the original camera files. If we shot to the internal codec with the intention of Catalyst stabilization in post, I don't think that would work with re-wrapped files from EditReady2. We haven't yet decided how to handle that.

Sorry to revive a dead thread but I figured I would ask @joema 6 months into your production workflow of using Atomos Recorders and the FX6/FX9 - have you found a reliable solution for working with FCPX and Catalyst Browse's image stabilisation within your editing workflow?  In an ideal world someone would have licensed a stabilisation plugin to read the data within FCPX, but not holding my breath!

I too use the Atomos Prores/Raw workflow with the internal AVC as backup, and my FX6 finally arrived this week after months on backorder (YAY!)  I haven't tested anything yet, but ideally I would import all my Prores RAW footage to FCPX and when coming across a clip that needs stabilisation, work directly with the original Internally recorded AVC footage and Catalyst browse to stabilise and reimport back to FCPX.  

I generally always import my footage to library instead of keeping files in place.  In the above scenario I imagine I would simply make an exact hard copy of the entire SD card on my scratch disk for referencing the original internal files/ metadata when needed.

Two Questions:
1) Have I read this thread correctly that the only way to incorporate the original XML files within your FCP project is to import "with files in place", which would then mean just "Reveal files in finder" to reference the gyro metadata later for stabilisation in Catalyst Browse?

2) Can the stabilisation effect be copied over to a prores file within catalyst browse?  for example:

1. Import my graded/ and transcoded prores raw file (to Prores 422 or 4444) and the original AVC file
2. Stabilise the AVC the file
3. Copy that effect (similar to how final cut works) to the graded prores file within catalyst browse
4. Export the stabilised prores file back into FCP

FCP (nor any other NLE) can use Sony gyro stabilization data. It doesn't matter how the camera files are imported.

Sony's gyro stabilization works very well, but there are several constraints:

- It doesn't work well at 24 fps due to frame blurring. To get good results requires 60 fps. I haven't tried 50 fps. Here is a test comparing 23.98 vs 59.94 fps:  vimeo.com/598896853/2bde2933ac
- All in-camera and lens stabilization must be turned off when shooting, else Catalyst will not process the gyro data. The FX6 doesn't have IBIS but all lens stabilization must be off. 
- Catalyst will only stabilize one clip at a time, they cannot be batched.
- While Catalyst processes the stabilization rapidly, the clips must be exported and encoded. Options include both ProRes or the original XAVC (which still requires encoding). Either takes a long time. That "bakes in" the stabilization to the exported files. 
- If exported as XAVC, they can be relinked to the original "in place" non-stabilized files you originally imported. If exported as ProRes those will not relink due to an audio conflict; I don't understand that because Invisor shows both XAVC and ProRes exports from Catalyst have the same audio channels and parameters.

Some previous versions of FCP would not allow importing from tree-oriented media using "leave files in place". It would only allow re-wrap and copy to library. However -- I just noticed that's no longer the case, at least for Sony XAVC-I from the FX6 and XAVC-S from the A7RIII. I don't know what version of FCP changed that, or if I somehow did not remember it correctly. 

The UI is a bit tricky in that area and varies based on what folder level you click on if using the FCP import dialog, or whether you import from Finder using drag/drop.

You cannot copy over the Catalyst gyro stabilization as you mentioned. It does not exist as an effect -- it is processed by Catalyst externally then baked into the encoded output file. Apparently the only option is to import unstabilized XAVC using "leave files in place", then later if gyro stabilization is needed to certain clips do that in Catalyst, export then relink in FCP.

Again -- to even have the gyro stabilization option you must plan ahead and all camera sensor and lens stabilization must be turned off when shooting those clips.

Test of Catalyst gyro stabilization on FX6 and 28-135 lens at 28mm, walking, hand held:  vimeo.com/617097074/aa54cef30c

Various tests at 23.98 and 59.94 fps of Catalyst vs optical vs FCP stabilization on FX6 using Sony 70-200 G-Master lens: