fbpx
Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1

TOPIC: Phantom library size - file corruption?

Phantom library size - file corruption? 10 Dec 2019 01:39 #103077

  • theDV8
  • theDV8's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 3
  • Thank you received: 0
I have a very odd problem that I'm not sure how to describe. I think it started with my late 2014 27-inch iMac which started exhibiting very strange behavior like having a file in the recycle bin that I couldn't purge, and random reboots, and extreme slowness (took 15 minutes to boot up). I ended up transferring all my files to a NAS and reinstalling MacOS which seems to have fixed the problems on the iMac for now, but the files I transferred are in a sorry state.

As it pertains to FCPX Libraries, some of my libraries are reading ridiculously large file sizes (multiple terabytes). For reference my average libraries are around 20GB. I can still load the file and the video and all my edits seem to be intact. Deleting render files, proxies, etc does not reduce the file size and if I peak into the package contents, I see things like the ".fcpcache" folder repeating itself (I'll attach a screen shot), where clicking ".fcpcache" shows the same contents as the parent folder, and then clicking the ".fcpcache" folder shows the contents of that parent folder ad infinitum.

One thing I tried to do is to make a copy of the original library which then reduced the size of the package to 0 bytes, but this isn't right either. These copied libraries open up and the videos and my edits are all in tact.

Any idea on how I can fix this? I'm not even sure what the problem is exactly. If it were file corruption I'd imagine the library wouldn't open properly. I can't seem to find any posts pertaining to this issue, so if anyone has any ideas I'd really appreciate the help.
Attachments:

Please Log in or Create an account to join the conversation.

Phantom library size - file corruption? 10 Dec 2019 13:20 #103091

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1629
  • Karma: 27
  • Thank you received: 352
In some cases space reported by the file system to Finder can be out of date. Run Disk Utility First Aid on all volumes. To run this on the system or boot volume, use Recovery Mode, else it will be read-only and not make any updates: support.apple.com/en-us/HT201314

This is for locally-attached storage. This does not apply to network-attached storage. Various items like Spotlight indexing may not work predictably on NAS storage, so for problems like this stick to troubleshooting on local volumes.

Also use only HFS+ (MacOS Extended Journaled) or APFS for FCPX libraries and data on local drives. Do not use ExFAT or any other local file system.

Please Log in or Create an account to join the conversation.

Last edit: by joema.

Phantom library size - file corruption? 11 Dec 2019 01:00 #103111

  • theDV8
  • theDV8's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 3
  • Thank you received: 0
Thanks for your response Joema, I think you're suggesting I copy the files back to the Mac to troubleshoot, but the problem is that the files are now so large, I don't have enough room on my local drive to house them. For example, I have one file that is showing as a 1.2 terabyte file, but I only have about 600GB of free space on my local drive.

One thing I should clarify is that the file sizes are only showing up too large when I check the file size through Finder on a Mac. When I look at the file through SSH or through the NAS web interface, it's showing up correctly as a 14.2GB file. Is there any way to fix these while they're on the NAS? I am accessing the NAS via SMB3.

Please Log in or Create an account to join the conversation.

Phantom library size - file corruption? 11 Dec 2019 17:08 #103134

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1629
  • Karma: 27
  • Thank you received: 352

theDV8 wrote: ...the file sizes are only showing up too large when I check the file size through Finder on a Mac. When I look at the file through SSH or through the NAS web interface, it's showing up correctly as a 14.2GB file. Is there any way to fix these while they're on the NAS? I am accessing the NAS via SMB3.


Google shows some references to problems like this on older versions of FCPX. What version of FCPX and MacOS are you running?

Before copying to the NAS, was any of the library or data on ExFAT or any other partition type besides HFS+ or APFS?

It's possible the libraries or related file structure were somehow damaged due to your previous machine hangs. If that was never addressed via First Aid Disk Utility, it's conceivable that orphaned space was allocated. How that would be handled if later copied to an SMB NAS share, I don't know.

The .fcpcache entry is a hidden file inside the library which is a symlink pointing either to the current directory (if default library-internal cache is used) or to the specified location if external cache is selected via library inspector>storage locations>modify settings>cache.

A FCPX library bundle is actually a directory tree, and how Finder handles space reporting for that, I don't know. It may have some kind of cached optimization to avoid walking the tree when showing space consumption. If that cached value is stale or wrong, I don't know how to update it, especially if the library is copied in that state to a NAS share.

How HFS+ or APFS symbolic links or hard links are handled when copied to a NAS, I don't know. I'm not experienced with NAS-hosted FCPX libraries and media. Maybe someone else could advise.

Please Log in or Create an account to join the conversation.

Phantom library size - file corruption? 12 Dec 2019 10:39 #103159

  • theDV8
  • theDV8's Avatar Topic Author
  • Offline
  • Fresh Boarder
  • Fresh Boarder
  • Posts: 3
  • Thank you received: 0
Hi Joema, thanks so much for your thorough response. I am running FCPX 10.4.8 on MacOS 10.15.2 on both machines so I don't think its an issue with running outdated versions of either.

I create a library per project and archive them to the NAS as soon as they're done, so the problem is with the projects that I was working on when I had the problem with the iMac last month. I can't remember exactly what version I was on on FCPX or MacOS at the time, but it was whatever was in public release at the time, and I've been using the APFS filesystem since it was released in High Sierra. I did try and run First Aid Disk Utility before I copied the files over and reinstalled MacOS but it found no errors. I also ran Diagnostic mode to check for memory errors and that didn't turn up any issues either.

I found some old posts on recovering corrupt libraries by deleting the FCPX database file and letting FCPX rebuild the files upon opening the library, but it looks like they've changed the way this is handled since those posts were published. Do you know if there's some similar work around for the current version of FCPX that might be able to force FCPX to rebuild the library and perhaps recalculate the disk space used?

Thanks again for taking the time to reply to my messages, it's very helpful!

Please Log in or Create an account to join the conversation.

Phantom library size - file corruption? 12 Dec 2019 17:37 #103175

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1629
  • Karma: 27
  • Thank you received: 352

theDV8 wrote: ... Do you know if there's some similar work around for the current version of FCPX that might be able to force FCPX to rebuild the library and perhaps recalculate the disk space used?...


If I understand correctly, your library and edits seem to function properly from the standpoint of FCPX. It's just an issue with the library bundle taking up (apparently) too much space after copying to the NAS? And that is only within Finder not other tools? Maybe there's also an issue with the hidden .fcpcache folder within the library showing recursive copies?

I wouldn't try to fix anything on a NAS share. We don't even understand why Finder seems to report excessive space consumption. You could try the terminal command du -s -h LibraryName.fcpbundle, and see what that returns (where LibraryName.fcpbundle is the name of your library in question.)

You would first have to navigate to the folder containing the library. See available terminal documentation for this.

If there is reason to believe the Finder-indicated space consumption is erroneous due to NAS or SMB issues, then you might have enough space to copy the library back to a local drive where we trust and understand the behavior better.

Please Log in or Create an account to join the conversation.

  • Page:
  • 1