Welcome, Guest
Username: Password: Remember me

TOPIC: HDDs: Max Capacity vs. recommended Capacity

HDDs: Max Capacity vs. recommended Capacity 30 Aug 2019 07:12 #101162

  • Michi
  • Michi's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 45
  • Thank you received: 1
  • Karma: 0
Hey fcp.co useers

I have read several times that it is not recommended to fill up more than 75% of a Hard disks capacity with data as higher volumes can impact performance, e.g. don't put more than 6TB of data on a 8TB drive...
Two questions in this regard:
- Does it only impact performance or also a drives reliability if I use more than 75 percent of the capacity? I have two 8TB backup drives that have more than 7TB of data on them. I can live with reduced performance but would not want to higher the risk of a failure...
- Does this "rule" count for SSDs too? I use several Samsung 1TB T5 for on the go work and sometimes have more than 900GB of footage on them. Bad idea?

Any feedback highly appreciated!
Cheers
Last Edit: 30 Aug 2019 07:20 by Michi.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 30 Aug 2019 19:07 #101172

  • VidGreg
  • VidGreg's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 932
  • Thank you received: 185
  • Karma: 28
Hi Michi
Keeping 25% free space on drives is a good idea, especially hdds. You will suffer performance loss as drives fill up. This performance loss will keep on increasing as drives get fuller and fuller until a spinning beach ball is normal response to anything you do. Hdds may start to see performance dropping even at 50% fill.
SSDs do perform more consistently up till 90% fill or so, then you start seeing performance issues. Also, ssd formatted as APFS do a copy on write and writes in blocks and "Trim" controllers may have an effect.
How external drives are connected can play a role. Connect using at least USB3 or TB.

How much thru-put you need depends on what types of work and video formats/ resolution you are doing. 4k and up needs fast bandwidth and slow drives will drive you crazy. If your bandwidth thru-put is too slow, you may have dropped frames. Multicam and effects contribute to bandwidth needs. If doing SD, don't worry about it, almost all drives would be fine.
If your back up drives are really just for back-up and not actual editing, then you can decrease the percentage of free space, this is especially true if you back up at end of day instead of background, but if you then need the drive for actual editing, you would face the performance issues. Even so, try to maintain some free space. Once a drive gets to single digit free space you will see some extreme performance issues.
Lots of sites have done performance testing showing drives as they fill up. One site I think is good is Lloyd Chambers macperformanceguide.com. He has graphs showing various drives performance as they fill. Other sites also do performance testing like Tom's Hardware.

Hope this Helps, Greg
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 31 Aug 2019 08:18 #101182

  • Michi
  • Michi's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 45
  • Thank you received: 1
  • Karma: 0
Hey VidGreg

Yes, that helps. Thank you for the detailed explanation.
I'm really just talking about Backup-Drives: 8TB drives from WD and Seagate, that I use 2-3 times a week to clone my Promise Raid (from which I do all my editing and media management). High Performance is not crucial for me on this drives. I was just wondering if it could have an impact on reliability too if I fill up the drives to 90%. But if I understand you right that should not be the case in general.
On a side note: you mentioned formatting SSDs as APFS. I have my Samsung T5s all formatted as EXFAT, the option for APFS doesn't even appear in Disk Utility for those drives... Am I missing something? I thought EXFAT is the best option since I sometimes use them to copy data to Microsoft machines too...

Cheers,
Michael
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 01 Sep 2019 15:34 #101209

  • VidGreg
  • VidGreg's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 932
  • Thank you received: 185
  • Karma: 28
Hi Michael
Do you use Windows for video editing as well as FCPX? Or is it just sometimes you need to copy some files between Mac and PC?
exFAT is okay and the most compatible when going between Mac and PCs. I think using native MacOS Extended HFS+ formatting for the spinning hard drives is a better and more reliable format. Personally, I'd stay away from NTFS (new default Windows format) unless only for PC's not Macs. There are translation programs to allow Mac to write to NTFS, but what a pain and will slow your thru-put and cause many issues. Macs can read NTFS, but the write scene is truly messed up.

Not sure why you do not see APFS option for your ssd's?? What version of macOS are you running? APFS is available at least from High Sierra forwards. If you are only using the ssd for editing on Macs, I would see about reformatting next time to APFS. Use separate drives for PC data transfers.

WARNING Do Not Reformat without having back-ups as Reformatting will ERASE your drive.

I would not ever use APFS for spinning hard drives, not good, and Time Machine cannot use APFS and needs HFS+.
Having said that, APFS can be faster and more reliable for internal and external ssd's.

Lastly, I feel that stuffing drives into the 90% full can reduce reliability as this will increase fragmentation of files and due to the way that the OS handles adding and copying files, you risk corrupted data as the system tries to find room and the drive heads will be going crazy trying to find and write space. Having said this, I don't always follow my own advice. Do as I say, not as I do :silly:
Drives, especially Hdd's are cheap enough to not take chances.

Hope this Helps, Greg
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 05 Sep 2019 21:22 #101288

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
I use exFAT. It seems to work OK. Having said that SSD do not take as big of hit in performance like mechanical HD when they get 75% full but because of the way the data is written it will degrade the hard drive faster. Having said that the newer SSD can be take more wear an tear.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 06 Sep 2019 12:22 #101300

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2849
  • Thank you received: 376
  • Karma: 34
This only applies to system drives, SSD and HDD both. The OS and apps need to create/delete and read/write invisible, temporary working files. Plus any VRAM that gets used on the system drive. The more apps, the more documents/project files, the more space they need for these files. Technically 15% minimum free space is needed to be safe. 25% is a bit overkill but can give you peace of mind.

When the drive doesn't have enough space for all these temp working files, it has to cache more and more to RAM and VRAM and slows the system down. If there's not enough space for even that, then you end up with symptoms that may not make a lot of sense and possibly lost data.

This applies equally to SSD and HDD, as it is about physical capacity, not speed.

External drives can run fine with very little of their capacity left as free space, but to be safe and keep a good performing system, we generalize the 15% (25%) to them, also. Like applying television broadcast action/title safe areas to web videos, it just makes them cleaner and looks more professional.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 06 Sep 2019 16:57 #101306

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
FCPX.guru wrote:
This only applies to system drives, SSD and HDD both. The OS and apps need to create/delete and read/write invisible, temporary working files. Plus any VRAM that gets used on the system drive. The more apps, the more documents/project files, the more space they need for these files. Technically 15% minimum free space is needed to be safe. 25% is a bit overkill but can give you peace of mind.

When the drive doesn't have enough space for all these temp working files, it has to cache more and more to RAM and VRAM and slows the system down. If there's not enough space for even that, then you end up with symptoms that may not make a lot of sense and possibly lost data.

This applies equally to SSD and HDD, as it is about physical capacity, not speed.

External drives can run fine with very little of their capacity left as free space, but to be safe and keep a good performing system, we generalize the 15% (25%) to them, also. Like applying television broadcast action/title safe areas to web videos, it just makes them cleaner and looks more professional.

?
For the older mechanical HD the read and write speed will be slower when 75% full. If you are doing cuts only editing of a single video layer as opposed to playing back 7 or 8 video layers you may not notice any difference.

The SSD have a limited amount of read and write. It does not mater if it is the OS drive or not. If you add videos and delete videos every day or even use an external SSD as a cache drive the result will be the same. SSD write the data to blocks. It has to write a new block and then delete the old one. If you only have 15% free space those blocks will wear out quicker than if you had 50% free space. I admit the new SSD are more durable but the blocks can still go bad over time.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 06 Sep 2019 23:35 #101310

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2849
  • Thank you received: 376
  • Karma: 34
"For the older mechanical HD the read and write speed will be slower when 75% full."

On very old drives when they were slow that was true, not on modern drives. I have a few very full that run just fine. HDDs are fast enough now that this is not the case today.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 07 Sep 2019 02:22 #101312

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
FCPX.guru wrote:
"For the older mechanical HD the read and write speed will be slower when 75% full."

On very old drives when they were slow that was true, not on modern drives. I have a few very full that run just fine. HDDs are fast enough now that this is not the case today.

You kind of miss quoted me. I also stated If you are doing cuts only editing of a single video layer as opposed to playing back 7 or 8 video layers you may not notice any difference.
I have a few very full that run just fine.

No one doubts your HD works at 85% capacity but the question is not if a HD that is 85% full will work. The question is will performance take a hit as the hard drive fills up. Having said that I am editing on a 1.5 terabyte drive that only has 80 GB of available space. I don't see much of a difference from when it was new but I am editing AVCHD at 1920X1080. You would have to do a benchmark at 10% capacity and 85% capacity and see if there is a difference. You will not get a huge 50% difference in performance but you should see a performance loss. Even if it is only 15%-20% it can cause someone's system to drop frames if they edit Apple Pro Res at 4K or uncompressed HD with a picture in picture.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 07 Sep 2019 13:02 #101316

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2849
  • Thank you received: 376
  • Karma: 34
15-20% is the minimum safe benchmark for a system drive.

We edit 4K H.264 multicams daily, yes, I'm talking about real world performance.

Carry on.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 07 Sep 2019 17:05 #101319

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
FCPX.guru wrote:
15-20% is the minimum safe benchmark for a system drive.

We edit 4K H.264 multicams daily, yes, I'm talking about real world performance.

Carry on.

No one is saying you cannot edit video on a HD this 90% full. I do it all the time. Having said that 4K h.264 only uses about 150 megabits per second. It is not hard to edit. Pro Res 4K and uncompressed HD will use over 100 Megabytes per second (big difference form megabits to Megabytes). Simply stating you can edit h.264 4K with your HD 90% full doesn't mean there isn't a performance loss when the HD capacity gets full. You would have to do a benchmark (below is a CPU and GPU benchmark). I have heard some HD will not write into the inner portion of the disks so there may be a few HD that don't take a performance hit but all of mine do. Other people's hard drives probably do as well.

The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 11 Oct 2019 16:20 #102007

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
More people may stumble onto this thread from time to time. The video might be worth watching,

The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 12 Oct 2019 01:58 #102012

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2849
  • Thank you received: 376
  • Karma: 34
And this capacity limit is for system drives, not secondary drives.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 12 Oct 2019 18:38 #102017

  • arc
  • arc's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 98
  • Thank you received: 12
  • Karma: 1
FCPX.guru wrote:
And this capacity limit is for system drives, not secondary drives.

How could the drive be at zero capacity if it is a system drive? Keep in mind the drives are USB. Who uses a USB drive for a system drive? My SATA drives don't take that big of hit in performance as they get full. It was not the most scientific testing but some people might notice similar results.
The administrator has disabled public write access.

HDDs: Max Capacity vs. recommended Capacity 12 Oct 2019 20:28 #102023

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2849
  • Thank you received: 376
  • Karma: 34
It's not the performance that takes a hit when a system drive gets overly full, it's erratic symptoms and the danger of data loss. As a retired IT engineer, I saw it a lot. I'm just offering data, nothing more.
The administrator has disabled public write access.