Welcome, Guest
Username: Password: Remember me

TOPIC: RAID or not to RAID?

RAID or not to RAID? 06 Jun 2019 12:59 #100128

  • gaddster
  • gaddster's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 260
  • Thank you received: 2
  • Karma: 1
Got a new iMac i9 64gb ram machine coming next week, looking at how I store my data.

Currently ib have about 7 different drives all linked up (besides a couple of Lacie rugged I use out of the office on my mbp)

I was going to buy a Lacie 2big TB3 = usb 20tb drive and use it to store libraries, archives, design elements etc all on the one drive, but not sure if this is the right solution ie should I run a raid 0 (2 x tb drives) and mirror them so I have a copy..

Then a company is offering a virtual server solution whereas I can store and edit from a cloud (has anyone done this??)

Any Idas or suggestions would be great
MBP retina 15" 2014
imac 27 i7 2013
G-tech
Panasonic GH4, AC90, Gopro
The administrator has disabled public write access.

RAID or not to RAID? 06 Jun 2019 15:48 #100129

  • JarrodMFay
  • JarrodMFay's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 138
  • Thank you received: 25
  • Karma: 1
I think you have to judge how long your data needs to be useable. Most post houses have solutions for current, near-term and long-term data storage. If that's your situation than I would seriously recommend LTO tapes (of whatever flavor you like) for long term storage. If not (but you still need solid data storage) than just live by the mantra "If it doesn't exist on at least 2 drives then it doesn't exist). Assume every drive you own will fail at some point. Mirrored RAID is viable but can be a super hassle to recover with some setups. A cloud solution makes sense if you have sufficient upload speeds and if the cost is right for you. This is a discussion that has been had over many threads on this forum so I suggest you do a search for more nuanced opinions. Best of luck.
The administrator has disabled public write access.

RAID or not to RAID? 06 Jun 2019 19:24 #100133

  • gaddster
  • gaddster's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 260
  • Thank you received: 2
  • Karma: 1
thanks, "If it doesn't exist on at least 2 drives then it doesn't exist'... what a great way to look at it, im sure at times I store for the sake of storing... I will look through the forum now
MBP retina 15" 2014
imac 27 i7 2013
G-tech
Panasonic GH4, AC90, Gopro
The administrator has disabled public write access.

RAID or not to RAID? 07 Jun 2019 03:49 #100138

  • arc
  • arc's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 20
  • Thank you received: 4
  • Karma: 0
RAID 1 would make a back up not RAID 0. Having said that it might best to use a software RAID.
The administrator has disabled public write access.

RAID or not to RAID? 07 Jun 2019 03:56 #100139

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2592
  • Thank you received: 340
  • Karma: 32
RAID redundancy is a buffer, not a backup.
The administrator has disabled public write access.

RAID or not to RAID? 08 Jun 2019 17:29 #100165

  • arc
  • arc's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 20
  • Thank you received: 4
  • Karma: 0
RAID 1 makes two identical copies on separate drives. If one drive fails you have the same data backed up on the other RAID 1 drive. You can refer to it as a backup copy. RAID 0 splits the data across multiple drives. If one drive fails all the data is lost.
The administrator has disabled public write access.

RAID or not to RAID? 08 Jun 2019 19:53 #100166

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2592
  • Thank you received: 340
  • Karma: 32
RAID 0 is fast.
RAID 1 if one drive fails, you MUST replace the failed drive and allow time for the RAID to rebuild. During that time the overall system will slow down greatly. It is not a back-up, it is a buffer. If anything goes wrong, that single remaining drive will corrupt and you lose it all.
A backup drive is a fully independent, full performance, usually "identical" copy.
In a two-drive system, the performance difference between RAID 0 and RAID 1 can be significant.
Last Edit: 08 Jun 2019 19:55 by FCPX.guru.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 04:58 #100172

  • arc
  • arc's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 20
  • Thank you received: 4
  • Karma: 0
FCPX.guru wrote:
RAID 0 is fast.
RAID 1 if one drive fails, you MUST replace the failed drive and allow time for the RAID to rebuild. During that time the overall system will slow down greatly. It is not a back-up, it is a buffer. If anything goes wrong, that single remaining drive will corrupt and you lose it all.
A backup drive is a fully independent, full performance, usually "identical" copy.
In a two-drive system, the performance difference between RAID 0 and RAID 1 can be significant.

You are confusing RAID 5 for RAID 1.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 12:59 #100176

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2592
  • Thank you received: 340
  • Karma: 32
RAID 1, if one drive fails, first you can't use it at all (there I confused it with level 5) until you replace the bad disk with a pre-formatted (most systems) drive and allow the RIAD to rebuild itself, or you risk overwriting data.

But the rest is correct, it's a buffer, not a backup copy, the RAID 1 system would need to be rebuilt to function safely.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 14:53 #100178

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1145
  • Thank you received: 238
  • Karma: 20
FCPX.guru wrote:
RAID 1, if one drive fails, first you can't use it at all (there I confused it with level 5) until you replace the bad disk with a pre-formatted (most systems) drive and allow the RIAD to rebuild itself, or you risk overwriting data...

I think RAID 1 allows work to continue after a single disk failure. When you replace the failed disk, the rebuild will copy the good data from the surviving disk to the replacement, while allow you to continue work -- although performance may be degraded during the rebuild phase.

The behavior may vary between various RAID 1 implementations. With some, rebuild can take a long time. With others's it's pretty fast: www.softraid.com/pages/features/raid_levels.html#raid1

Your main point is valid -- RAID 1 is not the best type of backup. In the software/hardware stack there are many points of failure. RAID 1 only protects against a physical hard drive failing.

E.g, an FCPX or Lightroom library database can get corrupted -- RAID 1 writes the corrupted data to both drives. An HFS+ filesystem error can corrupt data -- it gets written to both drives. I have had both of those happen. A RAID chassis can fail, damaging both drives. Or the RAID chassis can fail non-destructively (power supply, on-board logic, etc) taking down the entire array. I've experienced that also.

There is also user error. If files or folders are accidentally deleted or modified, RAID 1 dutifully copies this to both drives. I've seen several cases where after a RAID 5 disk failure, the user accidentally replaces the wrong drive. Depending on the RAID system this can "break" the volume and require an offline rebuild or might not be recoverable.

There is no perfect backup solution but it's important to know the limitations of RAID and not treat it as a sole backup.

Where uptime is crucial, redundant RAID can help *one* cause of storage failures. A periodic offline backup using Carbon Copy, etc. can be a good complement to RAID. Continuous differential backup by Time Machine is also useful, esp. for the system drive.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 18:16 #100180

  • gaddster
  • gaddster's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 260
  • Thank you received: 2
  • Karma: 1
thanks for the detailed response, so raid 1 isn't my best solution, RAID 5 would be better but I need at least 3 equal drives, yes?
MBP retina 15" 2014
imac 27 i7 2013
G-tech
Panasonic GH4, AC90, Gopro
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 20:31 #100183

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2592
  • Thank you received: 340
  • Karma: 32
If you want performance and real RAID buffer, 4 bay RAID 5, with a cheap USB 1 drive to back it up nightly. Backup drives don't need to be fast, just reliable.

All of our dual drive enclosures are RAID 0 for speed, with a cheap slow drive as it's nightly backup. USB 3 connections give us pretty nice performance.

Our 4 bay RAIDs are level 5, and our 8 bay RAIDS are level 6, all with enterprise quality 7200rpm drives in them, T'bolt connections. We just got a very large NAS system and there's a backup directory for each drive on it now. It's long term archival storage, not for editing off of.
Last Edit: 09 Jun 2019 20:35 by FCPX.guru.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 21:43 #100185

  • joema
  • joema's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 1145
  • Thank you received: 238
  • Karma: 20
gaddster wrote:
thanks for the detailed response, so raid 1 isn't my best solution, RAID 5 would be better but I need at least 3 equal drives, yes?

I agree with FCPX.guru. RAID-1 is OK, but you are giving up 1/2 your storage (plus performance) to cover one specific failure case. My documentary team does what he described: we use RAID-0 (for 2-drive arrays) and do nightly backups.

Even for 4-drive arrays we sometimes use RAID-0 with a twin nightly backup since media doesn't change that often. For 6/8 drive systems RAID-6 is OK, but even that should be backed up.

For a Lacie 2Big, I'd suggest considering RAID-0 and another backup drive via Carbon Copy, Time Machine, etc.

We formerly used RAID-5 for 4-drive arrays, but here is the dilemma: it must be backed up, regardless. You must also consider the business case for downtime to achieve full productivity and balance that against probability of failure.

E.g, if a RAID-5 array loses a drive, enters rebuild mode for 10 hr, during which performance is degraded by 70%, you lost a day of work. If the data on that array is only updated once per day you could have switched to a nightly backup, continued work, and only lose 10 minutes. If the data on the array is updated continuously then RAID-5 could be more valuable. But it *still* must be backed up. Hoping and praying the array won't have a fatal problem is not a valid strategy -- that is what FCPX.guru is saying, probably from experience.
Last Edit: 09 Jun 2019 21:43 by joema.
The administrator has disabled public write access.

RAID or not to RAID? 09 Jun 2019 22:14 #100186

  • arc
  • arc's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 20
  • Thank you received: 4
  • Karma: 0
FCPX.guru wrote:
RAID 1, if one drive fails, first you can't use it at all (there I confused it with level 5) until you replace the bad disk with a pre-formatted (most systems) drive and allow the RIAD to rebuild itself, or you risk overwriting data.

But the rest is correct, it's a buffer, not a backup copy, the RAID 1 system would need to be rebuilt to function safely.

I stated if one drive fails. That lets you know there must be more than one drive to start with correct? You don't have to rebuild the RAID 1 because there is no spanning of data over multiple drives it is just a copy at RAID 1. RIAD 5 does what you think RAID 1 does. RIAD one is mirroring the data making an exact copy. RAID 5 does not make a copy but instead stripes the data over multiple drives. It stripes the data across multiple drives in a way that if one drive fails you can rebuild the RAID.
The administrator has disabled public write access.

RAID or not to RAID? 10 Jun 2019 12:00 #100187

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2592
  • Thank you received: 340
  • Karma: 32
RAID 1 does have to rebuild, had to replace a RAID 1 drive last week.

As for backups, I would strongly advise using Carbon Copy Cloner, as it can read inside of Packages (FCPX Libraries) and do append backups on the contents that have changed, not have to re-copy the entire Library again and again and again. Saves time, very efficient, very reliable. Not a fan of TimeMachnie for many reasons, but I know folks who use it and like it, it's a valid alternative. Just that CCC offers so much more for a good price and I, at least, am talking about data that a large company relies on as its life's blood, not to mention liability with clients (who own most of the media). Not something I want a consumer product like TimeMachine to be trusted with. Just my personal opinion.
The administrator has disabled public write access.

RAID or not to RAID? 10 Jun 2019 21:07 #100191

  • gaddster
  • gaddster's Avatar
  • OFFLINE
  • Gold Boarder
  • Posts: 260
  • Thank you received: 2
  • Karma: 1
Thanks everyone for your input, I did not realise my question was so detailed, yet straight forward in parts.

Am I Right here in saying:-

Raid 0 - uses both drives to store all data across both drives on the Lacie 20tb 2 big.. one drive breaks and im knackered? read and write speeds very good

Raid 1 - mirrors one drive so 20tb becomes 2 x 10tb .. one drive breaks and you have another? read speed slower, write speed just as quick as raid 0?

Raid 5 - min 3 drives, 1 is used as copy so you lose 1/3 straight away (with 3 drives).. read speed great, write speed suffers?

raid 6 - min 4 drives, lose 50% but uses striping across all 4 drives so if one or 2 drives falter I still have my data?
read speed great but write speed poor compared o others?

raid 10 - not sure lol
MBP retina 15" 2014
imac 27 i7 2013
G-tech
Panasonic GH4, AC90, Gopro
The administrator has disabled public write access.

RAID or not to RAID? 10 Jun 2019 22:10 #100193

  • JarrodMFay
  • JarrodMFay's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 138
  • Thank you received: 25
  • Karma: 1
Here's a good article to read about various RAID types with the pros/cons of each.

www.prepressure.com/library/technology/raid
Last Edit: 10 Jun 2019 22:11 by JarrodMFay.
The administrator has disabled public write access.

RAID or not to RAID? 11 Jun 2019 04:21 #100195

  • arc
  • arc's Avatar
  • OFFLINE
  • Junior Boarder
  • Posts: 20
  • Thank you received: 4
  • Karma: 0
gaddster wrote:
Thanks everyone for your input, I did not realise my question was so detailed, yet straight forward in parts.

Am I Right here in saying:-

Raid 0 - uses both drives to store all data across both drives on the Lacie 20tb 2 big.. one drive breaks and im knackered? read and write speeds very good

Raid 1 - mirrors one drive so 20tb becomes 2 x 10tb .. one drive breaks and you have another? read speed slower, write speed just as quick as raid 0?

Raid 5 - min 3 drives, 1 is used as copy so you lose 1/3 straight away (with 3 drives).. read speed great, write speed suffers?

raid 6 - min 4 drives, lose 50% but uses striping across all 4 drives so if one or 2 drives falter I still have my data?
read speed great but write speed poor compared o others?

raid 10 - not sure lol

Soundslike you have it more or less. You can also run a RAID 10 (1+0) which is the combination of RAID 1 and RAID 0.
The administrator has disabled public write access.