[H]ard|Forum - View Single Post - ZFS: RAID-Z3 (raidz3) Recommended Drive Configuration :
RAID-Z3 is better than RAID 10 in most cases that do not require high performance for small, random writes. Certainly for a media server, the performance of RAID-10 is not required.
First, with 10 drives of capacity C, you get only 5C available capacity with RAID 10. You get 7C with RAID-Z3. That is 40% more space.
Then there is the probability of data loss.
Assume that the probability of a single drive failing during a rebuild (i.e., after you have already lost one drive) is F. Note that F is probably higher than the pro-rated annual failure rate over the rebuild time, since all of the data on at least one of the drives must be read during the rebuild, which creates a greater stress on the drive(s) than is seen by the average drive in AFR studies.
So, the probability of data loss with RAID 10, after one drive has already failed, is F.
After a single drive failure with RAID-Z3, you must lose at least three more drives for data loss to occur. To compute this, we need the general formula for probability of failure of exactly X drives out of K drives. It is
C[K, X] F^X (1 - F)^(K - X)
where C[K, X] is the combination formula, the number of ways of choosing X out of K:
C[K, X] = K! / ( X! (K - X)! )
The probability of losing at least 3 drives out of an N-drive RAID-Z3 that has had a single drive failure is (1 - P3), where P3 is the probability of losing 0, 1 or 2 drives.
P3 = (1 - F)^(N - 1) + (N - 1) F (1 - F)^(N - 2) + (N - 1)(N - 2) F^2 (1 - F)^(N-3) / 2
As long as we have that formula, it is easy to compute the corresponding P for RAID-Z1 and RAID-Z2:
P2 = (1 - F)^(N - 1) + (N - 1) F (1 - F)^(N - 2)
P1 = (1 - F)^(N - 1)
Let's take an example. We will assume F = 5%. A 16-drive RAID-10 will have a capacity of 8C, the same as an 11-drive RAID-Z3, a 10-drive RAID-Z2, or a 9-drive RAID-Z1. So, for a capacity of 8C, the probability of data loss during a rebuild after a single drive failure is:
RAID-10:
F = 5%
RAID-Z1:
1 - (1 - F)^(9 - 1) = 33.7%
RAID-Z2:
1 - (1 - F)^(10 - 1) - (10 - 1) F (1 - F)^(10 - 2) = 7.1%
RAID-Z3:
1 - (1 - F)^(11 - 1) - (11 - 1) F (1 - F)^(11 - 2) - (11 - 1)(11 - 2) F^2 (1 - F)^(11 - 3) / 2
= 1.15%
So, for a capacity of 8C, you need only 11 drives with RAID-Z3, as compared to 16 drives with RAID-10, and your probability of data loss during a rebuild after a single drive failure is only 1% with RAID-Z3, as compared to 5% with RAID-10.
If F were only 2%, then the corresponding data loss probabilities for RAID-10, -Z1, -Z2, -Z3 are: 2%, 14.9%, 1.3%, and 0.086%. So that is less than 0.09% for RAID-Z3, as compared to 2% for RAID-10 : 23 times higher chance of data loss with RAID-10 as compared to RAID-Z3 !
Bottom line is that RAID-Z3 is greatly superior to RAID-10, except in rare applications where the higher performance of RAID-10 is required.
