hello friends! new(ish)!
Home Server/Setting up your Storage: Difference between revisions
>DataHoarderAnon mNo edit summary |
>DataHoarderAnon (Added additional sections and some content. This is a skeleton page (for now).) |
||
| Line 1: | Line 1: | ||
== Setting up your RAID Solution == | |||
=== mdadm === | |||
=== ZFS === | |||
ZFS has a lot of really great features that make a a superb file system. It has file system level checksums for data integrity, file self healing which can correct silent disk errors, incremental snapshots and rollback, file deduplication, encryption, and more. | ZFS has a lot of really great features that make a a superb file system. It has file system level checksums for data integrity, file self healing which can correct silent disk errors, incremental snapshots and rollback, file deduplication, encryption, and more. | ||
There are however, some downsides to ZFS. Notably inflexibility and the upfront cost. ZFS | There are however, some downsides to ZFS. Notably inflexibility and the upfront cost. ZFS RAIDZ arrays CANNOT BE EXPANDED after being created. Parity cannot be added either (you cannot change a RAIDZ1 to a RAIDZ2 later on). You cannot use differently sized disks or disks with data already on them, even disks formatted as ZFS. In other words, you need to buy ALL of the disks you plan on using in your RAIDZ array at the same time, because unlike other software RAID (or even hardware RAID), you won't be able to change it later. This inherently makes ZFS costly to use and thus unfriendly to more budget oriented server builds. Now also add in the fact that running ZFS also requires a hefty amount of RAM, preferably ECC ram, which is expensive in and of itself and requires server hardware to utilise to it's fullest, and that some of the fancy features like dedup also require a good processor too... The price tag starts to add up really quickly. | ||
So when asking yourself "Should I use ZFS?" you really should be asking "Do I really need ZFS?" (Do I want long term data integrity and all those other fancy features?) and "Can I afford ZFS?". If the answer to both of those questions is "Yes", then you can and should use ZFS, otherwise use something else like Snapraid or mdadm. | So when asking yourself "Should I use ZFS?" you really should be asking "Do I really need ZFS?" (Do I want long term data integrity and all those other fancy features?) and "Can I afford ZFS?". If the answer to both of those questions is "Yes", then you can and should use ZFS, otherwise use something else like Snapraid or mdadm. | ||
[https://openzfs.github.io/openzfs-docs/index.html ZFS Documentation] | [https://openzfs.github.io/openzfs-docs/index.html ZFS Documentation] | ||
=== Btrfs === | |||
=== Snapraid === | |||
=== Hardware RAID === | |||
{{Warning| Using hardware RAID and software RAID at the same time is NOT recommended. If you wish to pool multiple hardware RAID arrays together into a logical volume use LVM.}} | |||
If you bought an old used server with a RAID controller already installed, or perhaps you don't feel like messing with software RAID solutions, you have the option of using hardware RAID rather than software RAID. | |||
== Choosing a file system == | |||
===XFS=== | |||
===ext4=== | |||
===NTFS=== | |||
If you are using snapraid as your raid solution, using NTFS formatted drives is perfectly fine. With Snapraid you are usually pulling out random drives you have lying around, which are most likely to be NTFS formatted. Otherwise, we do not recommend using NTFS unless you are running a Windows server for some reason. It does not have the same level of support on Linux and UNIX based systems as ext4 and XFS. | |||
unRAID '''does not''' support NTFS. If you are using unRAID you will need to use ext4 or XFS. | |||
== Distributed Filesystems == | |||
===CEPH=== | |||
===seaweedfs=== | |||
===lizardfs=== | |||
===moosefs=== | |||
== See also == | == See also == | ||
*[[Home server]] | *[[Home server]] | ||
*[[Storage devices]] | *[[Storage devices]] | ||
Revision as of 20:03, 22 December 2020
Setting up your RAID Solution
mdadm
ZFS
ZFS has a lot of really great features that make a a superb file system. It has file system level checksums for data integrity, file self healing which can correct silent disk errors, incremental snapshots and rollback, file deduplication, encryption, and more.
There are however, some downsides to ZFS. Notably inflexibility and the upfront cost. ZFS RAIDZ arrays CANNOT BE EXPANDED after being created. Parity cannot be added either (you cannot change a RAIDZ1 to a RAIDZ2 later on). You cannot use differently sized disks or disks with data already on them, even disks formatted as ZFS. In other words, you need to buy ALL of the disks you plan on using in your RAIDZ array at the same time, because unlike other software RAID (or even hardware RAID), you won't be able to change it later. This inherently makes ZFS costly to use and thus unfriendly to more budget oriented server builds. Now also add in the fact that running ZFS also requires a hefty amount of RAM, preferably ECC ram, which is expensive in and of itself and requires server hardware to utilise to it's fullest, and that some of the fancy features like dedup also require a good processor too... The price tag starts to add up really quickly.
So when asking yourself "Should I use ZFS?" you really should be asking "Do I really need ZFS?" (Do I want long term data integrity and all those other fancy features?) and "Can I afford ZFS?". If the answer to both of those questions is "Yes", then you can and should use ZFS, otherwise use something else like Snapraid or mdadm.
Btrfs
Snapraid
Hardware RAID
If you bought an old used server with a RAID controller already installed, or perhaps you don't feel like messing with software RAID solutions, you have the option of using hardware RAID rather than software RAID.
Choosing a file system
XFS
ext4
NTFS
If you are using snapraid as your raid solution, using NTFS formatted drives is perfectly fine. With Snapraid you are usually pulling out random drives you have lying around, which are most likely to be NTFS formatted. Otherwise, we do not recommend using NTFS unless you are running a Windows server for some reason. It does not have the same level of support on Linux and UNIX based systems as ext4 and XFS.
unRAID does not support NTFS. If you are using unRAID you will need to use ext4 or XFS.