Ceph
Reddit take
Not enough linked mention summaries yet; the source excerpts below are the evidence.
Pros
- I've found it to be rock-solid if you thoroughly understand how it works and what it's doing. In some failure modes (e.g. a machine or a disk fails), Ceph will automatically "self-heal" i.e. redistribute data and I/O to other devices as...
- CEPH has a very effective way of doing this, using placement Groups spread across many drives, so you don't recover a failed drive, but instead rebuild all the affected Placement Groups in parallel,using many different drives.
- BTW, don't let the haters say you shouldn't use CEPH in a homelab, it's great, I love it.
- Ceph is by far the most mature alternative and you can run it in single node.
- Other option with HA would be to fill your current servers with SSDs (change your RAID card for an HBA) and migrate to PROXMOX + CEPH for an HCI cluster
Cons
- Ceph could be a viable alternative, but is clustered.
- According to my research, 1G networking is unviable for Ceph, especially without a dedicated NIC. ... for ceph, you could add (or replace the wifi card) a m.2 2,5G network card and use it for ceph only, but yes, 3 nodes and 10G is the mi...
- Each node has a 256gb boot nvme drive but im not sure the network would be fast enough for ceph? Its only gigabit which i heard ceph would crawl on.
- I've played with ceph before, enough to know I do not want to manage ceph as a full time job lol.
- Ceph is kind of a high learning curve and to follow the spec means spending a lot of money for larger amounts of storage (anything over 2tb in my mind).
Sentiment
22 positive (63%) · 10 mixed (29%) · 3 negative (9%)
Trust + time weighted score: +53% · raw score 54%
What redditors said (10 of 35)
I've found it to be rock-solid if you thoroughly understand how it works and what it's doing. In some failure modes (e.g. a machine or a disk fails), Ceph will automatically "self-heal" i.e. redistribute data and I/O to other devices as necessary
CEPH has a very effective way of doing this, using placement Groups spread across many drives, so you don't recover a failed drive, but instead rebuild all the affected Placement Groups in parallel,using many different drives.
BTW, don't let the haters say you shouldn't use CEPH in a homelab, it's great, I love it.
Ceph is by far the most mature alternative and you can run it in single node.
Ceph could be a viable alternative, but is clustered.
According to my research, 1G networking is unviable for Ceph, especially without a dedicated NIC. ... for ceph, you could add (or replace the wifi card) a m.2 2,5G network card and use it for ceph only, but yes, 3 nodes and 10G is the minium requierment and scale with more nodes, more ssds/disks and faster network (you could use ceph with less, but the performance isn't great).
Other option with HA would be to fill your current servers with SSDs (change your RAID card for an HBA) and migrate to PROXMOX + CEPH for an HCI cluster
I agree that 1Gb is a bottleneck, but it certainly runs well, even with Frigate that is I/O intensive.
Each node has a 256gb boot nvme drive but im not sure the network would be fast enough for ceph? Its only gigabit which i heard ceph would crawl on.
Ceph storage is your best bet if each node has its dedicated NVME drive, and SSD to run the Proxmox OS.