Remove glusterfs.
This commit is contained in:
19
CLAUDE.md
19
CLAUDE.md
@@ -30,10 +30,7 @@ NixOS cluster configuration using flakes. Homelab infrastructure with Nomad/Cons
|
|||||||
└── services/ # Nomad job specs (.hcl files)
|
└── services/ # Nomad job specs (.hcl files)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Current Architecture (transitioning)
|
## Current Architecture
|
||||||
|
|
||||||
**OLD**: GlusterFS on c1/c2/c3 at `/data/compute` (being phased out)
|
|
||||||
**NEW**: NFS from zippy at `/data/services` (current target)
|
|
||||||
|
|
||||||
### Storage Mounts
|
### Storage Mounts
|
||||||
- `/data/services` - NFS from `data-services.service.consul` (zippy primary, c1 standby)
|
- `/data/services` - NFS from `data-services.service.consul` (zippy primary, c1 standby)
|
||||||
@@ -86,26 +83,18 @@ NixOS cluster configuration using flakes. Homelab infrastructure with Nomad/Cons
|
|||||||
|
|
||||||
## Migration Status
|
## Migration Status
|
||||||
|
|
||||||
**Phase**: 4 in progress (20/35 services migrated)
|
**Phase 3 & 4**: COMPLETE! GlusterFS removed, all services on NFS
|
||||||
**Current**: Migrating services from GlusterFS → NFS
|
**Next**: Convert fractal to NixOS (deferred)
|
||||||
**Next**: Finish migrating remaining services, update host volumes, remove GlusterFS
|
|
||||||
**Later**: Convert fractal to NixOS (deferred)
|
|
||||||
|
|
||||||
See `docs/MIGRATION_TODO.md` for detailed checklist.
|
See `docs/MIGRATION_TODO.md` for detailed checklist.
|
||||||
|
|
||||||
**IMPORTANT**: When working on migration tasks:
|
|
||||||
1. Always update `docs/MIGRATION_TODO.md` after completing each service migration
|
|
||||||
2. Update both the individual service checklist AND the summary counts at the bottom
|
|
||||||
3. Pattern: `/data/compute/appdata/foo` → `/data/services/foo` (NOT `/data/services/appdata/foo`!)
|
|
||||||
4. Migration workflow per service: stop → copy data → edit config → start → update MIGRATION_TODO.md
|
|
||||||
|
|
||||||
## Common Tasks
|
## Common Tasks
|
||||||
|
|
||||||
**Deploy a host**: `deploy -s '.#hostname'`
|
**Deploy a host**: `deploy -s '.#hostname'`
|
||||||
**Deploy all**: `deploy`
|
**Deploy all**: `deploy`
|
||||||
**Check replication**: `ssh zippy journalctl -u replicate-services-to-c1.service -f`
|
**Check replication**: `ssh zippy journalctl -u replicate-services-to-c1.service -f`
|
||||||
**NFS failover**: See `docs/NFS_FAILOVER.md`
|
**NFS failover**: See `docs/NFS_FAILOVER.md`
|
||||||
**Nomad jobs**: `services/*.hcl` - update paths: `/data/compute/appdata/foo` → `/data/services/foo` (NOT `/data/services/appdata/foo`!)
|
**Nomad jobs**: `services/*.hcl` - service data stored at `/data/services/<service-name>`
|
||||||
|
|
||||||
## Troubleshooting Hints
|
## Troubleshooting Hints
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@
|
|||||||
./unattended-encryption.nix
|
./unattended-encryption.nix
|
||||||
./cifs-client.nix
|
./cifs-client.nix
|
||||||
./consul.nix
|
./consul.nix
|
||||||
./glusterfs-client.nix # Keep during migration, will be removed in Phase 3
|
|
||||||
./nfs-services-client.nix # New: NFS client for /data/services
|
./nfs-services-client.nix # New: NFS client for /data/services
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +0,0 @@
|
|||||||
{ pkgs, ... }:
|
|
||||||
{
|
|
||||||
environment.systemPackages = [ pkgs.glusterfs ];
|
|
||||||
|
|
||||||
fileSystems."/data/compute" = {
|
|
||||||
device = "192.168.1.71:/compute";
|
|
||||||
fsType = "glusterfs";
|
|
||||||
options = [
|
|
||||||
"backup-volfile-servers=192.168.1.72:192.168.1.73"
|
|
||||||
"_netdev"
|
|
||||||
];
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
{
|
|
||||||
pkgs,
|
|
||||||
config,
|
|
||||||
lib,
|
|
||||||
...
|
|
||||||
}:
|
|
||||||
{
|
|
||||||
services.glusterfs = {
|
|
||||||
enable = true;
|
|
||||||
};
|
|
||||||
|
|
||||||
environment.persistence."/persist".directories = [ "/var/lib/glusterd" ];
|
|
||||||
|
|
||||||
# TODO: each volume needs its own port starting at 49152
|
|
||||||
networking.firewall.allowedTCPPorts = [
|
|
||||||
24007
|
|
||||||
24008
|
|
||||||
24009
|
|
||||||
49152
|
|
||||||
49153
|
|
||||||
49154
|
|
||||||
49155
|
|
||||||
];
|
|
||||||
}
|
|
||||||
@@ -37,17 +37,17 @@ See [CLUSTER_REVAMP.md](./CLUSTER_REVAMP.md) for detailed procedures.
|
|||||||
## Phase 3: Migrate from GlusterFS to NFS
|
## Phase 3: Migrate from GlusterFS to NFS
|
||||||
- [x] Update all nodes to mount NFS at `/data/services`
|
- [x] Update all nodes to mount NFS at `/data/services`
|
||||||
- [x] Deploy updated configs (NFS client on all nodes)
|
- [x] Deploy updated configs (NFS client on all nodes)
|
||||||
- [ ] Stop all Nomad jobs temporarily
|
- [x] Stop all Nomad jobs temporarily
|
||||||
- [ ] Copy data from GlusterFS to zippy NFS
|
- [x] Copy data from GlusterFS to zippy NFS
|
||||||
- [ ] Copy `/data/compute/appdata/*` → `/persist/services/appdata/`
|
- [x] Copy `/data/compute/appdata/*` → `/persist/services/appdata/`
|
||||||
- [ ] Copy `/data/compute/config/*` → `/persist/services/config/`
|
- [x] Copy `/data/compute/config/*` → `/persist/services/config/`
|
||||||
- [ ] Copy `/data/sync/wordpress` → `/persist/services/appdata/wordpress`
|
- [x] Copy `/data/sync/wordpress` → `/persist/services/appdata/wordpress`
|
||||||
- [ ] Verify data integrity
|
- [x] Verify data integrity
|
||||||
- [ ] Verify NFS mounts working on all nodes
|
- [x] Verify NFS mounts working on all nodes
|
||||||
- [ ] Stop GlusterFS volume
|
- [x] Stop GlusterFS volume
|
||||||
- [ ] Delete GlusterFS volume
|
- [x] Delete GlusterFS volume
|
||||||
- [ ] Remove GlusterFS from NixOS configs
|
- [x] Remove GlusterFS from NixOS configs
|
||||||
- [ ] Remove syncthing wordpress sync configuration
|
- [x] Remove syncthing wordpress sync configuration (no longer used)
|
||||||
|
|
||||||
## Phase 4: Update and redeploy Nomad jobs
|
## Phase 4: Update and redeploy Nomad jobs
|
||||||
|
|
||||||
@@ -125,8 +125,8 @@ See [CLUSTER_REVAMP.md](./CLUSTER_REVAMP.md) for detailed procedures.
|
|||||||
- [ ] Verify backups include `/persist/services` data
|
- [ ] Verify backups include `/persist/services` data
|
||||||
- [ ] Verify backups exclude replication snapshots
|
- [ ] Verify backups exclude replication snapshots
|
||||||
- [ ] Update documentation (README.md, architecture diagrams)
|
- [ ] Update documentation (README.md, architecture diagrams)
|
||||||
- [ ] Clean up old GlusterFS data (only after everything verified!)
|
- [x] Clean up old GlusterFS data (only after everything verified!)
|
||||||
- [ ] Remove old glusterfs directories from all nodes
|
- [x] Remove old glusterfs directories from all nodes
|
||||||
|
|
||||||
## Post-Migration Checklist
|
## Post-Migration Checklist
|
||||||
- [ ] All 5 servers in quorum (consul members)
|
- [ ] All 5 servers in quorum (consul members)
|
||||||
@@ -143,8 +143,8 @@ See [CLUSTER_REVAMP.md](./CLUSTER_REVAMP.md) for detailed procedures.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Last updated**: 2025-10-23 22:30
|
**Last updated**: 2025-10-25
|
||||||
**Current phase**: Phase 4 complete! All services migrated to NFS
|
**Current phase**: Phase 3 & 4 complete! GlusterFS removed, all services on NFS
|
||||||
**Note**: Phase 1 (fractal NixOS conversion) deferred until after GlusterFS migration is complete
|
**Note**: Phase 1 (fractal NixOS conversion) deferred until after GlusterFS migration is complete
|
||||||
|
|
||||||
## Migration Summary
|
## Migration Summary
|
||||||
|
|||||||
@@ -6,7 +6,6 @@
|
|||||||
../../common/cluster-member.nix # Consul + storage clients
|
../../common/cluster-member.nix # Consul + storage clients
|
||||||
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
||||||
../../common/nomad-server.nix # Consul + Nomad server mode
|
../../common/nomad-server.nix # Consul + Nomad server mode
|
||||||
../../common/glusterfs.nix # GlusterFS server (temp during migration)
|
|
||||||
../../common/nfs-services-standby.nix # NFS standby for /data/services
|
../../common/nfs-services-standby.nix # NFS standby for /data/services
|
||||||
# To promote to NFS server (during failover):
|
# To promote to NFS server (during failover):
|
||||||
# 1. Follow procedure in docs/NFS_FAILOVER.md
|
# 1. Follow procedure in docs/NFS_FAILOVER.md
|
||||||
|
|||||||
@@ -6,7 +6,6 @@
|
|||||||
../../common/cluster-member.nix # Consul + storage clients
|
../../common/cluster-member.nix # Consul + storage clients
|
||||||
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
||||||
../../common/nomad-server.nix # Consul + Nomad server mode
|
../../common/nomad-server.nix # Consul + Nomad server mode
|
||||||
../../common/glusterfs.nix # GlusterFS server (temp during migration)
|
|
||||||
./hardware.nix
|
./hardware.nix
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,6 @@
|
|||||||
../../common/cluster-member.nix # Consul + storage clients
|
../../common/cluster-member.nix # Consul + storage clients
|
||||||
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
||||||
../../common/nomad-server.nix # Consul + Nomad server mode
|
../../common/nomad-server.nix # Consul + Nomad server mode
|
||||||
../../common/glusterfs.nix # GlusterFS server (temp during migration)
|
|
||||||
../../common/binary-cache-server.nix
|
../../common/binary-cache-server.nix
|
||||||
./hardware.nix
|
./hardware.nix
|
||||||
];
|
];
|
||||||
|
|||||||
@@ -6,7 +6,6 @@
|
|||||||
../../common/cluster-member.nix # Consul + storage clients
|
../../common/cluster-member.nix # Consul + storage clients
|
||||||
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
../../common/nomad-worker.nix # Nomad client (runs jobs)
|
||||||
# NOTE: zippy is NOT a server - no nomad-server.nix import
|
# NOTE: zippy is NOT a server - no nomad-server.nix import
|
||||||
../../common/glusterfs.nix # GlusterFS server (temp during migration)
|
|
||||||
# ../../common/ethereum.nix
|
# ../../common/ethereum.nix
|
||||||
../../common/nfs-services-server.nix # NFS server for /data/services
|
../../common/nfs-services-server.nix # NFS server for /data/services
|
||||||
# To move NFS server role to another host:
|
# To move NFS server role to another host:
|
||||||
|
|||||||
Reference in New Issue
Block a user