Refactor.
This commit is contained in:
199
README.md
Normal file
199
README.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# alo-cluster NixOS Configuration
|
||||
|
||||
This repository contains the NixOS configuration for a distributed cluster of machines managed as a unified flake.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The configuration uses a **layered profile system** that enables code reuse while maintaining clear separation of concerns:
|
||||
|
||||
```
|
||||
minimal-node # Base system (SSH, users, boot, impermanence)
|
||||
↓
|
||||
cluster-node # Cluster services (Consul, GlusterFS, CIFS, encryption)
|
||||
↓
|
||||
server-node # Server workloads (future: MySQL, PostgreSQL)
|
||||
↓
|
||||
workstation-node # Development tools (Docker, deploy-rs, emulation)
|
||||
↓
|
||||
desktop-node # GUI environment (Hyprland, Pipewire, fonts)
|
||||
```
|
||||
|
||||
Each layer extends the previous one, inheriting all configurations. Hosts select a profile level that matches their role.
|
||||
|
||||
### Special Node Types
|
||||
|
||||
- **cloud-node**: Minimal + Consul only (cloud VPS deployments)
|
||||
- **compute-node**: Cluster + Nomad worker (container orchestration)
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── flake.nix # Main flake definition with all hosts
|
||||
├── common/
|
||||
│ ├── global/ # Global configs applied to all systems
|
||||
│ │ ├── console.nix # Linux console colors (Solarized Dark)
|
||||
│ │ ├── locale.nix # Timezone and locale settings
|
||||
│ │ └── nix.nix # Nix daemon and flake configuration
|
||||
│ ├── minimal-node.nix # Base layer: SSH, users, boot, impermanence
|
||||
│ ├── cluster-node.nix # Cluster layer: Consul, GlusterFS, CIFS
|
||||
│ ├── server-node.nix # Server layer: bare metal services (future)
|
||||
│ ├── workstation-node.nix # Workstation layer: dev tools
|
||||
│ ├── desktop-node.nix # Desktop layer: GUI environment
|
||||
│ ├── cloud-node.nix # Cloud VPS profile
|
||||
│ ├── compute-node.nix # Nomad worker profile
|
||||
│ ├── base-node.nix # [DEPRECATED] Alias for cluster-node
|
||||
│ └── [feature modules] # Individual feature configs
|
||||
├── hosts/
|
||||
│ ├── c1/ # Compute node 1
|
||||
│ ├── c2/ # Compute node 2
|
||||
│ ├── c3/ # Compute node 3
|
||||
│ ├── alo-cloud-1/ # Cloud VPS
|
||||
│ ├── chilly/ # Server node
|
||||
│ ├── zippy/ # Workstation node
|
||||
│ └── sparky/ # Desktop node
|
||||
├── home/
|
||||
│ ├── default.nix # Home-manager entry point
|
||||
│ ├── profiles/ # Per-profile package sets
|
||||
│ │ ├── server.nix
|
||||
│ │ ├── workstation.nix
|
||||
│ │ └── desktop.nix
|
||||
│ ├── programs/ # Per-profile program configurations
|
||||
│ │ ├── server.nix # CLI tools (fish, tmux, git, nixvim)
|
||||
│ │ ├── workstation.nix # + dev tools
|
||||
│ │ └── desktop.nix # + Hyprland, wofi
|
||||
│ └── common/ # Shared home-manager configs
|
||||
└── services/ # Nomad job definitions (not NixOS)
|
||||
```
|
||||
|
||||
## Profile System
|
||||
|
||||
### System Profiles
|
||||
|
||||
Profiles are automatically applied based on the `mkHost` call in `flake.nix`:
|
||||
|
||||
```nix
|
||||
# Example: Desktop profile includes all layers up to desktop-node
|
||||
mkHost "x86_64-linux" "desktop" [
|
||||
./hosts/sparky
|
||||
];
|
||||
```
|
||||
|
||||
**Available profiles:**
|
||||
- `"server"` → minimal + cluster + server
|
||||
- `"workstation"` → minimal + cluster + server + workstation
|
||||
- `"desktop"` → minimal + cluster + server + workstation + desktop
|
||||
|
||||
### Home-Manager Profiles
|
||||
|
||||
Home-manager automatically inherits the same profile as the system, configured in `home/default.nix`:
|
||||
|
||||
```nix
|
||||
imports = [ ./programs/${profile}.nix ];
|
||||
home.packages = profilePkgs.${profile};
|
||||
```
|
||||
|
||||
This ensures system and user configurations stay synchronized.
|
||||
|
||||
## Host Definitions
|
||||
|
||||
### Current Hosts
|
||||
|
||||
| Host | Profile | Role | Hardware |
|
||||
|------|---------|------|----------|
|
||||
| **c1, c2, c3** | compute-node | Nomad workers | Bare metal servers |
|
||||
| **alo-cloud-1** | cloud-node | Consul server | Cloud VPS |
|
||||
| **chilly** | server | Future bare metal services | Bare metal server |
|
||||
| **zippy** | workstation | Development machine | Bare metal workstation |
|
||||
| **sparky** | desktop | Desktop environment | Bare metal desktop |
|
||||
|
||||
### Adding a New Host
|
||||
|
||||
1. Create host directory:
|
||||
```bash
|
||||
mkdir -p hosts/newhost
|
||||
```
|
||||
|
||||
2. Create `hosts/newhost/default.nix`:
|
||||
```nix
|
||||
{ config, pkgs, ... }:
|
||||
{
|
||||
imports = [
|
||||
../../common/encrypted-btrfs-layout.nix # or your layout
|
||||
../../common/global
|
||||
./hardware.nix
|
||||
];
|
||||
|
||||
networking.hostName = "newhost";
|
||||
# Host-specific configs here
|
||||
}
|
||||
```
|
||||
|
||||
3. Generate hardware config:
|
||||
```bash
|
||||
nixos-generate-config --show-hardware-config > hosts/newhost/hardware.nix
|
||||
```
|
||||
|
||||
4. Add to `flake.nix`:
|
||||
```nix
|
||||
newhost = mkHost "x86_64-linux" "workstation" [
|
||||
./hosts/newhost
|
||||
];
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Using deploy-rs
|
||||
|
||||
Deploy to specific host:
|
||||
```bash
|
||||
deploy -s '.#sparky'
|
||||
```
|
||||
|
||||
Deploy to all hosts:
|
||||
```bash
|
||||
deploy
|
||||
```
|
||||
|
||||
Deploy with detailed logging:
|
||||
```bash
|
||||
deploy -s '.#sparky' -- --show-trace
|
||||
```
|
||||
|
||||
### Manual Deployment
|
||||
|
||||
```bash
|
||||
nixos-rebuild switch --flake .#sparky --target-host sparky
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### Impermanence
|
||||
All hosts use tmpfs root with selective persistence. Persistent paths configured per-host in `persistence.directories` and `persistence.files`.
|
||||
|
||||
### Unattended Encryption
|
||||
Cluster nodes support automatic unlocking via Tailscale network using `common/unattended-encryption.nix`.
|
||||
|
||||
### Cluster Services
|
||||
- **Consul**: Service discovery and distributed KV store
|
||||
- **GlusterFS**: Distributed filesystem client
|
||||
- **CIFS/Samba**: Network file sharing
|
||||
|
||||
### Desktop Environment (sparky only)
|
||||
- **Hyprland**: Wayland compositor with CapsLock→Super remapping
|
||||
- **wofi**: Application launcher (Super+D)
|
||||
- **foot**: Terminal emulator (Super+Q)
|
||||
- **greetd/tuigreet**: Login manager with console option
|
||||
|
||||
### Development Tools (workstation/desktop)
|
||||
- Docker with rootless mode
|
||||
- deploy-rs for NixOS deployments
|
||||
- ARM emulation via binfmt
|
||||
- Full NixVim configuration
|
||||
|
||||
## Future Work
|
||||
|
||||
- Migrate Nomad services (MySQL, PostgreSQL) to bare NixOS services under `server-node.nix`
|
||||
- Add monitoring stack (Prometheus, Grafana)
|
||||
- Document Tailscale key rotation process
|
||||
- Add automated testing for configuration changes
|
||||
Reference in New Issue
Block a user