alo-cluster/README.md

# alo-cluster NixOS Configuration

This repository contains the NixOS configuration for a distributed cluster of machines managed as a unified flake.

## Architecture Overview

The configuration uses a **layered profile system** that enables code reuse while maintaining clear separation of concerns:

```
minimal-node          # Base system (SSH, users, boot, impermanence)
    ↓
cluster-node          # Cluster services (Consul, GlusterFS, CIFS, encryption)
    ↓
server-node           # Server workloads (future: MySQL, PostgreSQL)
    ↓
workstation-node      # Development tools (Docker, deploy-rs, emulation)
    ↓
desktop-node          # GUI environment (Hyprland, Pipewire, fonts)
```

Each layer extends the previous one, inheriting all configurations. Hosts select a profile level that matches their role.

### Special Node Types

- **cloud-node**: Minimal + Consul only (cloud VPS deployments)
- **compute-node**: Cluster + Nomad worker (container orchestration)

## Directory Structure

```
.
├── flake.nix                 # Main flake definition with all hosts
├── common/
│   ├── global/               # Global configs applied to all systems
│   │   ├── console.nix       # Linux console colors (Solarized Dark)
│   │   ├── locale.nix        # Timezone and locale settings
│   │   └── nix.nix           # Nix daemon and flake configuration
│   ├── minimal-node.nix      # Base layer: SSH, users, boot, impermanence
│   ├── cluster-node.nix      # Cluster layer: Consul, GlusterFS, CIFS
│   ├── server-node.nix       # Server layer: bare metal services (future)
│   ├── workstation-node.nix  # Workstation layer: dev tools
│   ├── desktop-node.nix      # Desktop layer: GUI environment
│   ├── cloud-node.nix        # Cloud VPS profile
│   ├── compute-node.nix      # Nomad worker profile
│   ├── base-node.nix         # [DEPRECATED] Alias for cluster-node
│   └── [feature modules]     # Individual feature configs
├── hosts/
│   ├── c1/                   # Compute node 1
│   ├── c2/                   # Compute node 2
│   ├── c3/                   # Compute node 3
│   ├── alo-cloud-1/          # Cloud VPS
│   ├── chilly/               # Server node
│   ├── zippy/                # Workstation node
│   └── sparky/               # Desktop node
├── home/
│   ├── default.nix           # Home-manager entry point
│   ├── profiles/             # Per-profile package sets
│   │   ├── server.nix
│   │   ├── workstation.nix
│   │   └── desktop.nix
│   ├── programs/             # Per-profile program configurations
│   │   ├── server.nix        # CLI tools (fish, tmux, git, nixvim)
│   │   ├── workstation.nix   # + dev tools
│   │   └── desktop.nix       # + Hyprland, wofi
│   └── common/               # Shared home-manager configs
└── services/                 # Nomad job definitions (not NixOS)
```

## Profile System

### System Profiles

Profiles are automatically applied based on the `mkHost` call in `flake.nix`:

```nix
# Example: Desktop profile includes all layers up to desktop-node
mkHost "x86_64-linux" "desktop" [
  ./hosts/sparky
];
```

**Available profiles:**
- `"server"` → minimal + cluster + server
- `"workstation"` → minimal + cluster + server + workstation
- `"desktop"` → minimal + cluster + server + workstation + desktop

### Home-Manager Profiles

Home-manager automatically inherits the same profile as the system, configured in `home/default.nix`:

```nix
imports = [ ./programs/${profile}.nix ];
home.packages = profilePkgs.${profile};
```

This ensures system and user configurations stay synchronized.

## Host Definitions

### Current Hosts

| Host | Profile | Role | Hardware |
|------|---------|------|----------|
| **c1, c2, c3** | compute-node | Nomad workers | Bare metal servers |
| **alo-cloud-1** | cloud-node | Reverse proxy | Cloud VPS |
| **chilly** | server | Home Assistant in a VM | Bare metal server |
| **zippy** | workstation | Development machine, server | Bare metal server |
| **sparky** | desktop | Desktop environment | Bare metal desktop |

### Adding a New Host

1. Create host directory:
   ```bash
   mkdir -p hosts/newhost
   ```

2. Create `hosts/newhost/default.nix`:
   ```nix
   { config, pkgs, ... }:
   {
     imports = [
       ../../common/encrypted-btrfs-layout.nix  # or your layout
       ../../common/global
       ./hardware.nix
     ];

     networking.hostName = "newhost";
     # Host-specific configs here
   }
   ```

3. Generate hardware config:
   ```bash
   nixos-generate-config --show-hardware-config > hosts/newhost/hardware.nix
   ```

4. Add to `flake.nix`:
   ```nix
   newhost = mkHost "x86_64-linux" "workstation" [
     ./hosts/newhost
   ];
   ```

## Deployment

### Using deploy-rs

Deploy to specific host:
```bash
deploy -s '.#sparky'
```

Deploy to all hosts:
```bash
deploy
```

Deploy with detailed logging:
```bash
deploy -s '.#sparky' -- --show-trace
```

### Manual Deployment

```bash
nixos-rebuild switch --flake .#sparky --target-host sparky
```

## Key Features

### Impermanence
All hosts use tmpfs root with selective persistence. Persistent paths configured per-host in `persistence.directories` and `persistence.files`.

### Unattended Encryption
Cluster nodes support automatic unlocking via Tailscale network using `common/unattended-encryption.nix`.

### Cluster Services
- **Consul**: Service discovery and distributed KV store
- **GlusterFS**: Distributed filesystem client
- **CIFS/Samba**: Network file sharing

### Desktop Environment (sparky only)
- **Hyprland**: Wayland compositor with CapsLock→Super remapping
- **wofi**: Application launcher (Super+D)
- **foot**: Terminal emulator (Super+Q)
- **greetd/tuigreet**: Login manager with console option

### Development Tools (workstation/desktop)
- Docker with rootless mode
- deploy-rs for NixOS deployments
- ARM emulation via binfmt
- Full NixVim configuration

## Future Work

- Migrate Nomad services (MySQL, PostgreSQL) to bare NixOS services under `server-node.nix`
- Add monitoring stack (Prometheus, Grafana)
- Document Tailscale key rotation process
- Add automated testing for configuration changes