ppetru/alo-cluster
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Refactor.

Browse Source
This commit is contained in:
Petru
2025-10-20 22:27:58 +01:00
parent fe51f1ac5b
commit 99db96e449
14 changed files with 268 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

199
README.md Normal file
View File

@@ -0,0 +1,199 @@
# alo-cluster NixOS Configuration
This repository contains the NixOS configuration for a distributed cluster of machines managed as a unified flake.
## Architecture Overview
The configuration uses a **layered profile system** that enables code reuse while maintaining clear separation of concerns:
```
minimal-node # Base system (SSH, users, boot, impermanence)
↓
cluster-node # Cluster services (Consul, GlusterFS, CIFS, encryption)
↓
server-node # Server workloads (future: MySQL, PostgreSQL)
↓
workstation-node # Development tools (Docker, deploy-rs, emulation)
↓
desktop-node # GUI environment (Hyprland, Pipewire, fonts)
```
Each layer extends the previous one, inheriting all configurations. Hosts select a profile level that matches their role.
### Special Node Types
- **cloud-node**: Minimal + Consul only (cloud VPS deployments)
- **compute-node**: Cluster + Nomad worker (container orchestration)
## Directory Structure
```
.
├── flake.nix # Main flake definition with all hosts
├── common/
│ ├── global/ # Global configs applied to all systems
│ │ ├── console.nix # Linux console colors (Solarized Dark)
│ │ ├── locale.nix # Timezone and locale settings
│ │ └── nix.nix # Nix daemon and flake configuration
│ ├── minimal-node.nix # Base layer: SSH, users, boot, impermanence
│ ├── cluster-node.nix # Cluster layer: Consul, GlusterFS, CIFS
│ ├── server-node.nix # Server layer: bare metal services (future)
│ ├── workstation-node.nix # Workstation layer: dev tools
│ ├── desktop-node.nix # Desktop layer: GUI environment
│ ├── cloud-node.nix # Cloud VPS profile
│ ├── compute-node.nix # Nomad worker profile
│ ├── base-node.nix # [DEPRECATED] Alias for cluster-node
│ └── [feature modules] # Individual feature configs
├── hosts/
│ ├── c1/ # Compute node 1
│ ├── c2/ # Compute node 2
│ ├── c3/ # Compute node 3
│ ├── alo-cloud-1/ # Cloud VPS
│ ├── chilly/ # Server node
│ ├── zippy/ # Workstation node
│ └── sparky/ # Desktop node
├── home/
│ ├── default.nix # Home-manager entry point
│ ├── profiles/ # Per-profile package sets
│ │ ├── server.nix
│ │ ├── workstation.nix
│ │ └── desktop.nix
│ ├── programs/ # Per-profile program configurations
│ │ ├── server.nix # CLI tools (fish, tmux, git, nixvim)
│ │ ├── workstation.nix # + dev tools
│ │ └── desktop.nix # + Hyprland, wofi
│ └── common/ # Shared home-manager configs
└── services/ # Nomad job definitions (not NixOS)
```
## Profile System
### System Profiles
Profiles are automatically applied based on the `mkHost` call in `flake.nix`:
```nix
# Example: Desktop profile includes all layers up to desktop-node
mkHost "x86_64-linux" "desktop" [
./hosts/sparky
];
```
**Available profiles:**
- `"server"` → minimal + cluster + server
- `"workstation"` → minimal + cluster + server + workstation
- `"desktop"` → minimal + cluster + server + workstation + desktop
### Home-Manager Profiles
Home-manager automatically inherits the same profile as the system, configured in `home/default.nix`:
```nix
imports = [ ./programs/${profile}.nix ];
home.packages = profilePkgs.${profile};
```
This ensures system and user configurations stay synchronized.
## Host Definitions
### Current Hosts
| Host | Profile | Role | Hardware |
|------|---------|------|----------|
| **c1, c2, c3** | compute-node | Nomad workers | Bare metal servers |
| **alo-cloud-1** | cloud-node | Consul server | Cloud VPS |
| **chilly** | server | Future bare metal services | Bare metal server |
| **zippy** | workstation | Development machine | Bare metal workstation |
| **sparky** | desktop | Desktop environment | Bare metal desktop |
### Adding a New Host
1. Create host directory:
```bash
mkdir -p hosts/newhost
```
2. Create `hosts/newhost/default.nix`:
```nix
{ config, pkgs, ... }:
{
imports = [
../../common/encrypted-btrfs-layout.nix # or your layout
../../common/global
./hardware.nix
];
networking.hostName = "newhost";
# Host-specific configs here
}
```
3. Generate hardware config:
```bash
nixos-generate-config --show-hardware-config > hosts/newhost/hardware.nix
```
4. Add to `flake.nix`:
```nix
newhost = mkHost "x86_64-linux" "workstation" [
./hosts/newhost
];
```
## Deployment
### Using deploy-rs
Deploy to specific host:
```bash
deploy -s '.#sparky'
```
Deploy to all hosts:
```bash
deploy
```
Deploy with detailed logging:
```bash
deploy -s '.#sparky' -- --show-trace
```
### Manual Deployment
```bash
nixos-rebuild switch --flake .#sparky --target-host sparky
```
## Key Features
### Impermanence
All hosts use tmpfs root with selective persistence. Persistent paths configured per-host in `persistence.directories` and `persistence.files`.
### Unattended Encryption
Cluster nodes support automatic unlocking via Tailscale network using `common/unattended-encryption.nix`.
### Cluster Services
- **Consul**: Service discovery and distributed KV store
- **GlusterFS**: Distributed filesystem client
- **CIFS/Samba**: Network file sharing
### Desktop Environment (sparky only)
- **Hyprland**: Wayland compositor with CapsLock→Super remapping
- **wofi**: Application launcher (Super+D)
- **foot**: Terminal emulator (Super+Q)
- **greetd/tuigreet**: Login manager with console option
### Development Tools (workstation/desktop)
- Docker with rootless mode
- deploy-rs for NixOS deployments
- ARM emulation via binfmt
- Full NixVim configuration
## Future Work
- Migrate Nomad services (MySQL, PostgreSQL) to bare NixOS services under `server-node.nix`
- Add monitoring stack (Prometheus, Grafana)
- Document Tailscale key rotation process
- Add automated testing for configuration changes

11
common/base-node.nix
View File

@@ -1,13 +1,8 @@
{ pkgs, ... }:
{
# DEPRECATED: Use cluster-node.nix for cluster nodes or minimal-node.nix for minimal systems
# This file is kept for backward compatibility with existing configurations
imports = [
./cifs-client.nix
./consul.nix
./glusterfs-client.nix
./impermanence.nix
./sshd.nix
./user-ppetru.nix
./unattended-encryption.nix
./systemd-boot.nix
./cluster-node.nix
];
}

6
common/cloud-node.nix
View File

@@ -1,10 +1,8 @@
{ pkgs, ... }:
{
# Cloud node: Minimal system with Consul for cloud deployments
imports = [
./minimal-node.nix
./consul.nix
./impermanence.nix
./sshd.nix
./user-ppetru.nix
./systemd-boot.nix
];
}

13
common/cluster-node.nix Normal file
View File

@@ -0,0 +1,13 @@
{ pkgs, ... }:
{
# Cluster node configuration
# Extends minimal-node with cluster-specific services (Consul, GlusterFS, CIFS)
# Used by: compute nodes (c1, c2, c3)
imports = [
./minimal-node.nix
./unattended-encryption.nix
./cifs-client.nix
./consul.nix
./glusterfs-client.nix
];
}

4
common/compute-node.nix
View File

@@ -1,7 +1,9 @@
{ pkgs, ... }:
{
# Compute node: Cluster node with Nomad, GlusterFS server, and Syncthing
# Used by: c1, c2, c3
imports = [
./base-node.nix
./cluster-node.nix
./glusterfs.nix
./nomad.nix
./syncthing-data.nix

11
common/container-node.nix
View File

@@ -1,11 +0,0 @@
{ lib, ... }:
{
imports = [
./impermanence.nix # TODO: find a way to avoid needing this here
];
boot.isContainer = true;
custom.impermanence.enable = false;
custom.tailscale.enable = false;
networking.useDHCP = lib.mkForce false;
}

6
common/desktop-node.nix
View File

@@ -1,5 +1,11 @@
{ pkgs, ... }:
{
# Desktop profile: Graphical desktop with Hyprland
# Extends workstation-node with desktop environment
imports = [
./workstation-node.nix
];
# Enable Hyprland (Wayland compositor)
programs.hyprland = {
enable = true;

12
common/minimal-node.nix Normal file
View File

@@ -0,0 +1,12 @@
{ pkgs, ... }:
{
# Minimal base configuration for all NixOS systems
# Provides: SSH access, user management, boot, impermanence
# Note: unattended-encryption is NOT included by default - add it explicitly where needed
imports = [
./impermanence.nix
./sshd.nix
./user-ppetru.nix
./systemd-boot.nix
];
}

14
common/server-node.nix Normal file
View File

@@ -0,0 +1,14 @@
{ pkgs, ... }:
{
# Server profile: Cluster-enabled system for server deployments
# Extends cluster-node with server-specific configurations
# Future: Add bare NixOS services here (mysql, postgres, etc.) when migrating from Nomad
imports = [
./cluster-node.nix
];
# Server-specific configurations can be added here
# Example (for future use):
# services.mysql.enable = lib.mkDefault false;
# services.postgresql.enable = lib.mkDefault false;
}

7
common/dev-node.nix → common/workstation-node.nix
View File

@@ -1,5 +1,12 @@
{ pkgs, inputs, ... }:
{
# Workstation profile: Development workstation configuration
# Extends server-node with development tools and emulation
imports = [
./server-node.nix
./unattended-encryption.nix
];
environment.systemPackages = with pkgs; [
wget
deploy-rs

10
flake.nix
View File

@@ -56,6 +56,14 @@
mkHost =
system: profile: modules:
let
# Auto-import profile-specific module based on profile parameter
profileModule =
if profile == "server" then ./common/server-node.nix
else if profile == "workstation" then ./common/workstation-node.nix
else if profile == "desktop" then ./common/desktop-node.nix
else null;
in
nixpkgs.lib.nixosSystem {
system = system;
modules = [
@@ -87,7 +95,7 @@
};
};
}
] ++ modules;
] ++ nixpkgs.lib.optional (profileModule != null) profileModule ++ modules;
specialArgs = {
inherit inputs self;
};

1
hosts/chilly/default.nix
View File

@@ -8,7 +8,6 @@
imports = [
../../common/encrypted-btrfs-layout.nix
../../common/global
../../common/base-node.nix
./hardware.nix
];

3
hosts/sparky/default.nix
View File

@@ -3,9 +3,6 @@
imports = [
../../common/encrypted-btrfs-layout.nix
../../common/global
../../common/base-node.nix
../../common/dev-node.nix
../../common/desktop-node.nix
./hardware.nix
];

1
hosts/zippy/default.nix
View File

@@ -4,7 +4,6 @@
../../common/encrypted-btrfs-layout.nix
../../common/global
../../common/compute-node.nix
../../common/dev-node.nix
# ../../common/ethereum.nix
./hardware.nix
];