# llmdeal.me consortium — machine onboarding This document accompanies [`/setup/consortium.sh`](/setup/consortium.sh) — the interactive setup wizard for community members who've registered a GPU/server to contribute inference capacity to the llmdeal consortium routing pool. If you don't have a consortium token yet, DM the operator on Discord; this script is gated on a pre-issued `cstm_` token. --- ## What the script does Eight ordered steps, idempotent on re-run: 1. **Prereqs** — verifies `curl`, `jq`, `systemctl`, `sudo`, `lspci`; offers to install missing packages via your distro's package manager. Detects your GPU (NVIDIA via `nvidia-smi`, fallback `lspci`). 2. **Identity** — validates your `cstm_` token locally, then phones home to `POST /api/consortium/verify` to confirm the operator has registered it. Token is stored in `/etc/llmdeal/consortium.env` (mode `0600`, owner root) — never echoed to `ps`, shell history, or journal logs. 3. **Detect inference server** — probes for vLLM (`:8000`), text-generation-inference (`:3000`), ollama (`:11434`), and llama.cpp server (`:8080`). If multiple are found you pick one; if none, you can specify a URL manually. 4. **Models** — queries the chosen server for its model list, lets you tick which models to expose, and prompts for an llmdeal alias for each (e.g. `meta-llama/Llama-3.3-70B-Instruct` → `llama-3.3-70b`). 5. **Tunnel** — three options: - **Cloudflare tunnel** (recommended): generates an `/etc/cloudflared/llmdeal-consortium.yml` template; you finish with `cloudflared tunnel login` + `create` + DNS. - **Reverse SSH** to `tunnel.llmdeal.me` via autossh (requires the operator to issue you an SSH key separately). - **Manual**: you expose the inference URL however you like. 6. **Systemd** — installs `llmdeal-consortium.service` plus `/usr/local/bin/llmdeal-consortium-heartbeat.sh`. The heartbeat runs forever, posting a JSON ping to `/api/consortium/heartbeat` every 60 s with your model list and current `gpu_util` (when `nvidia-smi` is available). 7. **Smoke test** — probes your local inference endpoint with a 1-token completion to confirm the model alias works, then hits `POST /api/consortium/smoke-test` so the control plane can record the round trip. On failure, prints specific diagnostics + offers a retry. 8. **Summary** — prints your machine ID, exposed model list, tunnel info, and the systemctl commands you'll use day to day. ## CLI flags | Flag | Purpose | | ------------------- | -------------------------------------------------------- | | `--non-interactive` | Read all config from `--config` file; no prompts. | | `--config ` | Override `/etc/llmdeal/consortium.env`. | | `--dry-run` | Print every action, mutate nothing. | | `--quiet` | Suppress banner + progress; errors only. | | `--reconfigure` | Re-run setup, overwrite existing config. | | `--help`, `-h` | Show usage and exit 0. | For `--non-interactive`, the config file must contain at least: ```ini CONSORTIUM_TOKEN=cstm_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx CONSORTIUM_MODELS=meta-llama/Llama-3.3-70B-Instruct,qwen/qwen3-235b CONSORTIUM_TUNNEL_KIND=1 # 1=cloudflared, 2=autossh, 3=manual ``` ## Security implications The script opens a network path from the public internet to your inference server. Understand what you're agreeing to: - **Cloudflare tunnel** mode: your machine stays fully behind NAT. Cloudflare proxies traffic to your local inference URL. They see the wire bytes. Don't use this for confidential workloads. - **Reverse SSH** mode: opens an outbound SSH connection to `tunnel.llmdeal.me`. The operator's SSH bastion forwards traffic back to your `127.0.0.1:`. The bastion's `sshd_config` is locked to the `consortium` user with only `-R` allowed (no shell). - **Manual** mode: you take responsibility for the public listener. At minimum, restrict the source IP to llmdeal's egress range and rate-limit per-IP. ## Daily operations ```bash systemctl status llmdeal-consortium # is the heartbeat running? journalctl -u llmdeal-consortium -f # tail logs systemctl restart llmdeal-consortium # after editing /etc/llmdeal/consortium.env systemctl disable --now llmdeal-consortium # leave the consortium ``` To rotate your token, ask the operator to issue a new one, then re-run the script with `--reconfigure`. ## Troubleshooting | Symptom | Likely cause | | ------------------------------------------ | -------------------------------------------------- | | `unauthorized` on `/verify` | Operator hasn't activated your token yet. | | `model_not_registered` on smoke-test | The alias you picked isn't in the issued model list. Ask operator to update. | | Local probe returns non-200 | Your inference server isn't actually serving that model id — check `curl /v1/models`. | | `cloudflared` install loop | The script only auto-installs on Debian/Ubuntu via the official apt repo. Other distros: install from https://github.com/cloudflare/cloudflared/releases. | | Heartbeat 401 in journal | The operator revoked your token; DM them. | ## Operator support DM the llmdeal operator (`@llmdeal-ops` on Discord) with: - Your `machine_id` (printed in step 8). - The last 50 lines of `journalctl -u llmdeal-consortium -n 50`. - Your tunnel mode (cloudflared / autossh / manual). The operator can list your row via: ```bash curl -sS -H "X-Admin-Key: $ADMIN_SECRET" \ https://llmdeal.me/api/consortium/machines | jq ``` ## Known limitations (v0.1) - The control-plane "smoke test" returns `latency_ms: 0` because the actual tunnel-routed probe through `tunnel.llmdeal.me` isn't wired up yet — the operator currently trusts the script's local sanity check. The monitoring URL `https://llmdeal.me/consortium/m/` printed in step 8 is reserved but not yet implemented; the operator uses `/api/consortium/machines` (X-Admin-Key gated) for now. - Auto-install of `cloudflared` is wired only for Debian/Ubuntu. Fedora/ Arch users install it themselves and re-run with `--reconfigure`. - The reverse-SSH tunnel mode writes the autossh invocation to the heartbeat script but does **not** auto-start autossh — that needs the operator-issued SSH key, distributed out of band.