Manage workers from the web UI, persisted in SQLite

Replace the CROSS_PY_WORKERS env list with a DB-backed worker store so agents
can be added/removed (and their online status verified) from the browser.
- db.py: add workers table + list_workers/add_worker/remove_worker.
- workers.py: configured_workers() reads the DB instead of settings.
- app.py: POST /api/workers (add, probes reachability) and
  DELETE /api/workers/<port>/<host>; startup log uses DB worker count.
- UI: Workers card with add form, per-worker live status + OS/CPU, remove.
- settings.py: drop CROSS_PY_WORKERS/_get_json_list; DB is single source.
- README/DESIGN updated to describe UI-managed workers.
This commit is contained in:
Brett Williams
2026-08-30 23:37:31 -05:00
parent 5967b606c7
commit 18d8d2e398
10 changed files with 187 additions and 41 deletions
+24 -8
View File
@@ -33,10 +33,22 @@ The workers (`build_agent.py`) keep their HTTP API; only one small read-only end
builds/ downloaded artifacts (runtime)
```
**Worker discovery:** static config (list of `host:port`) in the controller settings. On each
job the controller queries every worker `/status`, filters to `status=="ready"` and matching
CPU/OS, and picks one. This replaces Zeroconf on the controller; workers can stop advertising,
and the Zeroconf code path can be removed with the CLI retirement (§7).
**Worker discovery:** configurable via the web UI and persisted in the controller's SQLite
`workers` table (host + port per agent). On each job the controller queries every worker
`/status`, filters to `status=="ready"` and matching CPU/OS, and picks one. This replaces
Zeroconf on the controller; workers can stop advertising, and the Zeroconf code path can be
removed with the CLI retirement (§7).
**Worker management API** (adds/removes agents without touching config files):
| Endpoint | Method | Purpose |
|---|---|---|
| `GET /api/workers` | GET | live `/status` for each configured worker |
| `POST /api/workers` | POST | add a worker: JSON `{"spec": "host:port"}` (or `host`+`port`) |
| `DELETE /api/workers/<port>/<host>` | DELETE | remove a configured worker |
The workers table is seeded on demand from the UI; there is no `CROSS_PY_WORKERS`
environment bootstrap — the DB is the single source of truth.
We deliberately do **not** rewrite the agent's HTTP API, and do **not** add auth in v1 (same
trusted-LAN posture as today). The worker endpoint contract used by the controller:
@@ -86,6 +98,8 @@ so results survive worker container restarts and remain available after the work
|---|---|---|
| GET | `/` | web UI (single page) |
| GET | `/api/workers` | live `/status` for each configured worker |
| POST | `/api/workers` | add a worker: JSON `{"spec": "host:port"}` |
| DELETE | `/api/workers/<port>/<host>` | remove a configured worker |
| GET | `/api/capabilities` | distinct (os, cpu) across ready workers, for dropdowns |
| POST | `/api/jobs` | create job: multipart upload `file` OR JSON `{repo_url, os, cpu}` |
| GET | `/api/jobs` | list jobs (newest first, with status) |
@@ -150,7 +164,7 @@ agent/ (worker agent: build_agent.py, zeroconf_server.py)
ctrl/
__init__.py
app.py (Flask factory, routes, waitress runner)
db.py (SQLite init + helpers)
db.py (SQLite init + helpers; jobs + workers tables)
scheduler.py (dispatch + progress-poll loop, worker client)
workers.py (worker config load + /status probe)
settings.py (env-driven config: DB path, builds dir, worker list)
@@ -160,13 +174,15 @@ ctrl/
style.css
```
Worker config example (env or JSON file):
Controller config example (env or JSON file):
```
CROSS_PY_WORKERS='["10.0.0.11:9001","10.0.0.12:9001"]'
CROSS_PY_DATA=/var/lib/cross-py-controller # holds jobs.db + builds/
CROSS_PY_DATA=/var/lib/cross-py-controller # holds jobs.db (jobs + workers) + builds/
CROSS_PY_PORT=8080
```
Workers are added/removed in the web UI and stored in the SQLite `workers` table (no
`CROSS_PY_WORKERS` env var).
---
## 7. What we reuse vs. retire