Commit Graph
27 Commits
Author SHA1 Message Date
Brett Williams 7a36ae083c Fix worker removal and improve web UI defaults
- Parse optional http:// scheme prefix on worker specs so a scheme-typed
  IP is stored cleanly and its remove button works.
- Delete workers by their stored DB host/port instead of re-parsing the
  rendered URL, and surface add/remove errors in the UI.
- Default new builds to git URL instead of zip upload.
- Show worker hostname/IP from agent status in the address column.
- Always include DB host/port in worker API output.
2026-08-31 15:40:01 -05:00
Brett Williams f73a5bbb2f Fix disappearing artifacts and guard against empty builds
The agent's background build threads called jsonify()/send_file() with no
Flask app context, so after a successful build they crashed with
'Working outside of application context', which triggered the error
handler that rmtree'd the whole build dir before the controller could
download the artifact. Wrap both runners in app.app_context().

Also guard all three layers against empty results: the agent refuses to
report success or serve a zip when dist/ has no files, and the scheduler
flags a downloaded empty zip as a failure rather than marking the job
done.
2026-08-31 10:07:04 -05:00
Brett Williams 1faa544ead Make async git checkout return immediately
Once the job is dispatched the agent cloned synchronously even with
async=1, so a slow clone blew past the controller's 5s dispatch timeout
and the job was marked failed while the worker kept building. Run the
clone in the background thread and return 202 + job id right away.
2026-08-31 00:46:36 -05:00
Brett Williams cde111e584 Read repo_url from multipart form in job creation
The web UI submits jobs as FormData, so request.get_json() was always
None and the repo_url was never seen.
2026-08-31 00:42:11 -05:00
Brett Williams aab0c9b12d Fix sorting of capabilities by os/cpu tuple 2026-08-31 00:35:16 -05:00
Brett Williams b85ff310e1 Ignore runtime data/ dir; untrack stray jobs.db 2026-08-30 23:37:49 -05:00
Brett Williams 18d8d2e398 Manage workers from the web UI, persisted in SQLite
Replace the CROSS_PY_WORKERS env list with a DB-backed worker store so agents
can be added/removed (and their online status verified) from the browser.
- db.py: add workers table + list_workers/add_worker/remove_worker.
- workers.py: configured_workers() reads the DB instead of settings.
- app.py: POST /api/workers (add, probes reachability) and
  DELETE /api/workers/<port>/<host>; startup log uses DB worker count.
- UI: Workers card with add form, per-worker live status + OS/CPU, remove.
- settings.py: drop CROSS_PY_WORKERS/_get_json_list; DB is single source.
- README/DESIGN updated to describe UI-managed workers.
2026-08-30 23:37:31 -05:00
Brett Williams 5967b606c7 Fix modal never dismissing and noisy 404 errors
The .modal CSS display:flex override the hidden attribute, so the empty job
modal was always visible on load and closeModal() could not hide it - the
'non-dismissable empty error'. Add .modal[hidden] { display:none } so the
hidden attribute wins.

The blanket errorhandler(Exception) also converted normal HTTP 404s (e.g.
/favicon.ico, /apple-touch-icon-*.png requested by the browser) into 500s,
flooding the log. Return HTTPException instances unchanged so only genuine
500s are logged as errors.
2026-08-30 22:41:59 -05:00
Brett Williams 8a6f02b7e9 Add controller logging and return readable errors
The controller ran silently and surfaced errors as opaque/empty bodies,
making it impossible to diagnose failures from either the UI or the server.
- Configure Python logging (timestamps, levels) and log job lifecycle events
  (created / dispatched / done / failed) plus startup and dispatch warnings.
- Add a catch-all error handler so unhandled exceptions return a JSON error
  with the exception type+message instead of an empty response.
- Remove the frontend submit bug that hand-set multipart Content-Type without
  a boundary (and built an opts object it never used); post the FormData
  directly so the browser sets the correct content type + boundary.
2026-08-30 22:39:16 -05:00
Brett Williams 6f38beaf92 Rewrite README for the controller/agent model
Replace the retired CLI usage docs with the current flow: install the
setuptools package (editable) on each host, then run cross-py-agent on the
workers and cross-py-controller on the UI host with a CROSS_PY_WORKERS list.
2026-08-30 22:04:03 -05:00
Brett Williams 59fb7739e4 Rename worker package cross_py_builder -> agent
The nested cross_py_builder package only contains the worker agent. Rename
it to agent/ to match its role and distinguish it from the ctrl package and
the PyPI distribution name. Internal imports are relative, so build_agent.py
and zeroconf_server.py are unaffected except for the cosmetic APP_NAME banner.
2026-08-30 22:01:54 -05:00
Brett Williams b5b1cd740b Stream live worker progress via async build dispatch
The agent's /upload and /checkout_git were synchronous, blocking until a
build finished, so the controller could not relay live per-step progress.
Add ?async=1 support: the agent saves/clones the source, starts the build in
a background thread (holding build_lock), and returns 202 + {"id": ...}
immediately. The controller dispatches async, then polls /progress/<job_id>,
appending each step/log line to the job log and SSE stream, and treats the
build as done when progress clears (404) and the worker reports ready.
2026-08-30 21:59:46 -05:00
Brett Williams 0e89919342 Add controller with web UI and retire CLI
- Add ctrl/ package: Flask app (waitress), SQLite job store, background
  scheduler that probes configured workers, dispatches via /upload and
  /checkout_git, fetches artifacts, and serves a single-page UI
- Supporting API: /api/jobs (create/list/detail/cancel), /api/workers,
  /api/capabilities, artifact download, and SSE log stream
- Workers keep the existing HTTP API; add /progress/<job_id> endpoint and
  per-step build tracking for live status
- Retire agent_manager.py and the cross-py-builder CLI; controller UI is
  the primary frontend
- Record design in DESIGN.md; add waitress dependency

Note: /progress is exposed on the worker but the synchronous /upload and
/checkout endpoints block until a build completes, so per-step worker
progress is not yet relayed live in the UI. Live streaming needs an
async-start worker endpoint as a follow-up.
2026-08-30 21:55:46 -05:00
Brett Williams 5f0d2db1a9 Serialize builds and add health endpoint to build agent
- Add a non-blocking build lock so only one job runs at a time;
  concurrent upload/checkout requests get a 409 response
- Add /healthz returning 200 when idle and 503 while a build is running
- Ignore .DS_Store artifacts
2026-08-30 21:32:01 -05:00
Brett Williams 1902d36ae7 Harden build agent against path traversal and add job cleanup
- Validate job IDs on /download and /delete; reject invalid identifiers
- Prevent zip-slip during /upload extraction via safe_extract
- Basename uploaded filenames in /update to block path traversal
- Route git-checkout jobs into pybuild-data so they can be downloaded/deleted
- Clean up job dirs on extract, checkout, and build failures
- Ignore Python bytecode and egg-info artifacts
2026-08-30 21:24:54 -05:00
Brett Williams b511962a1f This works 2025-03-12 00:48:46 -05:00
Brett Williams e07e2bd997 Another attempt #2 2025-03-12 00:40:33 -05:00
Brett Williams 3c92a0775f Another attempt 2025-03-12 00:30:47 -05:00
Brett Williams 80fdc887f0 Fix pip setup 2025-03-12 00:27:16 -05:00
Brett Williams b0d3660881 Ability to checkout git repos, custom ports, and setup for pip 2025-03-12 00:20:25 -05:00
Brett Williams 94e0ecda83 Update README.md 2025-03-06 23:45:40 -06:00
Brett Williams 345b4f6c2c Remove version.py 2025-03-06 15:00:03 -06:00
Brett Williams a568b5140b Remove buggy existing process code 2025-03-06 14:57:39 -06:00
Brett Williams 0f487640bd Argparse cleanup 2025-03-06 10:07:19 -06:00
Brett Williams a9ccafc25d Add shebang and make scripts executable 2025-03-06 00:19:14 -06:00
Brett Williams f06d418e29 Fixed issue with updates after builds and restarting process 2025-03-06 00:16:13 -06:00
Brett Williams cdbee30109 Initial commit 2025-03-05 16:45:21 -06:00