mirror of
https://github.com/blw1138/cross-py-builder.git
synced 2026-09-07 21:41:09 -05:00
Rewrite README for the controller/agent model
Replace the retired CLI usage docs with the current flow: install the setuptools package (editable) on each host, then run cross-py-agent on the workers and cross-py-controller on the UI host with a CROSS_PY_WORKERS list.
This commit is contained in:
@@ -1,78 +1,123 @@
|
|||||||
# Cross-Py-Builder
|
# Cross-Py-Builder
|
||||||
|
|
||||||
Cross-Py-Builder is a simple remote build tool for compiling Python projects with PyInstaller on different platforms on a local network.
|
Cross-Py-Builder is a remote build tool for compiling Python projects with PyInstaller on
|
||||||
|
different platforms (Linux, macOS, Windows) across a local network.
|
||||||
|
|
||||||
|
It has two parts:
|
||||||
|
|
||||||
|
- **Controller** — a small web service with a browser UI for submitting builds (upload a zip
|
||||||
|
or point at a git repo), tracking live progress, and downloading the resulting binaries.
|
||||||
|
- **Agent** (worker) — one per target OS/CPU. Runs the actual PyInstaller builds and exposes
|
||||||
|
them over HTTP; the controller dispatches builds to an agent matching the requested
|
||||||
|
OS + CPU.
|
||||||
|
|
||||||
## System Requirements
|
## System Requirements
|
||||||
- **Ubuntu/Debian & macOS:** 2GB+ RAM
|
- **Ubuntu/Debian & macOS:** 2GB+ RAM
|
||||||
- **Windows:** 4GB+ RAM
|
- **Windows:** 4GB+ RAM
|
||||||
- **Python 3.10 or later** installed
|
- **Python 3.10 or later** installed on each host
|
||||||
- **Local Network** for build agents
|
- **Local network** between the controller and the agents
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Example Usage
|
## How it's packaged
|
||||||
|
|
||||||
|
This is a Python package installed with `setuptools` (`setup.py`). Installing it (editable, for
|
||||||
|
development) makes two commands available on your PATH:
|
||||||
|
|
||||||
```sh
|
| Command | Runs | Purpose |
|
||||||
./cross-py-builder.py --build /path/to/repo -cpu x64 -os windows
|
|---|---|---|
|
||||||
|
| `cross-py-controller` | `ctrl.app:main` | Web UI + scheduler (the machine you browse to) |
|
||||||
|
| `cross-py-agent` | `agent.build_agent:main` | Worker agent (one per target OS/CPU) |
|
||||||
|
|
||||||
options:
|
Start by installing the package into a virtualenv on **every** machine that will act as a
|
||||||
-h, --help show this help message and exit
|
controller or agent:
|
||||||
--status Get status of available servers
|
|
||||||
--build BUILD Path to the project to build
|
```sh
|
||||||
-cpu CPU CPU architecture
|
cd cross-py-builder
|
||||||
-os OS Operating system
|
python3 -m venv venv
|
||||||
-d, --download Download after build
|
source venv/bin/activate
|
||||||
--delete-cache Delete cache
|
|
||||||
--update-all Update build agent
|
pip install -e . # editable install: installs deps + links to your working dir
|
||||||
--restart RESTART Hostname to restart
|
```
|
||||||
--restart-all Restart all agents
|
|
||||||
--shutdown SHUTDOWN Hostname to shutdown
|
`-e` (editable) links the install to your checkout, so code changes take effect without
|
||||||
--shutdown-all Shutdown all agents
|
reinstalling. On the LXC workers you can alternatively copy the source and just run the `.py`
|
||||||
|
files (see below).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Running a worker (agent)
|
||||||
|
|
||||||
|
On each target OS/CPU machine (e.g. each Ubuntu LXC):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd cross-py-builder
|
||||||
|
python3 -m venv venv
|
||||||
|
source venv/bin/activate
|
||||||
|
pip install -e .
|
||||||
|
|
||||||
|
cross-py-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
The agent binds on port `9001` and announces itself on the local network. To run it
|
||||||
|
directly without the package command:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pip install -r requirements.txt
|
||||||
|
python agent/build_agent.py
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Worker VM Setup Guide
|
## Running the controller
|
||||||
|
|
||||||
This guide provides steps to set up a worker VM on **Ubuntu/Debian, macOS, or Windows**.
|
On the machine that will serve the browser UI:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd cross-py-builder
|
||||||
|
python3 -m venv venv
|
||||||
|
source venv/bin/activate
|
||||||
|
pip install -e .
|
||||||
|
```
|
||||||
|
|
||||||
|
Set the static worker list (JSON array of `host:port`, matching where your agents are
|
||||||
|
listening) and start it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CROSS_PY_WORKERS='["192.168.1.40:9001","192.168.1.41:9001"]' \
|
||||||
|
cross-py-controller
|
||||||
|
```
|
||||||
|
|
||||||
|
Then open `http://<controller-ip>:8080` in a browser.
|
||||||
|
|
||||||
|
### Controller environment variables
|
||||||
|
|
||||||
|
| Variable | Default | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `CROSS_PY_WORKERS` | (none) | JSON array of agent `host:port` strings |
|
||||||
|
| `CROSS_PY_PORT` | `8080` | HTTP port the controller UI listens on |
|
||||||
|
| `CROSS_PY_MAX_CONCURRENT` | `1` | Max simultaneous builds (across distinct agents) |
|
||||||
|
| `CROSS_PY_DATA` | `<repo>/data` | Data dir (jobs DB, uploaded sources) |
|
||||||
|
| `CROSS_PY_BUILDS` | `<data>/builds` | Dir where finished build artifacts are saved |
|
||||||
|
| `CROSS_PY_DEBUG` | off | Use Flask dev server instead of waitress |
|
||||||
|
|
||||||
### Ubuntu/Debian or macOS
|
|
||||||
1. **Create a new VM** and install the OS.
|
|
||||||
2. **Update the system:**
|
|
||||||
- **Ubuntu/Debian:**
|
|
||||||
```sh
|
|
||||||
sudo apt update && sudo apt upgrade -y
|
|
||||||
sudo apt install python3-pip python3-venv -y
|
|
||||||
```
|
|
||||||
- **macOS:**
|
|
||||||
```sh
|
|
||||||
xcode-select --install # Install xcode tools
|
|
||||||
```
|
|
||||||
3. **Install Python** >=3.10 from [python.org](https://www.python.org/downloads/) (Windows and macOS - macOS)
|
|
||||||
4. **Set up a virtual environment:**
|
|
||||||
- **Ubuntu/Debian/macOS:**
|
|
||||||
```sh
|
|
||||||
python3 -m venv venv
|
|
||||||
source venv/bin/activate
|
|
||||||
```
|
|
||||||
- **Windows:**
|
|
||||||
```sh
|
|
||||||
python3 -m venv venv
|
|
||||||
venv\Scripts\activate
|
|
||||||
```
|
|
||||||
5. **Copy project files** and install dependencies:
|
|
||||||
```sh
|
|
||||||
pip install -r requirements.txt
|
|
||||||
```
|
|
||||||
6. **Start Build Agent**:
|
|
||||||
- **Ubuntu/Debian/macOS:**
|
|
||||||
```sh
|
|
||||||
python3 build_agent.py
|
|
||||||
```
|
|
||||||
- **Windows:**
|
|
||||||
```sh
|
|
||||||
python build_agent.py
|
|
||||||
```
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Using the UI
|
||||||
|
|
||||||
|
1. Open the controller in a browser.
|
||||||
|
2. Pick the target **OS** and **CPU** (or "any").
|
||||||
|
3. Either upload a `.zip` of your project (PyInstaller spec + source) or paste a git
|
||||||
|
`repo_url`.
|
||||||
|
4. Submit. The job is queued → dispatched to a matching agent → built → artifacts fetched.
|
||||||
|
5. Watch live per-step progress, then download the built binaries from the job detail.
|
||||||
|
|
||||||
|
A submitted project only needs the source plus at least one `*.spec` file; each spec file in
|
||||||
|
the archive produces a build.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Retired CLI
|
||||||
|
|
||||||
|
The original single-command CLI (`./cross-py-builder.py --build ...`, `agent_manager.py`) has been
|
||||||
|
removed and replaced by the controller + agent model above.
|
||||||
|
|||||||
Reference in New Issue
Block a user