Commit Graph

249 Commits

Author SHA1 Message Date
0271abf705 Serverproxy manager (#61)
* Create serverproxy_manager.py

* Replace use of direct RenderServerProxy with ServerProxyManager method
2023-11-05 01:00:36 -05:00
c3b446be8e Don't create empty output directories in the source path (#60) 2023-11-04 23:58:08 -05:00
06a613fcc4 Zeroconf reports system properties (#59)
* Zeroconf.found_clients() now returns dicts of clients, not just hostnames

* Adjustments to distributed_job_manager.py

* Undo config change

* Report system metrics (cpu, os, etc) via zeroconf_server.py

* Zeroconf.found_clients() now returns dicts of clients, not just hostnames

* Adjustments to distributed_job_manager.py

* Undo config change

* Zeroconf.found_clients() now returns dicts of clients, not just hostnames

* Adjustments to distributed_job_manager.py

* Undo config change

* Adjustments to distributed_job_manager.py

* Undo config change

* Rename ZeroconfServer.found_clients() to found_hostnames()
2023-11-04 20:46:27 -05:00
d3b84c6212 Remove legacy client (#58)
* Remove legacy client

* Misc cleanup
2023-11-04 16:13:40 -05:00
Brett Williams
014489e3bf Add engine_help_viewer.py 2023-11-04 10:41:33 -05:00
65c256b641 New UI Redesign in pyqt6 (#56)
* Initial commit for new UI

* Initial commit for new UI

* WIP

* Status bar updates and has an icon for online / offline

* Add log_viewer.py

* Use JSON for delete_engine_download API

* Fix class issue with Downloaders

* Move Config class to new ui

* Add engine_browser.py

* Add a close event handler to the main window

* Fix issue with engine manager not deleting engines properly

* Rearrange all the files

* Add icons and resources

* Cache system info in RenderServerProxy

* Toolbar polish

* Fix resource path in status bar

* Add config_dir to misc_helper.py

* Add try block to zeroconf setup

* Add add_job.py

* Add raw args to add_job.py
2023-11-04 09:52:15 -05:00
bc8e88ea59 Config class (#51)
* Add new Config class to handle loading config files

* Use new config class in api_server.py
2023-10-29 22:22:40 -05:00
6ce69c8d35 Thread Safe Downloads for Renderers (#49)
* Make engines download on another thread

* Fix merge issues
2023-10-29 22:22:29 -05:00
dcc0504d3c Engine and downloader refactoring (#50)
* Make downloaders subclass of base_downloader.py

* Link engines and downloaders together for all engines

* Replace / merge worker_factory.py with engine_manager.py
2023-10-29 20:57:26 -05:00
Brett Williams
22aaa82da7 Simplify database.db logic 2023-10-27 02:41:31 -05:00
Brett Williams
951bebb3a8 Save database.db to upload dir, not code dir 2023-10-27 02:35:21 -05:00
Brett Williams
deac943e4c Add UI notifications to a try block 2023-10-25 21:52:08 -05:00
Brett Williams
32d863f624 Cancel all children if parent job is cancelled 2023-10-25 06:37:46 -05:00
Brett Williams
760d239d0c Minor fixes 2023-10-25 06:29:30 -05:00
cc1cf92118 Add simple notifications on job errors, completion and start. (#46) 2023-10-25 06:15:52 -05:00
Brett Williams
917a15c60c Make engine updating multithreaded 2023-10-25 02:57:25 -05:00
f01192909d Split add job helper (#45)
* Fix issue with engine not being available to download

* Add version caches to ffmpeg downloader

* Remove some test parameters

* "releases" should be "release" in linux ffmpeg url

* CPU was incorrectly reported as OS

* Fix naming structure for FFMPEG downloads for linux

* More linux ffmpeg work

* Improved error handling

* WIP

* Consolidate platform reporting to not use platform directly

* Fix missing folder name

* Fix project output naming

* Undo config.yaml commit

* Add is_engine_available API call

* Fix issue where subjobs would not find servers
2023-10-25 02:49:07 -05:00
Brett Williams
03e7b95e1b Split add_job_handler out to all_job_helpers.py 2023-10-24 20:53:06 -05:00
782a1a4699 Added engine update check on launch (#43) 2023-10-23 08:26:11 -05:00
e52682c8b9 Engine downloader API for #31 (#42)
* Add is_engine_available_to_download API call

* Fix issue with worker never throwing error if engine is not found

* Add API call to get most recent engine version

* Fix some minor import issues

* Fix web urls

* Fix default server log level

* Add progress bar for project download worker_factory downloads missing engine versions

* Better error handling when invalid version is given

* Add timeouts to engine downloaders
2023-10-22 17:02:30 -05:00
Brett Williams
9603046432 Fix path to blender pack_project.py 2023-10-21 22:48:15 -05:00
Brett Williams
9027cd7202 More code re-organizing 2023-10-21 22:45:30 -05:00
7a52cce40a Windows File Path Fixes (#39)
* Added system_safe_path to convert paths to Windows

* Fix issue where engines would not be reported unless a system engine was installed

* Platform independent searching for binaries in engine directory

* Add missing package to requirements.txt

* Better error handling for ffmpeg.get_all_formats()

* Add system_safe_path to more locations in api_server.py

* Fix naming issue with Blender on macos

* Fix path lookups and add engine_path to workers

* Report installed renderers in status

* Remove files included by accident
2023-10-21 22:12:09 -05:00
0b6b971fbc Get detailed formats from engines (#38)
* Add get_detected_gpus() and supported_render_engines() to Blender class

* Parse help args for Blender

* Return dict instead of list

* Parse args for FFMPEG

* Add API to get renderer args

* Only return available renderers

* Parse help args for Blender

* Return dict instead of list

* Parse args for FFMPEG

* Rebase off master

* Rebase

* Change methods from class methods to instance methods

* FFMPEG format fetching
2023-10-20 21:06:16 -05:00
Brett Williams
c499096efc Update requirements.txt 2023-10-20 20:39:44 -05:00
af5eda97c5 FFMPEG downloader (#36)
* Add ffmpeg_downloader.py

* Move shared download logic to downloader_core.py

* Added Windows support and some misc cleanup

* Fix issue with copying contents of .dmg files

* Add FFMPEGDownloader to engine_manager.py
2023-10-20 18:46:54 -05:00
7d1ecf1fa5 Downloadable engines (#34)
* Add blender_downloader.py

* Add engine_manager.py

* Add additional methods to engine_manager.py

* Refactor file layout to make engines on par with workers

* Add system platform info to status response

* Default to using system platform / cpu if none are provided

* Add API to download an engine and some general cleanup

* Add method to delete downloaded engine

* Add API calls to download engines and delete downloads

* Misc fixes
2023-10-20 15:05:29 -05:00
4563dcb255 Fix #20 - Relaunch renderer when renderer crashes (#28)
* Render worker makes multiple attempts if file_path increases on each run

* Change blender_worker.py to not overwrite existing files

* Chmod +x to client
2023-07-02 23:50:15 -05:00
a475aa999a Major file reorganization (#26)
* Major file reorganization

* Rearrange imports

* Fix default log level
2023-06-30 21:24:40 -05:00
34fbdaa4d9 Refactor: DistributedJobManager with pub/sub status change notifications (#25)
* Add pubsub to render_queue and base_worker

* Refactor: Convert ZeroconfServer to Singleton with Class Methods

* New API for subjob servers to notify parent job servers of status changes

* Refactor: Move all subjob related methods to distributed_job_manager.py

* Rewrite for wait_for_subjobs

* Fix: DistributedJobManager.find_available_servers() takes 1 positional argument but 3 were given

* DistributedJobManager should now notify / be notified abotu background job changes

* Fix the make_ready api. Change children keyname to be id@hostname so it can be unique

* Fixes

* Image sequence to movie needs to find the actual start frame

* Fix: subjob_status_change did not return a valid response

* Fix client renderer selection

* Small fix for subjob status checking

* Fix issue with divide_frames_equally

* Fix issue where downloads were not occurring

* Fix issue where old status was being reported

* Add docstrings and code cleanup
2023-06-30 19:49:57 -05:00
0b0b410e76 Create python-app.yml 2023-06-30 03:53:32 -05:00
646fcf135d Create pylint.yml 2023-06-30 03:47:56 -05:00
Brett Williams
9791715b25 Fix issue where jobs would get stuck waiting for subjobs to finish 2023-06-16 01:42:54 -05:00
76e413c18d Assign frame ranges to servers based on their CPU count (#19)
* Expose renderer availability in status api

* Remove redundant is_available_for_job API call

* New server split logic by cpu and moved to server_helper.py

* Remove old dead code

* Add RenderStatus.WAITING to proxy categories
2023-06-16 00:04:02 -05:00
Brett Williams
0080cdb371 Use local_paths when submitting jobs to localhost 2023-06-15 20:45:46 -05:00
Brett Williams
79ff451af8 Hotfix: Failing to convert image sequences that do not start on frame 1 2023-06-15 19:35:39 -05:00
54ec5f0e9c Subjob Zip Files (#18)
* Transfer any uploaded zip files to subjobs instead of extracted zips

* Fix Blender naming scheme
2023-06-15 19:21:10 -05:00
Brett Williams
78a51ffea0 Hotfix: Make sure start and end frames are always ints 2023-06-15 19:09:11 -05:00
Brett Williams
46e65699cc Hotfix: Disable SSL for now 2023-06-15 17:47:20 -05:00
e6eb344d19 Wait for subjob completion and download render files to host (#17)
* Fix Blender image sequence -> video conversion and change video to use ProRes

* Wait for child jobs to complete

* Download and extract render files from subjobs

* Fix issue where zip was not removed

* Update client to use new method names in server proxy

* Fix minor download issue
2023-06-15 17:44:34 -05:00
0a0a228731 Thumbnail Generation Refactoring (#16)
* Fix file_list and convert save_first_frame to use FFMPEG directly

* All thumbnail generation now uses FFMPEG engine
2023-06-15 13:06:46 -05:00
Brett Williams
cae5dbb41c Fix failing thumbnail generation 2023-06-15 11:26:52 -05:00
69715e8afa Multi client jobs (#15)
* Add API to expose if RenderQueue is available to take new jobs for a given renderer and priority

* Fix issue with calculating Blender percent complete when not starting at 1

* Rename owner / client properties to parent / children

* Add make_ready method to API

* Create and submit subjobs to other servers

* Update make_ready to update children jobs and some misc fixes

* Misc GUI cleanup
2023-06-15 02:01:50 -05:00
78a389080c Ability to set custom start / end frames (#14)
* Accept start / end frames in job submissions. Start / end frame support for Blender

* Remove old render_all_frames variables and misc cleanup

* Client work - Client determines frame count for FFMPEG and shows frame picker UI
2023-06-11 20:45:16 -05:00
94bb1e4362 Remove Old Multi-Client Code / Refactoring (#13)
* Remove a lot of old code from render_queue regarding clients

* More code cleanup

* More code cleanup

* Move everything around

* Minor log change
2023-06-11 14:50:20 -05:00
86a1dae5b6 Submit URLs in new jobs - Also add_job cleanup. (#12)
* Add ability to submit render jobs as URLs

* Considated add_job and add_job_handler

* Cleaned up code flow and added back in support for local file path submissions

* Misc cleanup
2023-06-10 22:55:08 -05:00
Brett Williams
2763a0c97f Fix hostname issue on machines without ".local" in hostname and handle missed TimeOut exception 2023-06-10 16:07:58 -05:00
38936d40ab Async Server Status Fetch (#11)
* Add background fetching to server_proxy

* Update UI to use server_proxy fetched jobs

* Fix issue getting status with empty jobs_cache

* Fix issue with jobs not appearing after switching servers

* Remove job_cache from dashboard_window and utilize server_proxy caches

* Remove jobs from table that shouldn't be there

* Streamline how we're handling offline tracking and handle connection error when fetching thumbnail

* Add ability to remove any manually added servers
2023-06-09 18:38:58 -05:00
75de367153 Break get_time_elapsed into misc_helper (#10) 2023-06-09 11:10:42 -05:00
05c8ac506c Manually add servers (#9)
* Add ability to manually add servers to client

* Misc client cleanup

* Add availability check before adding server to client UI
2023-06-08 08:19:37 -05:00