kernelfoundry

kernelfoundry.algorithm.utils.resource_allocator¶

Classes

ResourceAllocator(device_ids[, ...])

class kernelfoundry.algorithm.utils.resource_allocator.ResourceAllocator(device_ids: list, max_inference_requests: int = 10)[source]¶

__init__(device_ids: list, max_inference_requests: int = 10)[source]¶

get_available_gpu()[source]¶: Find and reserve an available GPU.

release_gpu(gpu_id: int)[source]¶: Release a GPU back to the pool by device ID.

reserve_gpu()[source]¶: Context manager for GPU reservation.

get_available_server()[source]¶: Find and reserve an available inference server.

release_server(server_id: int)[source]¶: Release an inference server back to the pool by server ID.

reserve_server()[source]¶: Context manager for inference server reservation.