概览
目录
概览¶
Dask-jobqueue 项目提供了一个方便的接口，可以通过交互式系统（例如 Jupyter notebooks）或批处理作业进行访问。
示例¶
from dask_jobqueue import PBSCluster
cluster = PBSCluster()
cluster.scale(jobs=10) # Deploy ten single-node jobs
from dask.distributed import Client
client = Client(cluster) # Connect this local process to remote workers
# wait for jobs to arrive, depending on the queue, this may take some time
import dask.array as da
x = ... # Dask commands now use these distributed resources
自适应伸缩¶
Dask-jobqueue 还可以根据当前负载动态调整集群大小。这有助于在必要时扩展集群，并在不主动计算时缩小集群并节省资源。
cluster.adapt(minimum_jobs=10, maximum_jobs=100) # auto-scale between 10 and 100 jobs
cluster.adapt(maximum_memory="10 TB") # or use core/memory limits