Allocation Module

pipeline.allocate.assemble_allocations(args)

Function for assembling all allocations and bands for packing. Allocations contain multiple processes within a single SLURM job such that the estimates for time sum to less than the time allowed for that SLURM job. Bands are single process per job, based on a default time plus any previous attempts (use –allow-band-increase flag to enable band increases with successive attempts if previous jobs timed out)

Returns:

A list of tuple objects such that each tuple represents an array to submit to slurm with the attributes (label, time, number_of_datasets). Note: The list of datasets to apply in each array job is typcially saved under proj_codes/<repeat_id>/<label>.txt (allocations use allocations/<x>.txt in place of the label)

pipeline.allocate.create_allocations(groupID: str, workdir: str, bins: list, repeat_id: str, dryrun=False) → None

Create allocation files (N project codes to each file) for later job runs.

Returns:

None

pipeline.allocate.create_array_bands(groupID, workdir, bands, repeat_id, dryrun=False)

Create band-files (under repeat_id) for this set of datasets.

Returns:

None

pipeline.allocate.has_required_timings(detail) → bool

Check if the contents of this projects ‘detail’ dict has required timing estimates for allocation.

Returns:

True or False depending on the above condition.