Assess Module

assess.add_to_blacklist(savedcodes, groupdir, reason, logger)

Add a set of codes to the blacklist for a given reason

assess.analyse_data(g, workdir)

Show some statistics of netcdf and kerchunk data amounts for this particular group

assess.assess_main(args)

Main assessment function, different tools diverge from here.

assess.examine_log(workdir: str, proj_code: str, phase: str, groupID=None, repeat_id=None, error=None)

Open and examine a log file from a previous run of a given phase - Show full error log - Suggest rerun/examination command

assess.get_index_of_code(workdir: str, groupID: str, repeat_id: str, code: str) → int

Get the index of a project code within some repeat set of codes

Parameters:

  • workdir – (str) The current pipeline working directory.

  • groupID – (str) The name of the group which this project code belongs to.

  • repeat_id – (str) The subset within the group (default is main)

  • code – (str) The project code for which to get the index.

Returns pindex:

(int) The index of the project code within this group-subgroup.

assess.get_rerun_command(phase: str, ecode: str, groupID: str, repeat_id: str) → None

Print a rerun command for inspecting a single dataset using single_run.py

Parameters:

  • phase – (str) The current phase

  • ecode – (str) The index of the project code to be inspected by running in serial.

  • groupID – (str) The name of the group which this project code belongs to.

  • repeat_id – (str) The subset within the group (default is main)

Returns:

assess.match_type(args, logger)

Determine all project codes matching a specific type and save to a new_id file. Use existing -s flag for type?

assess.merge_old_new(old_codes, new_codes, reason=None)

Merge an existing list of project codes with a new set - Uses indexes if not a 1d list

assess.progress_check(args, logger)

Give a general overview of progress within the pipeline - How many datasets currently at each stage of the pipeline - Errors within each pipeline phase - Allows for examination of error logs - Allows saving codes matching an error type into a new repeat group

assess.save_selection(codes: list, groupdir: str, label: str, logger, overwrite=0, index=False)

Save selection of codes to a file with a given repeat label.

Requires a groupdir (directory belonging to a group), list of codes and a label for the new file.

assess.show_options(args, logger)

Use OS tools to list contents of relevant directories to see all jobids or labels.

List output or error directories (one per job id), or list all proj_codes text files.

assess.status_log_display(args, logger)

Display the full status log for a specific project code

assess.summary_data(args, logger)

Display summary info in terms of data representation

assess.upgrade_version(args, logger)

Upgrade the version info in the kerchunk file