alff.base

alff.base

Base classes for ALFF workflows and remote operations.

Classes:

• Workflow –

  Base class for workflows.

• RemoteOperation –

  Base class for operations on remote machines.

• KEY –

  A class to hold various constant keys used throughout the ALFF package.

Functions:

• init_alff_logger –

  Initializing the logger.

• get_logger –

Attributes:

• logger –

logger = cast(ColorLogger, _LoggerProxy()) module-attribute

Workflow(params_file: str, machines_file: str, schema_file: str | None = None)

Bases: ABC

Base class for workflows.

Workflow class is the central part of ALFF. Each workflow contains list of stages to be executed.

Subclass MUST reimplement:

- `__init__()`: initialize the workflow, need to override these attributes:
    - self.stage_map
    - self.wf_name
- `run()`: the main function to run the workflow. The default implementation is a loop over stages in `self.stage_map`, just for simple workflow. For complex workflow (e.g. with iteration like active learning), need to reimplement the `.run()` function.

Example

class WorkflowExample(Workflow):
    def __init__(self, params_file: str, machines_file: str):
        super().__init__(params_file, machines_file, SCHEMA_EXAMPLE)
        self.stage_map = {
            "stage_name1": stage_function1,
            "stage_name2": stage_function2,
            "stage_name3": stage_function3,
        }
        self.wf_name = "Name of the workflow"
        return

Note

• multi_mdicts in this class is a dictionary containing multiple remote machines, and will be used RemoteOperation class.
• All @abtractmethod must be reimplemented in subclasses.

Subclassed by:

• alff API reference
  • alff.al  al
    •  active_learning  WorkflowActiveLearning
    •  finetune  WorkflowFinetune
  • alff.gdata  gdata  gendata  WorkflowGendata
  • alff.pes  pes  pes_scan  WorkflowPes
  • alff.phonon  phonon  phonon  WorkflowPhonon

Methods:

• run –

  The main function to run the workflow. This default implementation works for simple workflow,

Attributes:

• params_file –
• machines_file –
• schema_file –
• multi_mdicts –
• pdict –
• stage_list –
• stage_map (dict[str, Callable]) –
• wf_name (str) –

params_file = params_file instance-attribute

machines_file = machines_file instance-attribute

schema_file = schema_file instance-attribute

multi_mdicts = config_machine.multi_mdicts instance-attribute

pdict = Config.loadconfig(self.params_file) instance-attribute

stage_list = self._load_stage_list() instance-attribute

stage_map: dict[str, Callable] instance-attribute

wf_name: str instance-attribute

run()

The main function to run the workflow. This default implementation works for simple workflow, for more complex workflow (e.g. with iteration like active learning), need to reimplement this .run() function.

RemoteOperation(work_dir, pdict, multi_mdicts, mdict_prefix='')

Bases: ABC

Base class for operations on remote machines.

Each operation includes atl east 3 methods:

- prepare
- run
- postprocess

Subclass must reimplement these methods:

- `__init__()`: initialize the operation, need to override these attributes:
- `prepare()`: prepare all things needed for the run() method.
- `postprocess()`: postprocess after the run() method.

Note

• Before using this class, must prepare a file work_dir/task_dirs.yml
• All paths (work_dir, task_dirs,...) are in POSIX format, and relative to run_dir (not work_dir).
• All @abtractmethod must be reimplemented in subclasses.
• Do not change the .run() method unless you know what you are doing.
• task_filter to filter task directories (filter already labelled structures).

  self.task_filter = {"has_files": ["file1.txt", "file2.txt"], "no_files": ["file3.txt"]}
  

Subclassed by:

• alff API reference
  • alff.al  al
    •  libal_md_ase  OperAlmdAseSevennet
    •  libal_md_lammps  OperAlmdLammpsSevennet
  • alff.gdata  gdata  libgen_gpaw
    •  OperGendataGpawAIMD
    •  OperGendataGpawOptimize
  • alff.pes  pes  libpes_lammps  OperPESLammpsOptimize

Methods:

• prepare –

  Prepare all things needed for run() method.

• postprocess –

  Postprocess after run() method.

• run –

  Function to submit jobs to remote machines.

Attributes:

• work_dir –
• pdict –
• mdict_list –
• task_dirs –
• op_name (str) –
• task_filter (dict[str, list[str]]) –
• commandlist_list (list[list[str]]) –
• forward_files (list[str]) –
• backward_files (list[str]) –
• forward_common_files (list[str]) –
• backward_common_files (list[str]) –

work_dir = work_dir instance-attribute

pdict = pdict instance-attribute

mdict_list = self._select_machines(multi_mdicts, mdict_prefix) instance-attribute

task_dirs = self._load_task_dirs() instance-attribute

op_name: str instance-attribute

task_filter: dict[str, list[str]] instance-attribute

commandlist_list: list[list[str]] instance-attribute

forward_files: list[str] instance-attribute

backward_files: list[str] instance-attribute

forward_common_files: list[str] instance-attribute

backward_common_files: list[str] = [] instance-attribute

prepare() abstractmethod

Prepare all things needed for run() method.

This method need to implement the following attributes

• self.commandlist_list: list[list[str]]
• self.forward_files: list[str]
• self.backward_files: list[str]
• self.forward_common_files: list[str]
• self.backward_common_files: list[str] # rarely used

postprocess() -> None | list abstractmethod

Postprocess after run() method.

run()

Function to submit jobs to remote machines.

Note

• Orginal taks_dirs is relative to run_dir, and should not be changed. But the sumbmission function needs taks_dirs relative path to work_dir, so we make temporary change here.

KEY

A class to hold various constant keys used throughout the ALFF package.

Attributes:

• time_str –
• DIR_LOG –
• FILE_LOG_ALFF –
• FILE_ITERLOG –
• DIR_TRAIN –
• DIR_MD –
• DIR_DFT –
• DIR_DATA –
• DIR_TMP –
• DIR_CANDIDATE –
• DIR_COLLECTDATA –
• DIR_FWDATA –
• FILE_DATAPATH –
• FILE_CHECKPOINTS –
• FILE_ARG_TRAIN –
• FILE_TRAJ_MD –
• FILE_TRAJ_MD_CANDIDATE –
• FILE_ITER_DATA –
• FILE_COLLECT_DATA –
• FMT_ITER –
• FMT_STAGE –
• FMT_MODEL –
• FMT_STRUCT –
• FMT_TASK_MD –
• FMT_TASK_DFT –
• RUNFILE_LAMMPS –
• FILE_ARG_LAMMPS –
• FILE_ARG_ASE –
• SCRIPT_ASE_PATH –
• SCHEMA_ASE_RUN –
• SCHEMA_LAMMPS –
• SCHEMA_ACTIVE_LEARN –
• SCHEMA_FINETUNE –
• DIR_MAKE_STRUCT –
• DIR_STRAIN –
• DIR_GENDATA –
• FILE_FRAME_UNLABEL –
• FILE_FRAME_LABEL –
• FILE_TRAJ_LABEL –
• SCHEMA_ASE_BUILD –
• SCHEMA_GENDATA –
• SCHEMA_PHONON –
• SCHEMA_ELASTIC –
• SCHEMA_PES_SCAN –
• DIR_SUPERCELL –
• DIR_PHONON –
• FILE_PHONOPYwFORCES –
• DIR_ELASTIC –
• DIR_SCAN –
• DIR_PES –

time_str = time.strftime('%y%m%d_%H%M%S') class-attribute instance-attribute

DIR_LOG = 'log' class-attribute instance-attribute

FILE_LOG_ALFF = f'{DIR_LOG}/{time_str}_alff.log' class-attribute instance-attribute

FILE_ITERLOG = '_alff.iter' class-attribute instance-attribute

DIR_TRAIN = '00_train' class-attribute instance-attribute

DIR_MD = '01_md' class-attribute instance-attribute

DIR_DFT = '02_dft' class-attribute instance-attribute

DIR_DATA = '03_data' class-attribute instance-attribute

DIR_TMP = 'tmp_dir' class-attribute instance-attribute

DIR_CANDIDATE = 'md_selected' class-attribute instance-attribute

DIR_COLLECTDATA = 'collect_data' class-attribute instance-attribute

DIR_FWDATA = 'fw_data' class-attribute instance-attribute

FILE_DATAPATH = 'data_paths.yml' class-attribute instance-attribute

FILE_CHECKPOINTS = 'checkpoints.yml' class-attribute instance-attribute

FILE_ARG_TRAIN = 'arg_train.yml' class-attribute instance-attribute

FILE_TRAJ_MD = 'traj_md.extxyz' class-attribute instance-attribute

FILE_TRAJ_MD_CANDIDATE = FILE_TRAJ_MD.replace('.extxyz', '_candidate.extxyz') class-attribute instance-attribute

FILE_ITER_DATA = 'label_data.extxyz' class-attribute instance-attribute

FILE_COLLECT_DATA = 'collect_label_data.extxyz' class-attribute instance-attribute

FMT_ITER = '04d' class-attribute instance-attribute

FMT_STAGE = '02d' class-attribute instance-attribute

FMT_MODEL = '02d' class-attribute instance-attribute

FMT_STRUCT = '05d' class-attribute instance-attribute

FMT_TASK_MD = '06d' class-attribute instance-attribute

FMT_TASK_DFT = '06d' class-attribute instance-attribute

RUNFILE_LAMMPS = 'cli_lammps.lmp' class-attribute instance-attribute

FILE_ARG_LAMMPS = 'arg_lammps.yml' class-attribute instance-attribute

FILE_ARG_ASE = 'arg_ase.yml' class-attribute instance-attribute

SCRIPT_ASE_PATH = f'{ALFF_ROOT}/util/script_ase' class-attribute instance-attribute

SCHEMA_ASE_RUN = f'{ALFF_ROOT}/util/script_ase/schema_ase_run.yml' class-attribute instance-attribute

SCHEMA_LAMMPS = f'{ALFF_ROOT}/util/script_lammps/schema_lammps.yml' class-attribute instance-attribute

SCHEMA_ACTIVE_LEARN = f'{ALFF_ROOT}/al/schema_active_learn.yml' class-attribute instance-attribute

SCHEMA_FINETUNE = f'{ALFF_ROOT}/al/schema_finetune.yml' class-attribute instance-attribute

DIR_MAKE_STRUCT = '00_make_structure' class-attribute instance-attribute

DIR_STRAIN = '01_strain' class-attribute instance-attribute

DIR_GENDATA = '02_gendata' class-attribute instance-attribute

FILE_FRAME_UNLABEL = 'conf.extxyz' class-attribute instance-attribute

FILE_FRAME_LABEL = 'conf_label.extxyz' class-attribute instance-attribute

FILE_TRAJ_LABEL = 'traj_label.extxyz' class-attribute instance-attribute

SCHEMA_ASE_BUILD = f'{ALFF_ROOT}/util/script_ase/schema_ase_build.yml' class-attribute instance-attribute

SCHEMA_GENDATA = f'{ALFF_ROOT}/gdata/schema_gendata.yml' class-attribute instance-attribute

SCHEMA_PHONON = f'{ALFF_ROOT}/phonon/schema_phonon.yml' class-attribute instance-attribute

SCHEMA_ELASTIC = f'{ALFF_ROOT}/elastic/schema_elastic.yml' class-attribute instance-attribute

SCHEMA_PES_SCAN = f'{ALFF_ROOT}/pes/schema_pes_scan.yml' class-attribute instance-attribute

DIR_SUPERCELL = '01_supercell' class-attribute instance-attribute

DIR_PHONON = '02_phonon' class-attribute instance-attribute

FILE_PHONOPYwFORCES = 'phonopy_with_forces.yml' class-attribute instance-attribute

DIR_ELASTIC = '02_elastic' class-attribute instance-attribute

DIR_SCAN = '01_scan' class-attribute instance-attribute

DIR_PES = '02_pes' class-attribute instance-attribute

init_alff_logger() -> ColorLogger

Initializing the logger.

get_logger()