io

thkit.io

Functions:

• write_yaml –

  Write data to a YAML file.

• read_yaml –

  Read data from a YAML file.

• combine_text_files –

  Combine text files into a single file in a memory-efficient. Read and write in chunks to avoid loading large files into memory

• unpack_dict –

  Unpack one level of nested dictionary.

• download_rawtext –

  Download raw text from a URL.

write_yaml(jdata: dict, filename: Union[str, Path])

Write data to a YAML file.

read_yaml(filename: Union[str, Path]) -> dict

Read data from a YAML file.

combine_text_files(files: list[str], output_file: str, chunk_size: int = 1024)

Combine text files into a single file in a memory-efficient. Read and write in chunks to avoid loading large files into memory

Parameters:

• files (list[str]) –

  List of file paths to combine.

• output_file (str) –

  Path to the output file.

• chunk_size (int, default: 1024 ) –

  Size of each chunk in KB to read/write. Defaults to 1024 KB.

unpack_dict(nested_dict: dict) -> dict

Unpack one level of nested dictionary.

download_rawtext(url: str, outfile: str = None) -> str

Download raw text from a URL.