openairclim.utils.download_zenodo

Download files from a Zenodo record using the requests library.

openairclim.utils.download_zenodo.CHUNK_SIZE = 1048576

Chunk size (bytes) used when streaming file downloads.

openairclim.utils.download_zenodo.REQUEST_TIMEOUT = 30

Timeout (seconds) applied to every Zenodo API/download request.

openairclim.utils.download_zenodo.download(record_or_doi: str, output_dir: str | Path, file_glob: str = '*', force: bool = False, max_attempts: int = 3, backoff_seconds: float = 5.0) → None[source]

Download files from a Zenodo record matching file_glob into output_dir, skipping any file that already exists and passes checksum verification.

Raises:

  • ValueError – if no record ID can be found in record_or_doi

  • RuntimeError – if a downloaded file’s checksum doesn’t match the Zenodo record’s metadata

openairclim.utils.download_zenodo.download_file(url: str, dest: str | Path, max_attempts: int = 3, backoff_seconds: float = 5.0) → None[source]

Stream a single file from url to dest, retrying on transient errors.

Parameters:

  • url (str) – The URL to download.

  • dest (str or Path) – Local path to write the file to.

  • max_attempts (int) – Number of attempts before giving up.

  • backoff_seconds (float) – Base delay between retries; doubles each attempt (5s, 10s, 20s, …).

Raises:

  • requests.HTTPError – if the request does not succeed.

  • requests.exceptions.RequestException – if all attempts fail.

openairclim.utils.download_zenodo.fetch_json(url: str, max_attempts: int = 3, backoff_seconds: float = 5.0) → dict[source]

Fetch and parse JSON from a URL, retrying on transient errors.

Parameters:

  • url (str) – The URL to fetch.

  • max_attempts (int) – Number of attempts before giving up. Must be at least 1.

  • backoff_seconds (float) – Base delay between retries; doubles each attempt (5s, 10s, 20s, …).

Returns:

The parsed JSON response.

Return type:

dict

Raises:

  • requests.HTTPError – if the request does not succeed.

  • ValueError – if max_attempts is less than 1.

openairclim.utils.download_zenodo.fetch_record_json(record_or_doi: str) → dict[source]

Fetch a Zenodo record’s metadata.

Parameters:

record_or_doi (str) – Zenodo record ID, or a DOI/URL containing one.

Returns:

The record’s metadata, as returned by the Zenodo API.

Return type:

dict

openairclim.utils.download_zenodo.fetch_record_versions(record_or_doi: str) → list[dict][source]

Fetch every published version of a Zenodo concept record.

Parameters:

record_or_doi (str) – Concept record ID, or a DOI/URL containing one.

Returns:

The version records (each a full Zenodo record dict).

Return type:

list[dict]

openairclim.utils.download_zenodo.main()[source]

Parse command-line arguments and download the matching files

openairclim.utils.download_zenodo.verify_checksum(path: str | Path, expected: str) → bool[source]

Verify a local file’s checksum against a Zenodo-format checksum string.

Parameters:
Returns:

True if the file exists and its checksum matches, False

otherwise.

Return type:

bool