Welcome to midas_open_parser’s documentation!

This is the documentation for the midas_open_parser library, which is designed to parse BADC-CSV files from the MIDAS Open dataset.

Installation

You can install midas_open_parser using pip:

pip install midas_open_parser

Usage

Parsing BADC-CSV Files

The midas_open_parser library provides functions to parse BADC-CSV files and extract data records and metadata.

from midas_open_parser import parse_badc_csv, parse_badc_csv_metadata

# Parse the BADC-CSV file
metadata = parse_badc_csv_metadata('path/to/file.csv')
data_rows = parse_badc_csv('path/to/file.csv')

Extracting Metadata

Once you have parsed the BADC-CSV file and obtained the metadata, you can extract the MIDAS metadata using the extract_midas_metadata function.

from midas_open_parser import extract_midas_metadata

# Extract MIDAS metadata
midas_metadata = extract_midas_metadata(metadata)

The midas_metadata dictionary contains the extracted metadata, organized into global metadata (midas_metadata['global']) and field-level metadata (midas_metadata[field_name]).

Handling Metadata Labels

The midas_open_parser library provides a flexible and extensible architecture for handling various metadata labels. Each label is associated with a handler class that defines how the metadata should be processed.

The library includes default handlers for commonly used metadata labels, such as ‘Conventions’, ‘title’, ‘source’, ‘creator’, ‘activity’, ‘feature_type’, ‘observation_station’, ‘location’, ‘collection_name’, ‘collection_version_number’, ‘date_valid’, ‘history’, ‘last_revised_date’, ‘comments’, ‘coordinate_variable’, ‘long_name’, ‘type’, ‘src_id’, ‘historic_county_name’, ‘height’, ‘midas_qc_version_number’, ‘midas_station_id’, and ‘missing_value’.

Handling Unknown Metadata Labels

If the library encounters an unknown metadata label, it will raise an UnknownMetadataLabelError. You can handle this error by catching the exception and taking appropriate action.

from midas_open_parser import extract_midas_metadata, UnknownMetadataLabelError

try:
    midas_metadata = extract_midas_metadata(metadata)
except UnknownMetadataLabelError as e:
    print(f"Error: {e}")

Extending the Library

You can extend the midas_open_parser library by creating new handler classes for additional metadata labels. To create a new handler class, you need to inherit from the LabelHandler base class and implement the handle_global or handle_field methods, depending on whether the label is a global or field-level metadata label.

from midas_open_parser.midas import LabelHandler

class NewLabelHandler(LabelHandler):
    def handle_global(self, values):
        # Implement logic for handling global metadata
        return processed_values

    def handle_field(self, field_name, values, label):
        # Implement logic for handling field-level metadata
        return {field_name: processed_values}

API reference

midas_open_parser

A Python library for parsing BADC-CSV files from the MIDAS Open dataset.

This library provides a flexible and extensible framework for handling various metadata labels and structures found in the MIDAS Open dataset.

exception midas_open_parser.UnknownMetadataLabelError

Bases: Exception

An exception raised when an unknown metadata label is encountered during metadata extraction.

midas_open_parser.extract_midas_metadata(metadata, label_handlers={'Conventions': <midas_open_parser.midas.ConventionsHandler object>, 'activity': <midas_open_parser.midas.ActivityHandler object>, 'collection_name': <midas_open_parser.midas.CollectionNameHandler object>, 'collection_version_number': <midas_open_parser.midas.CollectionVersionNumberHandler object>, 'comments': <midas_open_parser.midas.CommentsHandler object>, 'coordinate_variable': <midas_open_parser.midas.CoordinateVariableHandler object>, 'creator': <midas_open_parser.midas.CreatorHandler object>, 'date_valid': <midas_open_parser.midas.DateValidHandler object>, 'feature_type': <midas_open_parser.midas.FeatureTypeHandler object>, 'height': <midas_open_parser.midas.HeightHandler object>, 'historic_county_name': <midas_open_parser.midas.HistoricCountyNameHandler object>, 'history': <midas_open_parser.midas.HistoryHandler object>, 'last_revised_date': <midas_open_parser.midas.LastRevisedDateHandler object>, 'location': <midas_open_parser.midas.LocationHandler object>, 'long_name': <midas_open_parser.midas.LongNameHandler object>, 'midas_qc_version_number': <midas_open_parser.midas.MidasQcVersionNumberHandler object>, 'midas_station_id': <midas_open_parser.midas.MidasStationIdHandler object>, 'missing_value': <midas_open_parser.midas.MissingValueHandler object>, 'observation_station': <midas_open_parser.midas.ObservationStationHandler object>, 'source': <midas_open_parser.midas.SourceHandler object>, 'src_id': <midas_open_parser.midas.SrcIdHandler object>, 'title': <midas_open_parser.midas.TitleHandler object>, 'type': <midas_open_parser.midas.TypeHandler object>})

Extract MIDAS metadata from the parsed BADC-CSV metadata.

Args:: metadata (dict): The parsed BADC-CSV metadata dictionary. label_handlers (dict, optional): A dictionary containing instances of LabelHandler subclasses for handling specific metadata labels.
Returns:: dict: A dictionary containing the extracted MIDAS metadata, organized into global metadata (midas_metadata[‘global’]) and field-level metadata (midas_metadata[field_name]).
Raises:: UnknownMetadataLabelError: If an unknown metadata label is encountered and there is no corresponding handler in label_handlers.

midas_open_parser.parse_badc_csv(file_path)

Parse a BADC-CSV file and extract data records.

Args:: file_path (str): The path to the BADC-CSV file.
Returns:: list: A list of dictionaries, where each dictionary represents a data record.

midas_open_parser.parse_badc_csv_metadata(file_path)

Parse the metadata section of a BADC-CSV file.

Args:: file_path (str): The path to the BADC-CSV file.
Returns:: dict: A dictionary containing the parsed metadata, where the keys are the metadata labels, and the values are lists of tuples containing the reference and associated values.

Contributing

Contributions to the midas_open_parser project are welcome! If you encounter any issues or have suggestions for improvements, please open an issue on the project’s GitHub repository. If you would like to contribute code, please submit a pull request.

Licence

This project is licensed under the MIT License.