Admin 06 Jun 2026 05:14

admin #general

Generic Validation Checks for S100 GML and HDF5 Data Formats

Introduction

The S100 standard defines an international framework for the exchange of hydrographic and maritime data. Two of the most common encodings used within this framework are GML (Geography Markup Language) and HDF5 (Hierarchical Data Format version5). While the underlying models differXMLbased for GML and binary hierarchical for HDF5both require rigorous validation to ensure data integrity, interoperability, and compliance with regulatory expectations.

This page outlines a set of generic validation checks that can be applied to either format, highlights formatspecific considerations, and provides guidance for building automated validation pipelines.

S100 GML Overview

GML is an XML grammar designed to encode geographic features, geometry, and associated metadata. Within the S100 suite, GML files are typically packaged as .gml or .xml documents that reference a set of XML Schema Definition (XSD) files defining the allowed structure, data types, and constraints.

Key characteristics:

Schemadriven: Validation against XSD is mandatory for structural compliance.
Namespace usage: Each S100 product specification defines its own namespace.
Coordinate reference systems (CRS): GML elements must reference an EPSG code or a gml:ReferenceSystem definition.
Feature attributes: Attribute values may be constrained by enumerations, ranges, or pattern facets.

HDF5 Overview

HDF5 stores data in a selfdescribing, portable binary format. The format uses groups (similar to directories) and datasets (arrays, tables, images, etc.) that can contain attributes. Within S100, HDF5 files are used for large raster or gridded data such as bathymetry, hydrography, or environmental layers.

Important traits:

Hierarchical organization: Data is stored in a treelike structure.
Metadata attributes: Each group or dataset may carry attributes that describe units, CRS, fill values, etc.
Chunking & compression: Allows efficient access to large datasets.
External references: HDF5 can embed references to other files or datasets.

Generic Validation Checks

1. Schema / Structure Validation

GML: Validate the XML document against the official S100 XSD set using an XML validator (e.g., xmllint, Xerces, or Pythons lxml). The validator must be run with namespace awareness and must resolve any xs:import statements.

HDF5: Verify that required groups and datasets exist. A simple schema can be expressed as a JSON or YAML description that lists expected paths, data types, and required attributes. Tools such as h5check or custom scripts using the h5py library can perform this check.

2. DataType Consistency

Ensure that the actual data type of each element matches the declared type. For GML, this means checking that numeric values conform to xsd:decimal or xsd:integer and that strings obey any pattern constraints. In HDF5, confirm that dataset dtype matches the specification (e.g., float32 for depth values) and that attribute types are consistent.

3. Range and Enumerated Value Checks

Many S100 attributes have defined min/max ranges or enumerated value sets. Validation should:

Check numeric values against documented limits (e.g., depth values between 11000m and 0m).
Verify that coded attributes (e.g., featureType) contain only values listed in the corresponding code list.

4. Coordinate Reference System (CRS) Validation

For both formats, the CRS must be explicitly declared and must be a recognized EPSG identifier. Validation steps:

Extract the CRS reference (e.g., gml:srsName in GML or the crs attribute in HDF5).
Confirm that the identifier exists in the EPSG database.
If the dataset includes a transformation matrix, verify that its dimensions match the CRS definition.

5. Attribute Presence and Cardinality

The specification often defines mandatory attributes and limits on repetitions. Validation must:

Ensure mandatory attributes (e.g., gml:id, featureCode) are present.
Check that optional attributes do not exceed the allowed maximum occurrences.

6. File Integrity and Completeness

Detect truncated or corrupted files. For XML, check that the document parses without errors. For HDF5, use the h5check utility or the HDF5 librarys builtin validation function to ensure the files internal checksum is valid.

7. CrossReference Consistency

S100 datasets may reference external resources (e.g., image files, auxiliary tables). Validation should confirm that:

All xlink:href references in GML resolve to reachable URIs.
HDF5 external links point to existing files and the linked objects have the expected types.

8. Units and Dimensional Consistency

Attributes that describe physical quantities must carry unit specifications, typically using uom (unit of measure) URIs. Validation checks include:

Presence of a unit attribute for each numeric field.
Conformance of the unit URI to the ISO19100 series or the OGC Units of Measure database.
Consistency across related fields (e.g., depth and vertical datum must share compatible units).

9. PerformanceRelated Checks (Optional)

Large S100 products often require efficient access. While not strictly a validation step, checking that:

HDF5 datasets are chunked appropriately for typical access patterns.
Compression filters (e.g., gzip, szip) are applied where recommended.
GML files are split into logical sections to avoid excessively large XML trees.

10. Documentation and Metadata Completeness

Both formats rely on accompanying metadata (ISO19115/19139 for GML; HDF5 attributes for datasets). Validation should verify that required metadata elementssuch as title, date, responsibleParty, and lineageare present and populated.

Tip: Implement the checks as modular functions so they can be reused across different product types. A typical workflow is: parse schema check attribute checks CRS verification crossreference resolution final report.

Sample Validation Table

Check Category	GML Implementation	HDF5 Implementation
Schema Validation	XML validator against S100 XSD	JSON/YAML schema applied with `h5py`
DataType Consistency	XPath type checks; XSD facets	Dataset `dtype` inspection
Range/Enumeration	Pattern / enumeration facets in XSD	Python list of allowed values compared to attribute values
CRS Validation	`gml:srsName` vs EPSG database	Attribute `crs` vs EPSG database
CrossReference	Resolve `xlink:href`	Verify external links using `h5py.File(..., libver='latest')`

Best Practices for Building a Validation Pipeline

Separate Concerns: Keep schema validation, semantic checks, and performance checks in distinct stages.
Use OpenSource Libraries: Leverage lxml (Python), xercesc, or libxml2 for GML; use h5py and the HDF5 C library for binary checks.

Reference Files For Generic Validation Checks For S-100 GML And HDF5 Data Formats

Screenshoot

File Name

s100wg5_slides_rm15_20_21_22_1.pptx

File Size

0.10 MB

File Type

PPTX

File Site

Jagomart.net

Description

This file is just a reference file for Generic Validation Checks For S-100 GML And HDF5 Data Formats. Does not guarantee that the specific things you want are included in it.

Download on the Jagomart.net website

Direct download (wait 10 seconds)