> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataerai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Metadata extraction

> Extract structured metadata from 44 scientific data formats — array containers, microscopy, medical imaging, genomics, mass spec, spectroscopy and more — straight from Python with dataerai.metaextract.

`dataerai.metaextract` reads 44 scientific data formats across 76 file extensions — from array containers (HDF5, NetCDF, Zarr) and imaging (TIFF, FITS, DICOM) to genomics, mass spec and instrument files — and returns plain, JSON-ready metadata dictionaries with physical quantities tagged for unit-aware search. It can also convert supported files to [xarray](https://xarray.dev)/netCDF, classify the **data type**, compute checksums, and render interactive [Plotly](https://plotly.com/python/) visualizations.

It ships inside the [`dataerai-sdk`](/sdks/python) package — the same library DataErai runs server-side for [automatic metadata extraction](/data/metadata-extraction) on upload.

## Install

The scientific stack is optional, so a bare `pip install dataerai-sdk` stays light. Pull in only the capability you need:

```bash theme={null}
pip install "dataerai-sdk[metaextract]"             # core extractors
pip install "dataerai-sdk[metaextract-converters]"  # + xarray / netCDF conversion
pip install "dataerai-sdk[metaextract-viz]"         # + Plotly visualizations
pip install "dataerai-sdk[metaextract-checksums]"   # + fast blake3 checksums
pip install "dataerai-sdk[metaextract-utils]"       # + download / benchmark helpers
pip install "dataerai-sdk[metaextract-all]"         # everything above
```

<Note>
  The `[metaextract]` extra pulls compiled scientific dependencies (h5py, hyperspy, lxml, igor2). On Linux you may need a C toolchain (e.g. `build-essential`) for any wheels that aren't pre-built for your platform.
</Note>

## Supported formats

`dataerai.metaextract` extracts metadata from **44 scientific data formats** across
nine domains — array containers (HDF5, NetCDF, Zarr, MATLAB, Parquet), imaging
(TIFF/OME, FITS), microscopy (Nikon ND2, Zeiss CZI, Leica LIF, Olympus), medical &
neuro (DICOM, NIfTI, EDF, NWB), genomics (FASTA/FASTQ, VCF, SAM/BAM, GFF, FCS), mass
spec (mzML, imzML, Thermo RAW), spectroscopy (Bruker OPUS, Renishaw WDF, JCAMP-DX),
crystallography (CIF, VASP) and computational chemistry (Gaussian/ORCA, XYZ, MOL/SDF,
PDB/mmCIF) — alongside the original AFM/EM/XRD/RHEED instrument set.

See the full **[Supported formats](/metaextract/supported-formats)** table for
extensions and the reader each one needs. Every physical quantity is emitted in the
units engine's `{"value", "unit"}` shape, so extracted values are immediately
searchable in [unit-aware metadata search](/data/metadata-extraction).

Probe the live capability matrix at runtime with `get_supported_formats()` — entries degrade to `extract: false` when an optional dependency is missing rather than raising on import.

## Convert, classify and visualize

Beyond metadata, the package offers three further capabilities:

* **Convert to xarray/netCDF** — the instrument formats `.ibw`, `.xrdml` and `.imm` carry full converters, so `convert_file()` returns a labelled `xarray.Dataset` (and can save it as netCDF). Other formats are metadata-only.
* **Classify the data type** — `detect_data_type()` matches extracted metadata against built-in definitions (for example `Tapping Mode AFM`, `Single Frequency PFM`, `XRD Rocking Curve`) so you can route a file to the right pipeline. See [Data-type detection](/metaextract/data-types).
* **Visualize** — ready-made [Plotly](https://plotly.com/python/) plots ship for AFM, the three PFM variants (single-frequency, vector, DART) and XRD (rocking curve, 2θ–ω scan, reciprocal space map), plus generic metadata explorers. See [Visualization](/metaextract/visualization).

## 20-second example

```python theme={null}
from dataerai.metaextract import extract_metadata, detect_data_type

metadata = extract_metadata("scan.ibw")     # → dict of instrument settings
data_type = detect_data_type(metadata)      # → {'type': 'Tapping Mode AFM', 'confidence': 0.95, ...}
```

## Next steps

<CardGroup cols={2}>
  <Card title="Quickstart" icon="rocket" href="/metaextract/quickstart">
    Extract, convert, checksum, and batch-process files.
  </Card>

  <Card title="Data-type detection" icon="scan-search" href="/metaextract/data-types">
    Classify metadata and define your own data types.
  </Card>

  <Card title="Visualization" icon="chart-line" href="/metaextract/visualization">
    Interactive Plotly views of metadata and data.
  </Card>

  <Card title="Add a format" icon="puzzle" href="/metaextract/adding-formats">
    Write an extractor/converter for a new instrument file.
  </Card>
</CardGroup>