> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataerai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# NGA bulk imports

> Stage and run resumable National Gallery of Art imports from the Open Data snapshot.

Platform operators can import large National Gallery of Art Open Data slices
into a project with the same durable batch ledger used by Zenodo bulk imports.
Use this workflow for demo or backfill runs that are larger than the interactive
**Import from NGA** page.

NGA imports create metadata-first artwork assets. Dataerai stores the artwork
metadata and link-out URLs; it does not copy IIIF images into the project's
allocation.

## Start from the admin page

System administrators can open **Import Repositories**, choose **National
Gallery of Art**, provide the batch parameters, and click **Stage and run**. The
browser flow stages the same durable `BulkImportBatch` rows as the management
command and then starts `run_bulk_import_batch` for the new batch.

Use the browser form for normal operator starts. Use the commands below when
you need dry-run counts, shell access, resume/retry control, or recovery after a
web process stops.

## Stage from the snapshot

Build or reuse the local NGA Open Data snapshot, then stage object IDs:

```bash theme={null}
python manage.py stage_nga_bulk_import \
  --name "NGA paintings smoke" \
  --project-id <project-uuid> \
  --creator-email <operator-or-owner@example.org> \
  --classification Painting \
  --limit 100
```

The command creates a `BulkImportBatch` with `provider=nga` and one
`BulkImportRecord` per staged object ID. By default, it skips NGA assets already
materialized in the target project. Pass `--include-existing-assets` only when
you intentionally want to revisit existing rows.

Useful staging options:

| Option                      | Use                                            |
| --------------------------- | ---------------------------------------------- |
| `--classification Painting` | Restrict to one NGA classification.            |
| `--limit 100`               | Stage at most this many object IDs.            |
| `--offset 1000`             | Resume staging from a later snapshot position. |
| `--refresh-cache`           | Rebuild the local NGA snapshot before staging. |
| `--dry-run`                 | Report counts without writing batch rows.      |

## Run the batch

Drain staged records through the shared import path:

```bash theme={null}
python manage.py run_bulk_import_batch \
  --batch-id <batch-uuid> \
  --rate-limit-per-minute 600 \
  --progress-every 25
```

The runner calls the same `sync_import_page` body used by scheduled imports, one
NGA object ID at a time. It records each row as imported, updated, unchanged,
skipped, gone, or failed.

If a process stopped mid-run, resume safely:

```bash theme={null}
python manage.py run_bulk_import_batch \
  --batch-id <batch-uuid> \
  --reset-stale-running-minutes 60 \
  --retry-failed
```

## Monitor progress

Check a batch:

```bash theme={null}
python manage.py bulk_import_status --batch-id <batch-uuid>
```

For JSON output:

```bash theme={null}
python manage.py bulk_import_status --batch-id <batch-uuid> --json
```

The status payload includes `project_source_assets`, which reports target
project asset counts, distinct NGA object IDs, and staged records that have
materialized as `AssetContent.source = "nga"`.

## Metadata keys

Imported NGA assets keep provider provenance in `metadata.source = "nga"`, but
the artwork fields are unprefixed:

* `object_id`
* `accession_number`
* `classification`
* `medium`
* `artists`
* `styles`
* `iiif_image_url`
* `iiif_thumbnail_url`
* `object_page_url`