Configuration
How a GoArchive configuration file is organised, and how global settings and per-job overrides fit together.
GoArchive reads a single YAML file. It describes the servers to connect to, the jobs to run, and how those jobs should behave.
goarchive archive -c archiver.yaml --job archive_old_orders
The -c flag selects the file. Without it, GoArchive looks for archiver.yaml
in the current directory.
The blocks
| Block | Describes |
|---|---|
source | The database to read from |
destination | The database to write to |
replica | An optional replica to monitor for lag |
jobs | What to archive, and the relationships involved |
processing | Batch sizes and pacing |
safety | Replica thresholds and related settings |
verification | How copies are confirmed |
logging | Output level, format, and destination |
Global settings and per-job overrides
processing, verification, and logging can each be set once for the whole
file and then overridden by an individual job:
processing:
batch_size: 1000
sleep_seconds: 1
jobs:
archive_old_orders:
root_table: orders
primary_key: id
where: "created_at < DATE_SUB(NOW(), INTERVAL 2 YEAR)"
processing:
sleep_seconds: 0 # this job runs without pausing
A setting a job does not mention is inherited. A setting it does mention applies to that job alone.
Many jobs, one file
A single file holds as many jobs as you need, each with its own tables, relationships, and settings:
jobs:
archive_old_orders:
root_table: orders
...
archive_old_sessions:
root_table: sessions
...
goarchive list-jobs -c archiver.yaml
Reference pages
Database Connections
β source, destination, replica,
and the privileges each needs.
Jobs and Relations β defining what to archive and how tables relate.
Processing and Safety β batch sizes, pacing, verification, and logging.
Checking a file
goarchive validate -c archiver.yaml
This confirms the file is well-formed and that everything it refers to exists and is usable.
