Documentation menu

Version 1.8.0-community

Relation Graph

How GoArchive turns declared table relationships into a copy order, a delete order, and a set of rows that belong together.

Everything GoArchive does starts from the relationships you declare between tables.

Declaring relationships

A job names one root table and any number of relations. Each relation says: this table has rows that belong to the parent, joined by this column.

jobs:
  archive_old_orders:
    root_table: orders
    primary_key: id
    where: "created_at < DATE_SUB(NOW(), INTERVAL 2 YEAR)"
    relations:
      - table: order_items
        primary_key: id
        foreign_key: order_id
        dependency_type: "1-N"
      - table: shipments
        primary_key: id
        foreign_key: order_id
        dependency_type: "1-1"
        relations:
          - table: shipment_items
            primary_key: id
            foreign_key: shipment_id
            dependency_type: "1-N"

Relations nest. shipment_items belongs to shipments, which belongs to orders. The nesting reflects the real structure of your data.

From relationships to an order

GoArchive builds a graph from these declarations and sorts it topologically. That produces two orders, each the reverse of the other:

orders
  β”œβ”€β”€ order_items
  β”œβ”€β”€ order_payments
  └── shipments
        └── shipment_items

Copy order:    orders β†’ order_items β†’ order_payments β†’ shipments β†’ shipment_items
Delete order:  shipment_items β†’ shipments β†’ order_payments β†’ order_items β†’ orders

Parents are written first, so a child row never arrives at the destination before the row it refers to. Children are removed first, so a parent is never removed while something still points at it.

You can see both orders for any job:

goarchive plan -c archiver.yaml --job archive_old_orders

Finding the rows that belong together

The order tells GoArchive which table to process when. Discovery tells it which rows.

For each batch, GoArchive:

  1. Selects a set of root rows matching the job’s where clause.
  2. Follows each declared relation outward from those rows, level by level, collecting the children, then the children’s children, and so on.
  3. Ends with the complete set of rows belonging to that batch of root rows.

This set is what gets copied, verified, and removed together.

Why the declaration must match the database

The relationships in your configuration describe the same structure the database already enforces through its foreign keys. GoArchive checks that the two agree before it starts, because a mismatch would produce the wrong order.

If a table is declared as a sibling when the database says it is a child, the delete order would be wrong and the database would reject it. Checking up front turns that into a clear message before any data moves.

The check also looks outward: if a table outside your job holds a reference into it, GoArchive reports it, because removing the referenced rows would affect data the job never accounted for.

Requirements

For discovery and removal to be exact, every participating table needs a single-column primary key, and that column must be the table’s real primary key. The root table’s key must additionally be an integer, because progress is tracked as a position within that key.

Child tables may use any single-column key type.