Limitations

GoArchive is for cold data: rows that are no longer written by the application. It is not designed for hot, actively changing data.

Hard constraints #

These are refused before a run starts, at connection or in preflight, except the run contract and JSON values, which are not checked. Stored temporal key values are refused when met, before any delete.

Primary keys #

Every participating table needs a PRIMARY KEY, single-column or composite, and its configured primary_key must name exactly those columns, in key order. FLOAT/DOUBLE, ENUM/SET, BIT, any other type outside the allowed list , and prefix key parts are refused as keys. The root table’s key definition must stay the one the job was created with; a changed one is refused at startup.

Temporal values and identities #

DATE, DATETIME and TIMESTAMP key values must be valid: YYYY-MM-DD with year 0001–9999 and a real calendar day, plus HH:MM:SS and 0–6 fractional digits for DATETIME and TIMESTAMP. Zero dates, invalid clocks, time-zone suffixes and leap seconds are refused with TEMPORAL_READ_CONTRACT, whatever the server’s SQL modes; no override bypasses it. Non-key temporal values are copied as SQL text and never normalized.

No DDL and no concurrent writes during a run (contract) #

GoArchive’s correctness rests on these operating contracts. Violations are documented, not detected, except where recovery stops:

1. No DDL on participating tables while a job runs. Column lists are read once at startup. A renamed or dropped column fails an archive or copy-only batch before its copy commits; an added column is not detected and not copied. Run schema changes between runs.
2. Cold data only. The rows of a root’s tree must not change from the moment GoArchive reads them until it has finished that root, including across a crash and the recovery that follows. Recovery finishes a logged root whose source rows were deleted from outside without being able to tell.
3. Changed rows are not re-copied. A source row updated after GoArchive copied it is not re-copied, detected or reported.

Source and destination must be different databases #

SRC_DEST_IDENTITY_CHECK refuses a source and destination with the same server identity and the same schema name. The same server with a different schema is supported.

• Schema names are compared without regard to letter case on every server, so App and app are refused as one schema.
• Two servers that share a server identity (the same server_uuid, as happens when one was cloned from the other) are one server to GoArchive: with the same schema name on both, the job is refused.
• Each connection must reach one backend for the whole run; a proxy that routes one account’s statements to different servers is not supported.

InnoDB only #

Every participating source and destination table must be InnoDB.

Uncovered incoming foreign keys #

Any table outside the job holding a foreign key into it is fatal, for every ON DELETE rule. A table in another schema cannot join a job, so an incoming cross-schema foreign key is always fatal. For archive and purge, proving that none was missed needs PROCESS on the source account.

Destination schema must not be stricter than the source #

The destination may drop secondary indexes, auto_increment and defaults, and may relax NOT NULL. It must not add constraints the source lacks (Destination Checks ).

Table names on servers that store them in lower case #

On a server that stores table names in lower case, write every table name in the job in lower case. A mixed-case name is refused with TABLE_EXISTENCE_CHECK.

Character sets #

Every text column the job reads or writes must use a supported character set; preflight refuses the others (Character Sets ).

JSON values #

JSON is copied as its printed text, so a member’s MySQL type is not kept and some values can change in the copy. sha256 verification refuses a changed value; count and none do not compare values, and print a warning naming the job’s JSON columns.

Model limitations #

Supported relationship types #

1:1 and 1:N only.

• Many-to-many (N:M) cannot be expressed. A table appears once in a job, so a join table with two parents in the job carries only one relation; its second foreign key fails INTERNAL_FK_COVERAGE.
• Self-referencing hierarchies, a table referencing its own key, cannot be archived. Archive and purge refuse one with ON DELETE CASCADE or SET NULL (SELF_FK_DELETE_RULE_CHECK).

Relations must describe ownership #

Archive and purge delete the child keys discovered for the current root batch without checking whether another root still needs them.

Foreign key ON DELETE CASCADE #

GoArchive deletes children before their parents itself. A cascade between two job tables is redundant work that GoArchive does not account for; preflight warns about it.

Database triggers #

GoArchive cannot see logic in triggers: a source DELETE trigger that changes other tables has side effects GoArchive does not verify. Preflight refuses destination INSERT triggers and, unless overridden, source DELETE triggers.

Operational cautions #

Deep or wide trees can exhaust memory #

Discovery holds every descendant key of a root batch in memory. A deep tree with high fan-out at each level can grow it without bound: start with a small batch_size and raise it while watching memory.

A value larger than the destination’s max_allowed_packet fails its batch #

A row or an INSERT may exceed the destination’s max_allowed_packet, because large values are sent separately. A single value larger than it cannot be sent: nothing of the batch is logged or deleted, and the error names the table, column and size; raise the destination’s max_allowed_packet above it. An INSERT of many values each just under the long-data threshold can be refused whole: lower batch_size.

A transaction over Group Replication’s size limit fails its commit #

On a Group Replication source, a delete chunk larger than the group’s transaction size limit (group_replication_transaction_size_limit) fails its commit. Nothing is lost: the roots stay logged (copied, or deleting for purge). Lower batch_delete_size until each chunk fits, and rerun: recovery finishes them. On a Group Replication destination, the copy transaction can fail the same way: lower batch_size.

The copy transaction spans all tables #

One destination transaction copies, verifies and logs a batch across every table, holding row locks on tables already inserted while later ones are written. Avoid a destination that other workloads read concurrently.

Sequential by design #

One batch at a time, with no parallelism. Runs whose job trees share a source table run one at a time: a second run of the same job, a job on another root whose tree overlaps, the same tables sent to another destination, and two copy-only jobs reading the same tables are all refused while the first runs (Concurrency and Locking ).

A stopped job with unfinished work blocks every other job that shares one of its tables until its own rerun finishes that work. Its limits:

• Same-named databases on different servers block each other when they share one job_schema and the source cannot tell them apart: gtid_mode is not ON, the source is a Group Replication member, or the shards were provisioned from one another. Give each source server its own job_schema.
• Not detected: a server that received the stopped job’s data only before a GTID migration, and a server outside a group that holds the group’s data. Re-bind the stopped job before another job runs on its tables.
• Not detected: a stopped job recorded on a replica of this source.
• One registry: a stopped job tracked in another job_schema does not block.

The locks live on one server, so every connection must reach one backend for the whole run: a read/write-splitting proxy could send the locks to a replica while the deletes go to the primary. A copy-only job reading a replica and an archive writing its primary are not serialized; the run contract covers that case. On Group Replication in multi-primary mode, locks are local to a member, so runs connected to different members are not serialized.

Partial deletes after an interruption #

Delete chunks commit separately, to avoid long source locks. A run stopped between child and parent deletes leaves children removed while the parent remains, until the next start’s recovery finishes the root. The rows were copied and verified first, but monitoring that checks referential integrity during a run can see the gap.

Copy-only reruns rescan the whole range #

There is no checkpoint, so a copy-only rerun with resume: true reads the whole where range again to skip the roots it copied before. On a very large range, that costs a full scan per run.

No built-in metrics #

There is no metrics endpoint. Use the --progress display , the structured logs and the tracking tables .

Data loss on misconfiguration is possible #

Validation checks structure, not intent: a job with the wrong where deletes the wrong rows. Check the where against the data and dry-run’s first batch before running archive, and keep valid backups.

Trust model #

GoArchive defends against the operator’s mistakes, not against the operator. The configuration file is trusted input: whoever writes it already holds the credentials it names.

• Job where values are free SQL conditions by design. Only table and column identifiers are validated.
• When the person writing the configuration is not the credential holder, the boundary is the privileges of the accounts it names: grant only what the privilege matrix lists.
• Once a batch’s source rows are deleted, the archived rows exist only on the destination; keeping them safe, through durability, replication and backups, is outside GoArchive.

Do not expose configuration editing to untrusted users or untrusted automation.