Operations

Once a job is defined and tested, it usually becomes something that runs on a schedule. This page covers running GoArchive routinely.

Scheduling #

GoArchive runs to completion and exits, so any scheduler can drive it.

# weekly, Sunday at 02:00
0 2 * * 0 /usr/local/bin/goarchive archive \
            -c /etc/goarchive/archiver.yaml \
            --job archive_old_orders \
            --log-format json \
            --log-level info >> /var/log/goarchive/cron.log 2>&1

A few things make scheduled runs predictable:

• Use absolute paths for both the binary and the configuration file.
• Give a long job room. If a run may exceed the interval between runs, note that a second run cannot start while the first holds its lock — it reports the conflict and exits rather than running concurrently.
• Let an unfinished run continue next time. Because progress is recorded, the next scheduled run resumes rather than starting over. A job too large for one window can complete across several.

Running within a time window #

To confine work to a quiet period, schedule the start and use the pause switch to end it:

0 2 * * * /usr/local/bin/goarchive archive -c /etc/goarchive/archiver.yaml --job archive_old_orders
0 6 * * * /usr/bin/touch /var/run/goarchive/pause.flag
5 6 * * * /usr/bin/pkill -TERM goarchive; /bin/rm -f /var/run/goarchive/pause.flag

The pause takes effect at the next batch boundary, and the termination signal stops the run cleanly. Whatever remains is picked up by the next scheduled run.

Monitoring a run #

GoArchive writes structured logs. Send them to a file for scheduled runs:

logging:
  level: info
  format: json
  output: /var/log/goarchive/archive.log
  file_only: true

Every entry carries the job name, so several jobs can share a destination and remain distinguishable.

Progress is also visible in the tracking tables while a run is in flight:

SELECT job_name, job_status, last_processed_root_pk_id, last_heartbeat_at
FROM archiver_job
WHERE job_name = 'archive_old_orders';

A job that is running refreshes its heartbeat as it goes.

Controlling a running job #

Pause by creating the sentinel file; resume by removing it:

touch /var/run/goarchive/pause.flag
rm    /var/run/goarchive/pause.flag

Stop with Ctrl-C, or by sending a termination signal. The current batch finishes, its progress is recorded, and the process exits. Running the same command again continues from that point.

Log files #

GoArchive opens its log files in append mode and does not rotate them, so an external tool can manage rotation:

/var/log/goarchive/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    copytruncate
}

Routine maintenance #

• Re-run validate after schema changes. A job describes relationships between tables; if those tables change, confirm the description still matches.
• Review dry-run output periodically. Row counts drift as data grows, and the preview is the quickest way to see how much a scheduled run now moves.
• Keep the tracking records. They are the history of what was archived, and what makes an interrupted run resumable.