Performance Tuning and Monitoring

Performance work starts with measurement #

The first reflex on a slowing database is usually to add hardware resources. Adding resources gives temporary relief in some scenarios, but because it does not remove the root cause it makes the cost permanent and lays the ground for the problem to return at a larger scale.

A significant share of the slowdowns we meet in the field come from a handful of query patterns. Measuring that a single SELECT pattern consumes two thirds of the total query time on a system produces a result far more quickly than debating which server to grow. That is why we build our work on measurement.

Monitoring stack: PMM and Grafana #

Measuring requires data first. We install Percona Monitoring and Management (PMM) and gather MySQL and PostgreSQL metrics together with operating system metrics on a single dashboard. PMM’s Prometheus-based architecture keeps metrics in a queryable form and makes retrospective analysis possible during a post-incident review.

We arrange the Grafana dashboards around your workload and set alerts at meaningful thresholds. An alerting system tuned too sensitively starts being ignored before long; the purpose of an alert design is not to produce notifications but to single out the situation that needs a response.

If you already run Prometheus, Zabbix or a similar monitoring stack, we integrate the database metrics into your existing system.

Slow query analysis: GoFast #

The slow query log is the richest data source for performance problems, but it is not readable in its raw form. In a log file of thousands of lines the same query appears hundreds of times, with different parameters each time.

For the analysis of slow query records we use GoFast Enterprise , which we build ourselves.

GoFast Enterprise carries MySQL slow query logs into DuckDB with Go and makes them analytically queryable. The log is processed once, and every question after that is answered from the store. A hash of the records carried across is kept, and no duplicate record is created. Percentile values are calculated exactly. It runs as a single executable, with no dependencies, and inside Docker. Its report format is compatible with pt-query-digest.

The web interface shows the logs of more than one server on a single dashboard. Records are examined broken down by user, server, database and table. The same breakdowns are available in the command line reports. The course of a query fingerprint over time is followed on a graph. Daily and weekly views allow periods to be compared.

The tool does not need to connect to the database. You do not have to give access to the production system for an analysis; sending the log files is enough.

The parsing layer is published on GitHub as the MIT licensed open source gofast-cli. GoFast Enterprise is installed into customer infrastructure and provided under contract. The web interface and the gofast-mcp server are included in the licence.

AI-assisted analysis #

What has changed in performance analysis in recent years is not the data itself but the way questions are put to it. Once the slow query corpus and the metric history are opened to large language models, a question such as “what changed between 14:00 and 15:00 yesterday” can be answered in minutes.

We built two MCP (Model Context Protocol) servers for that purpose:

• gofast-mcp opens the slow query corpus produced by GoFast Enterprise to language models. It provides tools that can query the worst queries, the frequency-weighted impact score, lock contention per table, and the distribution of WHERE conditions in terms of index clustering. The component we use in incident analysis compares a time window with the reference period before it and ranks queries by how much they slowed down.
• PMM Prometheus MCP makes the metrics PMM collects queryable in the same way. It scans every saturation signal in an incident window — load average, CPU distribution, iowait, memory, disk latency and queue depth, network, and MySQL connection and QPS metrics — in a single call, and compares each metric’s peak with the median before it.

Used together, the two components allow the query-side and infrastructure-side traces of a performance incident to be examined at the same time. Both components are currently provided to our customers under contract.

Archiving is performance work too #

Tables that keep growing degrade performance step by step once the indexes can no longer be held in memory. Time gained through query optimisation is lost again as the table volume rises.

Removing data that is no longer actively used from the production system gives a more lasting gain than query optimisation alone in most scenarios. GoArchive, which we built for relational data archiving , moves and deletes records with verification.

Related services #

The performance effect of audit logging and the processor cost of encryption are assessed together: Database Security and KVKK Technical Measures

The effect of the backup window on the production load is within the scope of performance work too: Database Backup and Recovery

Replication lag and load distribution between nodes are assessed together with the cluster design: High Availability and Cluster Solutions

The GoFast tool family

We build the tools we use in performance analysis ourselves.