Expose how far each instance, including the cross-Region replicas of an
Aurora global database, lags behind the writer, from
aurora_global_db_instance_status(), converted to seconds. The writer
reports NULL lag, so no lag is exported for it. aws_region is a label
because the same server_id can appear in several Regions.
The collector is disabled by default and returns no data on non-Aurora
servers.
Aurora collector names get their own block in config.go, so gofmt
does not realign the main list when longer names are added.
Signed-off-by: Pavel K <megativ3@gmail.com>
Expose where each database's block reads come from (Aurora storage,
the Optimized Reads cache or local storage) and the time spent on each,
from aurora_stat_database(), with times converted to seconds.
aurora_stat_database() also returns every pg_stat_database column; only
Aurora's own columns are exported, the rest is already covered by the
stat_database collector.
The function only exists in Aurora PostgreSQL 14.9+ and 15.4+, so the
collector checks for it first and returns no data on older versions.
The collector is disabled by default and returns no data on non-Aurora
servers.
Signed-off-by: Pavel K <megativ3@gmail.com>
* stat_activity_autovacuum: add datname and schemaname labels
relname is only the part of the autovacuum query after the first dot, so
workers on same-named tables in different schemas or databases emitted
identical series and failed the scrape as duplicates.
Fixes#953
Signed-off-by: spicysharkco <321699413+spicysharkco@users.noreply.github.com>
* stat_activity_autovacuum: keep spaces in schema names
The schemaname label was the last space-separated word before the first
dot, so schemas such as "A schema" and "B schema" both became "schema",
and two workers on same-named tables in them gave duplicate series again.
Strip the command prefix that autovac_report_activity and
autovac_report_workitem write (VACUUM, VACUUM ANALYZE, ANALYZE or BRIN
summarize) with an anchored REGEXP_REPLACE and take everything before
the first dot. relname is unchanged and the query stays valid on
PostgreSQL 13 and later. A dot inside a schema name still ends the
schema early.
The integration test gets two workers in schemas "A schema" and
"B schema".
Signed-off-by: spicysharkco <321699413+spicysharkco@users.noreply.github.com>
* stat_activity_autovacuum: test the ANALYZE and BRIN summarize prefixes
The integration test only had VACUUM and VACUUM ANALYZE rows, so it
still passed with ANALYZE or BRIN summarize dropped from the prefix
pattern. Add one row for each.
Signed-off-by: spicysharkco <321699413+spicysharkco@users.noreply.github.com>
---------
Signed-off-by: spicysharkco <321699413+spicysharkco@users.noreply.github.com>
Aurora PostgreSQL rejects pg_last_xact_replay_timestamp() and, on
readers, pg_last_wal_receive_lsn() as not supported (0A000), so the
replication collector failed on every scrape there, on the writer and
on readers alike.
On Aurora the collector now queries only pg_is_in_recovery() and emits
pg_replication_is_replica. The lag metrics aren't emitted there: the
replica lag on Aurora comes from the aurora_replica_status collector.
Errors of that query are returned as usual.
Aurora is checked up front rather than after the error: these functions
aren't supported on Aurora, so a fallback would only add a failed
query, and an error in the Aurora log, to every scrape.
Based on the contribution by @dannotripp in #1274.
Signed-off-by: Pavel K <megativ3@gmail.com>
* Add aurora_stat_logical_wal_cache collector for Amazon Aurora PostgreSQL
Expose WAL cache hits, misses and read requests of each logical
replication slot from aurora_stat_logical_wal_cache().
The function only exists in Aurora PostgreSQL 11.17+, 12.12+, 13.8+,
14.7+ and 15.2+. Checking for Aurora alone would make the collector fail
on every scrape on older versions, so it first checks that the function
exists with to_regproc, the same way Aurora itself is detected, and
returns no data when it doesn't. The check is a small helper that other
version-dependent Aurora collectors can reuse.
The collector is disabled by default and returns no data on non-Aurora
servers.
Aurora collector names get their own block in config.go, so gofmt
does not realign the main list when longer names are added.
Signed-off-by: Pavel K <megativ3@gmail.com>
* collector: rename auroraFunctionExists to functionExists
The helper isn't Aurora specific, so it could be used by collectors
for other types of instances as well.
Signed-off-by: Pavel K <megativ3@gmail.com>
---------
Signed-off-by: Pavel K <megativ3@gmail.com>
Closes#1372. Part of #1371.
Adds CLI flags for the parts of the Postgres connection config that
aren't secrets: --datasource.uri (replaces DATA_SOURCE_URI), plus
--datasource.uri-file, --datasource.user-file and --datasource.pass-file
(replacing the *_FILE env vars) for values sourced from mounted files.
The user and password values themselves stay env-var only
(DATA_SOURCE_USER / DATA_SOURCE_PASS), since CLI flags are readable via
/proc and the Kubernetes API and would leak the secret itself. The
*-file flags carry only a path, the same pattern Prometheus uses for
password_file, so no secret ever reaches the command line.
Flag and env var reading now lives in cmd/postgres_exporter rather than
the exporter package, which we're trying to shrink. exporter.GetDataSources
stays for embedders, marked deprecated with a migration note, and will be
removed in v0.22.0.
The four env vars that gained a flag are deprecated: the exporter warns
at startup when one is set, the CHANGELOG maps each to its replacement,
and the docs now configure the exporter by flag.
Each flag falls back to its env var when unset, so behavior is unchanged
when only env vars are set. Precedence is unchanged: DATA_SOURCE_NAME
wins over everything, *_FILE wins over the plain value.
Signed-off-by: Nicolas Takashi <nicolas.takashi@dash0.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Expose the storage (durability) lag and RPO lag of each secondary
cluster in an Aurora global database from aurora_global_db_status(),
converted to seconds. The primary cluster reports -1, so no lag is
exported for it.
The collector is disabled by default and returns no data on non-Aurora
servers.
Aurora collector names get their own block in config.go, so gofmt
does not realign the main list when longer names are added.
Signed-off-by: Pavel K <megativ3@gmail.com>
* wal: skip the WAL metrics where Aurora doesn't support pg_ls_waldir()
Some Amazon Aurora PostgreSQL versions reject pg_ls_waldir() with
feature_not_supported (SQLSTATE 0A000), so the wal collector, which is
enabled by default, failed on every scrape there. On Aurora, detected
with instance.isAurora from #1311, treat that error as no WAL metrics;
Aurora versions where pg_ls_waldir() works keep reporting them, and any
other error is still returned.
Fixes#1309.
Signed-off-by: Pavel K <megativ3@gmail.com>
* wal: return ErrNoData where Aurora doesn't support pg_ls_waldir()
Other collectors return ErrNoData when there is no data for a known
reason, e.g. Aurora collectors on instances that aren't Aurora, so the
collector is reported as not successful with a debug log instead of an
error log.
Signed-off-by: Pavel K <megativ3@gmail.com>
---------
Signed-off-by: Pavel K <megativ3@gmail.com>
Expose replica lag, replay latency and pending read IOs from Aurora's
aurora_replica_status() function. Metrics are converted to base units
(seconds).
Aurora is detected in instance.setup() alongside current_database(),
using to_regproc('aurora_version') so that regular PostgreSQL servers
don't log an error on every scrape. The collector is disabled by
default and returns no data on non-Aurora servers.
Signed-off-by: Pavel K <megativ3@gmail.com>
Report settings that have been changed but only take effect after a
server restart as pg_settings_pending_restart{name} from the existing
settings collector. The flag is selected in the settings query itself
and carried in the pgSetting struct, so it is emitted from the same
loop that reports the value metrics, without an extra query.
The vartype filter moves from the SQL WHERE clause into
pgSetting.metric(): rows of every vartype are scanned, so pending
restarts are reported for settings beyond the bool/integer/real ones
exported as value metrics (e.g. wal_level is an enum); rows without a
numeric vartype are skipped silently, and the existing warning is kept
for broken values.
Signed-off-by: MarcWort <113890636+MarcWort@users.noreply.github.com>
* Create user facing documentation
Create a docs/ directory with user facing documentation. This covers getting started, running with docker, specifying secrets, configuration, and database permissions. This is all focused on the perspective of a user or someone deploying the exporter.
Signed-off-by: Joe Adams <github@joeadams.io>
* Update getting started to show multiple options
Clean up duplication on docker commands. Make the docs README more useful as a standalone page.
Signed-off-by: Joe Adams <github@joeadams.io>
* Update configuration docs
Add more details on each flag and collector to help the user understand the purpose.
Signed-off-by: Joe Adams <github@joeadams.io>
* Add frontmatter
Signed-off-by: Joe Adams <github@joeadams.io>
* Review feedback
Remove deprecated option doc and /-/reload endpoint (unimplemented).
Signed-off-by: Joe Adams <github@joeadams.io>
* Add 2 new flags in user docs configuration
Signed-off-by: Joe Adams <github@joeadams.io>
* Fix docs link
Signed-off-by: Joe Adams <github@joeadams.io>
* Fix correctness in docs
Signed-off-by: Joe Adams <github@joeadams.io>
---------
Signed-off-by: Joe Adams <github@joeadams.io>
