From a8e7b27e32ded8fe7f8717647131568896bc7533 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 6 Oct 2026 14:32:42 -0500 Subject: [PATCH] Use a non-locking initial test in TAS_SPIN on AArch64. Our testing showed that this is helpful at sufficiently high contention levels and doesn't hurt performance on smaller machines. The new TAS_SPIN macro for AArch64 is identical to the ones added for PPC and x86_64 (see commits bc2a050d40 and b03d196be0). Back-patch of commit 3d0b4b1068, which first shipped in v18. Salvatore Dipietro reports that this change is beneficial on newer Linux kernels, and the original commit has aged long enough to provide some confidence that it won't cause problems. v14 is left alone since it lacks commit a82a5eee31 and will soon be out of support. Reported-by: Salvatore Dipietro Reviewed-by: Jingtang Zhang Reviewed-by: Andres Freund Tested-by: Tom Lane Discussion: https://postgr.es/m/ZxgDEb_VpWyNZKB_%40nathan Discussion: https://postgr.es/m/20260915114623.431827-1-dipiets%40amazon.it Backpatch-through: 15 --- src/include/storage/s_lock.h | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 4d3ffc767f4..19bdce51233 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -337,18 +337,24 @@ tas(volatile slock_t *lock) #define S_UNLOCK(lock) __sync_lock_release(lock) -/* - * Using an ISB instruction to delay in spinlock loops appears beneficial on - * high-core-count ARM64 processors. It seems mostly a wash for smaller gear, - * and ISB doesn't exist at all on pre-v7 ARM chips. - */ #if defined(__aarch64__) || defined(__aarch64) +/* + * On ARM64, it's a win to use a non-locking test before the TAS proper. It + * may be a win on 32-bit ARM, too, but nobody's tested it yet. + */ +#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock)) + #define SPIN_DELAY() spin_delay() static __inline__ void spin_delay(void) { + /* + * Using an ISB instruction to delay in spinlock loops appears beneficial + * on high-core-count ARM64 processors. It seems mostly a wash for smaller + * gear, and ISB doesn't exist at all on pre-v7 ARM chips. + */ __asm__ __volatile__( " isb; \n"); }