1
0
mirror of https://github.com/coturn/coturn.git synced 2026-10-24 00:08:57 +03:00
Files
coturn/CLAUDE.md
Pavel Punsky ac601f832d key OAuth MESSAGE-INTEGRITY with the whole mac_key (#2096)
RFC 7635 Section 6.2 keys HMAC-SHA1 MESSAGE-INTEGRITY with the token's
full 160-bit mac_key. The OAuth path went through the
long-term-credential helpers, which hash only get_hmackey_size(SHA1) =
16 bytes, so requests from RFC 7635 compliant clients failed with 401
and coturn's responses could not be verified by them. turnutils_uclient
shared the truncation, so the bundled client never noticed.

Add integrity helpers that take an explicit key length and use the whole
mac_key for OAuth sessions, both when checking requests (userdb and
check_stun_auth) and when signing responses. --oauth-legacy-mac-key on
turnserver (and on turnutils_uclient) restores the 16-byte truncation
for deployments that depend on it; the server logs a warning when it is
on. A mac_key whose last 4 bytes are zero hashes the same in both modes.

Fixes #2091 and #644

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:59:49 -07:00

27 KiB
Raw Permalink Blame History

AGENT.md — Coturn

Coturn is a TURN/STUN server written in C11, implementing RFC 5766, RFC 5389, and related NAT traversal protocols. It supports multiple database backends (SQLite, PostgreSQL, MySQL, Redis, MongoDB), multiple auth mechanisms, and a fuzzing harness for OSS-Fuzz.

Build

# Standard build
mkdir build && cd build
cmake ..
make -j$(nproc)

# Debug build
cmake -DCMAKE_BUILD_TYPE=Debug ..
make -j$(nproc)

# Fuzzer build (requires clang or AppleClang)
CC=clang CXX=clang++ cmake -S . -B build -DFUZZER=ON
cmake --build build -j$(nproc)

Key CMake options:

  • -DFUZZER=ON — build OSS-Fuzz targets (requires Clang or AppleClang)
  • -DCMAKE_BUILD_TYPE=Debug|Release — defaults to Release (-O3 -DNDEBUG) when unset, so benchmark and load-test builds are optimized without opting in. -DFUZZER=ON builds are exempt: they take their -O/-fsanitize flags from the OSS-Fuzz CFLAGS environment.
  • -DWITH_MYSQL=ON/OFF, -DWITH_PGSQL=ON/OFF, -DWITH_MONGO=ON/OFF, -DWITH_REDIS=ON/OFF

Required validation

Every code change must be validated before it is considered complete. Run the unit tests, the local/system tests, and the fuzzing smoke tests. When working from macOS, also validate the Linux build and Docker image tests inside Docker containers.

Clean stale build output first. Before running the system tests, copying the checkout into a container, or building Docker images for testing, remove leftover build output from earlier runs — both the legacy autotools build and CMake:

  • The legacy ./configure && make build writes Makefile, bin/, lib/, include/, and sqlite/ at the repo root (all gitignored, so git status looks clean).
  • examples/run_tests*.sh resolve ../bin/turnserver before the CMake build/bin/, so a stale root bin/ silently shadows the freshly built server. On the host you end up testing old binaries; inside a Linux container a copied macOS bin/ fails with cannot execute binary file: Exec format error.
  • cp -a / Docker build contexts copy gitignored dirs too, so stale CMake trees (build/, build-win/) and legacy output ride along into containers and image builds, causing wrong-arch errors and longer build/test cycles.
# Legacy-build output at the repo root (all gitignored):
rm -rf Makefile bin lib include sqlite
# CMake trees from other platforms/configs (keep the one you are using):
rm -rf build-win
# Local build + unit tests
cmake -S . -B build -DBUILD_TESTING=ON
cmake --build build --parallel $(nproc)
ctest --test-dir build --output-on-failure

# Local/system tests
cd examples
./run_tests.sh                  # default protocols, both legacy and
                                # threaded uclient (--listener-threads /
                                # --sender-threads). On Linux it also
                                # enables --udp-recvmmsg on the server
                                # and runs a -Y packet load-gen smoke
                                # that checks non-zero send_pps +
                                # recv_pps. GSO/multiplex live in
                                # run_tests_multiplex_peer.sh.
./run_tests_conf.sh             # same protocols, conf-file driven;
                                # mirrors run_tests.sh via config keys.
./run_tests_multiplex_peer.sh   # exercises --multiplex-peer with shared
                                # per-thread relay sockets (UDP/TCP/TLS/
                                # DTLS) on the small port range opened by
                                # --multiplex-peer-port. Auto-enables
                                # --udp-recvmmsg on Linux.
./run_tests_mobile.sh           # starts the server with --mobility (MICE)
                                # and drives -M clients so the
                                # MOBILITY-TICKET session-resume branch in
                                # handle_turn_refresh gets automated
                                # coverage. UDP/TCP only (legacy + threaded);
                                # -M over TLS/DTLS re-handshakes mid-run and
                                # is non-deterministic on loopback.
./run_tests_rfc5780.sh          # starts the server with --rfc5780 (NAT
                                # behavior discovery) on two loopback IPs and
                                # drives turnutils_stunclient -f, asserting the
                                # OTHER-ADDRESS / CHANGE-REQUEST / RESPONSE-PORT
                                # path. Needs a second bindable loopback IP
                                # (native on Linux; SKIPs on macOS without a
                                # 127.0.0.2 alias).
./run_tests_stateless_nonce.sh  # starts the server with --stateless-nonce
                                # (derived challenge nonces, issue #1999) and
                                # runs the standard relay workload over
                                # UDP/TCP (+TLS/DTLS on Linux), asserting
                                # wire-transparency plus the listener
                                # fast-path marker in the server log.
./run_tests_ipv6_relay.sh       # pins that an IPv6 allocation is advertised
                                # under its real IPv6 relayed transport address
                                # (not an IPv4 --external-ip) and that uclient
                                # accepts an IPv6 relay it did not explicitly
                                # request (-A keep). SKIPs without a usable
                                # IPv6 loopback.
./run_tests_dtls_default.sh     # pins that the DTLS listeners stay down
                                # unless --dtls is given, that --dtls brings
                                # them up
./run_tests_oauth.sh            # starts an --oauth server on a SQLite DB of
                                # the turndb test keys and drives uclient -J,
                                # pinning that OAuth MESSAGE-INTEGRITY uses the
                                # whole 20-byte mac_key and that
                                # --oauth-legacy-mac-key (server + client)
                                # restores the 16-byte truncation. Needs sqlite3
                                # or python3; SKIPs otherwise.
./run_tests_prom.sh             # only when Prometheus support is built
cd ..

# Fuzzing smoke tests (increase runs/time for fuzzing-related changes)
fuzzing/run-local.sh ASan 0 -runs=1
fuzzing/run-local.sh ASan 1 -runs=1

Treat any FAIL line from the example scripts as a test failure even if the script exits with status 0.

On macOS, run the Linux validation in Docker as well. The fuzzing smoke tests build the reusable coturn-fuzz-local image; then run a clean Linux build and system-test pass against a copied checkout so no root-owned build artifacts are left in the working tree:

docker run --rm \
  -v "$PWD:/src:ro" \
  --entrypoint bash \
  coturn-fuzz-local \
  -lc 'apt-get update && apt-get install -y --no-install-recommends git libmicrohttpd-dev python3 && \
       cp -a /src /tmp/coturn && \
       git config --global --add safe.directory /tmp/coturn && \
       cd /tmp/coturn && \
       rm -rf Makefile bin lib include sqlite build build-win && \
       cmake -S . -B build-linux -DBUILD_TESTING=ON && \
       cmake --build build-linux --parallel $(nproc) && \
       ctest --test-dir build-linux --output-on-failure && \
       rm -rf build && ln -s build-linux build && \
       cd examples && ./run_tests.sh && ./run_tests_conf.sh && \
       ./run_tests_mobile.sh && ./run_tests_multiplex_peer.sh && \
       ./run_tests_rfc5780.sh && ./run_tests_stateless_nonce.sh && \
       ./run_tests_ipv6_relay.sh && ./run_tests_dtls_default.sh && \
       ./run_tests_oauth.sh'

Also validate the packaged Docker image. Run the same stale-output cleanup at the repo root first (see above) so the image build context does not pick up prebuilt host binaries:

cd docker/coturn
make docker.image dockerfile=debian tag=codex-local platform=linux/arm64
make test.docker tag=codex-local platform=linux/arm64/v8
cd ../..

Use platform=linux/amd64 on x86_64 hosts. On Apple Silicon, build with platform=linux/arm64 and run the Bats image tests with platform=linux/arm64/v8, which is the spelling expected by docker/coturn/tests/main.bats.

Code style

All C source — including src/, fuzzing/, and tests/ — must be formatted with clang-format-15 using the project's .clang-format. The CI job .github/workflows/clang.yml runs make lint and fails the build on any formatting drift, so any commit containing C/H files must be formatted before it is created.

# Format the entire repo (uses the Makefile target — equivalent to
# `find . \( -iname "*.c" -o -iname "*.h" \) -not -name "ns_turn_khash.h" \
#    | xargs clang-format -i`. ns_turn_khash.h is a vendored 3rd-party
# header and is excluded from both format and lint):
make format

# Verify formatting matches CI (zero output = clean):
make lint

Mandatory pre-commit step for any session that edits C/H files:

find . \( -iname "*.c" -o -iname "*.h" \) -not -name "ns_turn_khash.h" | xargs clang-format -i

Run this before git commit whenever the staged diff touches *.c or *.h, even when only one file was edited. The find form above does not require ./configure to have been run, so it works in worktrees and fresh clones.

Key style rules (LLVM-based):

  • Indent: 2 spaces, no tabs
  • Column limit: 120
  • Pointer alignment: right (int *p)
  • Brace style: attach (K&R)
  • Zero-initialize stack buffers at declaration: uint8_t buf[N] = {0} or SomeStruct s = {0}
  • Prefix new identifiers (functions, types, macros) with turn_, not ns_ — the ns_ names are legacy; do not add new ones

Comments

Keep comments short — one line, two at most. A comment explains why the code is there, not what it does; if the code already reads clearly, omit it. Cite the normative source when that is the reason for the code:

/* RFC 8656 Section 12: channel numbers are restricted to 0x4000-0x4FFF. */

A comment must stand on its own, read cold, with no knowledge of the change that introduced it. Do not write: that a change is part of a larger or multi-part effort, references to a prior discussion or review, what the code used to do, an issue/PR number as the only justification, or attribution.

Memory allocation

MANDATORY. All heap allocation in coturn-owned C code MUST go through the fail-fast wrappers declared in ns_turn_utils.h:

  • turn_malloc(size)
  • turn_calloc(number, size)
  • turn_realloc(ptr, size)
  • turn_strdup(str)

Raw malloc / calloc / realloc / strdup are not allowed — do not add them, and do not review/merge code that uses them. Any file that allocates must include ns_turn_utils.h. Release memory with plain free() (there is no turn_free; free(NULL) is a no-op).

The wrappers log the failing call site and abort() on allocation failure; they never return NULL for a real failure. Therefore callers MUST NOT add a NULL/failure check on the result, and "fixing" an allocation by adding a NULL check is the wrong fix — route it through a wrapper instead. (turn_malloc, turn_calloc, turn_realloc return NULL only for a zero-size request; turn_strdup returns NULL only for a NULL argument.)

Rationale: an allocation that fails mid-request cannot be recovered correctly in a long-running TURN server, so a controlled, logged abort is safer than a NULL dereference or a silently degraded process. The logger writes into a stack buffer (no heap allocation), so it stays usable when the heap is exhausted.

The only permitted exceptions — do not convert these, and exclude them from any bare-allocator check:

  • src/apps/relay/libtelnet.c — vendored, public-domain third-party code.
  • src/prometheus/prom.c — self-contained module with its own NULL-return error contract; does not link turncommon.
  • ns_turn_utils.c — the wrapper implementation itself, plus its turn_mutex_* helpers, which deliberately degrade (return an error) rather than abort.
  • fuzzing/, tests/, and Windows-only shims.

Tests

# Protocol conformance (RFC 5769 test vectors)
cd examples && ./scripts/rfc5769.sh

# Basic TURN relay test (run server first, then client)
cd examples && ./scripts/basic/relay.sh
cd examples && ./scripts/basic/udp_c2c_client.sh

# Full test suite
cd examples && ./run_tests.sh

Load Test on DigitalOcean

Topology — 3 same-region droplets

Run uclient, turnserver, and peer on separate droplets so the loadgen and reflector never compete with the server under test. Last known setup was three Ubuntu 24.04 c-4 (4 vCPU / 8 GB) droplets in nyc1, all in the default-nyc1 VPC (10.116.0.0/24):

Role Droplet Private IP
turnserver coturn-server-perf (id 570437076) 10.116.0.2
uclient (loadgen) coturn-loadgen-perf (id 570437077) 10.116.0.3
peer (reflector) coturn-peer-perf (id 571385939) 10.116.0.4

The 2-droplet shape (peer co-located on the loadgen) is fine for smoke runs but burns ~20 % of loadgen CPU on the peer reflector and skews send_pps comparisons across configs.

Never paste DigitalOcean tokens into logs or files. Use a local environment variable such as DIGITALOCEAN_TOKEN.

Flag inventory (relay-side)

  • --udp-recvmmsg — Linux recvmmsg() batched receive on the shared fan-in sockets only: the client listener and, under --multiplex-peer, the per-thread shared relay socket (udp_recvmmsg_eligible). Per-session relay sockets stay on the single-recv path. On by default on Linux; disable with --udp-recvmmsg=false (or =0). DTLS session sockets still go through the OpenSSL read path.
  • --udp-gso — Linux UDP-GSO (UDP_SEGMENT cmsg) when a sendmmsg batch shares destination and segment size. Requires --multiplex-peer — that mode is what enables the sendmmsg batching GSO piggybacks on; passing --udp-gso alone is a silent no-op.
  • --multiplex-peer — replace the per-allocation relay-port bind with a per-thread shared IPv4+IPv6 relay socket pair. Implies sendmmsg batching on Linux; --udp-recvmmsg (on by default) supplies the recvmmsg window the batching piggybacks on. Port layout: thread i binds --multiplex-peer-port + 2*i (IPv4) and +1 (IPv6); a 4-thread server with default base 3480 uses 3480–3487. See docs/multiplex-peer.md for the design.
  • --udp-recvmmsg-log — emit udp-recvmmsg stats every 10 s; useful to confirm per-thread batch occupancy.

Flag inventory (uclient)

  • -Y packet|invalid|alloc — load-generator mode.
  • -c — no rtcp; in clnet_allocate the EVEN-PORT slot is then chosen randomly between 0 and -1 (startuclient.c:440).
  • --no-even-port — force ep = -1 unconditionally. Required for alloc-flood runs against --multiplex-peer, which strictly rejects EVEN-PORT with error 508 (ns_ioalib_engine_impl.c:1603).
  • -K N / --listener-threads N, --sender-threads N — loadgen-side receive/send pools. Auto: 0 for -m < 4, bumped to 1 listener / 2 sender for -m >= 4. Max 4 each. Use --sender-threads 4 to push the loadgen harder when you're trying to saturate the server.

Source upload + build

# locally
git archive --format=tar HEAD -o /tmp/coturn.tar
for ip in TURN_PUBLIC_IP UCLIENT_PUBLIC_IP PEER_PUBLIC_IP; do
  scp /tmp/coturn.tar root@$ip:/root/coturn.tar &
done; wait

# on each droplet (apt step needed once per droplet; peer needs at least
# turnutils_peer; uclient needs turnutils_uclient; server needs turnserver)
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y --no-install-recommends build-essential cmake pkg-config \
  libssl-dev libevent-dev libsqlite3-dev libhiredis-dev git iproute2 sysstat

rm -rf /root/coturn && mkdir /root/coturn
tar -xf /root/coturn.tar -C /root/coturn
cmake -S /root/coturn -B /root/coturn/build -DCMAKE_BUILD_TYPE=Release
cmake --build /root/coturn/build \
  --target turnserver turnutils_uclient turnutils_peer -j$(nproc)

Sysctl + ulimit on every droplet

sysctl -w net.core.rmem_max=134217728 net.core.wmem_max=134217728 \
  net.core.netdev_max_backlog=250000
ulimit -n 1048576

Peer fleet (peer droplet, 10.116.0.4)

Bring up 8 turnutils_peer processes on ports 3480..3487 so the loadgen can target multiple distinct peer ports. The Linux peer transparently batches inbound datagrams via recvmmsg() and echoes with GSO-segmented sendmsg() when the batch shares source IP and segment size (see src/apps/peer/udpserver.c), so 8 processes on a 4-vCPU peer droplet stay well below saturation:

pkill -x turnutils_peer 2>/dev/null || true
for p in 3480 3481 3482 3483 3484 3485 3486 3487; do
  nohup /root/coturn/build/bin/turnutils_peer -L 10.116.0.4 -p $p \
    > /root/peer_${p}.log 2>&1 &
done
ss -ulnp | grep -E ':(348[0-7])\b' | wc -l  # expect >= 8

turnserver (server droplet, 10.116.0.2)

Baseline (no fast paths). Add flags from the table above to compare:

pkill -x turnserver 2>/dev/null || true
mkdir -p /root/runs
LABEL=baseline   # or recvmmsg / mpx_peer / mpx_gso
EXTRA=""         # "--udp-recvmmsg" |
                 # "--multiplex-peer --multiplex-peer-port=3480" |
                 # "--multiplex-peer --multiplex-peer-port=3480 --udp-gso"

nohup /root/coturn/build/bin/turnserver \
  --use-auth-secret --static-auth-secret=secret --realm=north.gov \
  --allow-loopback-peers \
  --listening-ip=10.116.0.2 --relay-ip=10.116.0.2 \
  --min-port=49152 --max-port=65535 \
  --no-tls \
  --log-file=stdout \
  $EXTRA \
  > /root/runs/${LABEL}.turnserver.log 2>&1 &
echo $! > /root/runs/${LABEL}.pid

# Wait for readiness before driving load.
until grep -q "Total auth threads:" /root/runs/${LABEL}.turnserver.log; do
  sleep 0.5
done

Packet-flood matrix workload (uclient droplet, 10.116.0.3)

8 parallel uclient processes, each -m 1 against a distinct peer port (3480..3487). The 1-session-per-(thread, peer-port) shape is what keeps --multiplex-peer's per-thread mp_table collision-free regardless of which relay thread the server's round-robin assigns each session to.

mkdir -p /root/runs
LABEL=baseline
PIDS=()
for pp in 3480 3481 3482 3483 3484 3485 3486 3487; do
  LOG=/root/runs/${LABEL}.par_p${pp}.log
  (timeout -s INT 45s /root/coturn/build/bin/turnutils_uclient \
     -Y packet -m 1 -n 50000000 -l 120 -c --no-even-port \
     --listener-threads 1 --sender-threads 4 \
     -e 10.116.0.4 -r "$pp" -X -g \
     -u user -W secret \
     10.116.0.2 > "$LOG" 2>&1) &
  PIDS+=("$!")
done
for p in "${PIDS[@]}"; do wait "$p"; done

turnutils_uclient logs send_pps=… , recv_pps=… , total_sent=… , total_recv=… periodically (uclient.c:945). Always take the median of in-flight progress lines (drop the first 2-3 warm-up samples and any trailing zero-pps lines), then sum across the 8 streams. Two derived values that matter:

  • recv_pps = end-to-end relayed throughput. The canonical comparison metric across configs.
  • send_pps − recv_pps = packets dropped somewhere on the round trip. At parity throughput this should match across configs.

3-host CPU instrumentation

Run mpstat on all three droplets (host-wide CPU breakdown) and pidstat on the server droplet (turnserver process slice), pinned to the duration of the loadgen run:

DUR=45

# on turnserver
nohup mpstat 1 $DUR > /root/runs/${LABEL}.server_mpstat.txt 2>&1 & disown
nohup pidstat -h -u -r -p $(cat /root/runs/${LABEL}.pid) 1 $DUR \
  > /root/runs/${LABEL}.server_pidstat.txt 2>&1 & disown

# on uclient
nohup mpstat 1 $DUR > /root/runs/${LABEL}.uclient_mpstat.txt 2>&1 & disown

# on peer
nohup mpstat 1 $DUR > /root/runs/${LABEL}.peer_mpstat.txt 2>&1 & disown

Parse the Average: line — note field offsets differ from the per-sample lines because there's no time column:

awk '$1=="Average:" && $2=="all" {
  printf "user=%5.1f sys=%5.1f softirq=%5.1f idle=%5.1f\n", $3, $5, $8, $NF
}' /root/runs/${LABEL}.server_mpstat.txt

# turnserver process CPU only:
tail -1 /root/runs/${LABEL}.server_pidstat.txt | awk '{
  print "usr="$4" sys="$5" CPU="$8"%"
}'

Reading the matrix (what "good" looks like)

A last clean run (2026-05-16, 3-droplet, 8 streams × 45 s, c-4):

Config recv pps (8-stream sum) turnserver CPU server host idle server host softirq
baseline 91 k 367 % 24 % 14.1 %
--udp-recvmmsg 99 k 229 % (−38 %) 48 % 2.6 %
--multiplex-peer 97 k 293 % (−20 %) 42 % 13.3 %
--multiplex-peer --udp-gso 96 k 134 % (−63 %) 65 % 0.8 %

The recv-pps ceiling (~95-100 k) is the loadgen-side cap on a c-4 uclient — %sys + %softirq on the uclient host runs at ~70 % per core across all configs, indicating the kernel network stack is saturated. The win shows up as server-side CPU savings, not as more relayed pps. To push the server toward saturation, resize the uclient to c-8 (or larger) first.

Bottleneck identification

For any run, look at the three mpstat Average: lines:

Host idle % Reading
< 20 % on uclient uclient is the cap; you're measuring loadgen, not server. Bump uclient size or thread pools.
< 20 % on peer peer reflector saturated; add more peer processes or move peer to a bigger droplet.
> 50 % on server server has headroom; you can push it harder if uclient/peer allow.
< 10 % on server server is the bottleneck — this is what you want when measuring server-side perf changes.

Allocation throughput

# on uclient — alloc-flood; --no-even-port keeps multiplex-peer from
# rejecting every other request with 508.
timeout -s INT 60s /root/coturn/build/bin/turnutils_uclient \
  -Y alloc -m 200 -n 1000 -c --no-even-port \
  -L 10.116.0.3 \
  -u user -W secret \
  10.116.0.2 > /root/runs/${LABEL}.alloc.log 2>&1

# rate:
grep -oE "total_allocations=[0-9]+" /root/runs/${LABEL}.alloc.log \
  | awk -F= '{print $2}' | sort -n | tail -1

Sessions are torn down between iterations, so this measures alloc/s throughput, not concurrent-session cap. Baseline and --multiplex-peer deliver roughly the same rate (~700 alloc/s on c-4); allocation is control-plane and isn't where the fast paths help. The cap-removal property of --multiplex-peer is structural (2 ports per relay thread, total) — cite docs/multiplex-peer.md rather than trying to demo it via a load test.

Invalid-packet flood

# on uclient
timeout -s INT 12s /root/coturn/build/bin/turnutils_uclient \
  -Y invalid -m 1 -l 16 \
  10.116.0.2 > /root/runs/invalid_m1.log 2>&1 || true

-Y invalid bypasses the TURN allocate handshake and sprays malformed datagrams at the listener; useful for the parse/reject hot path. The uclient pacer drops its interval to 100 µs and lifts the per-burst send cap to 4096 in flood modes (uclient.c:899), so even -m 1 produces real load. Restart turnserver after an invalid flood before any allocation test — past runs saw rapid RSS growth.

CPU profile (perf, server droplet)

Build with -DCMAKE_BUILD_TYPE=RelWithDebInfo for readable symbols. Capture once per config you're comparing (fastpath_off, fastpath_on, optionally one-flag-at-a-time):

sysctl -w kernel.perf_event_paranoid=-1 kernel.kptr_restrict=0
pid=$(cat /root/runs/${LABEL}.pid)
perf record -F 99 -g -p "$pid" -o /root/runs/${LABEL}.perf.data -- sleep 30
perf report --stdio -i /root/runs/${LABEL}.perf.data --no-children \
  --sort comm,dso,symbol > /root/runs/${LABEL}_perf.report

Useful summaries

# per-stream median send/recv pps across all 8 streams of a run:
for f in /root/runs/${LABEL}.par_p*.log; do
  grep "send_pps=" "$f" | awk -F'[=, ]' '
    {for (i=1;i<=NF;i++) {if ($i=="send_pps") s=$(i+1)+0;
                          if ($i=="recv_pps") r=$(i+1)+0}
     print s" "r}' | sort -n | awk -v f="$f" '
    {a[NR]=$0} END {print f": median="a[int(NR/2)]}'
done

# turnserver process RSS over time:
ps -o pid,rss,vsz,pcpu,pmem,comm -p $(cat /root/runs/${LABEL}.pid)

# server NIC counters (drops show up as nonzero rx_dropped on eth1):
ssh root@TURN_PUBLIC_IP 'cat /proc/net/dev | column -t | head -5'

Unit tests (Unity, opt-in via BUILD_TESTING=ON)

Unity is fetched on demand via CMake FetchContent; nothing is vendored. Tests live under tests/ and link against the existing turnclient static library.

# CMake direct
cmake -S . -B build -DBUILD_TESTING=ON
cmake --build build -j --target check     # builds tests, runs ctest
cmake --build build -j --target test_ioaddr  # build a single binary
ctest --test-dir build --output-on-failure   # run already-built tests

# Legacy Makefile bridge (after ./configure; requires cmake on PATH)
make unit-tests   # bootstraps build/unit-tests/, builds + runs Unity tests

Adding a new test: drop tests/test_<name>.c and append coturn_add_test(test_<name>) in tests/CMakeLists.txt. The check target picks it up automatically.

See docs/Testing.md for database setup and extended test scenarios.

Source layout

src/
  client/          # TURN client library (C)
  client++/        # TURN client library (C++)
  server/          # Core TURN/STUN server logic
  apps/
    relay/         # turnserver main process, listeners, netengine
    uclient/       # CLI test client
include/turn/      # Public headers
fuzzing/           # OSS-Fuzz targets and seed corpora
examples/          # Test scripts and sample configs
turndb/            # Database schema and setup scripts
docs/              # Protocol notes and configuration docs

Common patterns

  • Port types: use uint16_t for port fields and parameters (not int); port 0 means OS-assigned ephemeral
  • Buffer initialization: zero-initialize stack buffers at declaration (= {0}), not just before first use
  • HMAC output buffers: declare as uint8_t buf[MAXSHASIZE] = {0} — the buffer is written into the message before HMAC runs, so uninitialized bytes would be briefly present in the packet
  • Uninitialized structs: use = {0} for stack-allocated address structs (e.g., ioa_addr)
  • Counter overflow in turn_ports.c: _turnports uses uint32_t low/high counters; comparisons must be overflow-safe (use subtraction, not >=)
  • Port bounds checks: use <= USHRT_MAX (not < USHRT_MAX) when validating that an int holds a valid port — port 65535 is valid
  • Wire-format field access: read/write multi-byte STUN/TURN wire fields with the alignment-safe helpers in src/ns_turn_defs.h — turn_read_u16/u32/u64(field) and turn_write_u16/u32/u64(field, value) (memcpy + network-byte-order swap). Never cast a wire byte pointer to uint16_t */uint32_t */uint64_t * and dereference it: attributes sit at arbitrary offsets, so the cast is undefined behavior and a SIGBUS on strict-alignment targets. For fields that are not byte-swapped (CONNECTION-ID's as-is encoding, values already in network order), use a plain memcpy, not the swapping helpers
  • Error handling: check return values of all OpenSSL/libevent calls; use ERR_clear_error() before HMAC operations
  • Logging: use TURN_LOG_FUNC macros, not fprintf/perror