mirror of
https://github.com/ossrs/srs.git
synced 2026-10-24 09:59:20 +03:00
develop
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Winlin
|
7b737e64df |
Perf: Load and profile tools, and the first SRS and Go proxy fixes. v8.0.63 (#4785)
SRS gets performance tools that load and profile SRS in Docker for every protocol: a lean load tool, an internal skill that runs an improvement round, and a measurement method that holds up to a 10% verdict. The same tools then measure the Go proxy. The first runs found one SRS fix and three Go proxy fixes. The load tool, a standalone Go module in cmd/perf (bin/srs-perf): - Per-core worker processes shard the clients and report counters each second. Lean clients for every protocol: RTMP publish and play, HTTP-FLV, HTTP-TS, HLS, DASH, RTSP, SRT, WHIP and WHEP. No decode; frames and lag come from timestamps. - 17 scenarios: each protocol's fan-out, publishers (rtmp-pub, whip-pub), many-streams, cross-protocol bridges (srt-rtmp, srt-whep, whip-rtmp, whip-flv), and delay probes for the RTMP pause, forward and edge. - run ramps by doubling to a capacity, then holds at 50% and 90% of it. compare checks that the setups match and reports capacity and CPU per client at the 10% line. - make perf-image builds an optimized SRS with symbols and frame pointers, with perf, FlameGraph and FFmpeg. SRS runs on one pinned core and is configured only by env files, one per scenario. The measurement method: - A flat CBR source, checked to ±3% per second, since SRS CPU follows the bitrate. - SRS CPU from its cgroup once a second. Holds have no profiler in the measuring window; each count gets its own separate profile hold. - perf-ab.sh builds two revisions and runs them side by side on separate cpusets. compare gives the 95% CI of the paired per-second difference. - perf-verify.sh checks every run: capacity, healthy steps, profiles and symbols, noise, and what bound the run. The internal-perf-for-srs skill: - SKILL.md holds the rules, the shared tools and the round workflow. A Target Router loads references/cpp-server.md for the C++ media server or references/go-server.md for the Go proxy. - Each reference covers its tools, topology, scenarios, measuring, how to read its profiles (Linux perf, or Go pprof and trace), A/B, the verified scenarios with their capacity and top bottleneck, and how to start a round. - srs-develop routes performance, benchmark, load test and profile tasks to it. perf-skill-test.sh checks the text against the scenario registry. - Reports are git-ignored, since their numbers hold only for the machine that made them. - The skills now name each server by its job: the C++ media server and the Go proxy, with no "generation". C++ media server, one core, Linux arm64 in Docker: - Fix: an empty SrsSharedPtr holds no reference count, so each fan-out copy of a message does one malloc instead of three. Covered by srs_utest_ai05.cpp. - Capacity: rtmp-rtmp, rtmp-flv, rtmp-hls, whip-rtmp and whip-flv 12,800; rtmp-dash and srt-rtmp 6,400; rtmp-ts and rtmp-rtsp 3,200; whip-whep 2,000 (cap); rtmp-whep, srt-whep, srt-srt and rtmp-pub 1,280; whip-pub 1,000 (cap); many-streams 640 streams. - Top bottlenecks found, not fixed: the RTMP fan-out copy, _st_epoll_dispatch with EAGAIN reads, 4 KB reads and writes for HLS and DASH segments, one write per message for HTTP-TS and two per packet for RTSP, the SRTP replay window in libsrtp, and libsrt for SRT. - Delays: the paused RTMP player's 500 ms sleep, and the forward's and edge publish's 500 ms receive timeout, each adding about 220 ms of lag. The Go proxy, cmd/proxy, on one core in front of 1 to 4 SRS origins: - perf-up.sh -target proxy, run -target proxy, which samples the proxy and each origin, Go profiles in each profile hold (CPU, allocations, mutex, block, trace, GC), and proxy A/B. - Fix: HTTP-FLV, HTTP-TS and HLS relay the origin's error status, headers and body, rather than a 500 with the origin's address. - Fix: a WebRTC STUN binding routes by its ufrag, not a cached client address. WHIP and WHEP failed from 160 clients on three origins. - Capacity on three origins: rtmp-flv and rtmp-hls 12,800; rtmp-rtmp and rtmp-ts 6,400; whip-whep 2,000 (cap); srt-srt and rtmp-pub 1,280; whip-pub 1,000 (cap); many-streams 1,280 streams. --------- Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Winlin
|
551ee22d1f |
Integ: Add the srs-integ test suite, and fix 135 bugs. v8.0.61 (#4772)
Adds `srs-integ`, a black-box integration suite for the SRS media server, and fixes the SRS bugs it found: over 120 of them, in every protocol. The suite tests SRS from outside the process, only through what a client, an operator or a callback server can see, so a rewritten server could pass the same cases. The suite now has about 1,450 cases (1,353 main and 99 timer), with no known bugs left: every bug a case found is fixed in this PR. It also removes some unused features, adds env support for a few more directives, and changes a few hook and API fields. ## srs-integ suite `cmd/integ` is a Go module of its own, `srs-integ`, separate from the root `srsx` module. It imports nothing from the repository. - **Run:** `make integ` makes the test media, then runs the cases in two `go test` processes at once: the main cases and the `TestTimer_` cases. `make integ-unit` runs the unit tests of the clients and analyzers, and `bin/srs-integ` gives `probe`, `publish` and `prepare` from the command line. - **Server adapter:** each server is `objs/srs -e`, configured only by env vars, on ports from `SRS_INTEG_PORTS` (14000-14999 in the Port Plan). Most cases share a Common server, split into 4 shards. Cases that need other settings run on variant servers, and groups share these where they can. - **Case classes:** Fast (≤100ms), Normal (≤500ms) and Timer (≤10s, only in `TestTimer_` groups). A case that runs over its limit fails. `KnownBug` marks a case that fails because of an SRS bug, and the case fails once the bug is fixed, so the mark can't go stale. - **Clients and analyzers written for the suite:** RTMP and RTMPS (both handshakes, any chunk size, enhanced RTMP, malformed input), FLV, TS, AAC/MP3, MP4, HLS (TS, fMP4, AES-128, SAMPLE-AES) and DASH. Each one fills a shared stream report: codecs, picture size, sequence headers, keyframes, timestamps and A/V sync. - **Clients built on libraries:** WebRTC WHIP/WHEP on pion (UDP and TCP, NACK/RTX/TWCC recorder, raw DTLS sessions), SRT on gosrt, RTSP on gortsplib, and a GB28181 PS-over-RTP publisher with no SIP. - **Test media:** generated from `doc/source.flv` by `make integ-prepare` and never committed. The files cover H.264, HEVC, VP9, AV1, AAC, Opus, MP3 and G.711, in FLV, MP4 and TS. A manifest keeps the media from being made again when nothing changed. - **Feature coverage:** the HTTP API, HTTPS API and exporter, reload, heartbeat, RTMP, RTMPS, HTTP-FLV, HTTP-TS, HTTP AAC and MP3, HTTPS, HLS, DASH, WebRTC, SRT, RTSP, GB28181, the MPEG-TS over UDP caster, every HTTP hook, security rules, forward, edge, the origin cluster, DVR, static files and VOD, signals, the command line and env settings. - **Conversion matrix:** each source (RTMP/RTMPS, WHIP, SRT, GB28181) is played by each player (RTMP, HTTP-FLV, HTTP-TS, MP3, AAC, HLS ts and fMP4, DASH, WHEP, SRT, RTSP) with each codec it can carry. The README has the matrix. - **Scale:** `-integ.scale=N` runs each case N times. A run of about 1,000 cases passes in about 11s. ## SRS bug fixes Every fix has a utest and, where a client can see it, an srs-integ case that failed before the fix. The main ones by area: - **HLS:** fMP4: - The default init file is now listed by its dir. - `hls_entry_prefix` now applies to fMP4 URIs. - MP3 streams now get an audio track in the init file. - Each publish now starts with a discontinuity. - `hls_keys` now encrypts the samples with cbcs. - A segment that fails to open is now dropped. Before, it crashed SRS. Other HLS fixes: key URIs, AES padding, HLS starts again after `hls_dispose`, the ratios, a TS republish, session names, and the `hls_ctx` of another stream's session is now checked as a new session. - **DASH:** the config, the MPD (window, update period, HEVC and MP3 codecs), init files of MP3 and of one-track streams, and dispose. - **WebRTC:** - Keyframes with an SEI, and Sender Reports and their mapping. - Opus timing, G.711, and WHEP SSRCs. - A refused WHIP publisher now frees its stream. - The start and hooks, and WebRTC over TCP. - **SRT:** only the configured latencies are set, as ffmpeg does. Also fixed: HEVC headers and CTS, frame counts, and the listener. - **RTSP:** a republish to a kept player, PLAY without SETUP, unknown methods, closes, DESCRIBE/TEARDOWN, request bodies and `on_stop`. - **GB28181:** - SSRC parsing. - HEVC to RTMP is no longer sent in legacy FLV tags. - A crash on a second media connection. - Latency with no audio, sliced frames, CTS, session lifetime, and the muxer and caster output. - **RTMP and HTTP:** - Timestamps that jump back without time correction no longer stop a player. - The republish param and the publish timeout after a pause. - The no-origin close and the RTMPS port. - HTTP ranges, including MP4 suffix ranges and VOD 416. - Redirects, mounts, unmounts and URL `..`. - CORS, API auth paths, JSONP 400 and 403 answers. - Static files and the request echo. - **Hooks:** - The heartbeat URL's query is now sent. - `on_play` clients and client types, and `on_dvr`. - A client that `on_connect` refused no longer gets `on_close`. - **Other:** DVR (MP4 info frame, audio rate, audio cut), listen ports above 65535, the env vhost check, the UDP caster, process stats, the tcmalloc rate, the encoder loop, the edge failover, STUN, coroutine joins, and the resource manager's fast-id cache (WebRTC and GB lookups never used it). ## Behavior changes - **Hooks:** - `on_publish`, `on_unpublish`, `on_play` and `on_stop` have a new `type` field after `client_id`, such as `rtmp-play` or `hls-play`, the same value as `/api/v1/clients`. - Every protocol now sends `param` with its leading `?`. - HTTP players send their own `http(s)://` tcUrl, and WHIP/WHEP clients send `webrtc://<host>/<app>`. - **API:** `/api/v1/features` adds `srt`, `rtsp`, `gb28181`, `rtc`, `simulator` and `sanitizer`. The `/api/v1/` index lists only the URLs SRS serves. - **Env support:** - The vhost security rules. - The edge protocol. - The origin cluster and coworkers. - The DVR apply filter. - **Timers in seconds:** `hls_dispose`, `dash_dispose`, `dvr_duration`, `queue_length`, `stun_timeout` and the heartbeat `interval` now accept fractions such as `0.5`. A fraction used to be cut to whole seconds, so `0.5` read as 0. - **Removed features:** a config or env var that names one of these, on or off, now makes SRS refuse to start (and `-t` fail) with `<name> is removed`: - HDS, including `--hds=on`. - The FLV POST caster and `asprocess`. - `query_latest_version` and `first_wait_for_qlv`. - The vhost `bandcheck`, `refer` and `atc_auto`. - Edge `token_traverse`. Check clients with the edge's HTTP hooks instead. ## Tests, CI and skills - **CI:** a new `integration-test` job builds `trunk/Dockerfile.test` (now with `--simulator=on`) and runs srs-integ. - It caches the test media. - It runs the main cases and the timer cases as separate steps, each retried on its own. - Limits are scaled by 4 with `-integ.limit-scale=4`. - **Unit tests:** more stable. They use random seeds, the test servers no longer share ports or leak fds, and a gtest listener fails a test that leaves a handler on a global resource manager. - **Skills:** - The `srs-develop` Gate and Full tiers run `make integ`. - `integration-tests.md` explains how to add cases. - The new `internal-audit-for-srs` skill audits each commit blind before review. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> |
||
|
Winlin
|
cd11a6720f |
Proxy: Harden internal/env tests and add counterfeiter fakes. v7.0.144 (#4665)
- **Refactor `internal/env` for testability.** Route every `os`/filesystem call in `env.go` through swappable package-level function variables (`getEnv`, `setEnv`, `lookupEnv`, `openFile`). Split `parseEnvFile` into a thin file-opening wrapper plus a pure `parseEnvReader(io.Reader)` so the parser can be tested directly without touching disk. - **Hermetic tests, 96.9% coverage.** Rewrite `internal/env/env_test.go` to install in-memory fakes via `withFakeEnv` / `withFakeOpen` helpers that swap the package vars and restore them on `t.Cleanup`. Tests no longer mutate real process env or write temp `.env` files, removing a source of flakiness under parallel test execution. New cases cover `NewEnvironment`, `setEnvDefault`, `loadEnvFile` error paths, and edge cases in the parser. - **Counterfeiter-based fake generation.** Add `counterfeiter` as a Go tool dependency, a `//go:generate` directive for the `Environment` interface (`internal/env/gen.go`), and commit the generated `internal/env/envfakes/fake_environment.go` so downstream packages can test against a spec-faithful fake instead of hand-rolling stubs. Expose the step as `make generate`. - **Tooling.** `scripts/proxy-utest.sh` gains a `--coverage` / `-c` flag that runs `go test -coverprofile=...` across `./cmd/...` and `./internal/...` and prints per-function coverage via `go tool cover -func`. The `srs-develop` skill doc is updated to include the regenerate-fakes step and the new coverage flag. - **Go version.** Bump `go.mod` to Go 1.25 (required for the `go tool` directive used to pin the counterfeiter CLI as a tool dep). --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Winlin
|
460412c4b5 |
Move build output to bin/, replace godotenv with custom .env parser, and update docs. v7.0.143 (#4661)
- Move build output from `./srs-proxy` to `bin/srs-proxy` following Go project conventions, updating Makefile, .gitignore, and all documentation references - Replace third-party `godotenv` dependency with a custom `.env` parser that supports comments, `export` prefix, quoted values, escape sequences, and inline comments — with full unit tests - Remove `ignore-worklog.md` and update `README.md` with skill-based AI prompts - Bump copyright year from 2025 to 2026 across all source files --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Winlin
|
ebf8b712c9 |
Proxy: restructure repo as Go project with proxy as first module (#4652)
Reorganize the SRS (Simple Realtime Server) repository to follow a conventional Go project structure, setting the stage for a progressive transition from a C++ project to a Go project. The proxy, which was once contained within its own `proxy/` subdirectory, will now be converted into the initial Go module located at the root of the repository, serving as a template for subsequent Go modules. - **Go module at repo root:** `go.mod` moved to repo root, module renamed from `proxy` to `srsx`. The repo is now a proper Go project with `cmd/` and `internal/` at the top level. - **Elevation of Proxy Code:** Move the proxy code from `proxy/cmd/proxy-go/` to `cmd/proxy/`, and from `proxy/internal/` to `internal/`. The proxy serves as the inaugural application; subsequent modules (for instance, `cmd/origin`) will mimic this arrangement. - **Documentation Restructured:** Transfer the documentation from `proxy/docs/` to `docs/proxy/`, revise the main README to endorse OpenClaw as the preferred AI tool, and update `proxy/README.md` to point to the new documentation locations. - **Build and config:** `Makefile` moved to root, `PROXY_STATIC_FILES` default path corrected for the new layout, `.gitignore` consolidated. - **Cleanup:** removed standalone `proxy/LICENSE` (repo-level license applies), all internal imports updated to `srsx/internal/...`. - **OpenClaw workspace:** added community bot info, git workflow conventions, and support group behavior guidance. This restructuring was performed by OpenClaw orchestrating Claude Code and Codex via ACP. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> |