Summary:
Migrates mvfst's qlog implementation from the legacy "draft-00" format
to the official IETF qlog main schema v0.4 specification (draft-ietf-quic-qlog-
main-schema and draft-ietf-quic-qlog-quic-events). This ensures compatibility
with standard qlog tooling and adheres to the ratified IETF format.
Key changes to the qlog structure:
1. Event format: Migrated from array format [time, category, event, data] to
object format {time: X, name: "category:event_type", data: {...}}. Event
names now follow the "category:event_type" convention (e.g.,
"transport:packet_sent", "connectivity:connection_closed").
2. File structure: Updated top-level fields to include file_schema
(urn:ietf:params:qlog:file:contained), qlog_version ("0.4"),
serialization_format ("application/qlog+json"), and proper event_schemas
array. Added summary field with max_duration, total_event_count, and
trace_count.
3. Common fields: Restructured common_fields to follow IETF spec with ODCID,
scid, protocol_type, reference_time ("0"), time_format ("relative"), and
time_units ("microseconds").
4. Vantage point: Changed from name-based to type-based representation
({"type": "client"} or {"type": "server"}).
5. Streaming support: Updated finishStream() to properly inject the summary
field before the final closing brace for both pretty and non-pretty JSON
formats. Non-pretty JSON is handled as single-line output.
Implementation spans QLoggerTypes.cpp (event name mappings), FileQLogger.cpp
(summary generation and streaming), and updated test expectations throughout
QLoggerTest.cpp to validate the new format.
Reviewed By: sharmafb
Differential Revision: D86882919
fbshipit-source-id: bad3cf3667ce0a3ad8de207bb3a67a7bc5404a2b
Summary:
This commit completely removes the DSR (Direct Server Return) feature from the codebase. DSR was a zero-copy QUIC packet transmission mechanism that allowed backend servers to directly packetize and send data to clients, bypassing the transport layer.
Components removed:
**QUIC (fbcode/quic/)**
- Deleted quic/dsr/ directory with all DSR implementation files
- Removed BufferMeta and WriteBufferMeta from StreamData
- Removed writeBufMeta() and setDSRPacketizationRequestSender() from QuicSocket
- Removed DSR tracking from stream state (dsrSender, writeBufMeta, retransmission/loss buffers)
- Removed DSR packet tracking fields (isDSRPacket, nonDsrPacketSequenceNumber)
- Removed DSR logic from loss detection, flow control, packet scheduling
- Removed TperfDSRSender tool and DSR test infrastructure
**ti/bigcache**
- Deleted ti/bigcache/dsr/ directory (16 files: Cookie, PacketizationContext, SocketProvider, etc.)
- Removed ~1000 lines of DSR code from BigCacheCommandHandler (header/impl/inline)
- Removed XSK (AF_XDP) initialization and container management from MCBigCache.cpp
- Removed DSR RPC methods: async_eb_bigcacheGetDSR, async_eb_establishPacketizationSink, async_eb_release
- Removed getDSRBigcacheValueAsync(), createDSRDelegationRequest() from McrouterCache
- Removed DSR-specific stats namespace (dsrstats) and metrics
- Cleaned up BUCK dependencies removing quic/dsr and XSK references
**proxygen HTTP infrastructure**
- Removed allowDSR() virtual method from HTTPMessageFilter base class and 50+ filter implementations
- Removed setDSRRequestSender() interface from HTTPTransaction::Transport and HTTPSink
- Removed DSRRequestSender class and sendHeadersWithDelegate() from HTTPTransaction
- Removed BufferMeta struct and all related methods from HTTPTransaction (getBufferMeta, clearBufferMeta, etc.)
- Removed bufMeta_ member and references from HQSession::HQStreamTransportBase
- Removed isDelegated_ flag and delegated transaction checks from HTTPTransaction
- Removed cache DSR methods: canDSR(), performDSRDelegateAndScheduleCallbacks(), onDelegateSuccess(), onDelegateMiss()
- Removed DSR delegation from HTTPResponseCache and McrouterCache
**Test infrastructure fixes**
- Removed quic/dsr/test:mocks dependency from proxygen session test TARGETS
- Fixed PTM test to remove nonDsrPacketSequenceNumber field references
- Cleaned up MockQuicSocketDriver to remove DSR mock infrastructure (BufferMeta, writeBufMeta)
- Removed sendHeadersWithDelegate() from HTTPSessionMocks test helpers
- Commented out DSR-specific tests in HQDownstreamSessionTest (DelegateResponse, DelegateResponseError)
- Removed MockQuicDSRRequestSender from HQSessionMocks
- Disabled BigcacheSRPacketizer build targets (depends on removed quic/dsr infrastructure)
- Fixed WebTransportFilter, CdnFilters, and other components after removing DSR infrastructure
The codebase now exclusively uses standard QUIC stream writes through pendingWrites buffers.
Reviewed By: kvtsoy
Differential Revision: D86992558
fbshipit-source-id: a3814eaf735accdce989c45da8101aac8e8c831f
Summary:
This is an API break, but it should mostly be a manageable one. We want to be able to compile mvfst internally without exceptions, and folly::Optional is one dependency that makes this challenging. Additionally, we already have an imported secondary optional type for performance/struct size reasons, tiny-optional.
This second optional interface is mostly compatible in an API sense (including the use of std::nullopt) with std::optional. Thus our approach is to remove the dependency on folly::Optional, and offer a quic::Optional instead.
The next diff will properly vendor tiny-optional so that quic::Optional is an independent version of it.
Reviewed By: sharmafb, kvtsoy
Differential Revision: D74133131
fbshipit-source-id: 715f8bb5043ba3bb876cacfe54236887e0686b30
Summary: This introduces a more generic typealias so that we can, for instance, write `BufHelpers::createCombined` instead of `folly::IOBuf::createCombined`.
Reviewed By: jbeshay
Differential Revision: D73127508
fbshipit-source-id: d585790904efc8e9f92d79cbf766bafe0e84a69f
Summary:
Migrating mvfst priority API to be abstract, based on new classes is quic/priority. For now, it requires applications use `HTTPPriorityQueue::Priority`, to be compatible with the hardcoded `deprecated::PriorityQueue` implementation and apps cannot yet change the queue impl. Eventually the application will have full control of the queue.
There are minor functional changes in this diff:
1. Priority QLog types changed from int/bool to string
2. Any PAUSED stream has priority `u=7,i` if paused streams are disabled (previously explicitly settable to any priority)
Reviewed By: jbeshay
Differential Revision: D68696110
fbshipit-source-id: 5a4721b08248ac75d725f51b5cb3e5d5de206d86
Summary:
Adds more logging for:
- Number of initial packets received
- Number of unique crypto frames received
- Time elapsed to get the last meaningful initial crypto frame.
- named groups, psk status, and ech status from TLS layer.
Reviewed By: mjoras
Differential Revision: D71910754
fbshipit-source-id: ea8524213ba296727079800bb167ec080143c10b
Summary: Several "Network Path Model" parameters are described in Sec. 2.9 of the BBRv2 IETF draft. These quantities evolve throughout the connection and they are useful to analyze BBRv2 performance. This diff adds inflight_hi, inflight_lo, bandwidth_hi, and bandwidth_lo to the qlog.
Reviewed By: mjoras
Differential Revision: D61414936
fbshipit-source-id: 2862db2a6aab336fd8a60e4ae5b358e9ab5588b4
Summary:
The idea here is to make it so we can swap out the type we are using for optionality. In the near term we are going to try swapping towards one that more aggressively tries to save size.
For now there is no functional change and this is just a big aliasing diff.
Reviewed By: sharmafb
Differential Revision: D57633896
fbshipit-source-id: 6eae5953d47395b390016e59cf9d639f3b6c8cfe
Summary:
This helps with tuning l4s reaction in Cubic and debugging l4s in general.
To avoid spamming the qlog, an l4s weights update is only logged if new CE marks are echoed from the peer.
Reviewed By: sharmafb
Differential Revision: D57874704
fbshipit-source-id: 9c3f139aec73fc0dbd8d38e2237d6a6478cf1e3d
Summary: Add compression support to FileQlogger. When compression is enabled, the logger will produce gzipped *.qlog.gz files.
Reviewed By: avasylev
Differential Revision: D31287597
fbshipit-source-id: 178a65646f5091f2e159632c411c5abf77ff89d1
Summary: It's useful at the end of a connection to know if we tried DSR.
Reviewed By: jbeshay
Differential Revision: D30545282
fbshipit-source-id: bbb2f3408f7a2d5666676c9e2583bf8fd9f18911
Summary:
- Removed packetNum field from CipherUnavailable struct.
- Removed all instances referring to the field and fixed tests accordingly.
Reviewed By: mjoras
Differential Revision: D29968168
fbshipit-source-id: 9802b8cd66f43f2a8d54340f2d00639ee4679aaf
Summary: The kitchen sink must grow. Nice to know when the conn ends what the version was.
Reviewed By: lnicco
Differential Revision: D27868680
fbshipit-source-id: 2770ed2c647d76affbb52597a5a2a6720eabfe66
Summary: Turns out I really just need to axe it completely otherwise we end up with some of the packets relative from 0.
Reviewed By: afrind, lnicco
Differential Revision: D24631969
fbshipit-source-id: 9b84f382aec2cc557d59ad7de0034882ed7cd62c
Summary: In the spec these are now lowercase.
Reviewed By: avasylev
Differential Revision: D22104880
fbshipit-source-id: adc9bbcd7bd9e7babb1946648d2867ec7dc9336b
Summary: This is now an optional field on individual events.
Reviewed By: lnicco
Differential Revision: D22102853
fbshipit-source-id: ecfb4b4d9c2e594f10e93c8dfbff96b1dad4422d
Summary:
This adds a QLog event for Retry packets. There are three fields:
- packetType: This is always "RETRY"
- packetSize: The size of the entire packet
- tokenSize: The size of the retry token
Reviewed By: mjoras
Differential Revision: D20910017
fbshipit-source-id: feb4abbf0d15ff7b24fd11f00634e9da84bf8333
Summary: In the constructor of FileQlogger adds a bool that sets streaming mode. If it's set it creates a thread that reads events from the log and immediatly outputs them to an output file.
Reviewed By: mjoras
Differential Revision: D20250881
fbshipit-source-id: 2af3ff0aeaa5b62f90b0c01570c96c92fdab4412
Summary:
this isn't part of the QLog draft. The ack frame information in
already included in the packet_received event. Qvis also doesn't need this.
The only additional information this gives us is that the ack frame in the
packet_received has more packets than current outstanding packets. packet_ack
only includes those still outstanding.
For example, it's possible that only packets [5, 10] are outstanding since peer
already acked [0, 4] before. But peer hasn't received the ack of ack. Then peer
sends another packet which will ack [0, 10]. In that packet_received event, the
ack frame will have [0, 10]. But we will only generate packet_ack for [5, 10].
If such information is important, then we should keep this, but make it concise
at least.
Right now i'm not even sure if such information is important. We can get to the
same conclusion by backtracking to the previously received [0, 4] ack frame.
This is a lot of data.
Thoughts?
Reviewed By: mjoras
Differential Revision: D19237301
fbshipit-source-id: 95dfe2f9f6942cc30434b1fc05dd4929607d9c95
Summary:
QLogConnectionMigrationEvent:
Allow observing client-side ConnectionMigration attempts (replacing the
socket), and observing the server-side changing the peer address it
is writing to.
QLogPathValidationEvent:
Allow observing successful/failed path validation attempts.
Success is considered as a correct PathResponse being returned.
A Failure is only published on the timeout expiring, not an invalid
PathChallenge frame being returned (we do not cancel this).
There are already QlogEvents for PathChallenge/PathResponse that
can be observed.
Reviewed By: JunqiWang
Differential Revision: D18340999
fbshipit-source-id: 512108f82a6e082021c0bd3254f108c128b17ba3
Summary: this prints a ton if someone logs one connection per file
Reviewed By: sharma95
Differential Revision: D18048613
fbshipit-source-id: 126f770e0f8ca0ea492df2fefb96fa89d1c0fb7f
Summary:
to make the log easier to parse. Otherwise i will need a script to
scan the whole log and calculate refTime differences.
Reviewed By: sharma95
Differential Revision: D18042798
fbshipit-source-id: 1573f60eacaedb361fc8cdc2332333ff8bee6a13
Summary:
This diff adds a new event for bandwidth estimation update, and
changed the app limited/unlimited into its own events instead of piggybacking
onto the congestion update
Reviewed By: mjoras
Differential Revision: D17602684
fbshipit-source-id: 48b855c68b58992d07b8c9b872aea637c4267137
Summary:
Add a pacing observer interface in QLogger . An implementation is
added for tperf.
Reviewed By: mjoras
Differential Revision: D17816559
fbshipit-source-id: c9079a24a6dab88df83f32fec35dc287c58989cf
Summary:
Move major QLogger code into BaseQLogger so that mobile clients can choose not
to depend on it.
Reviewed By: mjoras
Differential Revision: D17781130
fbshipit-source-id: 3e35c552c8948a8c15ddb0170d28a92326aea9e1
