1
0
mirror of https://github.com/facebookincubator/mvfst.git synced 2025-11-09 10:00:57 +03:00
Commit Graph

47 Commits

Author SHA1 Message Date
Joseph Beshay
a73eade468 Add more fields to Qlog Transport Summary to track spurious retransmissions
Summary:
Add three new fields to the qlog transport summary to help analyze spurious transmissions and how they are affected by using adaptive loss thresholds.

Sample transport summary with added fields:
```
          "transport_summary",
          {
            "current_conn_flow_control": 1068498609,
            "current_writable_bytes": 106330,
            "dsr_packet_count": 0,
            "final_packet_loss_reordering_threshold": 77,
            "final_packet_loss_time_reordering_threshold_dividend": 29,
            "quic_version": 4207849474,
            "sum_cur_stream_buffer_len": 0,
            "sum_cur_write_offset": 5243215,
            "sum_max_observed_offset": 67,
            "total_bytes_cloned": 0,
            "total_bytes_recvd": 38157,
            "total_bytes_retransmitted": 244982,
            "total_bytes_sent": 5605121,
            "total_crypto_data_recvd": 342,
            "total_crypto_data_written": 1548,
            "total_packets_spuriously_marked_lost": 203,
            "total_stream_bytes_cloned": 0,
            "used_zero_rtt": false
          }
```

Reviewed By: afrind, mjoras

Differential Revision: D33354583

fbshipit-source-id: 55d9880ef02f6914b74c1b6508863bea7807950b
2022-01-07 18:42:12 -08:00
Joseph Beshay
dadf8652c2 FileQLogger Improvements
Summary:
- Gracefully catch errors when QLog file cannot be created.
- Remove an unused variable.

Reviewed By: avasylev

Differential Revision: D31671608

fbshipit-source-id: 058d852eb2dc1c513dc6c6eb22aee29b77a1b96d
2021-10-15 12:36:45 -07:00
Joseph Beshay
a7776d9d9e Add compression support to FileQLogger
Summary: Add compression support to FileQlogger. When compression is enabled, the logger will produce gzipped *.qlog.gz files.

Reviewed By: avasylev

Differential Revision: D31287597

fbshipit-source-id: 178a65646f5091f2e159632c411c5abf77ff89d1
2021-10-06 10:01:57 -07:00
Matt Joras
40cc0ca93e Add DSR packet count to state and qlog.
Summary: It's useful at the end of a connection to know if we tried DSR.

Reviewed By: jbeshay

Differential Revision: D30545282

fbshipit-source-id: bbb2f3408f7a2d5666676c9e2583bf8fd9f18911
2021-08-26 14:06:57 -07:00
Hani Damlaj
da36437098 Remove packetNum From CipherUnavailable
Summary:
- Removed packetNum field from CipherUnavailable struct.

- Removed all instances referring to the field and fixed tests accordingly.

Reviewed By: mjoras

Differential Revision: D29968168

fbshipit-source-id: 9802b8cd66f43f2a8d54340f2d00639ee4679aaf
2021-08-04 12:13:21 -07:00
Matt Joras
19339ef38f Add quic version to transport summary.
Summary: The kitchen sink must grow. Nice to know when the conn ends what the version was.

Reviewed By: lnicco

Differential Revision: D27868680

fbshipit-source-id: 2770ed2c647d76affbb52597a5a2a6720eabfe66
2021-04-19 17:58:35 -07:00
Matt Joras
98298496a2 Add some more info to transport summary qlog
Summary: Same as before!

Reviewed By: lnicco

Differential Revision: D27785992

fbshipit-source-id: 0160fb1572c8afc3dbbcf1dac50fc73ddbfbad83
2021-04-14 20:05:53 -07:00
Chetan Ambekar
cd754c2499 Revert D27768829: Add some more info to transport summary qlog
Differential Revision:
D27768829 (3ed777d04f)

Original commit changeset: 43cbf88f38ce

fbshipit-source-id: 4c678ebccbb7c9c33b1c7ab9ed98f6bdbc9c7e49
2021-04-14 18:45:06 -07:00
Matt Joras
3ed777d04f Add some more info to transport summary qlog
Summary: Specifically interested in seeing whether 0RTT was used, writable bytes, and conn flow control.

Reviewed By: yangchi

Differential Revision: D27768829

fbshipit-source-id: 43cbf88f38ce56065cad785344dedc279a0af7d1
2021-04-14 18:08:13 -07:00
Yang Chi
7c85871c3e HTTP Priority update logging in QLog
Summary: as title

Reviewed By: mjoras

Differential Revision: D24903759

fbshipit-source-id: 39127b658e6cb06025272c1c5ab0f7de2ae1a54a
2020-11-17 20:21:49 -08:00
Matt Joras
6d799fafcb Remove refTimePoint completely
Summary: Turns out I really just need to axe it completely otherwise we end up with some of the packets relative from 0.

Reviewed By: afrind, lnicco

Differential Revision: D24631969

fbshipit-source-id: 9b84f382aec2cc557d59ad7de0034882ed7cd62c
2020-10-29 18:08:17 -07:00
Matt Joras
015ff4a808 Lowercase qlog strings.
Summary: In the spec these are now lowercase.

Reviewed By: avasylev

Differential Revision: D22104880

fbshipit-source-id: adc9bbcd7bd9e7babb1946648d2867ec7dc9336b
2020-07-14 16:12:25 -07:00
Matt Joras
a21f1c2f68 Remove trigger event field.
Summary: This is now an optional field on individual events.

Reviewed By: lnicco

Differential Revision: D22102853

fbshipit-source-id: ecfb4b4d9c2e594f10e93c8dfbff96b1dad4422d
2020-06-17 19:21:26 -07:00
Matt Joras
448b707798 Remove max_outgoing_loss_rate.
Summary: We aren't using it anyway.

Reviewed By: lnicco

Differential Revision: D22102854

fbshipit-source-id: 66cbd86de3e6101bde6e425b8945692793ab9421
2020-06-17 19:21:26 -07:00
Aman Sharma
4a7ddd92af Add "RetryPacket" QLogger event
Summary:
This adds a QLog event for Retry packets. There are three fields:
- packetType: This is always "RETRY"
- packetSize: The size of the entire packet
- tokenSize: The size of the retry token

Reviewed By: mjoras

Differential Revision: D20910017

fbshipit-source-id: feb4abbf0d15ff7b24fd11f00634e9da84bf8333
2020-04-21 13:40:53 -07:00
Zachary Weinberger
88e3fdadc0 Add streaming option for FileQLogger
Summary: In the constructor of FileQlogger adds a bool that sets streaming mode. If it's set it creates a thread that reads events from the log and immediatly outputs them to an output file.

Reviewed By: mjoras

Differential Revision: D20250881

fbshipit-source-id: 2af3ff0aeaa5b62f90b0c01570c96c92fdab4412
2020-03-12 13:47:35 -07:00
TJ Yin
a396f62335 Replace folly::Optional::hasValue() by has_value()
Differential Revision: D19882830

fbshipit-source-id: 031217f9890351022bc8d171f0ccd7e045dd6972
2020-02-26 08:40:44 -08:00
Yang Chi
fa942df8b3 Remove packet ack event from QLogs
Summary:
this isn't part of the QLog draft. The ack frame information in
already included in the packet_received event. Qvis also doesn't need this.

The only additional information this gives us is that the ack frame in the
packet_received has more packets than current outstanding packets. packet_ack
only includes those still outstanding.

For example, it's possible that only packets [5, 10] are outstanding since peer
already acked [0, 4] before. But peer hasn't received the ack of ack. Then peer
sends another packet which will ack [0, 10]. In that packet_received event, the
ack frame will have [0, 10]. But we will only generate packet_ack for [5, 10].
If such information is important, then we should keep this, but make it concise
at least.

Right now i'm not even sure if such information is important. We can get to the
same conclusion by backtracking to the previously received [0, 4] ack frame.

This is a lot of data.

Thoughts?

Reviewed By: mjoras

Differential Revision: D19237301

fbshipit-source-id: 95dfe2f9f6942cc30434b1fc05dd4929607d9c95
2020-01-03 11:03:38 -08:00
Viktor Chynarov
45e71f737a Add QLog events for ConnMigration, PathValidation
Summary:
QLogConnectionMigrationEvent:
Allow observing client-side ConnectionMigration attempts (replacing the
socket), and observing the server-side changing the peer address it
is writing to.

QLogPathValidationEvent:
Allow observing successful/failed path validation attempts.
Success is considered as a correct PathResponse being returned.
A Failure is only published on the timeout expiring, not an invalid
PathChallenge frame being returned (we do not cancel this).

There are already QlogEvents for PathChallenge/PathResponse that
can be observed.

Reviewed By: JunqiWang

Differential Revision: D18340999

fbshipit-source-id: 512108f82a6e082021c0bd3254f108c128b17ba3
2019-11-08 08:39:36 -08:00
Yang Chi
a0b6f5a038 Remove excessive FileQLogger logging message
Summary: this prints a ton if someone logs one connection per file

Reviewed By: sharma95

Differential Revision: D18048613

fbshipit-source-id: 126f770e0f8ca0ea492df2fefb96fa89d1c0fb7f
2019-10-24 09:53:22 -07:00
Yang Chi
b0088bb4e2 Add time since stream creation into Stream level qlog events
Summary:
to make the log easier to parse. Otherwise i will need a script to
scan the whole log and calculate refTime differences.

Reviewed By: sharma95

Differential Revision: D18042798

fbshipit-source-id: 1573f60eacaedb361fc8cdc2332333ff8bee6a13
2019-10-24 09:53:21 -07:00
Yang Chi
9247b8f49b Change Quic qlogger vantage point to enum
Summary: To prevent adhoc string passed in

Reviewed By: sharma95

Differential Revision: D18013615

fbshipit-source-id: 6f5512468755c2b1ecbba3ba42188fa22f4e94b9
2019-10-23 10:16:58 -07:00
Yang Chi
e12fe5287a AppLimited and Bandwidth update QLog event in Quic BBR
Summary:
This diff adds a new event for bandwidth estimation update, and
changed the app limited/unlimited into its own events instead of piggybacking
onto the congestion update

Reviewed By: mjoras

Differential Revision: D17602684

fbshipit-source-id: 48b855c68b58992d07b8c9b872aea637c4267137
2019-10-15 21:19:05 -07:00
Yang Chi
faae04331c Quic pacing observer
Summary:
Add a pacing observer interface in QLogger . An implementation is
added for tperf.

Reviewed By: mjoras

Differential Revision: D17816559

fbshipit-source-id: c9079a24a6dab88df83f32fec35dc287c58989cf
2019-10-10 22:08:10 -07:00
Subodh Iyengar
8750462043 Separate out qlogger code into baseqlogger
Summary:
Move major QLogger code into BaseQLogger so that mobile clients can choose not
to depend on it.

Reviewed By: mjoras

Differential Revision: D17781130

fbshipit-source-id: 3e35c552c8948a8c15ddb0170d28a92326aea9e1
2019-10-07 22:43:32 -07:00
Bonnie Xu
db349541f2 Update qlog format to be more complete
Summary: Update qlog format to be more complete. Adding the summary section (with extra fields like title, description, etc). This diff is just to make the format more on par with Robin's schema.

Reviewed By: mjoras

Differential Revision: D16499808

fbshipit-source-id: 56cfbb95404f7e3c6638bffda18b53f7d83048a1
2019-07-30 12:48:57 -07:00
Bonnie Xu
f72ab2cfd6 Implement HTTP3 event interface
Summary: Implement HTTP3 event interface for qlog.

Reviewed By: yangchi

Differential Revision: D16424215

fbshipit-source-id: 041cd0a999eea49fdf98218348bd2a62e467a0b0
2019-07-27 14:07:58 -07:00
Bonnie Xu
63a79f278b Implement metricUpdate interface
Summary: Implement metricUpdate interface as part of spec.

Reviewed By: mjoras

Differential Revision: D16447278

fbshipit-source-id: 662361f257a336066a9f7ffb4e31eea6012add3f
2019-07-25 22:49:24 -07:00
Bonnie Xu
ce7ff9c215 Implement packetAck interface
Summary: Implement packetAck interface as part of spec.

Reviewed By: sharma95

Differential Revision: D16422768

fbshipit-source-id: f053a3b89ec080603d6568e37ad9281cf4834186
2019-07-25 11:52:21 -07:00
Bonnie Xu
5d871f45e0 Implement packetBuffered interface
Summary: Implement packetBuffered interface, as part of spec.

Reviewed By: yangchi

Differential Revision: D16415617

fbshipit-source-id: ece8e9fcd05c3cb7b7c63f7d652ed8df4c1a355b
2019-07-25 11:52:19 -07:00
Bonnie Xu
ed6d61690f Implement transportStateUpdate interface
Summary: Implement transportStateUpdate interface as part of [spec](fburl.com/proxygen-qlog).

Reviewed By: yangchi

Differential Revision: D16294320

fbshipit-source-id: 74f024782fc17b4706791c468866567238fac278
2019-07-20 10:09:22 -07:00
Bonnie Xu
14203ac53f Implement packetsLost interface
Summary: Implement packetsLost interface, as part of [spec](fburl.com/proxygen-qlog).

Reviewed By: yangchi

Differential Revision: D16292615

fbshipit-source-id: 7d313df209ac18f1deec92e32f557b0a1e97415c
2019-07-18 12:45:03 -07:00
Bonnie Xu
c550969a81 Implement lossAlarm interface
Summary: Implement lossAlarm interface according to [spec](fburl.com/proxygen-qlog).

Reviewed By: yangchi

Differential Revision: D16285188

fbshipit-source-id: f588f92230a0f0d9bf5d5fa7ad5673583b49fc1b
2019-07-18 09:47:51 -07:00
Bonnie Xu
ded2044f39 Implement datagramReceived interface
Summary: Implement datagramReceived interface, as part of [spec](fburl.com/proxygen-qlog).

Reviewed By: sharma95

Differential Revision: D16270834

fbshipit-source-id: fc112600b7d72a3ffdbb3b88584b2e17ed9c1ce1
2019-07-17 22:49:59 -07:00
Bonnie Xu
65dea29c7c Implement packetDrop interface for Quic
Summary: Implement packetDrop interface for Quic, as part of [spec](fburl.com/qlog-schema).

Reviewed By: yangchi

Differential Revision: D16233099

fbshipit-source-id: cf1fb67cea6f2599a6960593eace8122475c3557
2019-07-17 10:24:47 -07:00
Bonnie Xu
fb54bd8f06 Implement AppIdleUpdate interface for Quic
Summary: Implement AppIdleUpdate interface for Quic, as part of [spec](fburl.com/qlog-schema).

Reviewed By: yangchi

Differential Revision: D16231072

fbshipit-source-id: 09a759ad06388b7e8322179fbb9368b0015ae5a0
2019-07-16 16:37:38 -07:00
Lin Huang
28e7b0d837 Resolve land conflict between dirsync and quic pacing
Reviewed By: yangchi, xubonnie

Differential Revision: D16281430

fbshipit-source-id: 4d10b9cf5880cfcc4bb1c946e4234cf80ab68a12
2019-07-16 09:49:49 -07:00
Lin Huang
b4baf99a14 Move xplat/quic from autosyncer to dirsync
Summary:
* remove quic from autosyncer
* set up dirsync for quic
* mirror source file for xplat/quic fbcode/quic

Reviewed By: JunqiWang

Differential Revision: D16254573

fbshipit-source-id: 24414d275de310b5d0cf6bb2702a203e5aed260c
2019-07-16 01:09:48 -07:00
Bonnie Xu
9e4052c375 Implement pacingMetricUpdate interface for Quic
Summary: Implement pacingMetricUpdate interface for Quic. See [spec](fburl.com/qlog-schema) for details.

Reviewed By: yangchi

Differential Revision: D16205933

fbshipit-source-id: 858e7227ff2e3fae55d9de410818b7afe99fb0a5
2019-07-15 21:58:20 -07:00
Bonnie Xu
4b2b63248e Implement metricUpdate interface for Quic
Summary: Implement metricUpdate interface for Quic. See [spec](fburl.com/qlog-schema) for details.

Reviewed By: yangchi

Differential Revision: D16170779

fbshipit-source-id: 3c4121a953aa93b7de7485f99a78765da1302d6f
2019-07-12 10:09:24 -07:00
Bonnie Xu
a5e59690bd Implemented TransportSummaryEvent interface for QLogger
Summary: Implemented TransportSummaryEvent interface for QLogger. See [spec](fburl.com/qlog-schema) for details.

Reviewed By: mjoras

Differential Revision: D16115821

fbshipit-source-id: 92da0a75b3c53a9287c456bb34a066a105a0ded4
2019-07-12 09:28:45 -07:00
Bonnie Xu
fc6aa8b248 Implemented ConnectionCloseEvent interface for QLogger
Summary:
Implemented ConnectionCloseEvent interface for QLogger. As part of spec, [see here.](fburl.com/qlog-schema)

I intend to add this to ProxygenQLogger after [D15945914](https://our.intern.facebook.com/intern/diff/D15945914/) lands, and connect it to Quic in the next diff (ex. `qLogger->add...`).

Reviewed By: yangchi

Differential Revision: D16093813

fbshipit-source-id: 040be49f9c762eeb0439ae6cf22e2596987d8791
2019-07-09 00:10:12 -07:00
Bonnie Xu
c8fc46a7ff Rename add to addPacket
Summary: Rename `add` to `addPacket`, since `add` is an ambiguous function name, especially as we include more events.

Reviewed By: sharma95

Differential Revision: D16102141

fbshipit-source-id: 73196a5f1bcc681e3bb0a26fd9edaf34f59bf710
2019-07-08 03:34:33 -07:00
Bonnie Xu
55a5e9b43b Added dcid, scid, reference time to toDynamic output
Summary: Added dcid, scid, reference time to toDynamic output, for FileQLogger, [in accordance to QLog output](https://raw.githubusercontent.com/quiclog/internet-drafts/master/examples/draft-00/draft-00-basic_example.qlog).

Reviewed By: sharma95

Differential Revision: D16036483

fbshipit-source-id: 6f2cec9c8705c808ccca36926568635627898a5d
2019-07-02 12:37:32 -07:00
Bonnie Xu
df29adde47 Added FileLogger function to output QLogs to file
Summary: Added FileLogger function to output QLogs to file.

Reviewed By: mjoras

Differential Revision: D15952871

fbshipit-source-id: 554f21e12e9060eee7cfd64ce85dc8847c884d3a
2019-06-26 00:10:39 -07:00
Bonnie Xu
685ff77e22 s/size_t/uint64_t/g in qlogger
Reviewed By: vchynarov

Differential Revision: D15983098

fbshipit-source-id: 34f9c4f7c6256add6eb51888d55d7e8c953e92a0
2019-06-25 21:16:04 -07:00
Bonnie Xu
91963d1af4 Refactor QLogger to include derived QLogger classes depending on use case
Summary:
Refactor QLogger to include derived QLogger classes depending on use case.

The QLog output will differ depending on what is requesting a QLog.

`File QLogger`: This is connected to HQ, stores all of the events per connection, and then outputs them to a file.

Reviewed By: sharma95

Differential Revision: D15928447

fbshipit-source-id: b2cdcd4d6305585b0f4f0aa21a95d1398aa761a5
2019-06-20 19:00:55 -07:00