lib/mvfst
1
0
Fork 0
mirror of https://github.com/facebookincubator/mvfst.git synced 2025-08-01 01:44:22 +03:00
Code Activity

Use a single queue for scheduling DSR and non-DSR streams.

Browse Source 
Summary:
The write loop functions for DSR or non-DSR are segmented today. As such, so are the schedulers. Mirroring this, we also currently store the DSR and non-DSR streams in separate write queues. This makes it impossible to effectively balance between the two without potential priority inversions or starvation.

Combining them into a single queue eliminates this possibility, but is not entirely straightforward.

The main difficulty comes from the schedulers. The `StreamFrameScheduler` for non-DSR data essentially loops over the control stream queue and the normal write queue looking for the next stream to write to a given packet. When the queues are segmented things are nice and easy. When they are combined, we have to deal with the potential that the non-DSR scheduler will hit a stream with only DSR data. Simply bailing isn't quite correct, since it will just cause an empty write loop. To fix that we need check, after we are finished writing a packet, if the next scheduled stream only has DSR data. If it does, we need to ensure `hasPendingData()` returns false.

The same needs to be done in reverse for the DSR stream scheduler.

The last major compication is that we need another loop which wraps the two individual write loop functions, and calls both functions until the packet limit is exhausted or there's no more data to write. This is to handle the case where there are, for example, two active streams with the same incremental priority, and one is DSR and the other is not. In this case each write loop we want to write `packetLimit` packets, flip flopping between DSR and non DSR packets.

This kind of round robining is pathologically bad for DSR, and a future diff will experiment with changing the round robin behavior such that we write a minimum number of packets per stream before moving on to the next stream.

This change also contains some other refactors, such as eliminating `updateLossStreams` from the stream manager.

(Note: this ignores all push blocking failures!)

Reviewed By: kvtsoy

Differential Revision: D46249067

fbshipit-source-id: 56a37c02fef51908c1336266ed40ac6d99bd14d4
This commit is contained in:
Matt Joras
2023-06-01 14:11:31 -07:00
committed by Facebook GitHub Bot
parent 5b5082a3ee
commit 35a2d34843
20 changed files with 401 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
quic/api/QuicPacketScheduler.cpp
View File

@ -458,7 +458,11 @@ void StreamFrameScheduler::writeStreamsHelper(
level.iterator->begin();
do {
auto streamId = level.iterator->current();
auto stream = conn_.streamManager->findStream(streamId);
auto stream = CHECK_NOTNULL(conn_.streamManager->findStream(streamId));
if (!stream->hasSchedulableData() && stream->hasSchedulableDsr()) {
// We hit a DSR stream
return;
}
CHECK(stream) << "streamId=" << streamId
<< "inc=" << uint64_t(level.incremental);
if (!writeSingleStream(builder, *stream, connWritableBytes)) {
@ -476,30 +480,40 @@ void StreamFrameScheduler::writeStreams(PacketBuilderInterface& builder) {
DCHECK(conn_.streamManager->hasWritable());
uint64_t connWritableBytes = getSendConnFlowControlBytesWire(conn_);
// Write the control streams first as a naive binary priority mechanism.
const auto& writableControlStreams =
conn_.streamManager->writableControlStreams();
if (!writableControlStreams.empty()) {
const auto& controlWriteQueue = conn_.streamManager->controlWriteQueue();
if (!controlWriteQueue.empty()) {
conn_.schedulingState.nextScheduledControlStream = writeStreamsHelper(
builder,
writableControlStreams,
controlWriteQueue,
conn_.schedulingState.nextScheduledControlStream,
connWritableBytes,
conn_.transportSettings.streamFramePerPacket);
}
auto& writableStreams = conn_.streamManager->writableStreams();
if (!writableStreams.empty()) {
auto& writeQueue = conn_.streamManager->writeQueue();
if (!writeQueue.empty()) {
writeStreamsHelper(
builder,
writableStreams,
writeQueue,
connWritableBytes,
conn_.transportSettings.streamFramePerPacket);
// If the next non-control stream is DSR, record that fact in the scheduler
// so that we don't try to write a non DSR stream again. Note that this
// means that in the presence of many large control streams and DSR
// streams, we won't completely prioritize control streams but they
// will not be starved.
auto streamId = writeQueue.getNextScheduledStream();
auto stream = conn_.streamManager->findStream(streamId);
if (stream && !stream->hasSchedulableData()) {
nextStreamDsr_ = true;
}
}
} // namespace quic
}
bool StreamFrameScheduler::hasPendingData() const {
return conn_.streamManager->hasNonDSRLoss() ||
(conn_.streamManager->hasNonDSRWritable() &&
getSendConnFlowControlBytesWire(conn_) > 0);
return !nextStreamDsr_ &&
(conn_.streamManager->hasNonDSRLoss() ||
(conn_.streamManager->hasNonDSRWritable() &&
getSendConnFlowControlBytesWire(conn_) > 0));
}
bool StreamFrameScheduler::writeStreamFrame(