webrtc: Support FIN/FIN_ACK handshake for substream shutdown #4

timwu20 · 2026-01-09T17:03:25Z

Summary

Implements proper FIN/FIN_ACK handshake for WebRTC substream shutdown according to the libp2p WebRTC-direct specification, with timeout handling to prevent indefinite waiting.

Closes paritytech#476

Changes

1. Proper FIN/FIN_ACK Handshake

poll_shutdown now waits for FIN_ACK acknowledgment before completing
Added waker mechanism to wake shutdown task when FIN_ACK is received
Ensures graceful shutdown with delivery guarantees

2. Timeout Protection

Added 5-second timeout for FIN_ACK (matching go-libp2p's behavior)
Prevents indefinite waiting if remote never sends FIN_ACK
Automatically completes shutdown after timeout to avoid resource leaks

3. Shutdown Requirements Fulfilled

✅ Stops accepting new writes during shutdown (transitions through Closing state)
✅ Flushes pending data before sending FIN
✅ Sends FIN flag with empty payload
✅ Waits for FIN_ACK acknowledgment from remote
✅ Triggers data channel closure and resource cleanup

4. Code Refactoring

Unified Message event variant to include optional flag parameter (instead of separate MessageWithFlags variant)
Unified encode functions into single function with optional flag parameter
Fixed flag handling bug: changed from bitwise operations to equality checks
Renamed plural "flags" to singular "flag" throughout codebase for consistency with protobuf schema

Implementation Details

The shutdown process now follows this state machine:

Open → Closing (blocks new writes)
Flush pending data
Send FIN flag
Closing → FinSent
Wait for FIN_ACK or timeout (5 seconds)
FinSent → FinAcked
Drop substream → channel closes → data channel cleanup

The timeout task is spawned immediately when entering FinSent state and wakes the shutdown future after 5 seconds if no FIN_ACK is received.

Testing

Added comprehensive tests for FIN/FIN_ACK handshake
Added test for timeout behavior when FIN_ACK is not received

Compatibility

Follows libp2p WebRTC specification
5-second timeout matches go-libp2p's implementation

… for acknowledgment

src/transport/webrtc/substream.rs

…f inbound to self

dimartiro · 2026-01-12T22:51:02Z

src/transport/webrtc/substream.rs

+        let current_state = *self.state.lock();
+
+        match current_state {


I think the correct way to keep the lock during the match is:

Suggested change

let current_state = *self.state.lock();

match current_state {

let current_state = self.state.lock();

match *current_state {

Otherwise, the lock will be released while you are reading the state, which could lead to a race condition

The state gets updated for certain matched variants, so it needs to be unlocked again later. It's actually fine if lock isn't held since it would never get updated by another thread since poll_shutdown is never called concurrently.

haikoschol

When testing this with a libp2p dialer and a litep2p listener, I noticed the following:

Apparently libp2p does not send FIN_ACK. It doesn't even exist in their flag enum. Just an observation, still works due to the timeout.
litep2p does not use the Substream type for the noise handshake datachannel and therefore the flag handling code added in this PR is not executed. Additionally, right after sending the last handshake message, litep2p closes the datachannel directly. I tried commenting out this line, but that caused a panic in WebRtcConnection::on_inbound_data(). The spec just says "On success of the authentication handshake, the used datachannel is closed". It seems more logical to me that the dialer closes the channel. But given that connection establishment works, there is only ever one handshake channel per connection and changing this on the litep2p side seems to be a bit involved, I think this is a low priority issue.
I added a call to close the perf substream in libp2p here and logging calls (warn) when libp2p sends the FIN flag and when litep2p receives it, as well as when it sends FIN_ACK back. I see the first message, but not the latter two. I changed this log from trace to warn in order to see all flags of incoming messages and it seems to be always None.

Full dialer log:

$ SSLKEYLOGFILE=key.log RUST_LOG="info" cargo run --bin libp2p-perf -- client --transport-layer "webrtc" --server-address "/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q" --upload-bytes 1024 --download-bytes 1024
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.19s
     Running `target/debug/libp2p-perf client --transport-layer webrtc --server-address /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q --upload-bytes 1024 --download-bytes 1024`
2026-01-13T11:20:44.825444Z  INFO libp2p_swarm: local_peer_id=12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid
2026-01-13T11:20:44.828496Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/127.0.0.1/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828655Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.1.146/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828683Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.139.3/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828709Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/169.254.133.87/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.831052Z  INFO webrtc_ice::mdns: mDNS is using 0.0.0.0:5353 as dest_addr
2026-01-13T11:20:44.832012Z  INFO webrtc_mdns::conn: Looping and listening Ok(0.0.0.0:5353)
2026-01-13T11:20:44.832532Z  INFO webrtc::peer_connection: signaling state changed to have-local-offer
2026-01-13T11:20:44.833122Z  INFO webrtc::peer_connection: signaling state changed to stable
2026-01-13T11:20:44.833685Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Checking
2026-01-13T11:20:44.833711Z  INFO webrtc::peer_connection: ICE connection state changed: checking
2026-01-13T11:20:44.833723Z  INFO webrtc::peer_connection: peer connection state changed: connecting
2026-01-13T11:20:45.041060Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Connected
2026-01-13T11:20:45.041359Z  INFO webrtc::peer_connection: ICE connection state changed: connected
2026-01-13T11:20:45.075581Z  INFO webrtc::peer_connection: peer connection state changed: connected
2026-01-13T11:20:45.287422Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:20:45.288343Z  INFO libp2p_perf: Event: Some(ConnectionEstablished { peer_id: PeerId("12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"), connection_id: ConnectionId(1), endpoint: Dialer { address: /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q, role_override: Dialer, port_use: Reuse }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 459.91925ms })
2026-01-13T11:20:45.293837Z  INFO litep2p-perf: Uploaded 1.00 KiB bytes in 0.0000s bandwidth 546.67 Mbit/s
2026-01-13T11:20:45.297546Z  INFO litep2p-perf: Downloaded 1.00 KiB bytes in 0.0036s bandwidth 2.20 Mbit/s
2026-01-13T11:20:45.297613Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:20:45.297851Z  INFO libp2p_perf: Event: Some(Behaviour(Event { id: 1, result: Ok(()) }))
2026-01-13T11:20:45.298186Z  WARN webrtc_sctp::association: [] failed to write packets on net_conn: io error: io error: oneshot canceled

Full listener log:

$ RUST_LOG="info" cargo run --bin litep2p-perf -- server --listen-address "/ip4/127.0.0.1/udp/8888/webrtc-direct" --node-key "secret" --transport-layer webrtc
   Compiling litep2p v0.12.3 (/Users/haiko/code/polkadot/litep2p)
   Compiling litep2p-perf v0.1.0 (/Users/haiko/code/polkadot/litep2p-perf/litep2p)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.75s
     Running `target/debug/litep2p-perf server --listen-address /ip4/127.0.0.1/udp/8888/webrtc-direct --node-key secret --transport-layer webrtc`
2026-01-13T11:20:35.158021Z  INFO litep2p_perf: Using WebRTC transport layer
2026-01-13T11:20:35.161763Z  INFO litep2p::webrtc: start webrtc transport listen_addresses=["/ip4/127.0.0.1/udp/8888/webrtc-direct"]
2026-01-13T11:20:35.173753Z  INFO litep2p_perf: Server listening on address: ["/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"]
2026-01-13T11:20:45.085174Z  INFO litep2p_perf: Event: ConnectionEstablished { peer: PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid"), endpoint: Listener { address: "/ip4/127.0.0.1/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg/p2p/12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid", connection_id: ConnectionId(1) } }
2026-01-13T11:20:45.289427Z  WARN str0m: Drop ChannelData event for id: 0
2026-01-13T11:20:45.293199Z  WARN str0m: Drop BufferedAmountLow for id: 0
2026-01-13T11:20:45.295765Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=8
2026-01-13T11:20:45.295947Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=1024
2026-01-13T11:20:45.296132Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=8

src/transport/webrtc/substream.rs

haikoschol · 2026-01-13T11:34:12Z

I added a call to close the perf substream in libp2p [...]

After moving the substream closure before receiving the download bytes (right after this line), I see the flag and log message show up on the litep2p side.

Full dialer log:

$ SSLKEYLOGFILE=key.log RUST_LOG="info" cargo run --bin libp2p-perf -- client --transport-layer "webrtc" --server-address "/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiDkiVIC23hLqgGIRwSfB3MVbR8U0NjCAauOpeMAq2hU_Q/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q" --upload-bytes 1024 --download-bytes 1024
   Compiling libp2p-perf v0.1.0 (/Users/haiko/code/polkadot/litep2p-perf/libp2p)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.90s
     Running `target/debug/libp2p-perf client --transport-layer webrtc --server-address /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiDkiVIC23hLqgGIRwSfB3MVbR8U0NjCAauOpeMAq2hU_Q/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q --upload-bytes 1024 --download-bytes 1024`
2026-01-13T11:29:41.420941Z  INFO libp2p_swarm: local_peer_id=12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin
2026-01-13T11:29:41.423770Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/127.0.0.1/udp/56354/webrtc-direct/certhash/uEiAR1PJvzYZKDzUfIXaNlgi_PCzUf8FKOSIJRnEzw6l-mQ })
2026-01-13T11:29:41.424653Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.1.146/udp/56354/webrtc-direct/certhash/uEiAR1PJvzYZKDzUfIXaNlgi_PCzUf8FKOSIJRnEzw6l-mQ })
2026-01-13T11:29:41.424692Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.139.3/udp/56354/webrtc-direct/certhash/uEiAR1PJvzYZKDzUfIXaNlgi_PCzUf8FKOSIJRnEzw6l-mQ })
2026-01-13T11:29:41.424722Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/169.254.133.87/udp/56354/webrtc-direct/certhash/uEiAR1PJvzYZKDzUfIXaNlgi_PCzUf8FKOSIJRnEzw6l-mQ })
2026-01-13T11:29:41.425939Z  INFO webrtc_ice::mdns: mDNS is using 0.0.0.0:5353 as dest_addr
2026-01-13T11:29:41.426905Z  INFO webrtc_mdns::conn: Looping and listening Ok(0.0.0.0:5353)
2026-01-13T11:29:41.427432Z  INFO webrtc::peer_connection: signaling state changed to have-local-offer
2026-01-13T11:29:41.427960Z  INFO webrtc::peer_connection: signaling state changed to stable
2026-01-13T11:29:41.428344Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Checking
2026-01-13T11:29:41.429508Z  INFO webrtc::peer_connection: ICE connection state changed: checking
2026-01-13T11:29:41.429520Z  INFO webrtc::peer_connection: peer connection state changed: connecting
2026-01-13T11:29:41.634736Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Connected
2026-01-13T11:29:41.634866Z  INFO webrtc::peer_connection: ICE connection state changed: connected
2026-01-13T11:29:41.659586Z  INFO webrtc::peer_connection: peer connection state changed: connected
2026-01-13T11:29:41.872556Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:29:41.873559Z  INFO libp2p_perf: Event: Some(ConnectionEstablished { peer_id: PeerId("12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"), connection_id: ConnectionId(1), endpoint: Dialer { address: /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiDkiVIC23hLqgGIRwSfB3MVbR8U0NjCAauOpeMAq2hU_Q/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q, role_override: Dialer, port_use: Reuse }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 449.820083ms })
2026-01-13T11:29:41.878442Z  INFO litep2p-perf: Uploaded 1.00 KiB bytes in 0.0000s bandwidth 549.83 Mbit/s
2026-01-13T11:29:41.878515Z  WARN libp2p_perf::perf: closing the substream before receiving the download bytes
2026-01-13T11:29:41.878532Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:29:41.882684Z  INFO litep2p-perf: Downloaded 1.00 KiB bytes in 0.0041s bandwidth 1.92 Mbit/s
2026-01-13T11:29:41.883598Z  INFO libp2p_perf: Event: Some(Behaviour(Event { id: 1, result: Ok(()) }))

Full listener log:

$ RUST_LOG="info" cargo run --bin litep2p-perf -- server --listen-address "/ip4/127.0.0.1/udp/8888/webrtc-direct" --node-key "secret" --transport-layer webrtc
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.17s
     Running `target/debug/litep2p-perf server --listen-address /ip4/127.0.0.1/udp/8888/webrtc-direct --node-key secret --transport-layer webrtc`
2026-01-13T11:29:27.377482Z  INFO litep2p_perf: Using WebRTC transport layer
2026-01-13T11:29:27.379770Z  INFO litep2p::webrtc: start webrtc transport listen_addresses=["/ip4/127.0.0.1/udp/8888/webrtc-direct"]
2026-01-13T11:29:27.388211Z  INFO litep2p_perf: Server listening on address: ["/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiDkiVIC23hLqgGIRwSfB3MVbR8U0NjCAauOpeMAq2hU_Q/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"]
2026-01-13T11:29:41.669696Z  INFO litep2p_perf: Event: ConnectionEstablished { peer: PeerId("12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin"), endpoint: Listener { address: "/ip4/127.0.0.1/udp/56354/webrtc-direct/certhash/uEiAR1PJvzYZKDzUfIXaNlgi_PCzUf8FKOSIJRnEzw6l-mQ/p2p/12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin", connection_id: ConnectionId(1) } }
2026-01-13T11:29:41.874576Z  WARN str0m: Drop ChannelData event for id: 0
2026-01-13T11:29:41.877167Z  WARN str0m: Drop BufferedAmountLow for id: 0
2026-01-13T11:29:41.880297Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin") channel_id=ChannelId(1) data_len=8
2026-01-13T11:29:41.880492Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin") channel_id=ChannelId(1) data_len=1024
2026-01-13T11:29:41.880575Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin") channel_id=ChannelId(1) data_len=8
2026-01-13T11:29:41.880795Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooWRXvHDkUXKhRGqgyRBZF8GUv155TW2deKrq2tqvxsryin") channel_id=ChannelId(1) flag=0 data_len=0
2026-01-13T11:29:41.880914Z  WARN litep2p::transport::webrtc::substream: received FIN flag, sending FIN_ACK

I'm not sure what to make of this. 🤔

…ss of state

timwu20 · 2026-01-14T03:02:16Z

When testing this with a libp2p dialer and a litep2p listener, I noticed the following:

Apparently libp2p does not send FIN_ACK. It doesn't even exist in their flag enum. Just an observation, still works due to the timeout.
litep2p does not use the Substream type for the noise handshake datachannel and therefore the flag handling code added in this PR is not executed. Additionally, right after sending the last handshake message, litep2p closes the datachannel directly. I tried commenting out this line, but that caused a panic in WebRtcConnection::on_inbound_data(). The spec just says "On success of the authentication handshake, the used datachannel is closed". It seems more logical to me that the dialer closes the channel. But given that connection establishment works, there is only ever one handshake channel per connection and changing this on the litep2p side seems to be a bit involved, I think this is a low priority issue.
I added a call to close the perf substream in libp2p here and logging calls (warn) when libp2p sends the FIN flag and when litep2p receives it, as well as when it sends FIN_ACK back. I see the first message, but not the latter two. I changed this log from trace to warn in order to see all flags of incoming messages and it seems to be always None.

Full dialer log:

$ SSLKEYLOGFILE=key.log RUST_LOG="info" cargo run --bin libp2p-perf -- client --transport-layer "webrtc" --server-address "/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q" --upload-bytes 1024 --download-bytes 1024
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.19s
     Running `target/debug/libp2p-perf client --transport-layer webrtc --server-address /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q --upload-bytes 1024 --download-bytes 1024`
2026-01-13T11:20:44.825444Z  INFO libp2p_swarm: local_peer_id=12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid
2026-01-13T11:20:44.828496Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/127.0.0.1/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828655Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.1.146/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828683Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/192.168.139.3/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.828709Z  INFO libp2p_perf: Event: Some(NewListenAddr { listener_id: ListenerId(1), address: /ip4/169.254.133.87/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg })
2026-01-13T11:20:44.831052Z  INFO webrtc_ice::mdns: mDNS is using 0.0.0.0:5353 as dest_addr
2026-01-13T11:20:44.832012Z  INFO webrtc_mdns::conn: Looping and listening Ok(0.0.0.0:5353)
2026-01-13T11:20:44.832532Z  INFO webrtc::peer_connection: signaling state changed to have-local-offer
2026-01-13T11:20:44.833122Z  INFO webrtc::peer_connection: signaling state changed to stable
2026-01-13T11:20:44.833685Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Checking
2026-01-13T11:20:44.833711Z  INFO webrtc::peer_connection: ICE connection state changed: checking
2026-01-13T11:20:44.833723Z  INFO webrtc::peer_connection: peer connection state changed: connecting
2026-01-13T11:20:45.041060Z  INFO webrtc_ice::agent::agent_internal: [controlling]: Setting new connection state: Connected
2026-01-13T11:20:45.041359Z  INFO webrtc::peer_connection: ICE connection state changed: connected
2026-01-13T11:20:45.075581Z  INFO webrtc::peer_connection: peer connection state changed: connected
2026-01-13T11:20:45.287422Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:20:45.288343Z  INFO libp2p_perf: Event: Some(ConnectionEstablished { peer_id: PeerId("12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"), connection_id: ConnectionId(1), endpoint: Dialer { address: /ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q, role_override: Dialer, port_use: Reuse }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 459.91925ms })
2026-01-13T11:20:45.293837Z  INFO litep2p-perf: Uploaded 1.00 KiB bytes in 0.0000s bandwidth 546.67 Mbit/s
2026-01-13T11:20:45.297546Z  INFO litep2p-perf: Downloaded 1.00 KiB bytes in 0.0036s bandwidth 2.20 Mbit/s
2026-01-13T11:20:45.297613Z  WARN libp2p_webrtc_utils::stream: sending FIN flag to close stream
2026-01-13T11:20:45.297851Z  INFO libp2p_perf: Event: Some(Behaviour(Event { id: 1, result: Ok(()) }))
2026-01-13T11:20:45.298186Z  WARN webrtc_sctp::association: [] failed to write packets on net_conn: io error: io error: oneshot canceled

Full listener log:

$ RUST_LOG="info" cargo run --bin litep2p-perf -- server --listen-address "/ip4/127.0.0.1/udp/8888/webrtc-direct" --node-key "secret" --transport-layer webrtc
   Compiling litep2p v0.12.3 (/Users/haiko/code/polkadot/litep2p)
   Compiling litep2p-perf v0.1.0 (/Users/haiko/code/polkadot/litep2p-perf/litep2p)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.75s
     Running `target/debug/litep2p-perf server --listen-address /ip4/127.0.0.1/udp/8888/webrtc-direct --node-key secret --transport-layer webrtc`
2026-01-13T11:20:35.158021Z  INFO litep2p_perf: Using WebRTC transport layer
2026-01-13T11:20:35.161763Z  INFO litep2p::webrtc: start webrtc transport listen_addresses=["/ip4/127.0.0.1/udp/8888/webrtc-direct"]
2026-01-13T11:20:35.173753Z  INFO litep2p_perf: Server listening on address: ["/ip4/127.0.0.1/udp/8888/webrtc-direct/certhash/uEiCFl4FpV9gPv_LDA9lVLLQ4QM71y7jPXdH9qPGYBPnKVQ/p2p/12D3KooWBpZHDZu7YSbvPaPXKhkRNJvR7MkTJMQQAVBKx9mCqz3q"]
2026-01-13T11:20:45.085174Z  INFO litep2p_perf: Event: ConnectionEstablished { peer: PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid"), endpoint: Listener { address: "/ip4/127.0.0.1/udp/51413/webrtc-direct/certhash/uEiCNhSA3RD-4v77WuvuKWE7Spa6C4cPln-mWv0M0pSWMSg/p2p/12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid", connection_id: ConnectionId(1) } }
2026-01-13T11:20:45.289427Z  WARN str0m: Drop ChannelData event for id: 0
2026-01-13T11:20:45.293199Z  WARN str0m: Drop BufferedAmountLow for id: 0
2026-01-13T11:20:45.295765Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=8
2026-01-13T11:20:45.295947Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=1024
2026-01-13T11:20:45.296132Z  WARN litep2p::webrtc::connection: handle inbound message peer=PeerId("12D3KooW9s5y8JzwVMYB4Mw13d2ZLuG8cYcjP5AfDdcphhYoWAid") channel_id=ChannelId(1) data_len=8

This is good to know. That means we're relying on the timeout logic to function properly.
I'll revisit this later. I think we if we revamp how we do the message framing and utilize the same negotiation protocol as the other transports, that would be the ideal solution.
We need to explicitly call substream.close() on both server side and client side given it's a 4 way handshake. Each side sends FIN to close it's own writing side, but read side is still open. I made the changes on my fork of litep2p-perf here. The sequence diagram when closing a channel can be found in the "Closing an RTCChannel" section of the WebRTC spec.

Co-authored-by: Copilot <[email protected]>

…waker

…ents

haikoschol · 2026-01-23T11:20:46Z

Upstream PR has been opened. Closing this to avoid email spam.

timwu20 added 3 commits January 8, 2026 15:42

webrtc: Implement proper FIN/FIN_ACK handshake and fix flag handling bug

5a64e7e

webrtc: Unify event variants and encoding to use optional flag parameter

057eac8

webrtc: Implement proper shutdown with FIN/FIN_ACK handshake and wait…

0027ec7

… for acknowledgment

timwu20 force-pushed the tim/support-flags branch from 90d927a to 2fc4a2c Compare January 12, 2026 18:40

webrtc: Add timeout for FIN_ACK to prevent indefinite waiting

6de9d33

timwu20 force-pushed the tim/support-flags branch from 2fc4a2c to 6de9d33 Compare January 12, 2026 18:41

webrtc: Fix clippy and fmt

4c5917d

timwu20 changed the title ~~Tim/support flags~~ webrtc: Support FIN/FIN_ACK handshake for substream shutdown Jan 12, 2026

timwu20 marked this pull request as ready for review January 12, 2026 18:53

timwu20 requested review from dimartiro and haikoschol January 12, 2026 18:56

dimartiro reviewed Jan 12, 2026

View reviewed changes

src/transport/webrtc/substream.rs Outdated Show resolved Hide resolved

dimartiro reviewed Jan 12, 2026

View reviewed changes

src/transport/webrtc/substream.rs Outdated Show resolved Hide resolved

webrtc: Fix FIN_ACK response to be sent outbound to network instead o…

4695143

…f inbound to self

timwu20 requested a review from dimartiro January 12, 2026 22:22

dimartiro reviewed Jan 12, 2026

View reviewed changes

haikoschol reviewed Jan 13, 2026

View reviewed changes

src/transport/webrtc/substream.rs Show resolved Hide resolved

webrtc: Fix data channel closure by detecting substream drop regardle…

8578487

…ss of state

timwu20 force-pushed the tim/support-flags branch from b413ef4 to 8578487 Compare January 13, 2026 18:36

timwu20 and others added 6 commits January 22, 2026 11:46

webrtc: Use protobuf-generated Flag enum instead of i32

c7f4ea8

Update src/transport/webrtc/substream.rs

833da9b

Co-authored-by: Copilot <[email protected]>

Update src/transport/webrtc/substream.rs

35b99f4

Co-authored-by: Copilot <[email protected]>

webrtc: Use AtomicWaker instead of Mutex<Option<Waker>> for shutdown_…

42de0ab

…waker

webrtc: Use pinned Sleep instead of tokio::spawn for FIN_ACK timeout

2ec58e4

webrtc: Optimize WebRtcMessage encode/decode to reduce allocations

a4effe2

timwu20 force-pushed the tim/support-flags branch from d9d9225 to a4effe2 Compare January 22, 2026 20:25

webrtc: Wake blocked writers and close both sides on RESET_STREAM

73f76ec

timwu20 force-pushed the tim/support-flags branch from 1127166 to 73f76ec Compare January 22, 2026 22:35

timwu20 added 5 commits January 22, 2026 18:01

webrtc: Fix race condition in shutdown waker registration

7aa6084

webrtc: Log warning for unknown flag values in message decoding

e3a436e

webrtc: Fix typo in WebRtcMessage doc comment

0a24d6b

webrtc: Address review feedback for FIN/FIN_ACK handshake

1cc104a

webrtc: Additional review feedback fixes

2440797

timwu20 force-pushed the tim/support-flags branch from 2d78d7e to 2440797 Compare January 23, 2026 03:30

timwu20 added 2 commits January 22, 2026 22:37

webrtc: Fix varint length calculation to use ilog2

a36a04e

webrtc: Add test for FIN with payload and clarify race condition comm…

d537573

…ents

haikoschol closed this Jan 23, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

webrtc: Support FIN/FIN_ACK handshake for substream shutdown #4

webrtc: Support FIN/FIN_ACK handshake for substream shutdown #4

Uh oh!

timwu20 commented Jan 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

dimartiro Jan 12, 2026 •

edited

Loading

Uh oh!

timwu20 Jan 13, 2026

Uh oh!

haikoschol left a comment •

edited

Loading

Uh oh!

Uh oh!

haikoschol commented Jan 13, 2026

Uh oh!

timwu20 commented Jan 14, 2026

Uh oh!

haikoschol commented Jan 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		let current_state = *self.state.lock();

		match current_state {

webrtc: Support FIN/FIN_ACK handshake for substream shutdown #4

webrtc: Support FIN/FIN_ACK handshake for substream shutdown #4

Uh oh!

Conversation

timwu20 commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

1. Proper FIN/FIN_ACK Handshake

2. Timeout Protection

3. Shutdown Requirements Fulfilled

4. Code Refactoring

Implementation Details

Testing

Compatibility

Uh oh!

Uh oh!

Uh oh!

dimartiro Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timwu20 Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

haikoschol left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

haikoschol commented Jan 13, 2026

Uh oh!

timwu20 commented Jan 14, 2026

Uh oh!

haikoschol commented Jan 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

timwu20 commented Jan 9, 2026 •

edited

Loading

dimartiro Jan 12, 2026 •

edited

Loading

haikoschol left a comment •

edited

Loading