russh: Post-decompression SSH packet size was not bounded, allowing remote oversized compressed packets

Upgrade to 0.61.1

Before the fix, Decompress::decompress() grew its output buffer by repeated doubling and did not enforce a separate post-decompression ceiling. That meant a peer could send a small compressed packet that passed the normal on-wire transport length checks and then inflate it into a much larger packet after decompression.

It was verified that an attacker-crafted compressed payload can stay below the normal 256 KiB implementation transport packet cap while still inflating above the intended post-decompression bound. In other words, this is not only a "large on-wire packet" issue.

Version detail:

• The underlying post-decompression bounds bug appears to affect russh as far back as 0.34.0.
• In historical releases >= 0.34.0, < 0.58.0, the remote decompression path still used CryptoVec. Remote compressed SSH traffic could drive that path, and under constrained memory that historical code path could abort the process.
• In current-style releases >= 0.58.0, non-secret packet/decompression buffers were moved off CryptoVec and onto Vec<u8>, but the post-decompression size still remained unbounded. So the bug class remained reachable remotely, but the maintained-line impact is a current remote DoS / oversized-packet-acceptance issue rather than the older CryptoVec-based abort story.
• The maintained-line fix was verified against 0.60.2.

Compression is not selected in a default-vs-default russh session because the default preference order puts none first. However, the default server configuration still advertises zlib and zlib@openssh.com, and server-side negotiation follows the client's preference order for common algorithms. A client that prefers compression can therefore negotiate it with a default russh server.

OpenSSH portable was checked at /home/mjc/projects/openssh-portable commit 45b30e0a5. OpenSSH enforces a 256 KiB transport packet cap before decompression, but it does not reuse that cap after decompression. Instead, decompression writes to an sshbuf, which is indirectly bounded by OpenSSH's SSHBUF_SIZE_MAX hard maximum of 0x8000000 bytes (128 MiB).

The patch direction should follow that model: add an explicit post-decompression ceiling of 128 MiB, rather than assuming the compressed transport packet cap also bounds decompressed payload size.

Relevant OpenSSH reference points:

• /home/mjc/projects/openssh-portable/packet.c: PACKET_MAX_SIZE (256 * 1024)
• /home/mjc/projects/openssh-portable/packet.c: uncompress_buffer() inflates into compression_buffer
• /home/mjc/projects/openssh-portable/sshbuf.h: SSHBUF_SIZE_MAX 0x8000000

RFC / OpenSSH Comparison

RFC 4253 section 6 defines the binary packet format:

• packet_length
• padding_length
• payload
• random padding
• MAC

RFC 4253 section 6.2 says that, when compression is negotiated, the payload field is compressed, and that packet_length and MAC are computed from the compressed payload. The RFC also says implementations should check that packet length is reasonable to avoid denial-of-service and buffer-overflow attacks.

That means the pre-decompression transport packet length check is necessary but not sufficient. A correct implementation still needs a reasonable bound on the decompressed payload that becomes parser input.

OpenSSH provides such a bound indirectly through sshbuf's hard maximum. The russh fix should make the corresponding post-decompression bound explicit.

PoC

There were two kinds of proof:

• a wire-cap sanity test showing an attacker-crafted best-compressed DEBUG payload can stay below the normal SSH transport packet cap while still inflating beyond the intended post-decompression bound
• direct client and server receive-path tests that exercise the oversized post-decompression behavior itself

The current in-tree regression tests are:

• tests::compress::oversized_debug_payload_can_stay_below_wire_cap
• compression::tests::oversized_decompressed_packet_is_rejected
• client::tests::compressed_debug_is_ignored_after_client_parses_it
• client::tests::oversized_compressed_debug_is_rejected_before_client_ignores_it
• server::session::tests::compressed_debug_is_ignored_after_server_parses_it
• server::session::tests::oversized_compressed_debug_is_rejected_before_server_ignores_it

The important behavior is:

1. An attacker-crafted best-compressed DEBUG payload can stay below the normal 256 KiB transport packet cap while still inflating beyond 128 MiB.
2. In the direct client and server receive paths, small compressed DEBUG packets are still ignored normally after parsing.
3. In the direct client and server receive paths, oversized compressed DEBUG packets are rejected before the implementation reaches the normal "ignore DEBUG" behavior.

The strongest PoC for severity is the unauthenticated server-side case. A malicious client can choose zlib in the initial key exchange, because the default server advertises it and server-side negotiation follows the client's preference order for common algorithms. After NEWKEYS, but before authentication, the client can send a transport-layer SSH_MSG_DEBUG packet whose compressed body is below the transport packet cap but whose decompressed body exceeds the post-decompression cap.

That demonstrates the AV:N/AC:L/PR:N/UI:N case directly: the attacker is a remote SSH client and does not need a successfully authenticated session.

fn compressed_debug_payload(payload_len: usize) -> Vec<u8> {
    let mut payload = vec![b'A'; payload_len];
    payload[0] = crate::msg::DEBUG;

    let mut encoder =
        flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::best());
    encoder.write_all(&payload).unwrap();
    let compressed = encoder.finish().unwrap();

    assert!(
        compressed.len() < 256 * 1024,
        "oversized post-decompression payload still fits under the wire cap"
    );
    compressed
}

fn incoming_packet(compressed: Vec<u8>) -> SSHBuffer {
    let mut buffer = SSHBuffer::new();
    // maybe_decompress() receives the clear SSHBuffer after packet framing,
    // and decompresses bytes after packet_length + padding_length.
    buffer.buffer.extend_from_slice(&[0; 5]);
    buffer.buffer.extend_from_slice(&compressed);
    buffer
}

#[test]
fn unauthenticated_client_zlib_debug_is_rejected_by_server_before_auth() {
    let mut server = preauth_server_session_after_newkeys_with_zlib_decompressor();
    let oversized = MAXIMUM_DECOMPRESSED_PACKET_LEN + 1024;
    let buffer = incoming_packet(compressed_debug_payload(oversized));

    let err = server.maybe_decompress(&buffer).unwrap_err();
    assert!(
        matches!(err, crate::Error::PacketSize(len) if len > MAXIMUM_DECOMPRESSED_PACKET_LEN)
    );
}

The equivalent wire-level attack shape is:

1. Connect to a russh server using th