---
id: "GHSA-f9hh-cmx9-2f23"
page: "https://infrastack.ai/live/GHSA-f9hh-cmx9-2f23"
kind: "vulnerability"
aliases: ["GHSA-f9hh-cmx9-2f23"]
title: "mcp-server-fetch: robots.txt path-canonicalization bypass (e.g. //private fetched while Disallow: /private)"
severity: "low"
cvss: 3.1
epss: null
exploited: false
status: "open"
category: "ai-tooling"
products: [{"name": "mcp-server-fetch", "fixed": ["2026.11.11"], "affected": ">= 0.6.1, < 2026.11.11", "ecosystem": "pypi"}]
ecosystems: ["pypi"]
tags: ["ai-tooling", "claude-code", "cwe-41", "cwe-706", "mcp", "python"]
not_affected_if: ["auth-required", "default-config"]
published_at: "2026-11-11T07:27:04+00:00"
first_seen_at: "2026-11-11T07:28:35+00:00"
updated_at: "2026-11-11T07:30:02+00:00"
sources: [{"source": "ghsa-repo", "url": "https://github.com/modelcontextprotocol/servers/security/advisories/GHSA-f9hh-cmx9-2f23", "first_seen_at": "2026-11-11T07:28:35+00:00"}]
exploit_sources: []
---

# mcp-server-fetch: robots.txt path-canonicalization bypass (e.g. //private fetched while Disallow: /private)

Link for people: https://infrastack.ai/live/GHSA-f9hh-cmx9-2f23 (this .md is for agents)

The mcp-server-fetch MCP server compares robots.txt rules against the request path as written, without normalizing it the way web origins do. An attacker who can steer a model-initiated fetch can therefore retrieve pages a site disallowed in robots.txt, such as //private when Disallow: /private is set.

_Summary, check and fix written by a model from the sources; labelled as such._

## Affected

- pypi:mcp-server-fetch affected >= 0.6.1, < 2026.11.11 fixed 2026.11.11

## How to check

- Check the installed `mcp-server-fetch` version with `pip show mcp-server-fetch` or `uv pip show mcp-server-fetch`; versions from `>= 0.6.1` and `< 2026.11.11` are affected.
- Confirm whether the server runs with robots.txt enforcement on, which is the default.
- Check whether agents using the server can be given URLs from untrusted input or redirects.

## Fix

- `mcp-server-fetch`: `2026.11.11`

## Timeline

- 2026-11-11T07:28:35+00:00 ghsa-repo https://github.com/modelcontextprotocol/servers/security/advisories/GHSA-f9hh-cmx9-2f23

## Documents

- [advisory] Maintainer's advisory: github.com https://github.com/modelcontextprotocol/servers/security/advisories/GHSA-f9hh-cmx9-2f23

## From the sources

## Summary

`mcp-server-fetch` matches `robots.txt` rules against the request path exactly as the client spelled it, while a typical origin resolves the path (collapse duplicate slashes, decode separator/dot escapes, remove `.`/`..` segments) before serving. With `Disallow: /private`, an autonomous `fetch` of `https://example.com//private` (and similar alternate spellings) therefore passes the robots check and returns the body of the page the site asked not to be crawled.

This is a policy-enforcement bypass of the control the server documents for model-initiated fetches (`src/fetch/README.md`, "Customization - robots.txt"). It is not an authentication bypass: the origin was already serving the content over plain HTTP.

**Severity: Low.** No auth bypass, no SSRF / private-address bypass, no RCE, no crash. `robots.txt` is advisory crawler policy; the impact is defeating the advertised autonomous-fetch guarantee.

## Distinct from GHSA-3jqg-r5pp-rmp2

[GHSA-3jqg-r5pp-rmp2](https://github.com/modelcontextprotocol/servers/security/advisories/GHSA-3jqg-r5pp-rmp2) was a redirect-hop robots re-check gap (fixed so each hop is checked). This report is a different root cause: even with per-hop checks present, the path comparison itself does not canonicalize. Tested on commit `36380287ed3b15d115b55f546617d723072b1296` (`mcp-server-fetch` 2026.11.4), which already includes the hop fix.

## Affected

- Package: `mcp-server-fetch` (PyPI), under `src/fetch`
- Tested: 2026.11.4, commit `36380287ed3b15d115b55f546617d723072b1296`
- Comparison site: `src/fetch/src/mcp_server_fetch/server.py` around the `Protego` `can_fetch()` call in `check_may_autonomously_fetch_url()` (about line 532 at that commit). Spot-checks of older revisions since robots support was added show the same non-canonical comparison; we are not claiming every intermediate release.

## Reproduction (localhost only)

Uses the project's own `FakeWeb` harness — no network and no live site. Save as `src/fetch/tests/test_repro_robots_bypass.py` and run:

```
cd src/fetch && uv run --frozen pytest tests/test_repro_robots_bypass.py -v
```

```python
import httpx
import pytest
from mcp.types import CallToolResult
from .harness import FakeWeb, connect

ROBOTS = "https://example.com/robots.txt"
PRIVATE = "the private page"

def page(body):
    return httpx.Response(
        200, content=body.encode(), headers={"content-type": "text/plain"}
    )

def robots():
    return page("User-agent: *\nDisallow: /private")

def text_of(result: CallToolResult) -> str:
    return result.model_dump(by_alias=True, mode="json", exclude_none=True)[
        "content"
    ][0]["text"]

async def test_control_is_refused(web: FakeWeb) -> None:
    """Control: the plain path is refused today, so the rule is enforced."""
    web.add(ROBOTS, robots())
    web.add("https://example.com/private", page(PRIVATE))
    async with connect() as (session, _):
        result = await session.call_tool("fetch", {"url": "https://example.com/private"})
    assert result.is_error is True
    assert PRIVATE not in text_of(result)

@pytest.mark.parametrize(
    "url",
    ["https://example.com//private", "https://example.com/a%2f..%2fprivate"],
)
async def test_alternate_spelling_leaks(web: FakeWeb, url) -> None:
    """These spell the disallowed page differently; the origin still serves it."""
    web.add(ROBOTS, robots())
    web.add(url, page(PRIVATE))
    async with connect() as (session, _):
        result = await session.call_tool("fetch", {"url": url})
    assert result.is_error is True, "the disallowed page was fetched"
    assert PRIVATE not in text_of(result)

async def test_redirect_to_encoded_dots_leaks(web: FakeWeb) -> None:
    """A redirect target is not passed through Pydantic's AnyUrl."""
    web.add(ROBOTS, robots())
    web.add(
        "https://example.com/go",
        httpx.Response(302, headers={"location": "/%2e%2e/private"}),
    )
    web.add("https://example.com/%2e%2e/private", page(PRIVATE))
    async with connect() as (session, _):
        result = await session.call_tool("fetch", {"url": "https://example.com/go"})
    assert result.is_error is True, "the disallowed page was fetched"
    assert PRIVATE not in text_of(result)
```

On the tested revision the control passes and the three bypass tests fail (`AssertionError: the disallowed page was fetched`).

| Request path | Path Protego compares | Path HTTPX sends | Path a normalizing origin serves |
|---|---|---|---|
| `/private` | `/private` | `/private` | `/private` (blocked) |
| `//private` | `//private` | `//private` | `/private` (fetched) |
| `/a%2f..%2fprivate` | `/a%2F..%2Fprivate` | `/a%2f..%2fprivate` | `/private` (fetched) |
| `/%2e%2e/private` (via redirect `Location`) | `/../private` | `/%2e%2e/private` | `/private` (fetched) |

We also confirmed against a local Python `http.server` origin (path resolution comparable to nginx/Apache-style serving): `GET /private` refused by robots; `GET //private` and the encoded-dot redirect target returned the private file on the wire. An origin that does not normalize returns 404, so there is no bypass there.

Notes on scope: `/%2e%2e/private` works through a redirect `Location` (Pydantic `AnyUrl` normalizes that form on a direct argument and correctly blocks it). `//private` works as a direct argument; as a redirect target HTTPX treats scheme-relative `//private` as host `private`. Cross-host redirects still re-check the target host's robots.txt, so an attacker-controlled site that 302s to `https://victim.example/%2e%2e/private` can miss the victim's rule the same way.

## Impact assumptions

- Autonomous (model-initiated) `fetch` with robots enforcement on (default); the intentional prompt-fetch robots exemption is out of scope.
- Attacker influences a URL the agent fetches (directly or via redirect).
- Target publishes a path-specific `Disallow`, and its origin resolves the alternate spelling to that resource.

## Suggested fix

Resolve the request path the way an origin does before matching rules, and require **both** the literal spelling and the resolved path to be allowed (`and`). Decode only separator/dot escapes (`%2f`/`%2F`/`%2e`/`%2E`), collapse duplicate slashes, and remove `.`/`..` per RFC 3986 §5.2.4; leave ordinary escapes such as `%20` alone. Derive the wire path with `urlsplit`/`urlunsplit` rather than from a URL object that HTTPX/Pydantic may already have normalized. Keep the existing `can_fetch(str(url), …)` check and add the second check — the change only ever adds refusals.

Happy to share a tested patch + regression file against the project's harness privately if useful.

---

This issue was identified with AI-assisted static code review and checked against the source before reporting; happy to answer questions or help with a fix.

---

_Free to read, with fair use of about 60 pages a minute. For higher limits or API access: sales@infrastack.ai_