Skip to content
mager-bench1.3
← historical challenges

async-fetch

Write async Python for concurrent HTTP fetching with timeout and retry

the prompt

$ cat async-fetch.prompt
Write a Python async function `fetch_all(urls: list[str], timeout: float = 5.0, retries: int = 2) -> list[dict]` that fetches all URLs concurrently using `aiohttp` and returns a list of result dicts.

Each result dict must have:
  - `url`: the original URL
  - `status`: HTTP status code (int), or None on error
  - `body`: response text (str), or None on error
  - `error`: error message string, or None on success

Requirements:
1. All URLs must be fetched concurrently (not sequentially)
2. Each fetch must respect the `timeout` (per-request, not total)
3. Retry up to `retries` times on network errors or 5xx responses (exponential backoff: 0.5s, 1s, ...)
4. Never raise — errors go into the result dict
5. Type-hint everything; add a docstring
6. Show a `__main__` block that fetches 3 URLs and prints results

rubric

correctness

Is `asyncio.gather` (or equivalent) used for actual concurrency? Is timeout applied per-request via `aiohttp.ClientTimeout`? Does retry logic actually retry on 5xx and network errors (not just any exception)? Is exponential backoff implemented (not fixed sleep)? Does the function never raise on individual URL errors?

quality

Is the session created once (not per-request)? Is the retry loop clean (not deeply nested)? Are asyncio patterns idiomatic (async with, await)? Is the main block runnable with `asyncio.run`?

documentation

Does the docstring explain all parameters including units (timeout in seconds)? Are result dict keys documented? Is the backoff behavior described?

results

GPT-6 Astra

7.3
correctness 9.0quality 8.0documentation 5.031507ms

# Concurrency, per-attempt timeouts, retry/backoff behavior, shared session use, and result ordering are strong, but cancellation is improperly swallowed, unlimited connector concurrency is risky, and the docstring does not explicitly document every parameter, timeout units, or all result keys.

inspect full trace →

GPT-5.6 Sol

6.3
correctness 9.0quality 8.0documentation 2.018006ms

# Concurrency, per-attempt timeouts, retry targeting, exponential backoff, shared session use, and runnable main are strong, but dict types are unparameterized, Optional is unused, and the docstring omits required parameter, units, result-key, and backoff documentation.

inspect full trace →