Skip to content
mager-bench1.3
← historical challenges

elixir-test

Write idiomatic Elixir ExUnit tests — tests knowledge of Elixir testing conventions

the prompt

$ cat elixir-test.prompt
Write an ExUnit test module for the following Elixir function:

```elixir
defmodule StringUtils do
  @doc """
  Truncates a string to at most `max_len` characters.
  If truncated, appends `suffix` (default: "...").
  Returns the original string if it is already within `max_len`.
  Raises ArgumentError if max_len is negative.
  """
  @spec truncate(String.t(), non_neg_integer(), String.t()) :: String.t()
  def truncate(str, max_len, suffix \\ "...")
end
```

Requirements:
- Module name: `StringUtilsTest`, `use ExUnit.Case`
- Use `describe` blocks to group related cases
- At least 2 `describe` blocks: one for happy path, one for edge/error cases
- At least 8 test cases total covering: string shorter than limit, exact length, longer string (suffix appended), empty string, custom suffix, max_len of 0, negative max_len (should raise), unicode string
- Use `assert`, `assert_raise`, and pattern matching appropriately
- Add at least one doctest-style `## Examples` in a module comment showing usage

rubric

correctness

Do the 8+ tests cover all specified cases with correct expected values? Does `assert_raise ArgumentError` correctly test the negative max_len case? Are unicode string lengths handled via `String.length/1`, not `byte_size/1`?

quality

Are `describe` blocks used to group related tests logically? Are test names written as plain English specs ('truncates when string exceeds limit')? Is pattern matching used where it adds clarity (e.g. matching on {:ok, _} if the API returns tuples)? Is the module correctly structured with `use ExUnit.Case`?

documentation

Are describe block labels and test names readable as a spec? Is the module-level Examples comment present and accurate? Does the overall file read like idiomatic Elixir test documentation?

results

GPT-6 Astra

9.7
correctness 10.0quality 9.0documentation 10.025408ms

# The response thoroughly covers every required case with correct Unicode grapheme expectations, clear describe blocks and test names, valid exception assertions, and accurate module-level examples; the only minor limitation is that pattern matching is used somewhat stylistically rather than adding substantial clarity.

inspect full trace →

GPT-5.6 Sol

9.3
correctness 10.0quality 9.0documentation 9.012699ms

# The module comprehensively covers every required case with correct Unicode behavior, idiomatic assertions and grouping, though the describe labels say truncate/3 while most tests exercise the default-argument truncate/2 form.

inspect full trace →