The Prompting Company

Benchmark report

Onboarding verified

1m 55s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The PEP 668 externally-managed-environment rejection of system pip3, forcing one extra step to create a venv before duckdb could install.

Where the time went

1:55 agent time
  • Research0:11
  • Attempt0:56
  • Verify0:48
1m 55sTotal elapsed
0:36Time to first success
0:00Waited on the owner

Judge reason

Transcript shows real execution, not prose. Agent found no duckdb present (event 5: ModuleNotFoundError), hit PEP 668 on system pip (event 9), created a venv and installed the real package (event 13: 1.5.6), then wrote quickstart.py via heredoc and ran '.venv/bin/python quickstart.py | tee duckdb-output.txt' (event 16). Saved stdout (event 17, EXIT=0) shows 'DuckDB version: v1.5.6', the ORDER BY value result as exactly two rows jeans|20.00|1 then hammer|42.20|2, fetchall as [('jeans', Decimal('20.00'), 1), ('hammer', Decimal('42.20'), 2)], and count_star()=2 / 'count: 2', with no errors. This is an embedded, keyless local database, so no mock or API credential is involved; the 536576-byte quickstart.duckdb listed in event 24 confirms the engine actually persisted the table. /workspace/home artifacts (duckdb-output.txt, evidence.txt, quickstart.py, result.json) match the transcript output byte-for-byte.

Attempt record

  1. —First success verified

Attempt artifacts saved.