Refact.ai ranked number one open-source on SWE-bench verified with 352 of 500 real issues resolved autonomously

Resolving 352 out of 500 real-world software engineering issues from GitHub autonomously is the benchmark that requires clarification to appreciate. SWE-bench Verified does not test code generation quality. It tests autonomous end-to-end issue resolution: read the issue, understand the codebase, implement the fix, pass the existing tests. That is a multi-step autonomous engineering task, not completion assistance.

The self-hosting and enterprise deployment options being available is the security consideration that determines whether Refact is viable for organisations with code confidentiality requirements. Your code stays on your infrastructure rather than passing through a cloud service.

The customisation via fine-tuning on your specific codebase being available changes the suggestion quality from generic to project-specific over time. An agent that has learned your team's patterns, naming conventions and architectural preferences is different from one suggesting generic implementations of standard algorithms.

For teams evaluating AI coding agents specifically for autonomous task completion rather than completion assistance: how are you currently measuring the quality of autonomous resolution versus needing to intervene and correct?