BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
arXiv:2406.15877v5 Announce Type: replace-cross Abstract: Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks ranging from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human developers, the majority of their evaluations are limited to short and self-contained algorithmic tasks or standalone function calls. Solving chall
Read original sourceSource record
Project analysis
Project analysis is not yet available. See the source record above.