Experts are raising concerns over the ethical and academic validity of crowdsourced AI benchmarking platforms like Chatbot Arena. Notable voices such as Emily Bender from the University of Washington argue that these benchmarks lack necessary construct validity. Critics claim these platforms can lead to exaggerated claims from AI labs, as seen with Meta’s Llama 4 Maverick model. They advocate for more dynamic, well-defined benchmarks and proper compensation for evaluators. Despite these critiques, some developers emphasize the importance of transparent community feedback in the evaluation process.