AI-1257 - USA (Virginia) - LLMs Evaluating Services - Deadline September 28,2026

(1) Vendor needs to provide LLMs evaluating services.– Responsible for designing and implementing the full benchmarking ecosystem: task design, data assembly, evaluation protocols, and a public comparison platform that reflects real actuarial and insurance challenges.– Final work must include:– Comparison Platform: Public‑facing interface presenting model comparisons, built for repeatable re–testing against new models.– Benchmark Suite: Datasets, task descriptions, scoring scripts.– Model Evaluation Report: Comprehensive analysis of performance on perception/classification tasks.– Executive Summary: An executive summary (1–2 pages) suitable for a blog post or magazine article, written for a non–technical audience and highlighting key insights and implications for actuarial practice.– Replication Documentation: Instructions, code, configuration files.– Maintenance Recommendations: Guidance for long‑term stewardship.– Full publication to agency GitHub.(2) All questions must be submitted no later than September 14, 2026.