SW-117274 - USA (Texas) - Artificial Intelligence Agent Testing and Evaluation Platform - Deadline August 31,2026

(1) Vendor needs to provide artificial intelligence agent testing and evaluation platform. – Provide an enterprise grade AI Agent Testing and Evaluation solution to support the testing, evaluation, and continuous improvement of artificial intelligence–driven agents, including conversational chatbots and virtual assistants. – Support pre–deployment testing, post–deployment monitoring, and ongoing quality assurance of AI agents to improve reliability, user experience, and governance.– Provide a Software–as–a–Service (SaaS) or equivalent commercial solution that enables agency to test, evaluate, and monitor AI agent responses across development, testing, and production environments.• Work includes:– Testing conversational and chat–based AI agent.– Testing task–based Agents (agentic AI).– Evaluating response accuracy, completeness, consistency, and relevance.– Evaluating semantic accuracy and response quality.– Identifying gaps between intended and actual responses.– Detecting response errors, hallucinations, policy inconsistencies, or unintended behavior.– Evaluate completion of agents' task/workflow.– Capture and verify the source citation links for response verification against a knowledge source.– Providing performance dashboards and reporting.– Supporting continuous improvement and regression analysis.• Functional requirements:– The solution must provide automated or semiautomated evaluation of AI agent responses.– Auto generated prompts and expected responses based on the knowledge source(s) for testing teams to review and use for testing.– Evaluation based on semantic meaning rather than exact text matching.– Semantic response analysis – The solution must leverage AI techniques to:– Compare the meaning and intent of actual AI responses to predefined expected outcomes– Highlight semantic gaps, omissions, or deviations in responses.• Dashboards and reporting: The solution must include dashboards and reporting capabilities.• Performance, load and stress analysis: solution must include automated solution to capture performance of agents under testing.• Integrate with AI agents through APIs, logs, transcripts, or other supported mechanisms.(2) All the questions must be submitted no later than August 14, 2026. (3) The contract period will be for one year. (4) Non–mandatory pre–bid conference will be held on August 11, 2026.