Testing and evaluating AI-powered capabilities in mission-critical situations

Exploring the adoption of AI-enabled systems at the US Department of the Air Force

Please join Charlottesville Data Science for the talk Testing and Evaluating AI-powered Capabilities in Mission-Critical Situations from May Casterline! We'll be gathering in person at the Castle Hill Gaming office by the Shops at Stonefield.

About the talk

Many organizations are exploring integrating modern AI capabilities throughout their systems and operations. Organizations that work in high-stakes or "mission-critical" situations, however, face distinctive challenges when adopting AI technology. These organizations often have robust, long-standing policies and practices governing how to test and evaluate new tools and technologies to ensure they are reliable, safe, and effective. Given the "black box", self-learning, adaptive, and data-centric nature of AI, existing testing and evaluation procedures usually do not fully translate to new, AI-enabled systems. To build sufficient levels of confidence to deploy AI capabilities in mission-critical scenarios, organizations need to develop and promote new, AI-specific test and evaluation procedures.

May Casterline recently co-chaired a study for the National Academies of Sciences, Engineering, and Medicine exploring the adoption of AI-enabled systems at the US Department of the Air Force. In this talk, May will share and summarize her committee's findings and recommendations about how the Air Force can develop and field AI effectively, safely, and responsibly. Many of these insights may apply more broadly across any organization trying to harness the potential of AI in high-stakes, mission-critical situations.

About the speaker

May Casterline is a Principal Solutions Architect at NVIDIA in Charlottesville, Virginia. Her research interests include deep learning, novel modeling and visualization techniques for data exploitation, innovative machine learning applications to imagery data, data workflow design, and creative, distributed software solutions to challenging image processing problems. May earned both her doctorate and bachelor's degree in Imaging Science from the Rochester Institute of Technology.