Masterclass Testing Machine Learning (AI) Models

Gain basic understanding how to test Foundation Models, LLMs, Chatbots and AI bases systems.

Created by: Dan Andrei Bucureanu
Quality Transformation Consultant

What Will I Learn?

• Understand how AI is working
• Understand basic software testing and how to apply it to Artificial Intelligence Models
• Understand how Machine Learning Models are tested compared to traditional software
• Understand the Ethics behind Artificial Intelligence (AI) and how to validate biases in Large Language Models ( LLMs)
• Understand how to test reasoning abilities of AI
• Gain knowledge of reasoning types and how to validate Foundation AI Models against different logic and reasoning Types
• Gain knowledge on what is Natural Language Processing, and what tools can be used to test for it in a Machine Learning Model
• See how to benchmark AI against models "HellaSWAG, MMLU, CODEXGLUE, BLEU, Humaneval"
• Importance of test data and how model drifting degrades Large Language Model Performance
• See how chatbots can be tested with real Chatgpt examples
• Demo on testing Chat GPT with automated Tools
• Understand Adversial Testing techniques - Gain knowhow on how to perform attacks on Artificial Intelligence Models
• Understand the common and traditional metrcs what are used in Machine Learning field, such as F1 Score, Perplexity Recall and Accuracy

Requirements

• basic experience with software testing
• basic coding experience ( but not needed)
• optional - GPT model 4 subscription (but not needed)
• desire to learn the hottest skill on the market
• Basic programming in python
• VS Code, A mac/pc, OPEN AI API Key - not mandatory but desirable

Target audience

• Citizen Developer
• Software testers
• quality engineers
• Engineering Directors