From the course: CompTIA SecAI+ (CY0-001) Cert Prep

Unlock this course with a free trial

Join today to access over 25,500 courses taught by industry experts.

Introducing biases

Introducing biases

From the course: CompTIA SecAI+ (CY0-001) Cert Prep

Start my 1-month free trial Buy for my team

Introducing biases

“

Bias introduction is an attack where someone influences a model to produce unfair, unbalanced, or harmful outputs. This may be done as part of an intentional attack or it may be an unintentional part of the model development process. All AI models reflect their training data. An attacker can manipulate that data or the model's interactions to push outcomes in a specific direction. Let's talk about how this might work in a couple of case studies. First, imagine a sentiment analysis model trained on public reviews. An attacker might submit thousands of fake comments that associate certain ethnic names with negative language. Over time, the model begins to link those names with lower sentiment scores, even when the surrounding text remains neutral. Second, imagine an AI-driven hiring tool used by a human resources team. There, attackers could flood the system with fabricated resumes that skew data in favor of a particular school or gender, shifting how the model ranks future candidates…

Contents