Researchers have discovered a new threat in the AI security landscape, dubbed 'HalluSquatting', which exploits large language models' inability to distinguish between legitimate and malicious instructions.
This vulnerability allows attackers to inject malicious commands into AI systems, potentially leading to large-scale botnet attacks.
The lack of a clear boundary between trusted and untrusted sources makes it challenging for AI engine developers to prevent such attacks, and most current measures only mitigate the damage rather than addressing the root cause.
