Generative AI security risks

Generative Artificial Intelligence (GenAI) has emerged as a transformative technology, driving innovation across industries through advanced content generation, intelligent automation, software development, and decision support systems. The rapid evolution of Large Language Models (LLMs), multimodal foundation models, Retrieval-Augmented Generation (RAG), and autonomous AI agents has created unprecedented opportunities while simultaneously introducing new cybersecurity challenges. These AI systems are increasingly targeted by sophisticated attacks such as prompt injection, adversarial manipulation, model poisoning, data leakage, model theft, jailbreak attacks, hallucination exploitation, and AI-generated cyber threats. As organizations integrate GenAI into mission-critical applications, ensuring the confidentiality, integrity, availability, and trustworthiness of AI systems has become a fundamental research and industrial priority.

The International Conference on Cyber Security in Artificial Intelligence (ICCSAI 2027) invites original research contributions addressing the security, privacy, resilience, governance, and ethical challenges associated with Generative AI technologies. This track aims to provide an interdisciplinary platform for researchers, cybersecurity professionals, AI developers, industry practitioners, and policymakers to exchange innovative ideas, emerging technologies, practical solutions, and best practices for securing next-generation AI systems. Contributions covering theoretical foundations, novel security frameworks, attack detection, defense mechanisms, secure AI architectures, AI governance, regulatory compliance, and real-world case studies are particularly encouraged.

1. Foundation Model Security

• Security of Large Language Models (LLMs)

• Foundation Model Vulnerabilities

• Secure Fine-Tuning and Model Adaptation

• Model Alignment and Safety

• AI Model Robustness

2. Prompt and Input Security

• Prompt Injection Attacks

• Jailbreak Techniques and Defenses

• Prompt Leakage and System Prompt Protection

• Adversarial Prompting

• Secure Prompt Engineering

3. Adversarial AI Attacks

• Adversarial Machine Learning

• Evasion Attacks on Generative AI

• Model Poisoning Attacks

• Training Data Poisoning

• Backdoor Attacks on AI Models

4. Privacy and Data Security

• Privacy-Preserving Generative AI

• Sensitive Data Leakage

• Membership Inference Attacks

• Model Inversion Attacks

• Differential Privacy for Generative AI

5. AI Model Theft and Intellectual Property

• Model Extraction Attacks

• Model Stealing and Cloning

• Intellectual Property Protection

• Secure Model Sharing

• AI Watermarking and Fingerprinting

6. AI Supply Chain Security

• Secure AI Development Lifecycle

• AI Software Supply Chain Security

• Third-Party Model Risk Management

• Secure Model Distribution

• Dependency and Package Security

7. Retrieval-Augmented Generation (RAG) Security

• Secure RAG Architectures

• Vector Database Security

• Retrieval Poisoning Attacks

• Knowledge Base Integrity

• Secure Context Management

8. AI Agent Security

• Autonomous AI Agent Security

• Multi-Agent System Security

• Agent Authentication and Authorization

• Secure AI Decision-Making

• Agent Collaboration Security

9. AI-Generated Cyber Threats

• AI-Generated Malware

• AI-Assisted Phishing Attacks

• Deepfake Detection and Prevention

• Synthetic Identity Fraud

• AI-Enabled Social Engineering

10. AI for Cyber Defense

• AI-Based Threat Detection

• AI-Driven Intrusion Detection Systems

• AI-Assisted Security Operations (SOC)

• AI for Threat Intelligence

• Automated Incident Response

11. Explainability and Trustworthy AI

• Explainable AI (XAI) for Security

• Trustworthy Generative AI

• Hallucination Detection and Mitigation

• AI Transparency and Accountability

• Human-in-the-Loop Security

12. Secure AI Infrastructure

• Secure MLOps and AI DevSecOps

• Cloud AI Security

• Edge AI Security

• AI API Security

• Secure AI Deployment

13. Governance and Compliance

• AI Risk Assessment

• AI Governance Frameworks

• Regulatory Compliance for AI Systems

• AI Security Standards

• Ethical and Responsible AI

14. Emerging Challenges

• Security of Multimodal Generative AI

• Quantum-Resistant AI Security

• Federated Learning Security

• Digital Content Provenance

• Zero-Trust AI Architectures

• Resilient and Self-Healing AI Systems

• Security Evaluation and Benchmarking of Generative AI

• AI Security Testing, Validation, and Red Teaming