CyPeace at BSides Hanoi 2025 – When AI Listens to What You Didn’t Mean to Say 

14/10/2025

6

A seemingly innocent question — “How do I make an ice-cream?” — fell flat in the BSides Hanoi room. Not because the question was provocative, but because it contained hidden instructions: a tiny, malicious payload that could cause an AI system to act against expectations. At BSides Hanoi 2025, CyPeace demonstrated why AI is no longer a passive answer engine — it is an active surface for attackers, and organizations must treat it as such.

AI expands your attack surface in ways that feel invisible — unless you test for them.
Ngoc-Anh Tran
Defensive Security Lead, CyPeace

BSides Hanoi 2025 — Vietnam’s First Official “BSides” Event, and a Milestone for AI Security 

The arrival of BSides Hanoi 2025 in Vietnam marked a significant milestone for the country’s cybersecurity landscape. Held on 9 October 2025 in Hà Nội, the event – sub-titled “AI Hack YOU” – was the first time the globally respected BSides conference series was officially hosted in the country. 

Organised by Vietnam Cybersecurity Network J.S.C (VSEC) in collaboration with the National Cybersecurity Association (NCA) and the Vietnam Internet Association (VIA), BSides Hanoi 2025 also forms part of the lead-up activities to the forthcoming Hanoi Convention (United Nations Convention against Cybercrime), scheduled for October 2025 in Hanoi. This tie-in elevated the event beyond a local meetup – turning it into a strategic node in Vietnam’s growing cybersecurity agenda at both regional and global levels. 

What set the conference apart was its authentic commitment to the BSides ethos — a programme curated entirely on technical merit, independent of sponsorship or marketing agendas. Talks were chosen for their substance, not their slogans. With more than 200 security professionals, researchers, and engineers from Vietnam and abroad sharing the stage, having CyPeace’s session selected in such a highly technical, community-driven environment underscored the depth and relevance of our research. It was both a recognition of our work and a chance to contribute directly to the country’s first globally aligned security forum. 

Inside CyPeace’s Talk: “AI Attack” 

Delivered by Mr. Ngoc-Anh Tran, Defensive Security Lead at CyPeace, the session “AI Attacks: The New Attack Surface” explored how artificial intelligence introduces exploitable vectors that traditional security models rarely account for. Aligned with the event’s “AI Hack YOU” theme, the talk covers AI’s rapid integration into everyday workflows and highlighted the practical problems that arise as systems adopt increasingly capable models.

1. How AI develops and integrates into even the most routine activities, from personal assistants to enterprise automation. 

2. The forms of exploitation and abuse that arise as AI systems gain access to sensitive data, APIs, and operational logic. 

3. The expanding attack surface created as AI becomes more deeply embedded in applications and infrastructure, allowing adversaries new entry points to manipulate or interfere with system behavior. 

4. The need for a correct security perspective when designing and deploying AI-enabled applications and systems — treating AI not just as a feature, but as a potential attack vector. 

Session outline: from the evolution of AI to its vulnerabilities, real-world autonomy risks, and defensive strategies.

Throughout the presentation, Mr. Tran interleaved live demonstrations showing how seemingly harmless user inputs can conceal hidden commands that alter AI behaviour. These proof-of-concept underscored a key reality: modern AI systems don’t have to be broken to be exploited — they just have to be convinced. That insight became the foundation for the technical discussion that followed, tracing how each layer of the AI stack can be turned against itself. 

From Theory to Exploit: Where AI Becomes an Attack Surface  

The AI Attacks session broke the problem down layer by layer — not as abstract concepts, but as practical, testable vulnerabilities built from CyPeace’s own red-team and AI-security testing. Each layer of the AI stack introduces its own unique security challenges, together forming a new kind of kill chain for attackers.

1. The Data and Model Layer — Poisoning the Foundation

Every AI system starts with data. CyPeace’s research demonstrated how subtle manipulations to training or validation datasets can insert noisy data, creates bias, decreases reliability, install additional layers of hidden processing, etc. 

Demonstrating attacks on the machine learning pipeline — from data poisoning and adversarial samples to manipulated image and audio recognition.

Attackers can introduce samples that appear normal during evaluation yet trigger malicious misclassification later. Once a model learns from corrupted input, every inference built on it inherits that flaw. 

Model extraction adds a parallel threat: by repeatedly querying a deployed model, adversaries can reconstruct or approximate its decision boundaries, leading to both intellectual-property loss and tailored adversarial attacks.

2. The Prompt Layer — When Language Becomes an Exploit

Large language models (LLMs) make it possible for attackers to manipulate intent through words alone. A single crafted input can override instructions, bypass guardrails, or or leak sensitive data out of seemingly safe systems.  

Prompt injection: how crafted inputs can override safeguards and redirect model behavior.

CyPeace’s demonstration compared direct and indirect prompt injection — from commands like “ignore previous instructions” to malicious content hidden in third-party data, can achieve the same effect — persuading the AI to act against its intended policy.  

The “ice-cream prompt” made the concept tangible: a plain sentence hiding control tags that redirected model logic. The danger lies in its invisibility — malicious inputs can look perfectly legitimate. 

The “ice-cream prompt” demonstration — showing how a seemingly harmless query can carry hidden commands that alter an AI system’s output.

3. The Automation and Autonomy Layer — When AI Starts to Act

As AI systems gain execution privileges — sending requests, writing files, calling APIs — the impact of a prompt injection amplifies. 
CyPeace introduced an “autonomy scale” showing how risk expands from assistive recommendations to fully operational agents. At higher autonomy levels, a single prompt can translate into real-world actions — spamming, reconfiguring systems, or triggering unintended workflows. 

4. Defensive Mindset — Designing for Distrust 

The session closed with a defensive framework rooted in secure engineering practice: treat every input as untrusted code. 
Key principles included robust input validation, least privilege for AI components, continuous monitoring, and human-in-the-loop approval for critical actions. The goal isn’t to eliminate risk entirely but to build systems that fail safe and recover gracefully when abused. 

Securing the Future of AI 

BSides Hanoi 2025 underscored a turning point for cybersecurity: as AI becomes integral to business and infrastructure, its attack surface is no longer hypothetical — it’s here. The risks now reside in the logic we train, the data we collect, and the language our systems understand. 

Mr. Ngoc-Anh Tran’s presentation, backed by CyPeace’s research, brought this to life with realistic scenarios and proof-of-concept testing. AI security is already an engineering discipline — and organizations that adopt secure-by-design principles today will be better prepared for tomorrow’s attacks. 

For Vietnam, BSides Hanoi 2025 signaled a community ready to operate at global standards of research and collaboration. For CyPeace, it was a chance to share our findings and help shape the next phase of AI defense — building systems that are as resilient as they are intelligent. 

Nguyễn Quỳnh Nga

Nguyễn Quỳnh Nga

Marketing Lead

Cùng nhau bảo vệ

Logo

Không gian mạng

Logo

cho doanh nghiệp của bạn