The Scenario

Ananya is the founder of a small fintech startup in Hyderabad. In early 2025, she received a voice call from what appeared to be her bank’s relationship manager, requesting verbal authorisation to release funds from a current account for a vendor payment. The voice was familiar — warm, specific to her account, and contextually accurate. She authorised the transfer.

The call was a deepfake. An AI-generated voice, synthesised from recordings of the actual relationship manager’s previous interactions with Ananya and other clients, had been used to impersonate the banker with sufficient realism to bypass Ananya’s verification instincts. The transfer of ₹28 lakh was irreversible by the time the fraud was detected.

Investigation revealed that the relationship manager’s voice recordings — captured across customer service calls — had been exfiltrated from the bank’s call recording system in a prior data breach. Those recordings contained personal data: the manager’s voice, name, customer account references, and contextual details that made the synthesised interaction credible.

The DPDP Act Position

This scenario operates at the intersection of two DPDP Act obligations: the Data Fiduciary’s duty to prevent personal data breach, and the consequence of that breach being weaponised through generative AI.

First — Personal data breach enabling the deepfake under Section 2(u) and Section 8(5). The exfiltration of call recordings from the bank’s system was a personal data breach under Section 2(u) — unauthorised acquisition of personal data that compromised its confidentiality. Section 8(5) of the DPDP Act requires the Data Fiduciary to take reasonable security safeguards to prevent personal data breach. Rule 6 of the DPDP Rules specifies that these safeguards must include encryption or obfuscation of personal data, access controls over computer resources, and monitoring logs that enable detection of unauthorised access. If call recordings containing voice biometrics and customer account details were stored without adequate encryption, without access controls, and without monitoring — the bank failed its Section 8(5) obligation before a single deepfake was generated.

Second — Failure to classify voice biometrics as high-sensitivity personal data. Voice recordings that contain sufficient data to synthesise an individual’s voice are personal data under Section 2(t) of the DPDP Act — they are data about an identifiable individual. When used to train or feed a generative AI voice synthesis model, they become the direct input to a process that can impersonate that individual. The sensitivity of this category of personal data is therefore significantly higher than a simple audio archive might suggest, and the security architecture protecting it must reflect that sensitivity.

Third — Section 8(6) and Rule 7 breach notification failure. The prior data breach that enabled the deepfake should have triggered a notification obligation under Section 8(6) and Rule 7 — notification to the Data Protection Board and to each affected Data Principal, without delay, upon the bank becoming aware of the breach. Had notification occurred promptly, Ananya — and other customers whose interaction data was in the exfiltrated set — would have been on alert. The window for the deepfake fraud would have narrowed significantly.

Fourth — The deepfake generation itself as a further processing violation. The attacker who synthesised the relationship manager’s voice used the bank’s exfiltrated call data to train or prompt a voice generation model. Every step of that process — acquiring the recordings, training the model, generating the synthesised voice — constituted processing of personal data under Section 2(x) of the DPDP Act. This processing had no lawful basis, no consent, and no legitimate purpose. It was, by definition, unlawful processing — and the Data Fiduciary whose security failure enabled it bears legal responsibility under Section 8(5) for creating the conditions in which it occurred.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 4.2) explicitly names deepfake voice and video fraud as a primary AI-assisted threat vector, noting that AI technologies are increasingly used to generate highly convincing impersonation attempts and deepfake-enabled fraud that bypass traditional awareness-based detection due to their realism, contextual accuracy, and personalisation. Section 11 of the same document requires organisations to build deepfake and impersonation detection readiness, including verification procedures, executive impersonation monitoring, and escalation mechanisms.

Applied to banking and financial services, this means that verbal authorisation processes — for fund transfers, account changes, or contract executions — must not rely on voice alone as a verification factor in an era of generative voice synthesis. CERT-In’s framework, read alongside the DPDP Act’s security safeguard obligations, requires that financial institutions redesign their authorisation architectures to account for the synthetic impersonation risk that generative AI now makes operationally accessible to attackers.

The IS Audit Perspective

IS Audit Standard 420 (ICAI ISAS, Section 4.3) identifies opacity, embedded bias, and non-deterministic outcomes as risks specific to AI systems. In the context of voice synthesis, the opacity risk runs in two directions: the attacker’s model is opaque to the victim, and the bank’s internal monitoring was opaque to the attack. IS Audit Standard 420 requires that professionals maintain professional scepticism when using automated AI outputs and ensure that sensitive data handling meets privacy and legal requirements.

IS Audit 3.0 study material identifies data privacy and security — specifically the risk of ML systems being prone to data breach and identity theft — as a primary AI governance concern. For financial institutions, this risk now extends to the use of their customers’ and employees’ voice data as raw material for generative AI fraud.

The Compliance Fix

Banks, financial institutions, and any organisation that holds voice recordings or biometric personal data must urgently:

→ Classify all voice recordings containing sufficient data to enable voice synthesis as high-sensitivity personal data and apply maximum security standards under Rule 6: encryption at rest and in transit, access control scoped to minimum necessary, and comprehensive monitoring.

→ Implement Rule 7 breach notification workflows that specifically cover the exfiltration of voice and biometric data, with immediate notification to the Data Protection Board and affected Data Principals, without delay.

→ Redesign verbal authorisation processes for financial transactions: multi-factor authentication that cannot be bypassed by synthetic voice alone — including callback protocols, one-time codes, and out-of-band verification — must replace or supplement voice-only authorisation.

→ Build deepfake detection capabilities, as required by CERT-In CIGU20260002 (Section 11), for customer-facing and internal voice channels. AI-generated voices have detectable characteristics that real-time or near-real-time analysis can flag.

→ Train operational staff — particularly relationship managers and customer-facing teams — on the reality of AI voice synthesis fraud, the verification procedures to apply, and the escalation channels available when synthetic identity suspicion arises.

Ananya lost ₹28 lakh. The bank lost her trust. Both losses were preventable — if the bank had treated its call recording archive as the high-sensitivity personal data it was, and had built a security architecture proportionate to that sensitivity.

The compliance deadline is 13 May 2027. Voice data is personal data. Biometric impersonation through generative AI is a foreseeable risk. The DPDP Act requires organisations to design security against foreseeable risks — not merely react to them after loss.

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.