DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

September 22, 2026 Karthik & Sunil#DataPrivacyIndia, #DPDPAct, #DPDPCompliance, #EmergingTechnologies, #GenerativeAI, Karthik & SunilLeave a comment

The Scenario

Kiran is a 38-year-old consumer goods marketing manager in Chennai. In October 2024, she used the customer service chatbot of a major e-commerce platform to lodge a complaint about a defective product. During the conversation, she disclosed her name, order details, delivery address, phone number, and — because the issue involved a medication she had purchased — the name of a prescription drug and the reason she needed it urgently.

Unknown to Kiran, the platform’s terms of service contained a clause stating that “conversations with our assistant may be used to improve our AI services.” This clause was buried in a sixteen-page document she had agreed to at account registration, three years before the chatbot was deployed.

Twelve months later, Kiran received a targeted advertisement for a competing medication — delivered through a third-party advertising platform — that directly corresponded to the medication she had named in her complaint conversation. She had never searched for this medication online. The only digital record of its relevance to her was the chatbot conversation.

Investigation revealed that the e-commerce platform had been using customer service conversations — including Kiran’s — as fine-tuning data for its next generation customer service AI model, and had shared anonymised conversation datasets with an advertising technology partner. The anonymisation had been superficial: name and order ID had been removed, but the medication name, delivery address, and phone number — when cross-referenced against the advertising partner’s own data — were sufficient to re-identify Kiran.

The DPDP Act Position

Kiran’s situation is the most layered of the three stories because it involves three cascading compliance failures — consent fragmentation, inadequate anonymisation, and downstream re-identification — all enabled by generative AI training practices.

First — Buried terms of service do not constitute valid consent under Section 6(1). Section 6(1) of the DPDP Act requires consent to be free, specific, informed, unconditional, and unambiguous. Rule 3 of the DPDP Rules requires the notice to be presented and understandable independently of any other information — meaning it must stand alone, in clear and plain language, without requiring the Data Principal to extract relevant information from a sixteen-page document. A clause buried in terms of service agreed to three years before a chatbot was deployed does not give specific, informed consent to the use of chatbot conversations for AI model training. These are meaningfully different processing activities and require meaningfully different consent.

Furthermore, Section 6(1) limits consent to personal data necessary for the specified purpose. Kiran agreed to an account creation purpose. The collection of her complaint conversation for AI training is a different purpose — and the personal data involved, particularly health-related information, demands its own specific consent basis.

Second — The re-identification of pseudonymised data is a personal data breach under Section 2(u). The e-commerce platform argued that the data shared with its advertising partner had been anonymised. However, true anonymisation — as recognised in IS Audit Standard 430 (Section 4.6) — means that the data cannot be linked back to an individual through any means reasonably likely to be used. Removing a name and order ID while retaining a medication name, delivery address, and phone number does not constitute anonymisation when those elements, in combination, can be cross-referenced against other datasets to identify the individual.

When the advertising partner re-identified Kiran using this cross-referencing — evidenced by the targeted advertisement — a personal data breach occurred under Section 2(u): accidental or unauthorised disclosure of personal data that compromises its confidentiality. The platform bears responsibility for this breach under Section 8(1), because the sharing with the advertising partner was processing carried out on its behalf.

Third — Health information processed without specific consent. Kiran disclosed the name of a prescription medication — personal data directly relating to her health status. This is the most sensitive category of personal data in the context of the DPDP Act’s proportionality framework. Processing it for AI model training, without a specific notice disclosing health information as a training data category, and without specific consent covering that use, is processing without a lawful basis under Section 4(1). The advertising outcome — targeted medication advertising — demonstrates that this sensitive data was not merely processed for training: it was used to drive commercial targeting that affected Kiran directly.

Fourth — Third-party data sharing as undisclosed Data Processor engagement. Section 8(2) requires that every Data Processor engaged by the Data Fiduciary process personal data only for the specified purpose under a valid contract. The advertising technology partner received Kiran’s personal data for AI training purposes. However, by using it for advertising targeting — a different purpose — it operated outside its Data Processor mandate. Under Section 8(1), the e-commerce platform bears liability for its Data Processor’s actions. The inadequacy of the Data Processor contract — which apparently did not restrict downstream use to the training purpose — is a governance failure attributable to the platform.

The CERT-In and IS Audit Dimensions

CERT-In CIGU20260002 (May 2026, Section 12, Area 7) identifies data protection and privacy in AI systems as a governance priority requiring organisations to classify and protect sensitive data, define retention and deletion policies, and monitor AI-related data flows. The platform’s failure to classify health information disclosed in service conversations as sensitive personal data — and to restrict its downstream use accordingly — is precisely the governance gap CERT-In’s framework addresses.

IS Audit Standard 430 by ICAI requires professionals conducting DPDP audits to have technical knowledge and competence covering profiling of digital personal data, pseudonymisation, and anonymisation. The distinction between true anonymisation and pseudonymisation that can be reversed through cross-referencing is a core technical competence in this standard — and a core compliance obligation for any organisation using AI training datasets derived from customer interactions.

IS Audit Standard 420 notes that data lineage, completeness, transformations, and regulatory conditions must be assessed to determine whether technology-driven analyses produce valid outputs and whether data handling meets privacy and legal expectations. Kiran’s data travelled from a chatbot conversation through anonymisation, to a training dataset, to an advertising partner, and back to Kiran in the form of a targeted advertisement — a data lineage of five steps, none of which was disclosed to her, and none of which was covered by a lawful consent.

The Compliance Fix

E-commerce platforms, AI product companies, and any organisation using customer interaction data for generative AI training must:

→ Issue separate, specific consent notices — not buried terms of service — for the use of customer interaction data for AI model training. This notice must identify: the data categories (including any health information that may arise in service conversations), the training purpose, and the identity of any downstream partners who will receive training data.

→ Apply genuine anonymisation — not pseudonymisation — to any data shared with AI training partners. IS Audit Standard 430 defines anonymisation as the removal or modification of identifiable information so that it cannot be linked back to an individual. Re-identification through cross-referencing means the anonymisation was inadequate.

→ Implement Data Processor contracts under Section 8(2) that restrict downstream partners to the training purpose only, with explicit prohibition on using training datasets for advertising targeting, profiling, or any purpose beyond the stated training objective.

→ Build data lineage tracking for all AI training pipelines: every piece of personal data entering a training pipeline must be traceable to a specific consent record, a specific data category disclosure, and a specific purpose. If the lineage cannot be demonstrated, the processing has no lawful basis.

→ Apply AI data usage controls specifically to health information — flagging, quarantining, or excluding health-related disclosures from training datasets unless specific, separate consent has been obtained for that category.

→ Treat the re-identification of pseudonymised training data as a personal data breach under Section 2(u), and implement breach notification workflows accordingly.

Kiran disclosed a medical need in a service conversation. That conversation built a model, fed an advertising engine, and returned to her as a targeted advertisement. The DPDP Act calls every step of that journey processing — and the platform was the Data Fiduciary responsible for each one.

The Central Compliance Question for Generative AI Deployments

Generative AI presents a compliance challenge that is qualitatively different from every other technology in this series. With blockchain, the problem was that data could not be deleted. With IoT, the problem was that data was collected without awareness. With RPA, the problem was that automated processing created invisible data footprints. With data analytics, the problem was that consent architectures were outpaced by analytical capability.

With generative AI, all of these problems appear simultaneously — and a new one is added: the model can generate outputs that reveal, infer, or recreate personal data that was never explicitly in its prompt. IS Audit Standard 420 (ICAI ISAS) acknowledges this directly: generative AI models are non-deterministic. Their outputs cannot always be predicted, traced, or reproduced verbatim. IS Audit 3.0 course material by ICAI identifies this mathematical uncertainty as a governance gap that organisations have not yet resolved.

The DPDP Act does not exempt non-deterministic outputs from its coverage. Every output of a generative AI system that contains personal data — however that output was generated — is a processing event subject to the Act’s full consent, notice, and accountability framework.

Every organisation deploying generative AI on personal data must therefore answer six questions before and continuously during deployment:

One. What personal data is accessible to the model as context, training data, or retrieval input? Is every data category covered by a specific, itemised notice and a lawful consent under Section 5 and Rule 3?

Two. Does the model’s output filtering architecture prevent personal data from appearing in responses to users who are not authorised to receive it?

Three. Have we conducted a DPIA under Rule 13 that specifically assesses the risk of unintended personal data surfacing in AI-generated outputs — and the risk of model outputs being used to re-identify individuals from pseudonymised inputs?

Four. Are all generative AI vendors and downstream AI processing partners governed by Section 8(2) contracts limiting them to specified purposes — with explicit restrictions on using processed data for model training, advertising, or any purpose beyond the stated scope?

Five. Do we have audit logging at the inference level — recording every query and response in a form that allows a DPDP Act compliance review or breach investigation to trace which personal data appeared in which output?

Six. Have we assessed our generative AI deployment against CERT-In’s CIGU20260002 framework for AI system security — covering prompt injection risks, sensitive data leakage, training data integrity, and adversarial threats to AI inference systems?

IS Audit 3.0 course material by ICAI frames this governance requirement clearly: AI governance must establish accountability and oversight, ensure that responsible parties have the necessary skills and expertise, and ensure that AI activities result in decisions and actions aligned with the ethical, social, and legal responsibilities of the organisation. Under the DPDP Act, those legal responsibilities include every Data Principal whose personal data touches a generative AI system.

What Responsible Generative AI and DPDP Compliance Looks Like

The organisations that will navigate generative AI responsibly under the DPDP Act are not those with the most capable models. They are those that ask the hardest questions before deployment — and build governance architectures that survive those questions.

Responsible generative AI and DPDP compliance means:

Scoping the model’s data access to what is necessary. Issuing specific notices for every processing purpose the model serves. Building output controls that prevent unintended data surfacing. Logging every inference for compliance accountability. Treating AI vendors as Data Processors with binding contracts. Conducting DPIAs before deployment, not after incidents. And applying CERT-In’s AI security framework as a complement to DPDP compliance — because the same technology that poses a compliance risk inside the organisation is simultaneously being used by adversaries to attack it from outside.

The Puttaswamy judgment (2018) described the era we live in with extraordinary prescience: humans forget, but the internet does not. Data mining and knowledge discovery processes create new knowledge about individuals — including facts they did not themselves possess. In 2018, that observation described the risks of data analytics. In 2026, it describes generative AI with even greater precision. A well-trained LLM has absorbed patterns from personal data at a scale no previous technology has matched — and can generate outputs that reflect those patterns in ways no individual who contributed that data ever anticipated or agreed to.

The DPDP Act exists to govern that reality. The compliance deadline is 13 May 2027. The governance work begins now.

The Series in Retrospect: A Unified Compliance Principle

This is the seventh and final episode of DPDP Meets Emerging Technologies. Together, the seven episodes have covered AI, Cloud Computing, Blockchain, IoT, Data Analytics, RPA, and Generative AI — seven technologies that are transforming India’s digital economy, and seven compliance frontiers that the DPDP Act governs with equal force.

Looking across all seven episodes, one principle has run through every story, every violation, and every compliance fix: technology changes the mechanism of data processing, but it does not change the law that governs it.

Blockchain made data immutable — the erasure right still applied. IoT made data collection invisible — the consent requirement still applied. RPA made data processing automated — the Data Fiduciary’s liability still applied. Data analytics made inferences from data — those inferences were still personal data. And generative AI makes outputs non-deterministic — the accountability framework still applies.

The DPDP Act, 2023 was designed precisely for this reality. It does not enumerate technologies. It governs processing — in whatever form, at whatever scale, by whatever means. The definition in Section 2(x) is deliberately broad: any wholly or partly automated operation on digital personal data. No technology sits outside it.

The Series is Complete. Seven Technologies. Seven Compliance Frontiers. One Deadline.

Over the past seven episodes, DPDP Meets Emerging Technologies has examined how the Digital Personal Data Protection Act, 2023 applies to the technologies reshaping India’s digital economy.

Episode 1 — AI and DPDP: When the algorithm decides without consent.
Episode 2 — Cloud Computing and DPDP: When your data lives on someone else’s server.
Episode 3 — Blockchain and DPDP: When the ledger never forgets.
Episode 4 — IoT and DPDP: When every device becomes a witness.
Episode 5 — Data Analytics and DPDP: When the numbers know too much.
Episode 6 — RPA and DPDP: When the bot processes more than it should.
Episode 7 — Generative AI and DPDP: When the machine learns what it should not know.

Across all seven stories, one principle remained constant: technology changes the mechanism of data processing. The DPDP Act governs the processing, regardless of mechanism.

The compliance deadline is 13 May 2027. Every Indian organisation that processes personal data through any of these technologies is already in scope — today.

If your organisation needs a DPDP compliance assessment, a technology-specific privacy architecture review, or a staff awareness programme covering these emerging technology compliance risks, we are here to help.

📩 Reach us at: karthikandsunil@karthikandsunil.in
🔗 Full series:

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors: This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.7.2: DPDP & Generative AI – Ananya’s Voice Note and the Deepfake That Signed a Contract

September 17, 2026 Karthik & Sunil#DataFiduciary, #DataPrivacy, #DeepfakeFraud, #DPDPAct, #GenerativeAI, #VoiceBiometrics, Karthik & SunilLeave a comment

The Scenario

Ananya is the founder of a small fintech startup in Hyderabad. In early 2025, she received a voice call from what appeared to be her bank’s relationship manager, requesting verbal authorisation to release funds from a current account for a vendor payment. The voice was familiar — warm, specific to her account, and contextually accurate. She authorised the transfer.

The call was a deepfake. An AI-generated voice, synthesised from recordings of the actual relationship manager’s previous interactions with Ananya and other clients, had been used to impersonate the banker with sufficient realism to bypass Ananya’s verification instincts. The transfer of ₹28 lakh was irreversible by the time the fraud was detected.

Investigation revealed that the relationship manager’s voice recordings — captured across customer service calls — had been exfiltrated from the bank’s call recording system in a prior data breach. Those recordings contained personal data: the manager’s voice, name, customer account references, and contextual details that made the synthesised interaction credible.

The DPDP Act Position

This scenario operates at the intersection of two DPDP Act obligations: the Data Fiduciary’s duty to prevent personal data breach, and the consequence of that breach being weaponised through generative AI.

First — Personal data breach enabling the deepfake under Section 2(u) and Section 8(5). The exfiltration of call recordings from the bank’s system was a personal data breach under Section 2(u) — unauthorised acquisition of personal data that compromised its confidentiality. Section 8(5) of the DPDP Act requires the Data Fiduciary to take reasonable security safeguards to prevent personal data breach. Rule 6 of the DPDP Rules specifies that these safeguards must include encryption or obfuscation of personal data, access controls over computer resources, and monitoring logs that enable detection of unauthorised access. If call recordings containing voice biometrics and customer account details were stored without adequate encryption, without access controls, and without monitoring — the bank failed its Section 8(5) obligation before a single deepfake was generated.

Second — Failure to classify voice biometrics as high-sensitivity personal data. Voice recordings that contain sufficient data to synthesise an individual’s voice are personal data under Section 2(t) of the DPDP Act — they are data about an identifiable individual. When used to train or feed a generative AI voice synthesis model, they become the direct input to a process that can impersonate that individual. The sensitivity of this category of personal data is therefore significantly higher than a simple audio archive might suggest, and the security architecture protecting it must reflect that sensitivity.

Third — Section 8(6) and Rule 7 breach notification failure. The prior data breach that enabled the deepfake should have triggered a notification obligation under Section 8(6) and Rule 7 — notification to the Data Protection Board and to each affected Data Principal, without delay, upon the bank becoming aware of the breach. Had notification occurred promptly, Ananya — and other customers whose interaction data was in the exfiltrated set — would have been on alert. The window for the deepfake fraud would have narrowed significantly.

Fourth — The deepfake generation itself as a further processing violation. The attacker who synthesised the relationship manager’s voice used the bank’s exfiltrated call data to train or prompt a voice generation model. Every step of that process — acquiring the recordings, training the model, generating the synthesised voice — constituted processing of personal data under Section 2(x) of the DPDP Act. This processing had no lawful basis, no consent, and no legitimate purpose. It was, by definition, unlawful processing — and the Data Fiduciary whose security failure enabled it bears legal responsibility under Section 8(5) for creating the conditions in which it occurred.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 4.2) explicitly names deepfake voice and video fraud as a primary AI-assisted threat vector, noting that AI technologies are increasingly used to generate highly convincing impersonation attempts and deepfake-enabled fraud that bypass traditional awareness-based detection due to their realism, contextual accuracy, and personalisation. Section 11 of the same document requires organisations to build deepfake and impersonation detection readiness, including verification procedures, executive impersonation monitoring, and escalation mechanisms.

Applied to banking and financial services, this means that verbal authorisation processes — for fund transfers, account changes, or contract executions — must not rely on voice alone as a verification factor in an era of generative voice synthesis. CERT-In’s framework, read alongside the DPDP Act’s security safeguard obligations, requires that financial institutions redesign their authorisation architectures to account for the synthetic impersonation risk that generative AI now makes operationally accessible to attackers.

The IS Audit Perspective

IS Audit Standard 420 (ICAI ISAS, Section 4.3) identifies opacity, embedded bias, and non-deterministic outcomes as risks specific to AI systems. In the context of voice synthesis, the opacity risk runs in two directions: the attacker’s model is opaque to the victim, and the bank’s internal monitoring was opaque to the attack. IS Audit Standard 420 requires that professionals maintain professional scepticism when using automated AI outputs and ensure that sensitive data handling meets privacy and legal requirements.

IS Audit 3.0 study material identifies data privacy and security — specifically the risk of ML systems being prone to data breach and identity theft — as a primary AI governance concern. For financial institutions, this risk now extends to the use of their customers’ and employees’ voice data as raw material for generative AI fraud.

The Compliance Fix

Banks, financial institutions, and any organisation that holds voice recordings or biometric personal data must urgently:

→ Classify all voice recordings containing sufficient data to enable voice synthesis as high-sensitivity personal data and apply maximum security standards under Rule 6: encryption at rest and in transit, access control scoped to minimum necessary, and comprehensive monitoring.

→ Implement Rule 7 breach notification workflows that specifically cover the exfiltration of voice and biometric data, with immediate notification to the Data Protection Board and affected Data Principals, without delay.

→ Redesign verbal authorisation processes for financial transactions: multi-factor authentication that cannot be bypassed by synthetic voice alone — including callback protocols, one-time codes, and out-of-band verification — must replace or supplement voice-only authorisation.

→ Build deepfake detection capabilities, as required by CERT-In CIGU20260002 (Section 11), for customer-facing and internal voice channels. AI-generated voices have detectable characteristics that real-time or near-real-time analysis can flag.

→ Train operational staff — particularly relationship managers and customer-facing teams — on the reality of AI voice synthesis fraud, the verification procedures to apply, and the escalation channels available when synthetic identity suspicion arises.

Ananya lost ₹28 lakh. The bank lost her trust. Both losses were preventable — if the bank had treated its call recording archive as the high-sensitivity personal data it was, and had built a security architecture proportionate to that sensitivity.

The compliance deadline is 13 May 2027. Voice data is personal data. Biometric impersonation through generative AI is a foreseeable risk. The DPDP Act requires organisations to design security against foreseeable risks — not merely react to them after loss.

Disclaimer

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

September 12, 2026 Karthik & Sunil#DataPrivacy, #DPDPAct, #GenerativeAI, #LLMCompliance1 Comment

INTRODUCTION

Generative AI and DPDP compliance represent the most consequential frontier in India’s data protection landscape — and the one that is evolving fastest.

Generative AI refers to systems — large language models (LLMs), image generators, voice synthesisers, code generators, and multimodal models — that are trained on vast quantities of data and can produce new content: text, images, audio, video, and code that did not previously exist. These systems are already embedded in enterprise workflows across India: drafting documents, summarising records, answering customer queries, analysing contracts, generating reports, and supporting HR processes.

IS Audit 3.0 by ICAI identifies data privacy and security as a primary AI risk, noting that most AI applications depend on massive volumes of data — often sensitive and personal in nature — to learn and make intelligent decisions. As machine learning systems become more capable, they also become more capable of learning, retaining, and inadvertently reproducing the personal data on which they were trained or with which they were prompted.

IS Audit Standard 420 (ICAI ISAS) goes further, acknowledging that generative AI models present non-deterministic outputs — meaning the same input may produce different outputs across sessions. For AI models with these characteristics, it notes, reproducibility requires that parameters, prompts, and methodology be documented sufficiently to allow independent verification. This observation is not merely an audit standard. It is a compliance architecture requirement: if a generative AI system cannot reproduce how a personal data output was generated, the Data Fiduciary deploying it cannot adequately defend its processing decisions under the DPDP Act.

CERT-In’s Blueprint on AI-Assisted Cyber Threats (CIGU20260002, May 2026) adds a threat dimension that no organisation can afford to ignore: generative AI is being actively leveraged by threat actors to generate highly convincing phishing content, deepfake voice and video fraud, synthetic identities, and spear phishing campaigns that bypass traditional awareness-based detection — all of which directly target personal data of Indian citizens. Consequently, generative AI is not merely a compliance risk as a tool of enterprise productivity. It is simultaneously the instrument of some of the most sophisticated personal data threats India’s digital ecosystem has ever faced.

Against this backdrop, the DPDP Act, 2023 applies with full force — to every organisation that deploys generative AI on personal data, to every generative AI vendor that processes personal data on behalf of Indian organisations, and to every system that generates outputs touching identifiable individuals. The compliance deadline is 13 May 2027.

Three stories ground this in the reality of Indian organisations today.

Story 1 — Vikram’s HR Query and the LLM That Remembered Everyone

The Scenario

Vikram is an HR manager at a large IT services firm in Bengaluru. His company had deployed a generative AI chatbot — integrated with its internal HR system — to help managers answer HR policy questions, draft performance review language, and retrieve employee records. The system was built using a commercially available LLM, configured with access to the company’s HR data warehouse containing salary records, performance ratings, disciplinary histories, medical leave records, and confidential appraisal notes for approximately twelve thousand employees.

Over several months, Vikram and other managers used the system extensively. Gradually, however, a pattern emerged. When managers queried the chatbot about HR policies unrelated to specific employees, the model’s responses occasionally incorporated specific employee details — names, salary ranges, performance descriptors — drawn from the training context it had been given, even when those details were not requested and not relevant to the query.

In one instance, a senior manager querying about leave policy received a response that referenced a specific employee’s medical leave pattern as an illustrative example — without Vikram having requested that information. That employee had not been informed that her medical leave data was part of the AI system’s accessible context. She had not consented to its use as illustrative material in chatbot responses to other managers.

The DPDP Act Position

This scenario reveals a compliance failure that is specific to generative AI deployments: the gap between data that is technically accessible to the model and data that has lawful authorisation to appear in the model’s outputs.

First — Processing without lawful basis under Section 4. Section 4(1) of the DPDP Act permits processing only for a lawful purpose for which the Data Principal has given consent, or for certain legitimate uses under Section 7. Medical leave data, performance ratings, and salary information were collected for HR administration — a specific, documented purpose. Incorporating this data as illustrative context in chatbot responses to third-party managers is a materially different processing activity. It has neither a consent basis nor a legitimate use basis. The data appeared in the output not because it was requested but because the model’s context window encompassed it and its generation logic surfaced it as relevant. That is still processing — and it is processing without a lawful basis.

Second — Notice failure under Section 5 and Rule 3. The employee whose medical leave data appeared in the chatbot response had never been informed that her personal data would be processed by an AI system for the purpose of generating responses to queries by other managers. Section 5(1) requires that every Data Principal be given notice of the personal data to be processed and the purpose for which it will be processed. Rule 3 of the DPDP Rules requires that this notice be in clear and plain language, with an itemised description of personal data and a specific description of its processing purposes. “Your data will be used for HR administration” does not cover “your data may appear in AI-generated responses to your colleagues’ policy queries.”

Third — Rule 13(3) algorithmic software due diligence obligation. Rule 13(3) of the DPDP Rules places a specific obligation on Significant Data Fiduciaries to observe due diligence to verify that technical measures — including algorithmic software — adopted for storing, updating, or sharing personal data are not likely to pose a risk to the rights of Data Principals. A generative AI system configured with access to twelve thousand employee records, with no output filtering controls, no data minimisation at the inference layer, and no audit trail linking specific outputs to specific data inputs, is an algorithmic architecture that poses a risk to Data Principal rights. Due diligence requires its assessment before deployment — not after an incident surfaces.

Fourth — IS Audit Standard 420 (ICAI ISAS) governance failure. IS Audit Standard 420 requires that organisations deploying AI tools establish governance controls covering approval, validation, reproducibility, accreditation, and oversight. For AI models with non-deterministic outputs — which all current LLMs are — this requires documenting parameters, prompts, and methodology sufficiently to allow independent verification. Vikram’s company had no mechanism to trace which data inputs contributed to any specific chatbot output. When the problematic response appeared, there was no audit trail to investigate it, no log linking the output to the employee’s data, and no governance framework to determine whether a personal data breach had occurred.

IS Audit Standard 430 (ICAI ISAS, Section 3.3) requires that auditors understand the entire lifecycle of personal data — from collection through storage, processing, sharing, and eventual erasure. An LLM deployed on an HR data warehouse has a data lifecycle that extends into every inference it generates. Without lifecycle mapping specific to the AI layer, ISAS 430 compliance cannot be demonstrated.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 12) specifically identifies sensitive data leakage as a primary adversarial risk to AI systems. It notes that AI-assisted offensive tooling can target AI inference systems and AI-integrated workflows — meaning that an LLM with broad access to employee personal data is not merely a compliance liability. It is an attack surface. An adversary who manipulates the model’s input — through prompt injection — can potentially extract personal data that the model has in context, without any system alert being generated.

CERT-In’s framework requires organisations to restrict upload of sensitive information to public AI platforms, define acceptable usage policies, and implement access control and authentication on AI system workflows. Applied to enterprise LLM deployments, these requirements mean that personal data context must be scoped and governed — not simply made available to the model in bulk.

The Compliance Fix

Organisations deploying LLMs or generative AI systems on employee or customer personal data must:

→ Scope the model’s data access to the minimum necessary for each specific use case. An LLM answering HR policy questions does not need access to twelve thousand employees’ disciplinary histories and medical leave records simultaneously.

→ Issue a specific, itemised notice under Rule 3 to every Data Principal whose personal data forms part of the model’s accessible context — identifying the AI system as a processing activity and the inference generation as a purpose.

→ Implement output filtering controls at the inference layer: if the model’s response includes personal data not requested by the querying user, the response must be intercepted, logged, and reviewed before delivery.

→ Maintain an AI processing register that links every LLM deployment to the personal data categories it can access, the purposes for which it was deployed, and the consent records that authorise that processing.

→ Conduct a DPIA under Rule 13 before deploying any generative AI system on personal data at scale, with specific attention to the risk of unintended data surfacing in model outputs.

→ Implement audit logging at the inference level: every query and every response must be logged in a form that allows post-hoc investigation if a personal data concern is raised.

Vikram’s company gave their AI system a broad memory and no output accountability. Under the DPDP Act, the outputs that memory generated were processing events — each one requiring a lawful basis.

The compliance deadline is 13 May 2027. An AI system with a broad memory and no output accountability is not a productivity tool. It is an uncontrolled personal data processor.

(to be continued)

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Karthik & Sunil

Chartered Accountants

#GenerativeAI

DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

Like this:

DPDP 3.7.2: DPDP & Generative AI – Ananya’s Voice Note and the Deepfake That Signed a Contract

Like this:

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: