DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

The Scenario

Kiran is a 38-year-old consumer goods marketing manager in Chennai. In October 2024, she used the customer service chatbot of a major e-commerce platform to lodge a complaint about a defective product. During the conversation, she disclosed her name, order details, delivery address, phone number, and — because the issue involved a medication she had purchased — the name of a prescription drug and the reason she needed it urgently.

Unknown to Kiran, the platform’s terms of service contained a clause stating that “conversations with our assistant may be used to improve our AI services.” This clause was buried in a sixteen-page document she had agreed to at account registration, three years before the chatbot was deployed.

Twelve months later, Kiran received a targeted advertisement for a competing medication — delivered through a third-party advertising platform — that directly corresponded to the medication she had named in her complaint conversation. She had never searched for this medication online. The only digital record of its relevance to her was the chatbot conversation.

Investigation revealed that the e-commerce platform had been using customer service conversations — including Kiran’s — as fine-tuning data for its next generation customer service AI model, and had shared anonymised conversation datasets with an advertising technology partner. The anonymisation had been superficial: name and order ID had been removed, but the medication name, delivery address, and phone number — when cross-referenced against the advertising partner’s own data — were sufficient to re-identify Kiran.

The DPDP Act Position

Kiran’s situation is the most layered of the three stories because it involves three cascading compliance failures — consent fragmentation, inadequate anonymisation, and downstream re-identification — all enabled by generative AI training practices.

First — Buried terms of service do not constitute valid consent under Section 6(1). Section 6(1) of the DPDP Act requires consent to be free, specific, informed, unconditional, and unambiguous. Rule 3 of the DPDP Rules requires the notice to be presented and understandable independently of any other information — meaning it must stand alone, in clear and plain language, without requiring the Data Principal to extract relevant information from a sixteen-page document. A clause buried in terms of service agreed to three years before a chatbot was deployed does not give specific, informed consent to the use of chatbot conversations for AI model training. These are meaningfully different processing activities and require meaningfully different consent.

Furthermore, Section 6(1) limits consent to personal data necessary for the specified purpose. Kiran agreed to an account creation purpose. The collection of her complaint conversation for AI training is a different purpose — and the personal data involved, particularly health-related information, demands its own specific consent basis.

Second — The re-identification of pseudonymised data is a personal data breach under Section 2(u). The e-commerce platform argued that the data shared with its advertising partner had been anonymised. However, true anonymisation — as recognised in IS Audit Standard 430 (Section 4.6) — means that the data cannot be linked back to an individual through any means reasonably likely to be used. Removing a name and order ID while retaining a medication name, delivery address, and phone number does not constitute anonymisation when those elements, in combination, can be cross-referenced against other datasets to identify the individual.

When the advertising partner re-identified Kiran using this cross-referencing — evidenced by the targeted advertisement — a personal data breach occurred under Section 2(u): accidental or unauthorised disclosure of personal data that compromises its confidentiality. The platform bears responsibility for this breach under Section 8(1), because the sharing with the advertising partner was processing carried out on its behalf.

Third — Health information processed without specific consent. Kiran disclosed the name of a prescription medication — personal data directly relating to her health status. This is the most sensitive category of personal data in the context of the DPDP Act’s proportionality framework. Processing it for AI model training, without a specific notice disclosing health information as a training data category, and without specific consent covering that use, is processing without a lawful basis under Section 4(1). The advertising outcome — targeted medication advertising — demonstrates that this sensitive data was not merely processed for training: it was used to drive commercial targeting that affected Kiran directly.

Fourth — Third-party data sharing as undisclosed Data Processor engagement. Section 8(2) requires that every Data Processor engaged by the Data Fiduciary process personal data only for the specified purpose under a valid contract. The advertising technology partner received Kiran’s personal data for AI training purposes. However, by using it for advertising targeting — a different purpose — it operated outside its Data Processor mandate. Under Section 8(1), the e-commerce platform bears liability for its Data Processor’s actions. The inadequacy of the Data Processor contract — which apparently did not restrict downstream use to the training purpose — is a governance failure attributable to the platform.

The CERT-In and IS Audit Dimensions

CERT-In CIGU20260002 (May 2026, Section 12, Area 7) identifies data protection and privacy in AI systems as a governance priority requiring organisations to classify and protect sensitive data, define retention and deletion policies, and monitor AI-related data flows. The platform’s failure to classify health information disclosed in service conversations as sensitive personal data — and to restrict its downstream use accordingly — is precisely the governance gap CERT-In’s framework addresses.

IS Audit Standard 430 by ICAI requires professionals conducting DPDP audits to have technical knowledge and competence covering profiling of digital personal data, pseudonymisation, and anonymisation. The distinction between true anonymisation and pseudonymisation that can be reversed through cross-referencing is a core technical competence in this standard — and a core compliance obligation for any organisation using AI training datasets derived from customer interactions.

IS Audit Standard 420 notes that data lineage, completeness, transformations, and regulatory conditions must be assessed to determine whether technology-driven analyses produce valid outputs and whether data handling meets privacy and legal expectations. Kiran’s data travelled from a chatbot conversation through anonymisation, to a training dataset, to an advertising partner, and back to Kiran in the form of a targeted advertisement — a data lineage of five steps, none of which was disclosed to her, and none of which was covered by a lawful consent.

The Compliance Fix

E-commerce platforms, AI product companies, and any organisation using customer interaction data for generative AI training must:

→ Issue separate, specific consent notices — not buried terms of service — for the use of customer interaction data for AI model training. This notice must identify: the data categories (including any health information that may arise in service conversations), the training purpose, and the identity of any downstream partners who will receive training data.

→ Apply genuine anonymisation — not pseudonymisation — to any data shared with AI training partners. IS Audit Standard 430 defines anonymisation as the removal or modification of identifiable information so that it cannot be linked back to an individual. Re-identification through cross-referencing means the anonymisation was inadequate.

→ Implement Data Processor contracts under Section 8(2) that restrict downstream partners to the training purpose only, with explicit prohibition on using training datasets for advertising targeting, profiling, or any purpose beyond the stated training objective.

→ Build data lineage tracking for all AI training pipelines: every piece of personal data entering a training pipeline must be traceable to a specific consent record, a specific data category disclosure, and a specific purpose. If the lineage cannot be demonstrated, the processing has no lawful basis.

→ Apply AI data usage controls specifically to health information — flagging, quarantining, or excluding health-related disclosures from training datasets unless specific, separate consent has been obtained for that category.

→ Treat the re-identification of pseudonymised training data as a personal data breach under Section 2(u), and implement breach notification workflows accordingly.

Kiran disclosed a medical need in a service conversation. That conversation built a model, fed an advertising engine, and returned to her as a targeted advertisement. The DPDP Act calls every step of that journey processing — and the platform was the Data Fiduciary responsible for each one.

The Central Compliance Question for Generative AI Deployments

Generative AI presents a compliance challenge that is qualitatively different from every other technology in this series. With blockchain, the problem was that data could not be deleted. With IoT, the problem was that data was collected without awareness. With RPA, the problem was that automated processing created invisible data footprints. With data analytics, the problem was that consent architectures were outpaced by analytical capability.

With generative AI, all of these problems appear simultaneously — and a new one is added: the model can generate outputs that reveal, infer, or recreate personal data that was never explicitly in its prompt. IS Audit Standard 420 (ICAI ISAS) acknowledges this directly: generative AI models are non-deterministic. Their outputs cannot always be predicted, traced, or reproduced verbatim. IS Audit 3.0 course material by ICAI identifies this mathematical uncertainty as a governance gap that organisations have not yet resolved.

The DPDP Act does not exempt non-deterministic outputs from its coverage. Every output of a generative AI system that contains personal data — however that output was generated — is a processing event subject to the Act’s full consent, notice, and accountability framework.

Every organisation deploying generative AI on personal data must therefore answer six questions before and continuously during deployment:

One. What personal data is accessible to the model as context, training data, or retrieval input? Is every data category covered by a specific, itemised notice and a lawful consent under Section 5 and Rule 3?

Two. Does the model’s output filtering architecture prevent personal data from appearing in responses to users who are not authorised to receive it?

Three. Have we conducted a DPIA under Rule 13 that specifically assesses the risk of unintended personal data surfacing in AI-generated outputs — and the risk of model outputs being used to re-identify individuals from pseudonymised inputs?

Four. Are all generative AI vendors and downstream AI processing partners governed by Section 8(2) contracts limiting them to specified purposes — with explicit restrictions on using processed data for model training, advertising, or any purpose beyond the stated scope?

Five. Do we have audit logging at the inference level — recording every query and response in a form that allows a DPDP Act compliance review or breach investigation to trace which personal data appeared in which output?

Six. Have we assessed our generative AI deployment against CERT-In’s CIGU20260002 framework for AI system security — covering prompt injection risks, sensitive data leakage, training data integrity, and adversarial threats to AI inference systems?

IS Audit 3.0 course material by ICAI frames this governance requirement clearly: AI governance must establish accountability and oversight, ensure that responsible parties have the necessary skills and expertise, and ensure that AI activities result in decisions and actions aligned with the ethical, social, and legal responsibilities of the organisation. Under the DPDP Act, those legal responsibilities include every Data Principal whose personal data touches a generative AI system.

What Responsible Generative AI and DPDP Compliance Looks Like

The organisations that will navigate generative AI responsibly under the DPDP Act are not those with the most capable models. They are those that ask the hardest questions before deployment — and build governance architectures that survive those questions.

Responsible generative AI and DPDP compliance means:

Scoping the model’s data access to what is necessary. Issuing specific notices for every processing purpose the model serves. Building output controls that prevent unintended data surfacing. Logging every inference for compliance accountability. Treating AI vendors as Data Processors with binding contracts. Conducting DPIAs before deployment, not after incidents. And applying CERT-In’s AI security framework as a complement to DPDP compliance — because the same technology that poses a compliance risk inside the organisation is simultaneously being used by adversaries to attack it from outside.

The Puttaswamy judgment (2018) described the era we live in with extraordinary prescience: humans forget, but the internet does not. Data mining and knowledge discovery processes create new knowledge about individuals — including facts they did not themselves possess. In 2018, that observation described the risks of data analytics. In 2026, it describes generative AI with even greater precision. A well-trained LLM has absorbed patterns from personal data at a scale no previous technology has matched — and can generate outputs that reflect those patterns in ways no individual who contributed that data ever anticipated or agreed to.

The DPDP Act exists to govern that reality. The compliance deadline is 13 May 2027. The governance work begins now.

The Series in Retrospect: A Unified Compliance Principle

This is the seventh and final episode of DPDP Meets Emerging Technologies. Together, the seven episodes have covered AI, Cloud Computing, Blockchain, IoT, Data Analytics, RPA, and Generative AI — seven technologies that are transforming India’s digital economy, and seven compliance frontiers that the DPDP Act governs with equal force.

Looking across all seven episodes, one principle has run through every story, every violation, and every compliance fix: technology changes the mechanism of data processing, but it does not change the law that governs it.

Blockchain made data immutable — the erasure right still applied. IoT made data collection invisible — the consent requirement still applied. RPA made data processing automated — the Data Fiduciary’s liability still applied. Data analytics made inferences from data — those inferences were still personal data. And generative AI makes outputs non-deterministic — the accountability framework still applies.

The DPDP Act, 2023 was designed precisely for this reality. It does not enumerate technologies. It governs processing — in whatever form, at whatever scale, by whatever means. The definition in Section 2(x) is deliberately broad: any wholly or partly automated operation on digital personal data. No technology sits outside it.

The Series is Complete. Seven Technologies. Seven Compliance Frontiers. One Deadline.

Over the past seven episodes, DPDP Meets Emerging Technologies has examined how the Digital Personal Data Protection Act, 2023 applies to the technologies reshaping India’s digital economy.

Episode 1 — AI and DPDP: When the algorithm decides without consent.
Episode 2 — Cloud Computing and DPDP: When your data lives on someone else’s server.
Episode 3 — Blockchain and DPDP: When the ledger never forgets.
Episode 4 — IoT and DPDP: When every device becomes a witness.
Episode 5 — Data Analytics and DPDP: When the numbers know too much.
Episode 6 — RPA and DPDP: When the bot processes more than it should.
Episode 7 — Generative AI and DPDP: When the machine learns what it should not know.

Across all seven stories, one principle remained constant: technology changes the mechanism of data processing. The DPDP Act governs the processing, regardless of mechanism.

The compliance deadline is 13 May 2027. Every Indian organisation that processes personal data through any of these technologies is already in scope — today.

If your organisation needs a DPDP compliance assessment, a technology-specific privacy architecture review, or a staff awareness programme covering these emerging technology compliance risks, we are here to help.

📩 Reach us at: karthikandsunil@karthikandsunil.in
🔗 Full series:

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors: This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

Karthik & Sunil

Chartered Accountants

DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

Like this:

Related

Leave a ReplyCancel reply

Share this:

Like this:

Related

Leave a ReplyCancel reply

Discover more from Karthik & Sunil