DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

September 22, 2026 Karthik & Sunil#DataPrivacyIndia, #DPDPAct, #DPDPCompliance, #EmergingTechnologies, #GenerativeAI, Karthik & SunilLeave a comment

The Scenario

Kiran is a 38-year-old consumer goods marketing manager in Chennai. In October 2024, she used the customer service chatbot of a major e-commerce platform to lodge a complaint about a defective product. During the conversation, she disclosed her name, order details, delivery address, phone number, and — because the issue involved a medication she had purchased — the name of a prescription drug and the reason she needed it urgently.

Unknown to Kiran, the platform’s terms of service contained a clause stating that “conversations with our assistant may be used to improve our AI services.” This clause was buried in a sixteen-page document she had agreed to at account registration, three years before the chatbot was deployed.

Twelve months later, Kiran received a targeted advertisement for a competing medication — delivered through a third-party advertising platform — that directly corresponded to the medication she had named in her complaint conversation. She had never searched for this medication online. The only digital record of its relevance to her was the chatbot conversation.

Investigation revealed that the e-commerce platform had been using customer service conversations — including Kiran’s — as fine-tuning data for its next generation customer service AI model, and had shared anonymised conversation datasets with an advertising technology partner. The anonymisation had been superficial: name and order ID had been removed, but the medication name, delivery address, and phone number — when cross-referenced against the advertising partner’s own data — were sufficient to re-identify Kiran.

The DPDP Act Position

Kiran’s situation is the most layered of the three stories because it involves three cascading compliance failures — consent fragmentation, inadequate anonymisation, and downstream re-identification — all enabled by generative AI training practices.

First — Buried terms of service do not constitute valid consent under Section 6(1). Section 6(1) of the DPDP Act requires consent to be free, specific, informed, unconditional, and unambiguous. Rule 3 of the DPDP Rules requires the notice to be presented and understandable independently of any other information — meaning it must stand alone, in clear and plain language, without requiring the Data Principal to extract relevant information from a sixteen-page document. A clause buried in terms of service agreed to three years before a chatbot was deployed does not give specific, informed consent to the use of chatbot conversations for AI model training. These are meaningfully different processing activities and require meaningfully different consent.

Furthermore, Section 6(1) limits consent to personal data necessary for the specified purpose. Kiran agreed to an account creation purpose. The collection of her complaint conversation for AI training is a different purpose — and the personal data involved, particularly health-related information, demands its own specific consent basis.

Second — The re-identification of pseudonymised data is a personal data breach under Section 2(u). The e-commerce platform argued that the data shared with its advertising partner had been anonymised. However, true anonymisation — as recognised in IS Audit Standard 430 (Section 4.6) — means that the data cannot be linked back to an individual through any means reasonably likely to be used. Removing a name and order ID while retaining a medication name, delivery address, and phone number does not constitute anonymisation when those elements, in combination, can be cross-referenced against other datasets to identify the individual.

When the advertising partner re-identified Kiran using this cross-referencing — evidenced by the targeted advertisement — a personal data breach occurred under Section 2(u): accidental or unauthorised disclosure of personal data that compromises its confidentiality. The platform bears responsibility for this breach under Section 8(1), because the sharing with the advertising partner was processing carried out on its behalf.

Third — Health information processed without specific consent. Kiran disclosed the name of a prescription medication — personal data directly relating to her health status. This is the most sensitive category of personal data in the context of the DPDP Act’s proportionality framework. Processing it for AI model training, without a specific notice disclosing health information as a training data category, and without specific consent covering that use, is processing without a lawful basis under Section 4(1). The advertising outcome — targeted medication advertising — demonstrates that this sensitive data was not merely processed for training: it was used to drive commercial targeting that affected Kiran directly.

Fourth — Third-party data sharing as undisclosed Data Processor engagement. Section 8(2) requires that every Data Processor engaged by the Data Fiduciary process personal data only for the specified purpose under a valid contract. The advertising technology partner received Kiran’s personal data for AI training purposes. However, by using it for advertising targeting — a different purpose — it operated outside its Data Processor mandate. Under Section 8(1), the e-commerce platform bears liability for its Data Processor’s actions. The inadequacy of the Data Processor contract — which apparently did not restrict downstream use to the training purpose — is a governance failure attributable to the platform.

The CERT-In and IS Audit Dimensions

CERT-In CIGU20260002 (May 2026, Section 12, Area 7) identifies data protection and privacy in AI systems as a governance priority requiring organisations to classify and protect sensitive data, define retention and deletion policies, and monitor AI-related data flows. The platform’s failure to classify health information disclosed in service conversations as sensitive personal data — and to restrict its downstream use accordingly — is precisely the governance gap CERT-In’s framework addresses.

IS Audit Standard 430 by ICAI requires professionals conducting DPDP audits to have technical knowledge and competence covering profiling of digital personal data, pseudonymisation, and anonymisation. The distinction between true anonymisation and pseudonymisation that can be reversed through cross-referencing is a core technical competence in this standard — and a core compliance obligation for any organisation using AI training datasets derived from customer interactions.

IS Audit Standard 420 notes that data lineage, completeness, transformations, and regulatory conditions must be assessed to determine whether technology-driven analyses produce valid outputs and whether data handling meets privacy and legal expectations. Kiran’s data travelled from a chatbot conversation through anonymisation, to a training dataset, to an advertising partner, and back to Kiran in the form of a targeted advertisement — a data lineage of five steps, none of which was disclosed to her, and none of which was covered by a lawful consent.

The Compliance Fix

E-commerce platforms, AI product companies, and any organisation using customer interaction data for generative AI training must:

→ Issue separate, specific consent notices — not buried terms of service — for the use of customer interaction data for AI model training. This notice must identify: the data categories (including any health information that may arise in service conversations), the training purpose, and the identity of any downstream partners who will receive training data.

→ Apply genuine anonymisation — not pseudonymisation — to any data shared with AI training partners. IS Audit Standard 430 defines anonymisation as the removal or modification of identifiable information so that it cannot be linked back to an individual. Re-identification through cross-referencing means the anonymisation was inadequate.

→ Implement Data Processor contracts under Section 8(2) that restrict downstream partners to the training purpose only, with explicit prohibition on using training datasets for advertising targeting, profiling, or any purpose beyond the stated training objective.

→ Build data lineage tracking for all AI training pipelines: every piece of personal data entering a training pipeline must be traceable to a specific consent record, a specific data category disclosure, and a specific purpose. If the lineage cannot be demonstrated, the processing has no lawful basis.

→ Apply AI data usage controls specifically to health information — flagging, quarantining, or excluding health-related disclosures from training datasets unless specific, separate consent has been obtained for that category.

→ Treat the re-identification of pseudonymised training data as a personal data breach under Section 2(u), and implement breach notification workflows accordingly.

Kiran disclosed a medical need in a service conversation. That conversation built a model, fed an advertising engine, and returned to her as a targeted advertisement. The DPDP Act calls every step of that journey processing — and the platform was the Data Fiduciary responsible for each one.

The Central Compliance Question for Generative AI Deployments

Generative AI presents a compliance challenge that is qualitatively different from every other technology in this series. With blockchain, the problem was that data could not be deleted. With IoT, the problem was that data was collected without awareness. With RPA, the problem was that automated processing created invisible data footprints. With data analytics, the problem was that consent architectures were outpaced by analytical capability.

With generative AI, all of these problems appear simultaneously — and a new one is added: the model can generate outputs that reveal, infer, or recreate personal data that was never explicitly in its prompt. IS Audit Standard 420 (ICAI ISAS) acknowledges this directly: generative AI models are non-deterministic. Their outputs cannot always be predicted, traced, or reproduced verbatim. IS Audit 3.0 course material by ICAI identifies this mathematical uncertainty as a governance gap that organisations have not yet resolved.

The DPDP Act does not exempt non-deterministic outputs from its coverage. Every output of a generative AI system that contains personal data — however that output was generated — is a processing event subject to the Act’s full consent, notice, and accountability framework.

Every organisation deploying generative AI on personal data must therefore answer six questions before and continuously during deployment:

One. What personal data is accessible to the model as context, training data, or retrieval input? Is every data category covered by a specific, itemised notice and a lawful consent under Section 5 and Rule 3?

Two. Does the model’s output filtering architecture prevent personal data from appearing in responses to users who are not authorised to receive it?

Three. Have we conducted a DPIA under Rule 13 that specifically assesses the risk of unintended personal data surfacing in AI-generated outputs — and the risk of model outputs being used to re-identify individuals from pseudonymised inputs?

Four. Are all generative AI vendors and downstream AI processing partners governed by Section 8(2) contracts limiting them to specified purposes — with explicit restrictions on using processed data for model training, advertising, or any purpose beyond the stated scope?

Five. Do we have audit logging at the inference level — recording every query and response in a form that allows a DPDP Act compliance review or breach investigation to trace which personal data appeared in which output?

Six. Have we assessed our generative AI deployment against CERT-In’s CIGU20260002 framework for AI system security — covering prompt injection risks, sensitive data leakage, training data integrity, and adversarial threats to AI inference systems?

IS Audit 3.0 course material by ICAI frames this governance requirement clearly: AI governance must establish accountability and oversight, ensure that responsible parties have the necessary skills and expertise, and ensure that AI activities result in decisions and actions aligned with the ethical, social, and legal responsibilities of the organisation. Under the DPDP Act, those legal responsibilities include every Data Principal whose personal data touches a generative AI system.

What Responsible Generative AI and DPDP Compliance Looks Like

The organisations that will navigate generative AI responsibly under the DPDP Act are not those with the most capable models. They are those that ask the hardest questions before deployment — and build governance architectures that survive those questions.

Responsible generative AI and DPDP compliance means:

Scoping the model’s data access to what is necessary. Issuing specific notices for every processing purpose the model serves. Building output controls that prevent unintended data surfacing. Logging every inference for compliance accountability. Treating AI vendors as Data Processors with binding contracts. Conducting DPIAs before deployment, not after incidents. And applying CERT-In’s AI security framework as a complement to DPDP compliance — because the same technology that poses a compliance risk inside the organisation is simultaneously being used by adversaries to attack it from outside.

The Puttaswamy judgment (2018) described the era we live in with extraordinary prescience: humans forget, but the internet does not. Data mining and knowledge discovery processes create new knowledge about individuals — including facts they did not themselves possess. In 2018, that observation described the risks of data analytics. In 2026, it describes generative AI with even greater precision. A well-trained LLM has absorbed patterns from personal data at a scale no previous technology has matched — and can generate outputs that reflect those patterns in ways no individual who contributed that data ever anticipated or agreed to.

The DPDP Act exists to govern that reality. The compliance deadline is 13 May 2027. The governance work begins now.

The Series in Retrospect: A Unified Compliance Principle

This is the seventh and final episode of DPDP Meets Emerging Technologies. Together, the seven episodes have covered AI, Cloud Computing, Blockchain, IoT, Data Analytics, RPA, and Generative AI — seven technologies that are transforming India’s digital economy, and seven compliance frontiers that the DPDP Act governs with equal force.

Looking across all seven episodes, one principle has run through every story, every violation, and every compliance fix: technology changes the mechanism of data processing, but it does not change the law that governs it.

Blockchain made data immutable — the erasure right still applied. IoT made data collection invisible — the consent requirement still applied. RPA made data processing automated — the Data Fiduciary’s liability still applied. Data analytics made inferences from data — those inferences were still personal data. And generative AI makes outputs non-deterministic — the accountability framework still applies.

The DPDP Act, 2023 was designed precisely for this reality. It does not enumerate technologies. It governs processing — in whatever form, at whatever scale, by whatever means. The definition in Section 2(x) is deliberately broad: any wholly or partly automated operation on digital personal data. No technology sits outside it.

The Series is Complete. Seven Technologies. Seven Compliance Frontiers. One Deadline.

Over the past seven episodes, DPDP Meets Emerging Technologies has examined how the Digital Personal Data Protection Act, 2023 applies to the technologies reshaping India’s digital economy.

Episode 1 — AI and DPDP: When the algorithm decides without consent.
Episode 2 — Cloud Computing and DPDP: When your data lives on someone else’s server.
Episode 3 — Blockchain and DPDP: When the ledger never forgets.
Episode 4 — IoT and DPDP: When every device becomes a witness.
Episode 5 — Data Analytics and DPDP: When the numbers know too much.
Episode 6 — RPA and DPDP: When the bot processes more than it should.
Episode 7 — Generative AI and DPDP: When the machine learns what it should not know.

Across all seven stories, one principle remained constant: technology changes the mechanism of data processing. The DPDP Act governs the processing, regardless of mechanism.

The compliance deadline is 13 May 2027. Every Indian organisation that processes personal data through any of these technologies is already in scope — today.

If your organisation needs a DPDP compliance assessment, a technology-specific privacy architecture review, or a staff awareness programme covering these emerging technology compliance risks, we are here to help.

📩 Reach us at: karthikandsunil@karthikandsunil.in
🔗 Full series:

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors: This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.7.2: DPDP & Generative AI – Ananya’s Voice Note and the Deepfake That Signed a Contract

September 17, 2026 Karthik & Sunil#DataFiduciary, #DataPrivacy, #DeepfakeFraud, #DPDPAct, #GenerativeAI, #VoiceBiometrics, Karthik & SunilLeave a comment

The Scenario

Ananya is the founder of a small fintech startup in Hyderabad. In early 2025, she received a voice call from what appeared to be her bank’s relationship manager, requesting verbal authorisation to release funds from a current account for a vendor payment. The voice was familiar — warm, specific to her account, and contextually accurate. She authorised the transfer.

The call was a deepfake. An AI-generated voice, synthesised from recordings of the actual relationship manager’s previous interactions with Ananya and other clients, had been used to impersonate the banker with sufficient realism to bypass Ananya’s verification instincts. The transfer of ₹28 lakh was irreversible by the time the fraud was detected.

Investigation revealed that the relationship manager’s voice recordings — captured across customer service calls — had been exfiltrated from the bank’s call recording system in a prior data breach. Those recordings contained personal data: the manager’s voice, name, customer account references, and contextual details that made the synthesised interaction credible.

The DPDP Act Position

This scenario operates at the intersection of two DPDP Act obligations: the Data Fiduciary’s duty to prevent personal data breach, and the consequence of that breach being weaponised through generative AI.

First — Personal data breach enabling the deepfake under Section 2(u) and Section 8(5). The exfiltration of call recordings from the bank’s system was a personal data breach under Section 2(u) — unauthorised acquisition of personal data that compromised its confidentiality. Section 8(5) of the DPDP Act requires the Data Fiduciary to take reasonable security safeguards to prevent personal data breach. Rule 6 of the DPDP Rules specifies that these safeguards must include encryption or obfuscation of personal data, access controls over computer resources, and monitoring logs that enable detection of unauthorised access. If call recordings containing voice biometrics and customer account details were stored without adequate encryption, without access controls, and without monitoring — the bank failed its Section 8(5) obligation before a single deepfake was generated.

Second — Failure to classify voice biometrics as high-sensitivity personal data. Voice recordings that contain sufficient data to synthesise an individual’s voice are personal data under Section 2(t) of the DPDP Act — they are data about an identifiable individual. When used to train or feed a generative AI voice synthesis model, they become the direct input to a process that can impersonate that individual. The sensitivity of this category of personal data is therefore significantly higher than a simple audio archive might suggest, and the security architecture protecting it must reflect that sensitivity.

Third — Section 8(6) and Rule 7 breach notification failure. The prior data breach that enabled the deepfake should have triggered a notification obligation under Section 8(6) and Rule 7 — notification to the Data Protection Board and to each affected Data Principal, without delay, upon the bank becoming aware of the breach. Had notification occurred promptly, Ananya — and other customers whose interaction data was in the exfiltrated set — would have been on alert. The window for the deepfake fraud would have narrowed significantly.

Fourth — The deepfake generation itself as a further processing violation. The attacker who synthesised the relationship manager’s voice used the bank’s exfiltrated call data to train or prompt a voice generation model. Every step of that process — acquiring the recordings, training the model, generating the synthesised voice — constituted processing of personal data under Section 2(x) of the DPDP Act. This processing had no lawful basis, no consent, and no legitimate purpose. It was, by definition, unlawful processing — and the Data Fiduciary whose security failure enabled it bears legal responsibility under Section 8(5) for creating the conditions in which it occurred.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 4.2) explicitly names deepfake voice and video fraud as a primary AI-assisted threat vector, noting that AI technologies are increasingly used to generate highly convincing impersonation attempts and deepfake-enabled fraud that bypass traditional awareness-based detection due to their realism, contextual accuracy, and personalisation. Section 11 of the same document requires organisations to build deepfake and impersonation detection readiness, including verification procedures, executive impersonation monitoring, and escalation mechanisms.

Applied to banking and financial services, this means that verbal authorisation processes — for fund transfers, account changes, or contract executions — must not rely on voice alone as a verification factor in an era of generative voice synthesis. CERT-In’s framework, read alongside the DPDP Act’s security safeguard obligations, requires that financial institutions redesign their authorisation architectures to account for the synthetic impersonation risk that generative AI now makes operationally accessible to attackers.

The IS Audit Perspective

IS Audit Standard 420 (ICAI ISAS, Section 4.3) identifies opacity, embedded bias, and non-deterministic outcomes as risks specific to AI systems. In the context of voice synthesis, the opacity risk runs in two directions: the attacker’s model is opaque to the victim, and the bank’s internal monitoring was opaque to the attack. IS Audit Standard 420 requires that professionals maintain professional scepticism when using automated AI outputs and ensure that sensitive data handling meets privacy and legal requirements.

IS Audit 3.0 study material identifies data privacy and security — specifically the risk of ML systems being prone to data breach and identity theft — as a primary AI governance concern. For financial institutions, this risk now extends to the use of their customers’ and employees’ voice data as raw material for generative AI fraud.

The Compliance Fix

Banks, financial institutions, and any organisation that holds voice recordings or biometric personal data must urgently:

→ Classify all voice recordings containing sufficient data to enable voice synthesis as high-sensitivity personal data and apply maximum security standards under Rule 6: encryption at rest and in transit, access control scoped to minimum necessary, and comprehensive monitoring.

→ Implement Rule 7 breach notification workflows that specifically cover the exfiltration of voice and biometric data, with immediate notification to the Data Protection Board and affected Data Principals, without delay.

→ Redesign verbal authorisation processes for financial transactions: multi-factor authentication that cannot be bypassed by synthetic voice alone — including callback protocols, one-time codes, and out-of-band verification — must replace or supplement voice-only authorisation.

→ Build deepfake detection capabilities, as required by CERT-In CIGU20260002 (Section 11), for customer-facing and internal voice channels. AI-generated voices have detectable characteristics that real-time or near-real-time analysis can flag.

→ Train operational staff — particularly relationship managers and customer-facing teams — on the reality of AI voice synthesis fraud, the verification procedures to apply, and the escalation channels available when synthetic identity suspicion arises.

Ananya lost ₹28 lakh. The bank lost her trust. Both losses were preventable — if the bank had treated its call recording archive as the high-sensitivity personal data it was, and had built a security architecture proportionate to that sensitivity.

The compliance deadline is 13 May 2027. Voice data is personal data. Biometric impersonation through generative AI is a foreseeable risk. The DPDP Act requires organisations to design security against foreseeable risks — not merely react to them after loss.

Disclaimer

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

September 12, 2026 Karthik & Sunil#DataPrivacy, #DPDPAct, #GenerativeAI, #LLMCompliance1 Comment

INTRODUCTION

Generative AI and DPDP compliance represent the most consequential frontier in India’s data protection landscape — and the one that is evolving fastest.

Generative AI refers to systems — large language models (LLMs), image generators, voice synthesisers, code generators, and multimodal models — that are trained on vast quantities of data and can produce new content: text, images, audio, video, and code that did not previously exist. These systems are already embedded in enterprise workflows across India: drafting documents, summarising records, answering customer queries, analysing contracts, generating reports, and supporting HR processes.

IS Audit 3.0 by ICAI identifies data privacy and security as a primary AI risk, noting that most AI applications depend on massive volumes of data — often sensitive and personal in nature — to learn and make intelligent decisions. As machine learning systems become more capable, they also become more capable of learning, retaining, and inadvertently reproducing the personal data on which they were trained or with which they were prompted.

IS Audit Standard 420 (ICAI ISAS) goes further, acknowledging that generative AI models present non-deterministic outputs — meaning the same input may produce different outputs across sessions. For AI models with these characteristics, it notes, reproducibility requires that parameters, prompts, and methodology be documented sufficiently to allow independent verification. This observation is not merely an audit standard. It is a compliance architecture requirement: if a generative AI system cannot reproduce how a personal data output was generated, the Data Fiduciary deploying it cannot adequately defend its processing decisions under the DPDP Act.

CERT-In’s Blueprint on AI-Assisted Cyber Threats (CIGU20260002, May 2026) adds a threat dimension that no organisation can afford to ignore: generative AI is being actively leveraged by threat actors to generate highly convincing phishing content, deepfake voice and video fraud, synthetic identities, and spear phishing campaigns that bypass traditional awareness-based detection — all of which directly target personal data of Indian citizens. Consequently, generative AI is not merely a compliance risk as a tool of enterprise productivity. It is simultaneously the instrument of some of the most sophisticated personal data threats India’s digital ecosystem has ever faced.

Against this backdrop, the DPDP Act, 2023 applies with full force — to every organisation that deploys generative AI on personal data, to every generative AI vendor that processes personal data on behalf of Indian organisations, and to every system that generates outputs touching identifiable individuals. The compliance deadline is 13 May 2027.

Three stories ground this in the reality of Indian organisations today.

Story 1 — Vikram’s HR Query and the LLM That Remembered Everyone

The Scenario

Vikram is an HR manager at a large IT services firm in Bengaluru. His company had deployed a generative AI chatbot — integrated with its internal HR system — to help managers answer HR policy questions, draft performance review language, and retrieve employee records. The system was built using a commercially available LLM, configured with access to the company’s HR data warehouse containing salary records, performance ratings, disciplinary histories, medical leave records, and confidential appraisal notes for approximately twelve thousand employees.

Over several months, Vikram and other managers used the system extensively. Gradually, however, a pattern emerged. When managers queried the chatbot about HR policies unrelated to specific employees, the model’s responses occasionally incorporated specific employee details — names, salary ranges, performance descriptors — drawn from the training context it had been given, even when those details were not requested and not relevant to the query.

In one instance, a senior manager querying about leave policy received a response that referenced a specific employee’s medical leave pattern as an illustrative example — without Vikram having requested that information. That employee had not been informed that her medical leave data was part of the AI system’s accessible context. She had not consented to its use as illustrative material in chatbot responses to other managers.

The DPDP Act Position

This scenario reveals a compliance failure that is specific to generative AI deployments: the gap between data that is technically accessible to the model and data that has lawful authorisation to appear in the model’s outputs.

First — Processing without lawful basis under Section 4. Section 4(1) of the DPDP Act permits processing only for a lawful purpose for which the Data Principal has given consent, or for certain legitimate uses under Section 7. Medical leave data, performance ratings, and salary information were collected for HR administration — a specific, documented purpose. Incorporating this data as illustrative context in chatbot responses to third-party managers is a materially different processing activity. It has neither a consent basis nor a legitimate use basis. The data appeared in the output not because it was requested but because the model’s context window encompassed it and its generation logic surfaced it as relevant. That is still processing — and it is processing without a lawful basis.

Second — Notice failure under Section 5 and Rule 3. The employee whose medical leave data appeared in the chatbot response had never been informed that her personal data would be processed by an AI system for the purpose of generating responses to queries by other managers. Section 5(1) requires that every Data Principal be given notice of the personal data to be processed and the purpose for which it will be processed. Rule 3 of the DPDP Rules requires that this notice be in clear and plain language, with an itemised description of personal data and a specific description of its processing purposes. “Your data will be used for HR administration” does not cover “your data may appear in AI-generated responses to your colleagues’ policy queries.”

Third — Rule 13(3) algorithmic software due diligence obligation. Rule 13(3) of the DPDP Rules places a specific obligation on Significant Data Fiduciaries to observe due diligence to verify that technical measures — including algorithmic software — adopted for storing, updating, or sharing personal data are not likely to pose a risk to the rights of Data Principals. A generative AI system configured with access to twelve thousand employee records, with no output filtering controls, no data minimisation at the inference layer, and no audit trail linking specific outputs to specific data inputs, is an algorithmic architecture that poses a risk to Data Principal rights. Due diligence requires its assessment before deployment — not after an incident surfaces.

Fourth — IS Audit Standard 420 (ICAI ISAS) governance failure. IS Audit Standard 420 requires that organisations deploying AI tools establish governance controls covering approval, validation, reproducibility, accreditation, and oversight. For AI models with non-deterministic outputs — which all current LLMs are — this requires documenting parameters, prompts, and methodology sufficiently to allow independent verification. Vikram’s company had no mechanism to trace which data inputs contributed to any specific chatbot output. When the problematic response appeared, there was no audit trail to investigate it, no log linking the output to the employee’s data, and no governance framework to determine whether a personal data breach had occurred.

IS Audit Standard 430 (ICAI ISAS, Section 3.3) requires that auditors understand the entire lifecycle of personal data — from collection through storage, processing, sharing, and eventual erasure. An LLM deployed on an HR data warehouse has a data lifecycle that extends into every inference it generates. Without lifecycle mapping specific to the AI layer, ISAS 430 compliance cannot be demonstrated.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 12) specifically identifies sensitive data leakage as a primary adversarial risk to AI systems. It notes that AI-assisted offensive tooling can target AI inference systems and AI-integrated workflows — meaning that an LLM with broad access to employee personal data is not merely a compliance liability. It is an attack surface. An adversary who manipulates the model’s input — through prompt injection — can potentially extract personal data that the model has in context, without any system alert being generated.

CERT-In’s framework requires organisations to restrict upload of sensitive information to public AI platforms, define acceptable usage policies, and implement access control and authentication on AI system workflows. Applied to enterprise LLM deployments, these requirements mean that personal data context must be scoped and governed — not simply made available to the model in bulk.

The Compliance Fix

Organisations deploying LLMs or generative AI systems on employee or customer personal data must:

→ Scope the model’s data access to the minimum necessary for each specific use case. An LLM answering HR policy questions does not need access to twelve thousand employees’ disciplinary histories and medical leave records simultaneously.

→ Issue a specific, itemised notice under Rule 3 to every Data Principal whose personal data forms part of the model’s accessible context — identifying the AI system as a processing activity and the inference generation as a purpose.

→ Implement output filtering controls at the inference layer: if the model’s response includes personal data not requested by the querying user, the response must be intercepted, logged, and reviewed before delivery.

→ Maintain an AI processing register that links every LLM deployment to the personal data categories it can access, the purposes for which it was deployed, and the consent records that authorise that processing.

→ Conduct a DPIA under Rule 13 before deploying any generative AI system on personal data at scale, with specific attention to the risk of unintended data surfacing in model outputs.

→ Implement audit logging at the inference level: every query and every response must be logged in a form that allows post-hoc investigation if a personal data concern is raised.

Vikram’s company gave their AI system a broad memory and no output accountability. Under the DPDP Act, the outputs that memory generated were processing events — each one requiring a lawful basis.

The compliance deadline is 13 May 2027. An AI system with a broad memory and no output accountability is not a productivity tool. It is an uncontrolled personal data processor.

(to be continued)

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

DPDP 3.1.3 – Artificial Intelligence and DPDP

June 9, 2026June 9, 2026 Karthik & Sunil#AICompliance, #AIEthics, #AIGovernance, #ArtificialIntelligence, #ConsentManagement, #DataPrincipal, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DPDPAct, #KarthikAndSunil, #ResponsibleAI, #WearableData, ai, artificial-intelligence, chatgpt, DPDP, dpdp-act, news, technologyLeave a comment

Meena’s Story – She Consented to Track Her Health. She Did Not Consent to Losing Her Insurance.

Meena is a 38-year-old marketing professional. Health-conscious, financially aware, and careful about her choices.

She uses a health tracking wearable — diligently logging her steps, sleep, heart rate, and diet. She consented to the health app processing this data to help her monitor her own wellbeing. That felt like a fair exchange.

What she did not know is that her health app shares data with an insurance aggregator platform. That aggregator’s AI model analyses her sleep patterns, exercise consistency, dietary choices, and heart rate variability — and produces a health risk score. When Meena applies for health insurance, she is offered a premium three times higher than her colleague who does not use a wearable.

She tracked her health to take care of herself. The algorithm used that data to price her out of coverage.

The consent problem — specific purpose, specific basis

The DPDP Act, 2023 draws a hard line on this. Section 6(1) requires consent to be specific — each processing purpose requires its own distinct consent. Meena consented to health monitoring for her personal wellness. She did not consent to health risk profiling for insurance underwriting by a third party.

The telemedicine app illustration in the DPDP Act itself makes this principle concrete: when a telemedicine app asks for consent to access the user’s mobile contact list along with health services consent, the Act declares that second element invalid because it is not necessary for the stated purpose. The principle extends directly to Meena’s situation — health data collected for personal monitoring cannot be repurposed for insurance risk scoring without a separate, specific consent for that distinct purpose.

Section 6(6) reinforces this: once consent is withdrawn, the Data Fiduciary must within a reasonable time cease — and cause its Data Processors to cease — processing that personal data. The insurance aggregator is a Data Processor. If Meena withdraws her consent from the health app, that cessation must cascade downstream.

The Puttaswamy warning — AI creates knowledge people never gave

The Supreme Court’s landmark judgment in Justice K.S. Puttaswamy (Retd.) vs Union of India (2018) contains a prescient warning, referenced in the project knowledge base: the creation of new knowledge complicates data privacy law as it involves information the individual did not possess and could not disclose, knowingly or otherwise.

Meena’s health risk score is new knowledge — created by the AI from her data, about her, that she never produced or shared. She shared step counts. The algorithm inferred cardiovascular risk. She shared sleep data. The algorithm inferred stress patterns. She shared dietary logs. The algorithm inferred metabolic risk. None of these inferences are what she consented to share. Yet each is personal data under Section 2(t) of the DPDP Act — data about an identifiable individual — and the creation and commercial use of that inferred data requires a valid processing basis.

The data security dimension — what the AI pipeline holds is a high-value target

IS Audit Module 6 of the ICAI IS Audit 3.0 Course identifies this clearly: most AI applications are based on massive volumes of data to learn and make intelligent decisions. Machine learning systems depend on data which is often sensitive and personal in nature. Due to this systematic learning, these ML systems can become prone to data breach and identity theft.

The aggregated health data pipeline that feeds Meena’s risk score — wearable data, app analytics, insurance scoring parameters — is not just a compliance liability. It is a high-value breach target. If that pipeline is compromised, the personal data of thousands of health-conscious users is exposed, along with the inferred health risk scores that the algorithm produced from it. Under Rule 6(1) of the DPDP Rules, every layer of this pipeline — encryption, access control, logs, breach detection — must be implemented. Under Rule 7, a breach must be reported to the Data Protection Board within 72 hours.

The CERT-In Guidelines on Secure Adoption and Governance of AI Systems (Version 1.0, 25 May 2026) specifically require organisations to ensure secure and compliant handling of data processed by AI systems — classifying and protecting sensitive data, defining retention and deletion policies, and monitoring AI-related data movement and third-party handling. A health data pipeline shared with an insurance aggregator, without documented data handling obligations, fails this standard.

What Meena is entitled to — and what the platform must build

Under Section 11 of the DPDP Act, Meena has the right to access a summary of all personal data being processed about her, and the identities of all Data Fiduciaries and Processors with whom it was shared. She has the right to know that her wearable data reached an insurance aggregator. She was never told.

Under Section 12, she has the right to request erasure of her personal data. That erasure must cascade to the aggregator. The health app cannot fulfil the erasure obligation without a contractual mechanism to cause downstream processors to delete as well — which Rule 6(1)(f) requires to be built into the Data Processor contract.

Under Section 13, she has the right to grieve the processing. The Data Fiduciary must respond within the prescribed period. If the platform cannot explain how her health data reached an insurance pricing model — it cannot respond to that grievance.

The question every health tech and insurtech organisation must answer

Does your data-sharing agreement with downstream AI platforms define, in writing, the specific purposes for which shared personal data may be used? Does it prohibit repurposing of health data for insurance risk scoring without separate user consent? Does it require the downstream processor to honour erasure requests? Does it include security safeguards aligned with Rule 6?

If the answer to any of these is no — Meena’s situation is not a story. It is your organisation’s next compliance exposure.

Series 2, Episode 1 — Post 3 of 3 | DPDP Meets Emerging Technologies.

Sources: DPDP Act 2023 (Sections 2(t), 6(1), 6(6), 8, 11, 12, 13) | DPDP Rules 2025 (Rules 6(1), 6(1)(f), 7) | ICAI IS Audit 3.0 Course Materials | CERT-In Guidelines on Secure Adoption and Governance of AI Systems (Version 1.0, 25 May 2026) | Justice K.S. Puttaswamy (Retd.) vs Union of India (2018)

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors:
This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.1.2 – Artificial Intelligence and DPDP – The Algorithm That Inherited Someone Else’s Bias — and Gave It to Him

June 6, 2026June 5, 2026 Karthik & Sunil#AI, #AICompliance, #AIEthics, #AIGovernance, #AlgorithmicBias, #ArtificialIntelligence, #CISO, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DigitalIndia, #DPDPCompliance, #ITCompliance, #KarthikAndSunil, #ResponsibleAILeave a comment

Rajan is a 45-year-old entrepreneur from a small town in Bihar. He has built a profitable distribution business, has a strong local reputation, and is actively applying for senior roles through an AI-driven job platform.

He never gets shortlisted. Not once.

After months of rejections, a friend in tech takes a look at his profile. The friend tells him quietly: “The algorithm probably doesn’t recognise you. You don’t look like the people it was trained to select.”

Rajan did not know an algorithm was deciding his future. He assumed a recruiter had read his profile and found it wanting. The recruiter never saw it.

When the algorithm learns from biased history

AI systems learn patterns from historical data. The assumption is that historical data reflects good decisions. But what if those decisions were themselves biased — shaped by decades of geographic, socioeconomic, and institutional inequality?

IS Audit Module 6 of the ICAI IS Audit 3.0 Course is explicit: a big problem with AI systems is that their level of goodness or badness depends on how much data they are trained on. Bad data is often associated with ethnic, communal, gender or racial biases. Proprietary algorithms are used to find out information like who gets bail, whose loan is sanctioned. If the bias hidden in the algorithms — which take crucial decisions — goes unrecognised, it could lead to unethical and unfair results.

Rajan’s algorithm had learned that successful candidates, historically, came from certain geographies, certain institutions, and certain career trajectories. It had never been trained to question whether that pattern reflected genuine merit or merely reinforced historical exclusion. The algorithm was confident. The algorithm was wrong. And Rajan had no way of knowing — or challenging — either.

The Puttaswamy dimension — data creates new knowledge about peopleThe Supreme Court’s judgment in Justice K.S. Puttaswamy (Retd.) vs Union of India (2018), included in the project knowledge base, addresses exactly this: the creation of new knowledge complicates data privacy law as it involves information the individual did not possess and could not disclose, knowingly or otherwise.

The job platform’s algorithm created a new piece of knowledge about Rajan — a risk score, a fit score, a ranking — that he did not produce, did not verify, and did not consent to. That score is personal data under Section 2(t) of the DPDP Act: any data about an individual who is identifiable by or in relation to such data. Rajan is identifiable. The score is about him. It is personal data — and its creation and use must have a valid basis.

The DPDP and algorithmic bias — this is a legal obligation, not just an ethical one

Section 10(2)(c)(i) of the DPDP Act requires every Significant Data Fiduciary to conduct a Data Protection Impact Assessment that includes assessment and management of risk to the rights of Data Principals. Rajan’s right to non-discriminatory treatment flows from his fundamental rights under the Constitution and is directly implicated when a biased algorithm systematically excludes him from opportunity based on geographic origin.

Rule 13(3) of the DPDP Rules, 2025 goes further: a Significant Data Fiduciary must observe due diligence to verify that algorithmic software adopted for processing personal data is not likely to pose a risk to the rights of Data Principals. The word “pose a risk” is important. The organisation does not need to wait for Rajan to be demonstrably harmed. The obligation is preventive — to verify, before and during deployment, that the algorithm does not create this risk.

A hiring algorithm trained on historically biased data, operating without bias auditing, and producing systematically skewed outcomes for certain demographic groups poses a risk to Data Principal rights as a matter of its design. The DPDP Act requires that risk to be identified, documented, and mitigated.

The CERT-In AI governance framework — human oversight is mandatory

The CERT-In Guidelines on Secure Adoption and Governance of Artificial Intelligence Systems (Version 1.0, 25 May 2026) identify Human Oversight and Decision Governance as a core control area requiring organisations to: validate AI-generated outputs, restrict fully autonomous critical actions, and maintain auditability and approval mechanisms.

A hiring shortlist generated entirely by an AI model — with no human review of borderline cases, no audit trail of the factors that drove the ranking, and no mechanism to flag demographically anomalous patterns — fails the human oversight standard. AI-assisted decisions are permissible. Fully autonomous, unreviewed decisions that affect an individual’s livelihood are a governance failure.

What Rajan is entitled to — and what the platform owes him

Under Section 11 of the DPDP Act, Rajan has the right to access a summary of what personal data the platform is processing about him and what processing activities are being undertaken. He has the right to know that a risk score or fit score has been generated, even if he does not know to ask for it.

Under Section 13, he has the right to raise a grievance about how his personal data has been processed. If the platform cannot explain, at the individual level, why its algorithm ranked him as it did — and what data drove that ranking — it cannot respond to that grievance in any meaningful way.

The algorithm decided. DPDP says that decision must be auditable, explainable, and subject to human governance. Rajan deserves at least that much.

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors:
This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

Note: The images used are AI Generated Images

DPDP 3.1 – Artificial Intelligence and DPDP: When the Algorithm Decides

June 4, 2026June 4, 2026 Karthik & Sunil#AI, #AICompliance, #AIEthics, #AIGovernance, #AlgorithmicBias, #ArtificialIntelligence, #CISO, #DataPrincipal, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DigitalIndia, #DPDPAct, #DPDPCompliance, #ITCompliance, #KarthikAndSunil, #ResponsibleAI1 Comment

DPDP Series 2, Episode 1.1

Priya’s Story

The AI That Rejected Her Home Loan Without Reading Her File

Priya is a 31-year-old schoolteacher in a village in Tirunelveli. Clean credit history. Stable government salary. Zero defaults.

She applies for a home loan through a fintech platform. Within seconds, the response arrives: Rejected.

No reason. No human. No explanation. An AI credit-scoring algorithm made the call — silently, instantly, and without looking her in the eye.

She tries a second platform. Same outcome. She begins to wonder what is wrong with her, when the real question is: what is wrong with the algorithm?

Why AI credit scoring creates a DPDP problem

IS Audit Module 6 of the ICAI IS Audit 3.0 Course is direct: AI is widely used in banking apps to provide a faster, more accurate assessment of a potential borrower at less cost, accounting for a wider variety of factors. Credit scoring provided by AI is based on more complex and sophisticated rules compared to traditional systems.

More complex. More factors. And entirely invisible to Priya.

The problem is this: Priya’s loan application was rejected because the AI model had never meaningfully encountered a borrower profile like hers — a government employee in a Tier-3 city, with a savings-heavy profile and no credit card history — trained predominantly on urban, credit-card-using, high-transaction-volume data. IS Audit Module 6 names this explicitly: datasets applicable to AI applications to learn are really rare. Models trained on incomplete data produce biased outcomes for underrepresented groups.

The algorithm was not wrong about what it was trained to do. It was wrong about what it was trained on. And Priya paid the price.

The DPDP dimension — consent was not built for this

When Priya downloaded the fintech app and applied for the loan, she tapped “I Agree” to a terms-of-service document she likely did not read in full. That consent, under the DPDP Act, 2023, is not valid for everything the AI subsequently did with her data.

Section 6(1) of the DPDP Act is unambiguous: consent must be free, specific, informed, unconditional and unambiguous, limited to such personal data as is necessary for the specified purpose.

The specified purpose was loan evaluation. But the AI ingested Priya’s location history, app usage patterns, device behaviour, social interactions, and transaction metadata — far beyond what is necessary to evaluate creditworthiness. Every data element beyond the specified purpose is processing without a valid basis.

Furthermore, if her data was used to train or refine the AI model — improving the algorithm for future use — that is a separate processing purpose that required separate consent. She did not give it. Section 6(1) requires each distinct purpose to be separately consented to.

The right she did not know she had

Under Section 11 of the DPDP Act, Priya has the right to access a summary of all personal data being processed about her — including the processing activities undertaken. She has the right to ask what the algorithm used, what it concluded, and why.

Under Section 13, she has the right to raise a grievance with the Data Fiduciary. An AI system that cannot explain its decision — cannot identify what data points drove the rejection — cannot satisfy this right. A black-box model is, under DPDP, a grievance waiting to happen.

The CERT-In Guidelines on Secure Adoption and Governance of Artificial Intelligence Systems (Version 1.0, 25 May 2026) identify Human Oversight and Decision Governance as a mandatory control: validate AI-generated outputs, restrict fully autonomous critical actions, and maintain auditability and approval mechanisms. An AI that rejects a loan application with no human review and no audit trail fails every limb of this control.

The question every AI-first fintech must answer

Can you tell Priya — specifically, in relation to her file — what personal data the algorithm used, whether that data was within the scope of her consent, and how it contributed to the rejection decision?

If the answer is “our model doesn’t work that way” — the compliance gap is not in the algorithm. It is in the governance architecture around it.

The DPDP Act is not asking AI to stop working. It is asking AI to work accountably.

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors:
This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

Karthik & Sunil

Chartered Accountants

artificial-intelligence

DPDP 3.7.3: DPDP & Generative AI – She mentioned a prescription drug in a customer service chatbot. Twelve months later, she received a targeted advertisement for a competing medication. The only digital record of its relevance to her was that complaint conversation.

Like this:

DPDP 3.7.2: DPDP & Generative AI – Ananya’s Voice Note and the Deepfake That Signed a Contract

Like this:

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

Like this:

DPDP 3.1.3 – Artificial Intelligence and DPDP

Like this:

DPDP 3.1.2 – Artificial Intelligence and DPDP – The Algorithm That Inherited Someone Else’s Bias — and Gave It to Him

Like this:

DPDP 3.1 – Artificial Intelligence and DPDP: When the Algorithm Decides

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: