Month: September 2026

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

Karthik & Sunil, , , Leave a comment

INTRODUCTION

Generative AI and DPDP compliance represent the most consequential frontier in India’s data protection landscape — and the one that is evolving fastest.

Generative AI refers to systems — large language models (LLMs), image generators, voice synthesisers, code generators, and multimodal models — that are trained on vast quantities of data and can produce new content: text, images, audio, video, and code that did not previously exist. These systems are already embedded in enterprise workflows across India: drafting documents, summarising records, answering customer queries, analysing contracts, generating reports, and supporting HR processes.

IS Audit 3.0 by ICAI identifies data privacy and security as a primary AI risk, noting that most AI applications depend on massive volumes of data — often sensitive and personal in nature — to learn and make intelligent decisions. As machine learning systems become more capable, they also become more capable of learning, retaining, and inadvertently reproducing the personal data on which they were trained or with which they were prompted.

IS Audit Standard 420 (ICAI ISAS) goes further, acknowledging that generative AI models present non-deterministic outputs — meaning the same input may produce different outputs across sessions. For AI models with these characteristics, it notes, reproducibility requires that parameters, prompts, and methodology be documented sufficiently to allow independent verification. This observation is not merely an audit standard. It is a compliance architecture requirement: if a generative AI system cannot reproduce how a personal data output was generated, the Data Fiduciary deploying it cannot adequately defend its processing decisions under the DPDP Act.

CERT-In’s Blueprint on AI-Assisted Cyber Threats (CIGU20260002, May 2026) adds a threat dimension that no organisation can afford to ignore: generative AI is being actively leveraged by threat actors to generate highly convincing phishing content, deepfake voice and video fraud, synthetic identities, and spear phishing campaigns that bypass traditional awareness-based detection — all of which directly target personal data of Indian citizens. Consequently, generative AI is not merely a compliance risk as a tool of enterprise productivity. It is simultaneously the instrument of some of the most sophisticated personal data threats India’s digital ecosystem has ever faced.

Against this backdrop, the DPDP Act, 2023 applies with full force — to every organisation that deploys generative AI on personal data, to every generative AI vendor that processes personal data on behalf of Indian organisations, and to every system that generates outputs touching identifiable individuals. The compliance deadline is 13 May 2027.

Three stories ground this in the reality of Indian organisations today.

Story 1 — Vikram’s HR Query and the LLM That Remembered Everyone

The Scenario

Vikram is an HR manager at a large IT services firm in Bengaluru. His company had deployed a generative AI chatbot — integrated with its internal HR system — to help managers answer HR policy questions, draft performance review language, and retrieve employee records. The system was built using a commercially available LLM, configured with access to the company’s HR data warehouse containing salary records, performance ratings, disciplinary histories, medical leave records, and confidential appraisal notes for approximately twelve thousand employees.

Over several months, Vikram and other managers used the system extensively. Gradually, however, a pattern emerged. When managers queried the chatbot about HR policies unrelated to specific employees, the model’s responses occasionally incorporated specific employee details — names, salary ranges, performance descriptors — drawn from the training context it had been given, even when those details were not requested and not relevant to the query.

In one instance, a senior manager querying about leave policy received a response that referenced a specific employee’s medical leave pattern as an illustrative example — without Vikram having requested that information. That employee had not been informed that her medical leave data was part of the AI system’s accessible context. She had not consented to its use as illustrative material in chatbot responses to other managers.

The DPDP Act Position

This scenario reveals a compliance failure that is specific to generative AI deployments: the gap between data that is technically accessible to the model and data that has lawful authorisation to appear in the model’s outputs.

First — Processing without lawful basis under Section 4. Section 4(1) of the DPDP Act permits processing only for a lawful purpose for which the Data Principal has given consent, or for certain legitimate uses under Section 7. Medical leave data, performance ratings, and salary information were collected for HR administration — a specific, documented purpose. Incorporating this data as illustrative context in chatbot responses to third-party managers is a materially different processing activity. It has neither a consent basis nor a legitimate use basis. The data appeared in the output not because it was requested but because the model’s context window encompassed it and its generation logic surfaced it as relevant. That is still processing — and it is processing without a lawful basis.

Second — Notice failure under Section 5 and Rule 3. The employee whose medical leave data appeared in the chatbot response had never been informed that her personal data would be processed by an AI system for the purpose of generating responses to queries by other managers. Section 5(1) requires that every Data Principal be given notice of the personal data to be processed and the purpose for which it will be processed. Rule 3 of the DPDP Rules requires that this notice be in clear and plain language, with an itemised description of personal data and a specific description of its processing purposes. “Your data will be used for HR administration” does not cover “your data may appear in AI-generated responses to your colleagues’ policy queries.”

Third — Rule 13(3) algorithmic software due diligence obligation. Rule 13(3) of the DPDP Rules places a specific obligation on Significant Data Fiduciaries to observe due diligence to verify that technical measures — including algorithmic software — adopted for storing, updating, or sharing personal data are not likely to pose a risk to the rights of Data Principals. A generative AI system configured with access to twelve thousand employee records, with no output filtering controls, no data minimisation at the inference layer, and no audit trail linking specific outputs to specific data inputs, is an algorithmic architecture that poses a risk to Data Principal rights. Due diligence requires its assessment before deployment — not after an incident surfaces.

Fourth — IS Audit Standard 420 (ICAI ISAS) governance failure. IS Audit Standard 420 requires that organisations deploying AI tools establish governance controls covering approval, validation, reproducibility, accreditation, and oversight. For AI models with non-deterministic outputs — which all current LLMs are — this requires documenting parameters, prompts, and methodology sufficiently to allow independent verification. Vikram’s company had no mechanism to trace which data inputs contributed to any specific chatbot output. When the problematic response appeared, there was no audit trail to investigate it, no log linking the output to the employee’s data, and no governance framework to determine whether a personal data breach had occurred.

IS Audit Standard 430 (ICAI ISAS, Section 3.3) requires that auditors understand the entire lifecycle of personal data — from collection through storage, processing, sharing, and eventual erasure. An LLM deployed on an HR data warehouse has a data lifecycle that extends into every inference it generates. Without lifecycle mapping specific to the AI layer, ISAS 430 compliance cannot be demonstrated.

The CERT-In Dimension

CERT-In CIGU20260002 (May 2026, Section 12) specifically identifies sensitive data leakage as a primary adversarial risk to AI systems. It notes that AI-assisted offensive tooling can target AI inference systems and AI-integrated workflows — meaning that an LLM with broad access to employee personal data is not merely a compliance liability. It is an attack surface. An adversary who manipulates the model’s input — through prompt injection — can potentially extract personal data that the model has in context, without any system alert being generated.

CERT-In’s framework requires organisations to restrict upload of sensitive information to public AI platforms, define acceptable usage policies, and implement access control and authentication on AI system workflows. Applied to enterprise LLM deployments, these requirements mean that personal data context must be scoped and governed — not simply made available to the model in bulk.

The Compliance Fix

Organisations deploying LLMs or generative AI systems on employee or customer personal data must:

→ Scope the model’s data access to the minimum necessary for each specific use case. An LLM answering HR policy questions does not need access to twelve thousand employees’ disciplinary histories and medical leave records simultaneously.

→ Issue a specific, itemised notice under Rule 3 to every Data Principal whose personal data forms part of the model’s accessible context — identifying the AI system as a processing activity and the inference generation as a purpose.

→ Implement output filtering controls at the inference layer: if the model’s response includes personal data not requested by the querying user, the response must be intercepted, logged, and reviewed before delivery.

→ Maintain an AI processing register that links every LLM deployment to the personal data categories it can access, the purposes for which it was deployed, and the consent records that authorise that processing.

→ Conduct a DPIA under Rule 13 before deploying any generative AI system on personal data at scale, with specific attention to the risk of unintended data surfacing in model outputs.

→ Implement audit logging at the inference level: every query and every response must be logged in a form that allows post-hoc investigation if a personal data concern is raised.

Vikram’s company gave their AI system a broad memory and no output accountability. Under the DPDP Act, the outputs that memory generated were processing events — each one requiring a lawful basis.

The compliance deadline is 13 May 2027. An AI system with a broad memory and no output accountability is not a productivity tool. It is an uncontrolled personal data processor.

(to be continued)

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors: This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.7.1: DPDP & Generative AI – When the Machine Learns What It Should Not Know

DPDP 3.6.3 : DPDP & RPA – Sunita’s KYC Bot and the Credentials That Were Left Unlocked

Karthik & Sunil, , , , , , , , Leave a comment

The Scenario

Sunita is the compliance head of a mid-sized non-banking financial company (NBFC) in Mumbai. Her organisation had deployed an RPA solution to automate the KYC onboarding process — the bot collected applicant documents, cross-referenced identity data against government databases, triggered verification workflows, and updated the CRM with onboarding status. At its peak, the bot processed over eight hundred KYC applications per day.

The bot operated using a shared service account with administrative-level access to the customer data platform, the CRM, and the document management system. The credentials for this service account were stored in plain text in a configuration file on the deployment server — a common but critically insecure practice that IS Audit 3.0 had flagged as a known RPA risk.

In March 2025, an external attacker gained access to the deployment server, retrieved the bot’s credentials, and used them to access the customer data platform. Over three days, approximately 14,000 customer records — including names, Aadhaar numbers, PAN cards, income data, and bank account details — were exfiltrated. Because the bot routinely accessed the same systems under the same credentials, the access logs did not initially flag the intrusion as anomalous.

Sunita’s team discovered the breach only when a customer reported a fraudulent loan application made in her name.

The DPDP Act Position

The NBFC’s RPA deployment created four overlapping compliance failures — each of which independently triggered DPDP Act liability.

First — Rule 6(1)(a) security safeguard failure: absence of encryption or credential protection. Rule 6(1)(a) requires the Data Fiduciary to protect personal data through encryption, obfuscation, masking, or the use of virtual tokens. Storing bot credentials in plain text in a configuration file is the antithesis of this requirement. These credentials provided direct, unrestricted access to over 14,000 customer records containing sensitive personal data. The absence of any credential protection is a straightforward violation of the minimum security standard.

Second — Rule 6(1)(b) failure: inadequate access control. Rule 6(1)(b) requires appropriate measures to control access to computer resources. A shared service account with administrative-level access to multiple systems is structurally non-compliant. Principle of least privilege — a foundational security and governance concept — requires that the bot be granted only the minimum access necessary to complete its specific task. Administrative access across the customer data platform, CRM, and document management system simultaneously is access far in excess of what any individual process requires.

Third — Rule 6(1)(c) failure: inadequate monitoring and log review. Rule 6(1)(c) requires visibility on the accessing of personal data through appropriate logs, monitoring, and review to enable detection of unauthorised access. The breach went undetected for three days because the attacker’s access was indistinguishable from the bot’s normal access pattern. This means the NBFC had no mechanism to detect anomalous access against the baseline of expected bot behaviour — a direct failure of the monitoring obligation.

Fourth — Section 8(6) and Rule 7 breach notification failure. The NBFC discovered the breach only through a customer report. Had detection systems been in place, earlier identification would have been possible. Upon discovery, Rule 7 required immediate notification to affected Data Principals and notification to the Data Protection Board of India within seventy-two hours — including the nature and extent of the breach, its likely consequences, measures being taken to mitigate risk, and remedial steps to prevent recurrence. The delay caused by inadequate monitoring made timely compliance with these obligations impossible.

The penalty exposure under the Schedule to the DPDP Act is substantial. A breach of the obligation to take reasonable security safeguards under Section 8(5) attracts a penalty of up to ₹250 crore. A failure to notify under Section 8(6) attracts up to ₹200 crore. These are not theoretical upper limits — they are the statutory framework within which the Data Protection Board will exercise its discretion.

The IS Audit 3.0 Perspective

The study material on IS Audit 3.0 by ICAI identifies tool selection risk and operational execution risk as primary RPA governance failures. Specifically, it notes that when bots are deployed without a proper operating model, roles and responsibilities become blurred in production. In Sunita’s case, the operating model had no provision for credential management, access scoping, or anomaly detection — a governance gap that directly enabled the breach.

IS Audit 3.0 requires that RPA governance frameworks specifically address operational risk and data security through cross-functional team involvement, including legal and risk functions. Had this standard been applied, credential storage protocols, access control scoping, and monitoring requirements would have been part of the deployment framework — not afterthoughts discovered in a breach investigation.

Furthermore, IS Audit 3.0 notes that auditors engaging with RPA-enabled organisations must focus on control design opportunities and develop new testing approaches for bot-driven processes. Specifically in the DPDP compliance context, this means IS auditors should treat each RPA bot as a data processing entity and audit its access rights, credential management, output routing, and log review protocols as rigorously as they would audit any human-operated system.

The Compliance Fix

Any NBFC, financial institution, or organisation deploying RPA for processes involving sensitive personal data must urgently address:

→ Credential security: bot service account credentials must be stored in an encrypted secrets management vault — never in plain text in configuration files. This is the minimum standard under Rule 6(1)(a).

→ Access scoping: every bot must be assigned a service account with the minimum permissions necessary for its specific task. Administrative-level, cross-system access for a single bot is never appropriate when individual process-scoped access is achievable.

→ Anomaly detection baselines: organisations must establish normal bot behaviour profiles — typical access volumes, access times, data retrieval patterns — and configure monitoring alerts that fire when actual behaviour deviates from these baselines. Rule 6(1)(c) requires this level of visibility.

→ Breach response rehearsal: RPA-related breaches are difficult to detect quickly because bot access is indistinguishable from normal operations without baseline monitoring. Organisations must rehearse their breach detection, escalation, and notification workflows specifically for bot-related incidents — so that the Rule 7 seventy-two-hour notification obligation can be met when it matters.

→ IS Audit 3.0-aligned annual review: every RPA deployment processing personal data must be audited annually using IS Audit 3.0 standards, with specific attention to credential management, access control design, and log review sufficiency.

Sunita’s NBFC had an efficient KYC bot. However, it had no compliant security architecture around it. Fourteen thousand customers paid the price for that gap.

The Central Compliance Question for RPA Deployments

The unique DPDP compliance challenge that RPA creates is one of invisibility. Unlike a human employee who processes personal data visibly and accountably, an RPA bot processes data silently, continuously, and often without any mechanism for the organisation to know precisely what it touched, where it sent it, and what secondary data stores it created in the process.

The DPDP Act, however, applies equally to automated and human processing. Section 2(x) defines processing to include every operation performed on digital personal data — and Section 2(b) defines automated as any digital process capable of operating automatically. Together, these definitions mean that an RPA bot is not operating outside the Act’s scope. It is operating at the centre of it.

Every organisation deploying RPA on personal data must therefore answer five questions — not once at deployment, but continuously throughout the bot’s operational lifecycle:

One. Have we mapped the complete data footprint of every RPA process — including every system accessed, every intermediate file created, every output generated, and every downstream system that receives data?

Two. Does every bot service account operate on the principle of least privilege — with access strictly scoped to the minimum data necessary for the specific task it performs?

Three. Are bot credentials stored securely — encrypted in a secrets management system, rotated regularly, and audited for access?

Four. Do we have anomaly detection baselines for each bot’s data access behaviour, with alerts configured to flag deviations for human review?

Five. Can we honour a Data Principal’s Section 12(3) erasure request across the full data footprint of a bot process — including staging environments, output files, and communication logs?

IS Audit 3.0 states that a governance structure defining roles and responsibilities for automation activities is essential for successful RPA initiatives. Under the DPDP Act, that governance structure is not a success factor — it is a legal obligation.

What Responsible RPA and DPDP Compliance Looks Like

IS Audit 3.0 correctly observes that RPA is not replacing accountants but augmenting their effectiveness — creating space for professionals to focus on higher-value analytical and advisory work. That observation is true, and the productivity benefits of well-governed RPA are real. However, realising those benefits without simultaneously creating DPDP compliance liability requires a deliberate, structured approach to bot governance.

Responsible RPA and DPDP compliance means treating every bot as a named, scoped, audited data processor. It means mapping every process before deployment, monitoring every execution in production, and maintaining the ability to respond to any Data Principal rights request across every processing environment the bot touches.

The organisations that achieve this are not those with the most bots. They are those with the best governance around each one. Automation scales efficiency — but without proper controls, it also scales compliance risk at exactly the same rate.

The DPDP Act has made that tradeoff unambiguous. Organisations now need to make a deliberate choice. The compliance deadline is 13 May 2027.

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors: This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.6.3 : DPDP & RPA – Sunita’s KYC Bot and the Credentials That Were Left Unlocked