DPDP 3.1.3 – Artificial Intelligence and DPDP

June 9, 2026June 9, 2026 Karthik & Sunil#AICompliance, #AIEthics, #AIGovernance, #ArtificialIntelligence, #ConsentManagement, #DataPrincipal, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DPDPAct, #KarthikAndSunil, #ResponsibleAI, #WearableData, ai, artificial-intelligence, chatgpt, DPDP, dpdp-act, news, technologyLeave a comment

Meena’s Story – She Consented to Track Her Health. She Did Not Consent to Losing Her Insurance.

Meena is a 38-year-old marketing professional. Health-conscious, financially aware, and careful about her choices.

She uses a health tracking wearable — diligently logging her steps, sleep, heart rate, and diet. She consented to the health app processing this data to help her monitor her own wellbeing. That felt like a fair exchange.

What she did not know is that her health app shares data with an insurance aggregator platform. That aggregator’s AI model analyses her sleep patterns, exercise consistency, dietary choices, and heart rate variability — and produces a health risk score. When Meena applies for health insurance, she is offered a premium three times higher than her colleague who does not use a wearable.

She tracked her health to take care of herself. The algorithm used that data to price her out of coverage.

The consent problem — specific purpose, specific basis

The DPDP Act, 2023 draws a hard line on this. Section 6(1) requires consent to be specific — each processing purpose requires its own distinct consent. Meena consented to health monitoring for her personal wellness. She did not consent to health risk profiling for insurance underwriting by a third party.

The telemedicine app illustration in the DPDP Act itself makes this principle concrete: when a telemedicine app asks for consent to access the user’s mobile contact list along with health services consent, the Act declares that second element invalid because it is not necessary for the stated purpose. The principle extends directly to Meena’s situation — health data collected for personal monitoring cannot be repurposed for insurance risk scoring without a separate, specific consent for that distinct purpose.

Section 6(6) reinforces this: once consent is withdrawn, the Data Fiduciary must within a reasonable time cease — and cause its Data Processors to cease — processing that personal data. The insurance aggregator is a Data Processor. If Meena withdraws her consent from the health app, that cessation must cascade downstream.

The Puttaswamy warning — AI creates knowledge people never gave

The Supreme Court’s landmark judgment in Justice K.S. Puttaswamy (Retd.) vs Union of India (2018) contains a prescient warning, referenced in the project knowledge base: the creation of new knowledge complicates data privacy law as it involves information the individual did not possess and could not disclose, knowingly or otherwise.

Meena’s health risk score is new knowledge — created by the AI from her data, about her, that she never produced or shared. She shared step counts. The algorithm inferred cardiovascular risk. She shared sleep data. The algorithm inferred stress patterns. She shared dietary logs. The algorithm inferred metabolic risk. None of these inferences are what she consented to share. Yet each is personal data under Section 2(t) of the DPDP Act — data about an identifiable individual — and the creation and commercial use of that inferred data requires a valid processing basis.

The data security dimension — what the AI pipeline holds is a high-value target

IS Audit Module 6 of the ICAI IS Audit 3.0 Course identifies this clearly: most AI applications are based on massive volumes of data to learn and make intelligent decisions. Machine learning systems depend on data which is often sensitive and personal in nature. Due to this systematic learning, these ML systems can become prone to data breach and identity theft.

The aggregated health data pipeline that feeds Meena’s risk score — wearable data, app analytics, insurance scoring parameters — is not just a compliance liability. It is a high-value breach target. If that pipeline is compromised, the personal data of thousands of health-conscious users is exposed, along with the inferred health risk scores that the algorithm produced from it. Under Rule 6(1) of the DPDP Rules, every layer of this pipeline — encryption, access control, logs, breach detection — must be implemented. Under Rule 7, a breach must be reported to the Data Protection Board within 72 hours.

The CERT-In Guidelines on Secure Adoption and Governance of AI Systems (Version 1.0, 25 May 2026) specifically require organisations to ensure secure and compliant handling of data processed by AI systems — classifying and protecting sensitive data, defining retention and deletion policies, and monitoring AI-related data movement and third-party handling. A health data pipeline shared with an insurance aggregator, without documented data handling obligations, fails this standard.

What Meena is entitled to — and what the platform must build

Under Section 11 of the DPDP Act, Meena has the right to access a summary of all personal data being processed about her, and the identities of all Data Fiduciaries and Processors with whom it was shared. She has the right to know that her wearable data reached an insurance aggregator. She was never told.

Under Section 12, she has the right to request erasure of her personal data. That erasure must cascade to the aggregator. The health app cannot fulfil the erasure obligation without a contractual mechanism to cause downstream processors to delete as well — which Rule 6(1)(f) requires to be built into the Data Processor contract.

Under Section 13, she has the right to grieve the processing. The Data Fiduciary must respond within the prescribed period. If the platform cannot explain how her health data reached an insurance pricing model — it cannot respond to that grievance.

The question every health tech and insurtech organisation must answer

Does your data-sharing agreement with downstream AI platforms define, in writing, the specific purposes for which shared personal data may be used? Does it prohibit repurposing of health data for insurance risk scoring without separate user consent? Does it require the downstream processor to honour erasure requests? Does it include security safeguards aligned with Rule 6?

If the answer to any of these is no — Meena’s situation is not a story. It is your organisation’s next compliance exposure.

Series 2, Episode 1 — Post 3 of 3 | DPDP Meets Emerging Technologies.

Sources: DPDP Act 2023 (Sections 2(t), 6(1), 6(6), 8, 11, 12, 13) | DPDP Rules 2025 (Rules 6(1), 6(1)(f), 7) | ICAI IS Audit 3.0 Course Materials | CERT-In Guidelines on Secure Adoption and Governance of AI Systems (Version 1.0, 25 May 2026) | Justice K.S. Puttaswamy (Retd.) vs Union of India (2018)

Disclaimer

The contents of this post are intended for general awareness and informational purposes only. They do not constitute legal opinion, professional advice, consultancy, statutory interpretation, or a recommendation to act in any particular manner.

The Digital Personal Data Protection Act, 2023, related rules, notifications, regulatory guidance and judicial interpretations may evolve from time to time. The applicability of the law may also vary depending on the facts, sector, nature of data processing, organisational role, contractual terms and compliance framework.

Readers should not rely solely on this post for making legal, business, HR, technology, data-processing or compliance decisions. Specific advice from a qualified legal, privacy, cybersecurity, governance or compliance professional should be obtained before acting on any matter discussed.

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Authors:
This article has been co-authored by CA. Sunil Elayadath and CA. Karthik Narayanan S, Partners of Karthik & Sunil, together with Mr. Dhanesh P. K., Designated Partner, DSK Sustainability Tech.

DPDP 3.1.2 – Artificial Intelligence and DPDP – The Algorithm That Inherited Someone Else’s Bias — and Gave It to Him

June 6, 2026June 5, 2026 Karthik & Sunil#AI, #AICompliance, #AIEthics, #AIGovernance, #AlgorithmicBias, #ArtificialIntelligence, #CISO, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DigitalIndia, #DPDPCompliance, #ITCompliance, #KarthikAndSunil, #ResponsibleAILeave a comment

Rajan is a 45-year-old entrepreneur from a small town in Bihar. He has built a profitable distribution business, has a strong local reputation, and is actively applying for senior roles through an AI-driven job platform.

He never gets shortlisted. Not once.

After months of rejections, a friend in tech takes a look at his profile. The friend tells him quietly: “The algorithm probably doesn’t recognise you. You don’t look like the people it was trained to select.”

Rajan did not know an algorithm was deciding his future. He assumed a recruiter had read his profile and found it wanting. The recruiter never saw it.

When the algorithm learns from biased history

AI systems learn patterns from historical data. The assumption is that historical data reflects good decisions. But what if those decisions were themselves biased — shaped by decades of geographic, socioeconomic, and institutional inequality?

IS Audit Module 6 of the ICAI IS Audit 3.0 Course is explicit: a big problem with AI systems is that their level of goodness or badness depends on how much data they are trained on. Bad data is often associated with ethnic, communal, gender or racial biases. Proprietary algorithms are used to find out information like who gets bail, whose loan is sanctioned. If the bias hidden in the algorithms — which take crucial decisions — goes unrecognised, it could lead to unethical and unfair results.

Rajan’s algorithm had learned that successful candidates, historically, came from certain geographies, certain institutions, and certain career trajectories. It had never been trained to question whether that pattern reflected genuine merit or merely reinforced historical exclusion. The algorithm was confident. The algorithm was wrong. And Rajan had no way of knowing — or challenging — either.

The Puttaswamy dimension — data creates new knowledge about peopleThe Supreme Court’s judgment in Justice K.S. Puttaswamy (Retd.) vs Union of India (2018), included in the project knowledge base, addresses exactly this: the creation of new knowledge complicates data privacy law as it involves information the individual did not possess and could not disclose, knowingly or otherwise.

The job platform’s algorithm created a new piece of knowledge about Rajan — a risk score, a fit score, a ranking — that he did not produce, did not verify, and did not consent to. That score is personal data under Section 2(t) of the DPDP Act: any data about an individual who is identifiable by or in relation to such data. Rajan is identifiable. The score is about him. It is personal data — and its creation and use must have a valid basis.

The DPDP and algorithmic bias — this is a legal obligation, not just an ethical one

Section 10(2)(c)(i) of the DPDP Act requires every Significant Data Fiduciary to conduct a Data Protection Impact Assessment that includes assessment and management of risk to the rights of Data Principals. Rajan’s right to non-discriminatory treatment flows from his fundamental rights under the Constitution and is directly implicated when a biased algorithm systematically excludes him from opportunity based on geographic origin.

Rule 13(3) of the DPDP Rules, 2025 goes further: a Significant Data Fiduciary must observe due diligence to verify that algorithmic software adopted for processing personal data is not likely to pose a risk to the rights of Data Principals. The word “pose a risk” is important. The organisation does not need to wait for Rajan to be demonstrably harmed. The obligation is preventive — to verify, before and during deployment, that the algorithm does not create this risk.

A hiring algorithm trained on historically biased data, operating without bias auditing, and producing systematically skewed outcomes for certain demographic groups poses a risk to Data Principal rights as a matter of its design. The DPDP Act requires that risk to be identified, documented, and mitigated.

The CERT-In AI governance framework — human oversight is mandatory

The CERT-In Guidelines on Secure Adoption and Governance of Artificial Intelligence Systems (Version 1.0, 25 May 2026) identify Human Oversight and Decision Governance as a core control area requiring organisations to: validate AI-generated outputs, restrict fully autonomous critical actions, and maintain auditability and approval mechanisms.

A hiring shortlist generated entirely by an AI model — with no human review of borderline cases, no audit trail of the factors that drove the ranking, and no mechanism to flag demographically anomalous patterns — fails the human oversight standard. AI-assisted decisions are permissible. Fully autonomous, unreviewed decisions that affect an individual’s livelihood are a governance failure.

What Rajan is entitled to — and what the platform owes him

Under Section 11 of the DPDP Act, Rajan has the right to access a summary of what personal data the platform is processing about him and what processing activities are being undertaken. He has the right to know that a risk score or fit score has been generated, even if he does not know to ask for it.

Under Section 13, he has the right to raise a grievance about how his personal data has been processed. If the platform cannot explain, at the individual level, why its algorithm ranked him as it did — and what data drove that ranking — it cannot respond to that grievance in any meaningful way.

The algorithm decided. DPDP says that decision must be auditable, explainable, and subject to human governance. Rajan deserves at least that much.

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Note: The images used are AI Generated Images

DPDP 3.1 – Artificial Intelligence and DPDP: When the Algorithm Decides

June 4, 2026June 4, 2026 Karthik & Sunil#AI, #AICompliance, #AIEthics, #AIGovernance, #AlgorithmicBias, #ArtificialIntelligence, #CISO, #DataPrincipal, #DataPrincipalRights, #DataPrivacy, #DataProtection, #DigitalIndia, #DPDPAct, #DPDPCompliance, #ITCompliance, #KarthikAndSunil, #ResponsibleAILeave a comment

DPDP Series 2, Episode 1.1

Priya’s Story

The AI That Rejected Her Home Loan Without Reading Her File

Priya is a 31-year-old schoolteacher in a village in Tirunelveli. Clean credit history. Stable government salary. Zero defaults.

She applies for a home loan through a fintech platform. Within seconds, the response arrives: Rejected.

No reason. No human. No explanation. An AI credit-scoring algorithm made the call — silently, instantly, and without looking her in the eye.

She tries a second platform. Same outcome. She begins to wonder what is wrong with her, when the real question is: what is wrong with the algorithm?

Why AI credit scoring creates a DPDP problem

IS Audit Module 6 of the ICAI IS Audit 3.0 Course is direct: AI is widely used in banking apps to provide a faster, more accurate assessment of a potential borrower at less cost, accounting for a wider variety of factors. Credit scoring provided by AI is based on more complex and sophisticated rules compared to traditional systems.

More complex. More factors. And entirely invisible to Priya.

The problem is this: Priya’s loan application was rejected because the AI model had never meaningfully encountered a borrower profile like hers — a government employee in a Tier-3 city, with a savings-heavy profile and no credit card history — trained predominantly on urban, credit-card-using, high-transaction-volume data. IS Audit Module 6 names this explicitly: datasets applicable to AI applications to learn are really rare. Models trained on incomplete data produce biased outcomes for underrepresented groups.

The algorithm was not wrong about what it was trained to do. It was wrong about what it was trained on. And Priya paid the price.

The DPDP dimension — consent was not built for this

When Priya downloaded the fintech app and applied for the loan, she tapped “I Agree” to a terms-of-service document she likely did not read in full. That consent, under the DPDP Act, 2023, is not valid for everything the AI subsequently did with her data.

Section 6(1) of the DPDP Act is unambiguous: consent must be free, specific, informed, unconditional and unambiguous, limited to such personal data as is necessary for the specified purpose.

The specified purpose was loan evaluation. But the AI ingested Priya’s location history, app usage patterns, device behaviour, social interactions, and transaction metadata — far beyond what is necessary to evaluate creditworthiness. Every data element beyond the specified purpose is processing without a valid basis.

Furthermore, if her data was used to train or refine the AI model — improving the algorithm for future use — that is a separate processing purpose that required separate consent. She did not give it. Section 6(1) requires each distinct purpose to be separately consented to.

The right she did not know she had

Under Section 11 of the DPDP Act, Priya has the right to access a summary of all personal data being processed about her — including the processing activities undertaken. She has the right to ask what the algorithm used, what it concluded, and why.

Under Section 13, she has the right to raise a grievance with the Data Fiduciary. An AI system that cannot explain its decision — cannot identify what data points drove the rejection — cannot satisfy this right. A black-box model is, under DPDP, a grievance waiting to happen.

The CERT-In Guidelines on Secure Adoption and Governance of Artificial Intelligence Systems (Version 1.0, 25 May 2026) identify Human Oversight and Decision Governance as a mandatory control: validate AI-generated outputs, restrict fully autonomous critical actions, and maintain auditability and approval mechanisms. An AI that rejects a loan application with no human review and no audit trail fails every limb of this control.

The question every AI-first fintech must answer

Can you tell Priya — specifically, in relation to her file — what personal data the algorithm used, whether that data was within the scope of her consent, and how it contributed to the rejection decision?

If the answer is “our model doesn’t work that way” — the compliance gap is not in the algorithm. It is in the governance architecture around it.

The DPDP Act is not asking AI to stop working. It is asking AI to work accountably.

Disclaimer

The author / publisher shall not be responsible for any loss, liability, claim, penalty or consequence arising from reliance on the contents of this post without independent professional advice.

Share this:

Share this:

Share this: