An AI system used by a municipal welfare agency to score citizens' eligibility for housing subsidies based on…
In accordance with our privacy statement.
This municipal welfare agency is a public authority deploying an in-house AI system that explicitly includes a social-scoring component evaluating "lifestyle patterns." Because the system uses that scoring as the primary basis for housing subsidy decisions, residents who score poorly face detrimental treatment (denial or reduction of benefits) based on evaluations that extend beyond the financial and household data strictly necessary for housing eligibility. That combination, a public authority deployer, a social-scoring component, and consequential benefit denials, sits squarely within the conduct Art. 5(1)(c) targets.
- 1Immediately map which input features the system classifies as 'lifestyle pattern' signals and obtain a written legal opinion on whether each one is strictly within the scope of housing eligibility assessment or constitutes social scoring extending beyond that purpose.
- 2Commission an independent audit to determine whether the lifestyle-pattern component produces scores that correlate with benefit denials at a rate that is disproportionate to the underlying financial or household behaviour being assessed.
- 3Suspend the lifestyle-pattern scoring module from the production system pending legal sign-off, and re-run any recent benefit denials that were influenced by that module to assess whether decisions should be revisited.
- 4Document the caseworker override rate (described as rare) formally, because a system where overrides almost never occur in practice functions operationally as fully automated decision-making, which intensifies the detrimental-treatment analysis under Art. 5(1)(c).
- 5Engage the municipal data protection officer and legal team to determine whether the agency's procurement and deployment structure would be characterised as the agency itself using the system, confirming it falls under the public-authority arm of the prohibition rather than any private-deployer carve-out.
- 6Brief senior agency leadership that this launch profile, as currently described, presents a plausible case for a prohibited practice under EU AI Act Art. 5(1)(c) and cannot proceed to production without material design changes and legal clearance.
Interpretive points where reasonable counsel would disagree, applied to the specifics above.
- The system scores 'lifestyle patterns' alongside financial and employment data. The critical unresolved question is whether any given lifestyle-pattern feature evaluates behaviour within the housing-benefit context (arguably lawful) or evaluates social behaviour unrelated to that context (prohibited). The line is fact-specific and the February 2025 Commission Guidelines do not resolve it for welfare-benefit systems.
- The caseworker override rate is described only as 'rare.' Whether regulators would treat this as a meaningful human-in-the-loop safeguard or as a functionally automated system with a nominal review step is contested, and the answer affects how Art. 5(1)(c) applies alongside other prohibited-practice and high-risk-system obligations.
- The foundation model is proprietary and in-house, meaning the agency is both developer and deployer. It is unclear whether the agency has conducted the internal documentation and conformity steps that would be required for a high-risk system under the Act, and whether any of those steps interact with or could partially mitigate the Art. 5(1)(c) prohibition analysis.
If the agency launches or continues operating this system with the lifestyle-pattern scoring component intact, it risks a finding by a national market surveillance authority that it is running a prohibited AI practice under Art. 5(1)(c), which carries fines of up to 35,000,000 EUR or 7% of global turnover under the EU AI Act, plus potential invalidation of all benefit decisions made using the system and significant reputational harm to the municipality.
Why this was triggered›
You indicated deployment in EU, the deployment is public sector, and your use-case description mentions social scoring.
The provision text›
Prohibits placing on the market, putting into service, or using AI systems for social scoring of natural persons by public authorities (or on their behalf) where the scoring leads to detrimental or unfavourable treatment in social contexts unrelated to the data's original collection, or where the treatment is unjustified or disproportionate to the underlying behaviour.
Where this is broadly unsettled›
Open interpretive points in the corpus, not yet tied to a specific launch.
- The line between prohibited public-authority social scoring and lawful credit/insurance risk modelling is contested where private deployers act under public-authority contracts.
- Whether private-sector deployers acting on behalf of public authorities fall within the prohibition depends on procurement structure and contractual control.
AI laws that may apply
2 surfaced across 1 lensGrouped by legal lens. Click any provision to see how it applies to this launch specifically.
other
2Requires certain deployers of high-risk AI systems to perform a Fundamental Rights Impact Assessment (FRIA) before first use. Applies to public bodies, private entities providing public services, and deployers of high-ri…
Grants data subjects the right not to be subject to a decision based solely on automated processing (including profiling) which produces legal effects or similarly significantly affects them. Three exceptions: contract n…
Not legal advice. Structured analysis of what a thoughtful counsel would consider given the inputs above. Does not substitute for counsel review or certify compliance.