AI and Surveillance Capitalism: Zuboff’s Critique

Zuboff’s Argument

Zuboff’s core argument was that surveillance capitalism — her term for the economic logic that emerged in the 21st century, in which human experience is extracted as “behavioral surplus” and sold as behavioural predictions — was a new economic logic, distinct from industrial capitalism. Industrial capitalism extracted raw materials from nature and turned them into goods. Surveillance capitalism extracted behavioral surplus — data about human behaviour (what you searched, where you went, what you said, who you interacted with) — from people and turned it into predictions about what they would do next. These predictions were then sold to advertisers, insurers, lenders, and anyone else who wanted to influence human behaviour.

The extraction of behavioral surplus was invisible. Every time you used a search engine, checked a map, posted on social media, or walked past a security camera, data was being collected about you — what you searched for, where you went, what you said, who you interacted with. This data was fed into AI systems that built models of your behaviour, and these models were used to predict what you would do next — what you would buy, who you would vote for, whether you would default on a loan.

The predictions were valuable because they were actionable. If an advertiser knew you were likely to buy a car in the next month, they could show you car ads at the moment when you were most receptive. If an insurer knew you were likely to develop a health condition, they could adjust your premiums. If a political campaign knew you were persuadable on a particular issue, they could target you with ads designed to change your mind.

Zuboff argued that this system was fundamentally incompatible with human autonomy. If your behaviour is being predicted and influenced by systems you cannot see and do not understand, then your choices are not fully your own. You are being nudged, manipulated, and controlled — not by a dictator with a secret police, but by an algorithm with a profit motive.

Clearview AI: Surveillance Capitalism’s Sharpest Edge

The most vivid illustration of surveillance capitalism in the AI era is Clearview AI — a facial-recognition company founded in 2017 by Hoan Ton-That and Richard Schwartz. Clearview scraped billions of photos from social media platforms — Facebook, Instagram, Twitter, YouTube, LinkedIn, and Venmo — and built a searchable facial-recognition database. The company sold access to this database to law enforcement agencies, allowing them to identify anyone whose photo was in the database by uploading a single image.

Clearview’s technology was, in a sense, the logical conclusion of surveillance capitalism. The company had taken freely available data — photos that people had posted on social media, often without realising they could be used for facial recognition — and turned it into a surveillance tool. The people in the database had not consented to being in it. They had not been told it existed. And they had no way to remove themselves from it.

Clearview faced legal challenges from multiple directions. Facebook, Google, and Twitter sent cease-and-desist letters. The American Civil Liberties Union (ACLU) sued Clearview in Illinois on May 28, 2020, under the state’s Biometric Information Privacy Act (BIPA) — Illinois’ 2008 biometric privacy law, widely regarded as the strongest US state-level biometric privacy law and the only one of the three state-level biometric laws (Illinois, Texas, Washington) that grants a private right of action. Several countries took enforcement action: France’s CNIL (Commission nationale de l’informatique et des libertés, the national data-protection authority) fined Clearview €20 million in October 2022; Canada’s OPC and Australia’s OAIC conducted joint investigations finding Clearview breached privacy laws and ordered deletion of data, though neither imposed a monetary penalty. On September 3, 2024, the Dutch Data Protection Authority (Autoriteit Persoonsgegevens) fined Clearview €30.5 million — a substantial GDPR fine, though not among the largest (the top GDPR fines — Meta’s €1.2 billion in 2023, Amazon’s €746 million in 2021 (later annulled by a Luxembourg court in 2024) — are an order of magnitude larger).

Why Clearview is hard to stop

Clearview’s persistence illustrates a fundamental challenge of surveillance capitalism: once the data has been collected and the model has been built, it is very difficult to undo. You can fine the company, you can restrict its operations in specific jurisdictions, but the database still exists, and the technology still works. Clearview continued to operate in the US even after European regulators fined it, because GDPR enforcement ends at the EU’s borders. The asymmetry is structural: data extraction is fast and cheap; legal remedy is slow, jurisdictional, and incomplete.

The GDPR (General Data Protection Regulation) — the EU’s 2018 data-protection regulation (in force May 25, 2018), widely regarded as the most comprehensive in the world — has been partially effective against surveillance capitalism. It has forced companies to be more transparent about their data practices and has imposed significant fines for violations. But it has not fundamentally changed the surveillance-capitalist model, because it regulates how data is collected and used rather than prohibiting the underlying logic of behavioral-surplus extraction.

The Resistance

The resistance to surveillance capitalism has taken several forms.

Regulatory. The EU’s GDPR, which took effect in 2018, is the most comprehensive data-protection regulation in the world. It gives individuals the right to know what data is being collected about them, the right to request its deletion, and the right to object to its use for automated decision-making. The GDPR has been partially effective — it has forced companies to be more transparent about their data practices, and it has imposed significant fines for violations — but it has not fundamentally changed the surveillance-capitalist model. The European Union Artificial Intelligence Act (EU AI Act) goes further, prohibiting certain AI practices and imposing obligations on high-risk systems. Social scoring — the practice of scoring individuals based on aggregated behavioural data, associated with China’s broader surveillance approach — is fully prohibited by the EU AI Act (Article 5). Real-time remote biometric identification in publicly accessible spaces is also prohibited, but with narrow law-enforcement exceptions (such as targeted searches for victims of specific crimes or prevention of imminent terrorist threats).

Technical. Some researchers have developed technical approaches to resisting surveillance:

Three technical countermeasures

Differential privacy adds statistical noise to data so that no individual’s data can be identified, while aggregate patterns remain usable. It protects specific data points but does not change the fundamental logic of surveillance capitalism.
Federated learning trains models on local devices (phones, hospitals) without sending the raw data to a central server — only model updates are shared. It protects specific data points but does not change the broader surveillance-capitalist model.
Adversarial patterns are visual designs — makeup, clothing patterns — that disrupt facial-recognition systems, causing the model to misidentify or fail to detect a face. Projects like CV Dazzle (Adam Harvey, 2010) pioneered this form of technical resistance.

These approaches are promising but limited — they protect specific data points, but they do not change the fundamental logic of surveillance capitalism.

Commercial. Some companies have positioned themselves as privacy-focused alternatives to the surveillance capitalists. Apple, in particular, has made privacy a core brand value, introducing features like App Tracking Transparency — Apple’s 2021 iOS feature (launched with iOS 14.5 in April 2021) requiring apps to ask permission before tracking users across other apps — and on-device AI processing — running AI models on the user’s local device rather than sending data to the cloud. App Tracking Transparency hit the digital-ad economy hard: Meta/Facebook reportedly lost roughly $10 billion in ad revenue in 2022 as a result. These initiatives have been partially effective — they have given users more control over their data — but they have not changed the fundamental dynamics of the digital advertising market.

What It Means

Surveillance capitalism is, in some respects, the water that AI swims in. The AI systems that we use every day — search engines, social media feeds, recommendation algorithms, voice assistants — are powered by data that is collected through surveillance. The models that make these systems work are trained on behavioural data that has been extracted from users, often without their knowledge or consent.

The question is whether this is sustainable. Zuboff argues that it is not — that surveillance capitalism is fundamentally incompatible with democracy, because it concentrates power in the hands of the companies that control the data and the models, and because it undermines the autonomy of the individuals whose behaviour is being predicted and influenced.

Others argue that surveillance capitalism is the price of the digital services we have come to depend on — that the “free” services we use are paid for with our data, and that the trade-off is, on balance, worthwhile. This is a genuine debate, and the answer depends on what you value — convenience, innovation, and free services on one hand; privacy, autonomy, and democratic accountability on the other.

The AI era has intensified this debate. AI systems are more powerful than the systems that preceded them — they can extract more data, build more accurate models, and make more precise predictions. They can also be used for more invasive purposes — mass surveillance, behavioural manipulation, social control. The technology is not going away, and the question of how to govern it — how to ensure that AI serves human interests rather than undermining them — is the central political question of the AI era.