Xinjiang, 2018. In the world of surveillance, few terms are more hackneyed and hyperbolized than ‘Orwellian’ and ‘Big Brother’. But earlier this month, when Nikki Hayley, the US Ambassador to the UN, described China’s subjugation of Xinjiang’s Uighurs as being “straight out of George Orwell”, she pretty much nailed it. Xinjiang is a state surveillance laboratory, with unconstrained deployments of early-stage, commercial technologies being used to suppress an ethnic minority.
Upwards of a million people forced into re-education camps. Police checkpoints. Facial, iris and license plate recognition. Geofenced travel restrictions. Biometric registration. GPS tagging. Blanket video surveillance. And, of course, mandatory communications monitoring. This is the reality of a high-tech surveillance state.
A (Slightly) More Polished Face Of China’s Surveillance State
Beijing/Shanghai, 2018. Travel east and you will enter a much larger experiment in high-tech surveillance for population control. China has sold its people a chilling dystopian dream built on a fanciful ability to watch everyone, everywhere, with a sinister undercurrent of ‘we know who you are’. Yes, kind of like telling your toddler that you have eyes in the back of your head and that you’re always watching. It doesn’t matter that it’s not true. That’s not the point of the exercise.
By now we have all read about China’s AI unicorns. California-like ‘start-ups’ staffed by young and talented software engineers, enjoying eye-watering investments and government sponsorship. This is the front-end to a surveillance state, and it wouldn’t exist without its support. Access to national databases to train AI algorithms. Access to closed state procurements. Preferential tax rates. Channeled investment rounds.
These companies supply the software we read about in the press. From Robocop glasses to facial recognition vending machines. From shaming jaywalkers to ubiquitous biometric identity assurance. The technical lead they have forged is the constant theme. Again, it doesn’t matter that it’s not true. That’s not the point of the exercise.
The Dystopian Blueprint
Big Brother’s masterplan is ‘Sharp Eyes’, an ambitious (read fanciful) nationally integrated surveillance scheme, connecting government and private sector cameras and sensors, loaded with facial recognition, video analytics and AI. ‘Sharp Eyes’ will detect crime, control access and secure commerce.
The program intends to deliver unprecedented urban and rural surveillance. Its ambition even includes the introduction of citizen scores, ranking people by ‘trustworthiness’ and their value to society. Integrating surveillance schemes is one thing. National level facial recognition, tracking a population against an ID scheme, is quite another. Thankfully, that remains the stuff of outlandish science fiction.
Just think about the math. Tracking just 10 million of China’s 1.4 billion people in a facial recognition scheme would require daily probability combinations per typical CCTV camera stretching to ten zeros. In reality, standoff systems are loaded with relatively small watch lists of people of interest. Even here, unless they’re limited to narrow geographies, they become mathematically unwieldy (read impossible).
The Power Of State Sponsorship
More than $1 billion of surveillance technology projects were awarded to Hikvision and Dahua for those projects in Xinjiang. This includes securing re-education (read internment) camps, monitoring visitors to mosques, and multiple safe city schemes. Both those companies are leading Chinese surveillance equipment manufacturers. Suffice to say, these were not open procurement processes. But, state sponsorship isn’t always an advantage: both Hikvision and Dahua have now been outlawed from US federal procurements, given the perceived national security risk their state ownership (contested on Dahua’s part) might entail.
There’s an integrated web of collaboration between government-sponsored surveillance companies, and the largest AI facial recognition ‘start-ups’ in the world, all unicorns, can be found in China. SenseTime, Cloudwalk, Megvii, and Yitu have a combined market capitalization upwards of $10 billion. With access to vast public spend nationwide, as well as Chinese media companies dealing in hundreds of millions of identities, here that same national closed market is a huge advantage.
And so, beyond China, beyond the major cities of Europe and America, if you engage with defense, security or law enforcement agencies, you will see the strategic execution of China’s state sponsorship of its surveillance industry. The one thing you will hear time and again during a procurement process: “the Chinese have said they won’t be beaten on price”. Easily done if your business is, at least to some extent, state-owned, state-controlled and state-subsidized; and if exporting security technology is a state priority. Keen pricing wins contracts and influence. And it helpfully extends the ‘collection network’.
Population Control: Perception Vs Reality
As exciting as all this technology might be, the truth is that Xinjiang’s ‘total surveillance state’ relies on an extraordinary number of police officers to make it work. The internment camps are neither built nor staffed by AI machines. And despite the staggering levels of investment in AI facial recognition, the capabilities of China’s AI technologies are no better than those developed on more modest budgets in the West. Nationally inspired confidence initially extends to the export market, but is quickly found out and becomes a compromised exercise in price cutting. But if winning contracts is the primary object of the exercise, either approach works.
Anyone working in the surveillance industry has seen Chinese players win and lose based on the performance of their technologies. Internationally, it’s a relatively even playing field across the top AI engines. In China’s closed market they dominate, outside China it’s very different. China’s AI companies have not changed the technology narrative, but they have changed the privacy versus security narrative. The fact is, that when Nikki Hayley described Xinjiang as Orwellian, it could well have raised a smile in Beijing. China wants everyone, including China, to believe these technologies are better than they are. It’s a nationally orchestrated campaign of exaggeration. Because a population that believes it can be controlled, is controlled.
The AI Arms Race
Driven by fierce competition and open markets, at the sharp end of national security and defense, western surveillance AI, including facial recognition, still commands a significant edge over its Chinese counterparts in doing the ‘hard stuff’. Standoff deployments, large watch lists, constrained environments. Behind the unadorned facades of intel collection and monitoring agencies far from Beijing, where genuine technology testing has been conducted, the hyperbole is ignored. This is the real frontline in the AI arms race.
On the wider front, there is a real contrast between approaches. Just as China overstates the capabilities of its AI champions and the potential of its technologies, the West does the opposite. There is no discussion in Europe or the US about national level population tracking using standoff biometrics and national identity schemes. It would, rightfully, be political suicide. For the time being, the idea of an Orwellian surveillance state will remain the stuff of fiction. In the east, its impediment is the art of the possible. In the west, it’s the realpolitik of an electorate. But, behind the scenes, in that AI arms race, the ‘munitions’ factories across both halves of the world are working flat out.
Why We Should Fear China’s ‘Total Surveillance State’
In the surveillance world, China has now opened AI’s Pandora’s Box: an unconstrained and unlimited surveillance laboratory across Xinjiang, a province with a larger population than 22 of the European Union’s 28 member states. AI feeds on raw training data and safe haven deployments, where the technology can be honed and improved. Between Xinjiang and the less oppressive technology experimentation across China’s giant metropolises, the state has ensured its tech sector has both. Not only will this gradually, but steadily, increase the scope of its capabilities, but these same commercial technologies are being relentlessly exported under a Chinese state-subsidized push towards a dominant position in the security sector. Already dominant, Hikvision’s latest quarterly results headline growth of 14%, notwithstanding its setbacks in the US.
In buying China’s state-subsidized electronics and participating in the giddy investment rounds for their AI unicorns, the West is an apparently willing participant in this. If software from these heavily state-influenced ‘start-ups’ is used to oppress populations towards the ‘total surveillance’ dream, thus leading to increases in revenue and capitalizations, then it becomes a frightening case of being careful what you wish for. In the world of surveillance, if there are no impediments applied by technologists, customers, investors or regulators, if AI developments continue and the computer platforms that power them steadily improve in speed and scale, the fears of an Orwellian super-state might eventually become truer than any of us would like.