Ambient surveillance: Computer vision and animal camouflage

The evolution of computer vision has been rapid and intense. As early as the ’60s, the computer eye was able to extract three-dimensional information from images, then learning optical flow, motion estimation, and now categorizing pictorial elements according to their semantic concepts. In the animal kingdom, vision has evolved independently numerous times over millions of years, giving rise to a wide array of pinhole photoreceptors, compound sensors, and our own eyes whose aqueous structure is shared by all vertebrates. While animal eyesight develops in order to suit the demands of the environment—detecting darkened underwater forms or spotting prey from a distant aerial perspective—computer vision is progressed to gather information relevant for economic, medical, and security purposes. Increasingly, computer vision is tasked not only with aggregating imagery, but with deciphering and understanding it as well.

Animals and machines offer two bookends to the human experience—the domestication of animals to aid in agricultural labour changed not only the behaviour and social organization of animals, but also that of their human beneficiaries. Likewise, the development of new technology, spurred forth by different productive needs, serves to alter human societal practices and acts upon us as a form of machinic domestication. These recursive relationships form the environmental pressures under which tentative distinctions between human, animal, nature, and culture hold sway.

Central to computer vision is the notion of pattern recognition. In contemporary usage, this term refers to the algorithmic collation of innumerable data points—gathered via mass surveillance as a matter of routine by government agencies and commercial entities—collected, sorted, and filtered until they begin to reveal a pattern. While such an agglomeration of data is far too vast to be comprehended by any amount of human labour, humans can create training sets by attributing labels to a sample of data, and machine learning algorithms can extrapolate to apply their findings to an infinitude of incoming data. Some algorithms work without any oversight at all, comparing and contrasting each data point to come up with ad hoc relationships: distilling novelties into proprietary patterns even if they still look like noise to human eyes. The resulting patterns contain both rich predictive value as well as all of the latent biases and assumptions of their human engineers. That is, the patterns derived from surveillance fuel simulations predicting the future, albeit a future that owes its legibility to the interests and value systems of its beholder. In this way, the probabilistic accounting of surveillance–simulation systems borrow much from eugenic pseudo-sciences that view nature as a competitive (and manipulable) survival of the fittest.

On a psychological level, the ability to detect hazards and resources by extrapolating from past experience is crucial to habituation, and this form of pattern recognition is widespread amongst diverse fauna. But some creatures develop counter-techniques to avoid detection by others: in the place of pattern recognition there is the disruptive patterning of animal camouflage.

Such patterns serve to confuse the play of light and shadow amongst foliage, breaking up the silhouette of a creature as they navigate the naturally-occuring patterns of their home habitats. Although these tactics have been appropriated for military use—for example the Canadian Disruptive Pattern (CADPAT) uses pixelated forms that bear little resemblance to natural foliage to specifically act upon digital cameras and night vision—what is instructive here is the way in which animal colouration plays two patterns off each other to create a furtive effect. That is, the camouflage operates by situating an abstract colouration amongst the repeating noise of a natural environment. It’s like a visual polyrhythm, where two musical phrases overlap to create a syncopated rhythm operating in several time signatures at once. The combination of multi-scale patterns act upon parallel focal and ambient visual systems to disrupt both detection and recognition.

But then there are instances of animal species developing markings that resemble the eyes of predatory birds, or benign creatures mimicking the threatening patterns of poisonous ones. Cephalopods, in particular, possess a sophisticated array of pigmented and iridescent skin cells that can perfectly reproduce the colours of their local environment—an ability that has been proven to operate even in blind cephalopods. Such evolutions call into question the agency of prey: are these capacities evolved sheerly through environmental pressures and genetic variation? Or is there some willfulness, cleverly mirroring predatory eyes back at the beholder?

Furthermore, how can these tactics be understood outside the realm of vision, beyond the age-old hierarchies of predator and prey? We may think of the CCTV camera lens as the personified eye of so-called surveillance capitalism—and indeed computer vision has grown quite powerful, obtaining biometric data from the rhythm of your gait or reconstructing audio based on vibrations detected between video frames—but in reality a great deal of data is gleaned from routine behaviours, browser cookies, and invisible images hidden in emails that track when and where you loaded them. This data is best understood as forensic evidence, like dead skin cells and discarded indexes scraped from innocuous sources, pieced together not necessarily to prove what you did, but constantly collected to predict what you will do—or, better, to subtly shape behaviour toward a predetermined outcome. Here is where the actual is substituted for the virtual on a daily basis. A social media feed becomes a feedback loop, no longer intended for you but rather for “someone like you.”

Although the events of recent years have offered more insight into the operations of national security agency spyware, the use of aerial surveillance systems by domestic police, and personal data illicitly harvested for political interference, it remains true that advances in technology affect societal behaviour long before laws and rights can provide protective measures. And in the absence of such measures, conflicts arise between extant rights, often resulting in paradoxes between the rights of the individual and the social fabric as a whole. In prison, individuals who are judged to have broken the social contract are disenfranchised of some of their freedoms—including the freedom to information, such as access to the internet. As numerous penitentiaries begin making use of AI-driven surveillance to purportedly increase safety for those incarcerated, then prisoners not only lose access to information, they become commodified as information when their personal interactions are subject to data mining. Such practices date back to 19th century police archives that used anthropometric data to predict criminality based on physical features—ultimating fomenting an incentivized racism. Digital surveillance and data mining have become ubiquitous practices in an extremely profitable industry, while the balance between surveillance and privacy has yet to be enshrined as a human right—much less to what degree incarcerated individuals ought to be able retain this right.

In all cases, vision is never merely a passive reception of visual data; it is deeply shaped by contextual necessity and pre-existing value systems. You see what you are looking for. And in an economy where the processing of behavioural data yields industry-defining returns, one tends to see individuals as freely commodifiable resources, even if it means violating personal boundaries and overriding rights to privacy.

What we can learn from the sophisticated crypsis tactics developed by animals is the importance of context in obscuring legibility: the predatorial gaze is not simply negated or escaped, it is also redirected, something else is served up for scrutiny, lost in the noise of the local environment and all the overlapping rhythms of daily life. The countershading of the rat  snake or the ocellus of the owl-moth are woven into the existing patterns of other entities occupying the same habitat, all while fomenting a hidden, syncopated pattern—illegible to those who do not know what they are looking for.