Synthetic Confabulations

«It has been clear for several centuries now, it is not sufficient to describe the world by words, it is necessary to calculate the world. So that science has had ever more recurrence to numbers, which are images for thoughts. For example 2 is an ideogram for a concept of a pair, a couple. Now this ideographic code which is a code of numbers has been developed in a very fine way lately by computers, are themselves being transcoded into synthetic images. So it is my firm belief if you want to have a clear and distinct communication of your concept, you have to use synthetic images, no longer words.» [1]

Vilém Flusser, 1988

Text and images — Alexey Yurenev

I.

In the summer of 2019, I attended a lecture by American author, curator and Dean Emeritus of the International Center of Photography, Fred Ritchin in Moscow about the future of image making. Concerned with roles of authenticity that are threatened by proliferation of artificial intelligence and synthetic media, Ritchin showed a website thispersondoesnotexist.com: an endless online library of images of people generated by a neural network, styleGAN.

Commonly used for nefarious purposes and infamous for deep fakes, GAN, or Generative Adversarial Network, is an unsupervised set of neural networks consisting of a forger and a critic. Both are trained on an input of a dataset consisting of images. The forger’s function is to create new images to fool the critic, which verifies the results with the originals. Once the critic is misleadingly satisfied with the result, new image is produced. With more cycles or epochs of learning the images become more iconically referential, in relation to the original training dataset consisting of images that someone or something (a non-human agent) created, sourced, optimized and sorted.

Alexey Yurenev's grandfather (left) with his brother Naum, fallen outside Volgograd. Almost 80 years later, the photographer returns to the battlefield to find the «ghosts of the past».

II.

My grandfather, Grigoriy Lipkin, lived through four years of service in the Second World War, arguably the worst conflict in human history, and never talked about his own experience in it. Since his death his memory doesn’t exist anymore. The website thispersondoesnotexist.com challenged me to imagine what a synthesized memory of Grigoriy’s war exploits could bring to fill the void in collective and personal memory left by the silence of my grandfather and his peers.

I began by preparing the learning dataset for the machine to be able to work towards visualizing such a synthetic memory by downloading 35,000 photographs from waralbum.ru. This archive is assembled by war aficionados and amateur historians, who decipher the images and label the equipment visible in the photographs. Most of the photographs in the archive consisted of reactive iconographic depictions of war.

I sorted the photographs into categories for GAN to digest: Battlefields, Aftermath and War Actors, and by focal plane: closeup, medium, wide. Mostly black and white, thick grained and contrasted, categorizing the images of atrocities, destructions, industry of war and death felt like being Lilu in the scene of the film “Fifth Element”, absorbing the history of world civilization. [1]

After sorting thousands of images from the «Great Patriotic War», I sometimes thought I grew desensitized to violence yet seeing one brutal image would make me stop for the day. In this process I would ask myself whether I am training the machine, or the machine is training me. Since I had to review and ingest these photographs, perceiving them as «truth» and historical record a priori.

III.

It is well known that the famous image by Yevgeny Khaldei, planting a flag on Reichstag, was in fact staged and retouched.[3] Khaldei returned to Moscow after the war to retrieve the banner to be photographed. Unable to find the original flag, he sourced red tablecloths at TASS agency asking his uncle to sew a Soviet banner. He then returned to Berlin to stage what will become an iconic image. Smoke in the background was added in. A watch on the wrist of one of the soldiers was removed by Stalin’s censorship bureau. [4]

War is a religion in Russia. I was born in Moscow in 1986 and it felt like the war ended a few weeks before I was born. What American professor of comparative literature Marianne Hirsch refers to as post-memory[5], dominated the space for one’s own new life stories to take place because the overbearing heaviness of generational trauma caused by the war that affected each family in what used to be USSR. This trauma was parasitically implanted into my memory with films, parades, monuments, fireworks and carnations. I remember playing «Russians and Germans», a game that would always end with «Russians» pointing sticks at «Germans», while yelling «Hände hoch, Hitler caput!», German for «Hands up, Hitler’s dead!».

Growing up surrounded by the same footage that I am sorting to train the machine, I never questioned its authenticity. Today, with most of the veterans extinct, the memory is transitioning from communicative to cultural, where it is no longer passed down by the witnesses but controlled by mnemonic institutions such as archives and museums.

The heroic acts of military conquests are often used as prosthetics to construct a national identity.[6] These memories are manipulated to cultivate patriotism and national pride. In the context of the current Russo-Ukrainian war, Russian military of defense has published a website vozmezdie.ru (vozmezdie is Russian for retribution), a repository of documentation about Ukrainian nazi collaborators in World War II, to justify the horrific invasion and to feed into the government narrative of denazification of Ukraine.

IV.

Media philosopher Vilém Flusser, in his essay, «Towards a Philosophy of Photography» published in 1983, argues that photographs can only record states of things that are part of happenings that are perpetually in flux.[7] In a photograph, a happening is abstracted into an event, an all-in-once-ness. With photograph penetrating the space and time continuum, Flusser describes an apparatus producing technical images as «... boxes that devour history and spew out post-history.»[8]

Without any context, the photographs found in waralbum.ru could have been created at any time, staged, witnessed, manipulated, etcetera; 35,000 documents are mere 35,000 perspectives. GAN, unlike a human trying to decode the content of images, does not differentiate between truth and fiction. It is a convolutional model, propagating and predicting an assembly of pixels based on patterns it recognizes in its learning material. Whether who or what is depicted once was part of the photographic event as a Nazi, Soviet, dead child or a tank becomes irrelevant. It is a Rorschach alphabet soup of gazes, implicating four sites of interpretation of an image: camera, photographer, subject and audience[9]. When stepping back to look at this generative process we can witness the presence of an additional site. The one of the imagination. And this imagination is solely dependent on the synthetic experience of the audience in its encounter with the outcome of the process. The experience of relating to the world, to decode the meaning of the produced image.

The first set of photographs I used for training the machine consisted of «Battlefield» photographs. I prompted the rig to produce 100 images. Without being concrete in their indexicality, to me these synthetic images represent a dramatic battle scene, reminiscent of photographs I trained the networks on.

«Battlefield»: styleGan2 image trained on a dataset of archival photographs from World War II.

Looking at «Battlefield» (image above), I see a plume of black smoke rising from a burning object in the landscape, a sense of being in motion, a chase. It looks like a photograph that could have been exposed with a shutter from the early 20th century, where bottom and top of the plane was captured with a delay, distorting the horizon line into a winding vortex.

«Battlefield», as well as the other synthetic pictures appear as a subconscious landscapes, they are a fog of war, specters of battles returning in the shape of a monochromatic expressionist painting. These synthetic impressions let the imagination fill in the interpretation in an unconscious way and not trying to convince their audience of a fact via the kind of indexical and iconic quality, like a photograph would. Their likeness seems obscured, it is non-existent.[10] They are a shadow of a war archive. Unlike a silhouette, they do not signify an object backlit by an antumbral position of light, they are in fact, a distorted projection.

Nevertheless, in this state, synthetic pictures start appearing more truthful than the archival photographs. These images are emancipated from their responsibility and reliability to the physical world. Their form produces information, as powerful as the conventional photograph would, its imaginative strength does not lie in a misleading facticity but in a visceral emotion. I wonder whether this emotional response is primordial, reflexive and in this way more reliable.

V.

The word archive derives from the Greek work «Arkh», which means commencement and commandment, and also a dwelling, or to dwell. As any dwelling it has an inside and an outside and it also commands and rules of what is included as well as excluded. French philosopher Jacques Derrida, suggested approaching the archive with tools of psychoanalysis, comparing the archive to the human brain, that follows a rigid structures of classification.[11] It reveals as much as it re-veils.

When looking through the archives of WWII images I collected, I would never find a defective image, a mistake, an outtake, an image in between the frames that could have possibly been taken by accident from a shutter release hitting a hip, appearing out of focus and accidental. Deemed unarchivable by the agent in charge, during a process that Dutch author Erik Ketelaar coined as «archivalization».[12]

To confront my own bias of perceiving war themes in the synthetic images, I upload them into google API image recognition. Computer vision also recognizes war, violence, soldiers and history. Are these images synthetic suggestions or synthetic confabulations built out of jpg artefacts attempting to interpret the black and white negative grain that the originals were once made of? And to what extent, the dominance of such images in the collective memory creates a networked pre-conception for the image recognition engine?

Due to the world building potential of the abstraction, I take a subjective position and credit the synthetic images as what I perceive them to be based on the training set. I chose to caption the «scene» based on the input for that model: «Battlefield». I credit the neural network as a creator: StyleGan2 Image trained on a dataset of archival WWII photographs.

VI.

I set an impossible objective when I began collaborating with styleGAN. To reclaim something that is no longer: a memory lost with the death of the person. A paradoxical discovery encountered me during the archival investigation in search of materials to teach the machine and myself. I knew that my grandfather enlisted into the Red Army before reaching draft age to look for his brother Naum who disappeared shortly after the Nazi invasion. I came across an archive, www.pamyat-naroda.ru ( Russian for «memory of the people») that was recently declassified by the Russian Ministry of Defense. There, I found a deluge of data concerning my grandfather’s engagement in WWII. There, I was able to find out what happened to his brother. Naum was killed outside of Volgograd (formerly Stalingrad), fighting for a strategic section of railroad.

In January of 2022, I traveled to Volgograd for field research at the location of my granduncle’s death.

VII.

Note from my field journal January 20 2022.

Around me, the landscape for miles around consists of deserted prairies. Fertile but undeveloped, this place stands still as if in shame of what this earth saw 80 years before my arrival. Covered by several feet of snow, the horizon blends into the white sky, occasionally alternating with dead grass, a shrub, or a lonely tree. The piercing wind reminds me of the poetic lines inscribed on the Mamayev Kurgan monument in the center of Volgograd, a memorial to the fallen soldiers. «The iron wind was hitting their faces, they went ahead and ahead…», referring to the amount of ammunition fired in this place that is flat as a pancake. Being here, it is horrifying to imagine the brutality of the battles in a location with no possible shelter to hide from incoming fire. The wind blew the snow sideways, screeching against exposed skin on my face with a metallic tone.

VIII.

Channeling Roger Fenton and the pioneers of war photography that were often late to the scene and having to resort to staging and reenacting the action,[13] I used the same style apparatus. An 8x10 technical view camera with bellows, operating in a same principle as the first cameras that were designed. Working with this camera is a methodical and laborious process consisting of multiple steps that need to be executed in rigorous order for the exposure to be successfully produced.

This experience, made me think of my grandfather, Grigoriy Lipkin, whose first position upon enlisting was a 2nd person on artillery, meaning he had to carry the weapon.

Wounded 4 times he would return to the war, continuing on the campaign that would remain a secret until his death. As if this Sisyphean task had no meaning, there I was carrying a 20-kilo rig, in a white hell where over 2 million people have died and where now the composition lost any purpose or sense. Wherever I would point this imaginary weapon, it looked the same 360 degrees around me.

I used orthographic film which is a black and white medium that is perceptive exclusively to ultraviolet frequency. This technical film was traditionally used to make duplicates of other negatives, documents as well as masking in the darkroom for manipulation and retouching. Blind to what the human eye sees as the color red, which the emulsion of this film renders as black.

In my search for any traces of my relative in the infinity of a milk-white landscape, the film acts as a gesture to peel away an entire spectrum of light to find (or summon) the specters of the past that could leave an imprint on the thickness of the silver halides on the emulsion. The orthographic film, in this case became an internegative of sorts, a substrate between the image collection found online, the synthetic AI landscape and the real experience of the field of the prairies.

49°02'46.8 "N 44°08'08.6 "E: Taken with an 8x10 view camera with bellows, on orthographic film that only perceives ultraviolet frequencies. This technical film is usually used to make duplicates of other negatives and documents, but also to cover up manipulations and retouching in the darkroom. This film does not see what the human eye perceives as red. The emulsion reproduces it as black. Photo: Alexey Yurenev

IX.

While in the field, my family and friends kept texting me whether I feel anything. Any presence? What is this experience like? As much as I wanted to feel the spirit - I felt nothing of the trace of my uncle but being engulfed in a deafening silence under the dark cloth of the view camera and an inverted projection of the world on the ground glass. Standing in the middle of the land that is cursed by the perversion of war, death and killing, where nothing had survived including history and nothing was built upon it, two generations since.

Looking at the photographs I made, they are anonymous. Uneventful, still, flat they carry no trace of what happened there during the war. Two million people are buried under this snow. Two million gazes and perspectives are now abstracted into a banality of a snow-covered landscape. Evoking emotions that are as flat as a line of a horizon.

In 49°02’46.8”N 44°08’08.6”E (image above), dead vegetation is piercing through the white of the snow. A small ravine is leading the eye of the viewer towards the horizon, splitting the composition in two. In the distance far away, geometrically groomed withered fields are lined with electric pylons far away. The harsh contrast between the white of the sky and the snow is ripped by the blacks of the reeds and the dirt exposed on the edge of the ravine. The photograph is hyper-focal, everything appears tack sharp. The grain of orthographic film is extremely fine and barely visible. Photographically, it hides nothing within the rebate borders of its 8”x10” frame.

X.

The field had left me with a thought whether the large format depictions of the approximate resting place of my granduncle are even more artificial, synthetic and dislocated from the memories I was trying to render with an algorithm. The sublime digital images produced by the machine are much more reactive, full of action that I am triggered to imagine, while the pastural, banal landscape of the current state of the place transmutes nothing but stillness and sensation of a void larger than the one left by a disappearance of a relative.

For me silence and emptiness actually form the meaning of this experience. I choose to caption them with a GPS coordinate. Nameless, they carry a code, signifying a place of importance in history and myself. Spatially they locate a site of a battlefield, temporally they are dislocated from the battle by almost eight decades.

For that reason, I choose not to title them a battlefield. I let their importance dwell in the mystery of a codingcoordinate. This landscape now represents the same black box as the AI, in a discreet coding of meaning, I put into the image.

Amsterdam, September 2022

«Silent Hero» is an ongoing investigation into Alexey Yurenev's memory of his grandfather and whose WWII exploits that has been lost since his death in 2009.

Alexey Yurenev

Alexey Yurenev is a photographer, visual researcher and educator working on subjects of memory and the synthetic. HIs documentary projects have been published in The New York Times, National Geographic, Aljazeera, Yahoo!, Topic and Literary Hub. His work has been acknowledged by organizations such as Photographer of the Year International and Silurian Society of New York. In 2020, Yurenev co-founded a platform dedicated to innovative visual strategies FOTODEMIC.org and became a faculty member at The International Center of Photography in New York. Yurenev is currently based in The Netherlands attending a Photography & Society Master at the Royal Academy of Art in The Hague.

Patrick Ploschnitzki

Patrick Ploschnitzki is Visiting Assistant Professor of German Language and Culture at the University of Florida. His research examines sociocultural discourses in the context of dubbing, as well as so-called 'dubbed German' in the context of US television. Other research interests include the use of online translation tools in the foreign language classroom, and the connections between popular media and literary traditions, such as redefined images of home in contemporary punk and rap texts.

Footnotes and Bibliography

[1] Flusser, Vilém. Interview by Miklós Oeternák, the Osnabrük New Media Festival 1988. Other. The Osnabrük New Media Festival 1988, n.d.
[2] The Fifth Element . United States: Columbia Pictures, 1997.
[3] «Evgenii Khaldei.» International Center of Photography, December 7, 2018. https://www.icp.org/browse/archive/constituents/evgenii-khaldei.
[4] Lowndes, Coleman. «Why the Soviets Doctored This Iconic Photo.» Vox. Vox, October 2, 2018. https://www.vox.com/videos/2018/10/2/17928052/soviets-doctored-wwii-photo-reichstag-iwo-jima-world-war-ii.
[5] Marianne Hirsch, «The Generation of Postmemory,» Poetics Today, v. 29 #1, Spring 2008, 103.
[6] Landsberg, Alison. Prosthetic Memory: The Transformation of American Remembrance in the Age of Mass Culture. New York: Columbia University Press, 2006.
[7] Flusser, V. «Towards a philosophy of photography». Reaktion, p10
[8] Flusser, V. «Post-history», Univocal, p57
[9] Azoulay, A., «What is a photograph? What is photography?», Philosophy of Photography (2010) 1: 1, pp. 9–13
[10] Stultiens, A.G.E., «Ebifananyi : a study of photographs in Uganda in and through an artistic practice», Universiteit Leiden 2018, p15
[11] Derrida J. And Prenowitz E, Archive Fever: A Freudian Impression, Diacritics, Vol. 25, No. 2 (Summer, 1995), p9
[12] Ketelaar, Eric. «Tacit Narratives: The Meanings of Archives.» Archival Science 1, no. 2 (2001), p132
[13] «Truth, the First Casualty’ Part I/II Essay by Stephen Mayes.» FOTODEMIC. https://www.fotodemic.org/truth-the-first-casualty.

Mehr ReVue
passieren lassen?

Der ReVue Newsletter erscheint einmal im Monat. Immer dann, wenn ein neuer Artikel online geht. Hier en passant abonnieren.

Sie möchten unsere Arbeit
mit einer Spende unterstützen?
Hier en passant spenden!

Fotografie ist allgegenwärtig, wird aber in den journalistischen Medien noch wenig hinterfragt oder erklärt. Wer an Journalismus denkt, denkt an Texte. Das digitale Magazin ReVue verfolgt einen anderen Ansatz: Es nähert sich den Themen vom Bild her. In unseren Beiträgen untersuchen wir die Rolle und Funktion von Bildern im Verhältnis zum Text, zur Wahrheit, zum politischen oder historischen Kontext. Wie nehmen wir Bilder wahr? Welche Geschichte steckt dahinter?
Unsere Beiträge erscheinen auf Deutsch, wir übersetzen aber auch fremdsprachige Texte und erleichtern so den Wissenstransfer zu einer deutschsprachigen Leserschaft.
ReVue ist unabhängig. Die Redaktion arbeitet ehrenamtlich. ReVue ist ein Projekt der gemeinnützigen DEJAVU Gesellschaft für Fotografie und Wahrnehmung e.V. in Berlin.