How AI is hijacking the history of art

People tend to rejoice in the disclosure of a secret.

Or, at the very least, the media has come to realize that news of “mysteries solved” and “hidden gems revealed” generates traffic and clicks.

So I’m never surprised to see AI-assisted revelations of famous masters’ artwork go viral.

In the past year alone, I have read articles highlighting how artificial intelligence recovered a “secret” painting of a “lost lover” by the Italian painter Modigliani, “Brought back to life” a “hidden Picasso nude”, “Resuscitated” the destroyed works of the Austrian painter Gustav Klimt and “Restored” parts of Rembrandt’s 1642 painting “The Night Watch”. The list goes on.

As an art historian, I am more and more concerned about the coverage and dissemination of these projects.

They haven’t, in fact, revealed any secrets or solved a single mystery.

What they’ve done is generate AI wellness stories.

Are we really learning something new?

Take the reports on the paintings of Modigliani and Picasso.

These were projects carried out by the same company, Oxia Palus, which was founded not by art historians but by machine learning doctoral students.

In both cases, Oxia Palus relied on traditional X-rays, X-ray fluorescence and infrared imaging that had previously been produced and published years before – work which had revealed preliminary paintings under the visible layer on the artists’ canvases.

The company edited these x-rays and reconstructed them as new works of art by applying a technique called “neural style transfer. “This is a sophisticated sounding term for a program that breaks down works of art into extremely small units, extrapolates a style from them, and then promises to recreate images of other content in the same style.

Essentially, Oxia Palus assembles new works from what the machine can learn from existing x-ray images and other paintings by the same artist.

But aside from flexing the prowess of AI, is there any value – artistically, historically – in what the company does?

These recreations teach us nothing that we did not know about the artists and their methods.

Artists paint over their works all the time. It’s so common that art historians and restorers have a word for it: pentimento. None of these earlier compositions was an Easter egg deposited in the painting for later researchers to find out. The original x-ray images were certainly valuable in that they offered insight into the artists’ working methods.

But for me, what these programs do is not really interesting from an art history perspective.

The human sciences on the support of life

So when I see these reproductions catching the attention of the media, it strikes me as soft diplomacy for AI, showing a ‘cultured’ application of technology at a time when skepticism of its deceptions, of its prejudice and its abuse is on the rise.

When AI draws attention to the recovery of lost artwork, the technology looks a lot less scary than when it hits the headlines for creating deep counterfeits that falsify the discourse of politicians Where for using facial recognition for authoritarian surveillance.

These studies and projects also seem to promote the idea that computer scientists are better at historical research than art historians.

For years, university humanities departments were progressively deprived of funding, with more money pumped into science. With its claims to objectivity and empirically provable results, the sciences tend to command greater respect from funding agencies and the public, prompting humanities researchers to adopt computational methods.

Art historian Claire Bishop criticized this development, noting that when computer science integrates with the humanities, “[t]theoretical problems are overwhelmed by the weight of data ”, which generates deeply simplistic results.

Basically, art historians study the ways in which art can offer insight into how people once viewed the world. They explore how works of art have shaped the worlds in which they were created and will influence future generations.

A computer algorithm cannot perform these functions.

However, some academics and institutions have let themselves be subsumed by the sciences, adopting their methods and partnering with them in sponsored projects.

Literary critic Barbara Herrnstein Smith warned against giving too much ground to science. In his opinion, the sciences and the letters are not the antipodes that they are often presented publicly. But this representation benefited the sciences, valued for their supposed clarity and usefulness over the supposed obscurity and uselessness of the humanities. At the same time, she suggested that hybrid fields of study that merge the arts and sciences can lead to breakthroughs that would not have been possible if each had existed as a siled discipline.

I am skeptical. Not because I doubt the usefulness of expanding and diversifying our toolbox; of course some academics working in digital humanities have adopted computational methods with subtlety and historical awareness to add nuance or overturn entrenched narratives.

But my lingering suspicion emerges from an awareness of how public support for the sciences and denigration of the humanities mean that, in the effort to gain funding and acceptance, the humanities will lose what makes them. vital. The domain’s sensitivity to historical peculiarities and cultural differences makes the application of the same code to a wide variety of artefacts totally illogical.

How absurd it is to think that black and white photographs from 100 years ago would produce color the same way digital photographs do now. And yet that’s exactly what AI assisted colorization Is.

This particular example may seem like a small qualm, of course. But this effort to “revive the events”Systematically confuses representations with reality. Adding color does not show things as they were, but recreates what is already a recreation – a photograph – in our image, now labeled by IT.

Art as a toy in the scientists’ sandbox

Near the conclusion of a recent article devoted to the use of AI to unravel X-ray images by Jan and Hubert van EyckGhent altarpiece”, The mathematicians and engineers who wrote it refer to their method as being based on“ choosing the “best of all possible worlds” (borrowing the words of Voltaire) by taking the first output of two separate executions, do not differing only in the order of the entries. “

Perhaps if they had familiarized themselves more with the humanities, they would know how satirical those words were when Voltaire used to make fun of a philosopher who believed that endemic suffering and injustice were all part of God’s plan – that the world as it was represented the best we could hope for.

Maybe this “gotcha” is cheap. But it illustrates the problem of art and history becoming toys in the sandboxes of scientists with no humanities training.

At the very least, I hope the journalists and critics reporting on these developments take a more skeptical eye on them and change their framing.

In my opinion, rather than seeing these studies as heroic achievements, those charged with reporting their findings to the public should see them as opportunities to question what computer science does when it takes ownership of the study. art. And they should ask themselves if this is all for the good of someone or anything other than AI, its most zealous supporters and those who profit from it.


Sonja drimmer is a clinical physician and assistant professor at the University of Virginia. This article is republished from The conversation under one Creative Commons License. Read it original article.

Comments are closed.