Generative AI can turn your most precious memories into photos that never existed

🌈 Abstract

The article discusses the Synthetic Memories project by the Barcelona-based design studio Domestic Data Streamers, which uses generative AI models to create images of people's memories that were never captured on camera. The project aims to help immigrant and refugee communities around the world reclaim their lost memories and histories. The article also explores the potential use of these memory-based reconstructions in reminiscence therapy for dementia patients.

🙋 Q&A

[01] Synthetic Memories Project

1. What is the Synthetic Memories project?

• The Synthetic Memories project is a project run by the Barcelona-based design studio Domestic Data Streamers that uses generative image models, such as OpenAI's DALL-E, to bring people's memories to life and create images of scenes that have never been photographed or to re-create photos that were lost.

2. What is the goal of the Synthetic Memories project?

• The goal of the Synthetic Memories project is to help immigrant and refugee communities around the world reclaim their lost memories and histories by creating images of scenes and events that were never captured on camera.

3. How does the Synthetic Memories project work?

• The process involves an interviewer sitting down with a subject to get them to recall a specific scene or event, which a prompt engineer then uses to write a prompt for a generative image model to create an image.
• The team has built up a "glossary of prompting terms" that work well for evoking different periods in history and locations.
• There is often some back and forth, with the subject providing feedback to refine the image until it "clicks" with their memory.

4. What have the results of the Synthetic Memories project been so far?

• The project has been used to preserve the memories of people in various migrant communities, including Korean, Bolivian, and Argentine families living in São Paolo, Brazil.
• The team has also worked with a care home in Barcelona to explore how the memory-based reconstructions might help older people with dementia through reminiscence therapy.

[02] Use of Generative AI for Reminiscence Therapy

1. How is the Synthetic Memories project exploring the use of generative AI for reminiscence therapy?

• The team collaborated with researchers in Barcelona on a small pilot study with 12 subjects, applying the Synthetic Memories approach to reminiscence therapy for dementia.
• The researchers are now planning a larger clinical study with colleagues at the University of Toronto to compare the use of generative image models with other therapeutic approaches for dementia.

2. What did the pilot study discover about the use of generative images for reminiscence therapy?

• The researchers found that older versions of generative image models work better than newer, more detailed models, as the glitches and blurriness of the older models better match the nature of memories.
• They also discovered that older people connected with the images much better when they were printed out, rather than viewed on a screen.

3. What are the potential benefits and concerns around using generative AI for reminiscence therapy?

• The project explores whether these memory-based reconstructions can help reframe traumatic memories or if they could create a false sense of reality leading to confusion and cognitive dissonance.
• The team is aware of the need to clearly differentiate synthetic memories from real photographic records, which is why they prefer the less photorealistic images produced by older generative models.