Doubts about MyHeritage’s "LiveMemory" AI ancestor videos

In last week’s newsletter, I urged people to create ancestor videos using a new MyHeritage mobile app feature called LiveMemory. It was cute: upload an old group photo, and MyHeritage sends you an animated version of the photo showing the subjects moving around, chatting, and generally having a good time.

Jen from The Formidable Genealogist cooked up a doozy with an old 1927 photo, showing her ancestor playing musical instruments with friends:

Not everyone has been impressed. I showed Nicole an animated 1990s outdoor group photo with her friends posing on the steps of a monument. She was not happy with the results. Her face was changed to someone else’s, and an unknown boy suddenly pops out from the back of the group and runs to the front. The child was not in the original photo.

I had my own issues with the ancestor videos. A 1949 summertime photo showing my grandfather’s extended family had genders switched for several of the children when they started moving around. And for this portrait of unknown family that I found in a local antiques shop, nothing happened other than the father muttering something to the rest of the family, who remained frozen:

How are the animations made, and why are some clips showing things that didn’t happen? LiveMemory uses a generative AI engine to animate the photos:

“When a photo is uploaded, the AI model analyzes it to create the most suitable animation to reenact the scene. In photos with multiple people, the algorithm will animate the gestures and make them interact based on what it thinks will be most realistic.”

Unfortunately, AI models are limited in what they can depict, and will sometimes make substitutes based on thousands of other samples. This explains the mysterious boy popping out of the seated group of friends in Nicole’s 1990s photo.

This is not a “bug” in the sense of something breaking or a logic error. Rather, it’s an undesirable output of a system designed to animate still photos. The mistakes may look innocent, but they have serious implications for the way we understand the world around us, and the history that made us.

Criticism of ancestor videos

We’ve seen it before. Remember the AI “historical footage” of the 1849 Gold Rush, long before moving film cameras were invented?

Nicole had some pointed criticisms of the technology. “In the future, you won't know what's real and what's not real,” she warned. “Why would someone want make a video to sabotage my memory? Why create those fake feelings?”

That question I can answer. LiveMemory is basically a gimmick to get people to sign up and download the MyHeritage app. The app continuously prompts users to upload family tree data and eventually sign up for paid subscriptions. It also wants access to all of our photos, which can be used for further AI training.

Future of AI generated ancestor videos

What does the future hold for AI animations? It’s clearly not going away. As the technology improves, the quality of the animations will get better. Clips will get longer.

But then the videos will start to alter history in ways that are not accurate, to make them more interesting or serve some other motive. People will be allowed to customize their ancestors' emotions replacing frowns with smiles. Stout ancestors can be made thin. Your great-great-great grandfather can be dressed in a military uniform and placed on the battlefield at Gettysburg, even though he was too young to serve in the Civil War.

Maybe they’ll even start adding audio. First, it will be the murmur of conversation. Ancestors will begin to speak harmless phrases (“What time is supper?”) and then, potentially, things they never said at all.

As I said after the launch of OpenAI’s Sora tool, it’s exciting to view scenes that our ancestors may have experienced hundreds of years ago. However, we also have to be vigilant for seemingly authentic historical recreations based on flawed data or assumptions. It’s a slippery slope from harmless animations to revisionist history.