Another incident of Ancestry "data theft" scraping of local genealogy groups

Ancestry has once again been caught scraping data from a local genealogy society’s website without permission, according to a post dated May 3 on the Scottish Genes blog. I will explain what happened, why it’s so wrong, and what you should do if you belong to or support local genealogy and history societies.

Earlier in the spring, ManxBMD, a genealogy group located on the Isle of Man, accused Ancestry of “data theft,” republishing without permission an index of local vital records. The latest incident involves the Aberdeen and North-East Scotland Family History Society. Ancestry basically republished this group’s Index to Memorial Inscription Booklets without permission.

As someone with ancestry in Aberdeenshire, the index is a useful resource with inscriptions going back many centuries. I found names from kirkyards and plots that I wouldn't be able to visit otherwise without spending thousands of dollars in travel costs:

ManxBMD and the Aberdeen and North-East Scotland Family History Society are not the only victims. Ancestry.com unapologetically states that it has a program in place to scrape and republish data without permission.

Note that many of these societies depend on volunteer labor and dues and donations from members. But when Ancestry.com starts ranking higher in search results on Google, that translates to less awareness of these groups, fewer new members, and ultimately, fewer donations.

How to stop Ancestry from scraping data

Ancestry says that the only solution for a local group to proactively stop the for-profit company from scraping and republishing genealogy data is by changing its website's robots.txt file. This is a technical process that many volunteer groups can’t handle on their own. Further, stopping automated scrapers also stops Google from indexing the site in question, which may also harm future outreach and donations.

Another bad option to stop the scrapers: the groups can remove the data from the open web. That’s what ManxBMD did after it found out that Ancestry had strip-mined its online database, stating "Due to apparent data theft by Ancestry, a free search facility will no longer be available on ManxBMD":

As of early May 2024, the ManxBMD index is now paywalled.

Why is Ancestry scraping data?

Chris Paton from Scottish Genes asks, “one has to ask whether this is being done for charitable reasons, or for other purposes.”

The answer is: Other reasons. Ancestry doesn’t do charity. It doesn't care about the membership of the Aberdeen and North-East Scotland Family History Society or other groups. Its executives lose no sleep over the prospect of such groups' fundraising or online search profile being impacted by the unauthorized scraping and republishing of genealogy records on Ancestry.com.

Ancestry is owned by Blackstone Inc., a ruthless private equity group whose CEO was paid $897 million last year. The Ancestry business model is based on genealogists and government agencies and local history societies giving data to Ancestry for free, and then turning around and charging millions of people hundreds of dollars per year to access the data.

Ancestry will do anything to squeeze more money genealogists - including sneaky price increases and bait-and-switch tactics (see Ancestry piles on more fees, including new "AncestryDNA Plus" for formerly free features).

What should supporters of local genealogy groups do?

So what should you do, if you support or belong to local genealogy or history organizations?

1. Engage with their resources, whether online or in person.
2. See if your group’s data is being republished without permission by Ancestry or any other group, and demand that it be taken down.
3. Support these local groups in any way you can, whether it’s volunteering, writing a check, or participating in other activities.

It is the local genealogy and history socieities that take responsibility for preserving local community history and genealogy resources. It is these groups - not Ancestry or its billionaire owners - that need our support more than ever.