Part Two: Using AI in Genealogy Research Best Practices

In part one, we explain the basics of artificial intelligence and how you might begin to think about using AI in your genealogy research.

Here, in part two, we'll go more in depth with AI expert Steve Little to help you get a bigger picture about how AI could be useful in genealogy research now and in the future. Steve is a co-host of The Family History AI Show podcast, and AI educator with the National Genealogical Society (NGS).

LTG: What is prompt engineering?  

Steve: 

Prompt engineering is just a fancy term for “chatting with your chat bot,” that is, giving a language model the context to respond with the next right word.  By context, I like the analogy of short-term memory.  The technical jargon the experts use is “context window”. During the summer and fall of 2023, ChatGPT had a context window of about three pages, or a useful over-simplification would be to say, it had the short-term memory to process about three pages. It's significantly bigger now and every month it gets bigger.  

For example, when you lead the chatbot through a discussion of second cousins once removed and the formation of a GEDCOM file, you are setting the context. That is, you are putting words (and therefore ideas) about cousins and GEDCOM files in the model’s short-term memory, so to speak, so that the chatbot will have the words available to draw from and continue the discussion.  

When a user and a chatbot are talking about genealogy and second cousins and GEDCOM files and parents, grandparents, great-grandparents, then all the other statistically associated words in the model’s neural network are on standby, so to speak. That is, those genealogy related words are available to be selected as a probable next right word. When you lead a LLM through a discussion, you are shaping the context of the conversation. That's what prompt engineering is now: shaping a discussion, a dialog, in a useful way.  

In a sense, there are eight billion people on earth who know how to use these machines because by the time you are four years old, you know how to talk, and that's all that's required to use a language model. By the time you're four years, you know how to talk, and these tools just talk and listen.  

But all human communication is fraught with the possibility of misunderstanding. The reality is that people misunderstand each other all the time because words are slippery fish. If I say the word bank, for some people, a bank is where I put my money. But if you live in Wyoming and you enjoy fishing, you stand on the stream bank or the riverbank while you fish. 

If you're a pilot, how does a pilot steer an airplane? They bank the aircraft. The word bank is slippery – all words are slippery fish – and these tools are very good with slippery fish. But not perfect.  

What is different and difficult for us today is that we are not accustomed to imperfect machines, not in the strange ways these tools are imperfect. Because language models may not always understand the context of our conversations, it may surprise us. We think of these algorithms as imperfect, and we're not used to that. It may seem usefully creative in one context and factually challenged at the next moment.  

The first time you see one of these things you make a statement that you know is incorrect, not factually true, and doesn't correspond with the real world, it feels like being thrown off a cliff. We must learn to reconcile these imperfections with the usefulness of the tools.   

LTG:  Can you share a genealogy use case from a newspaper article? 

Steve: 

Let’s say you found a newspaper article from a hundred years ago, and it's a long profile of one of your ancestors. For example, imagine you are related to Charles Lindberg, and you discover a 20-page magazine profile of Charles Lindberg. You could give the language model that 20-page profile and tell it to extract every name that's mentioned in the profile and include how the named person is related to Charles Lindberg. Were they a family member? Were they another pilot? Were they a business partner? Were they someone Lindberg met on the street?

What you might ask the model to do would be, “Look at this article, generate a list of people mentioned in this article, and tell me their relationship to Charles Lindbergh.” And then you realize, well, gosh, now I'm going to have to go through and read the whole article to verify the relationship (not that reading the article is such a bad thing). 

But here is a tip: have the chatbot help you with the verification. You could say, “Add a third column, after the name in the first column and the relationship in the second column. Now add a third column, showing and quoting to me the exact sentence that led you to presume or make the claim that Bob Smith is Charles Lindbergh's nephew. Give me that sentence and tell me what page number that's on.” And the language model will process the 20-page profile in moments, creating a spreadsheet with names, relationships, and quoted evidence.  

 LTG:  How can genealogists collaborate with GPT to help the entire community? 

Steve: 

Collaborative learning is huge, and there's a long history of technology, of people sharing ideas and learning together.  

Once you figure out how to do a task, like analyzing an obituary, you can create a tool and save your prompt. That way the prompt is ready for you next time you need to analyze an obituary. You don’t have to recreate the wheel. OpenAI calls these saved prompts a custom GPT.  

So, a custom GPT is a way to save your prompts and to share your prompts. So, for example, if you and I process obituaries all day, and you tweak your obituary prompt so that it does something a little bit better, then we share that. And now we both have that better prompt. That's what these custom GPTs do. It's a way of sharing prompts. Like adding useful, helpful little tools to a Swiss Army knife. You discover a small task where the language model is helpful, efficient, or timesaving, then you save and share that use case prompt. And the whole community benefits. That is one of the purposes of my site AI Genealogy Insights.  

GeneaGPT and Custom GPTs: How they work

LTG:  Tell us about Open GeneaGPT.  

Steve: 

Open GeneaGPT is a long prompt I’ve created where I've asked the AI to pretend to be a genealogist.  It's just a prompt that says, “You are a professional genealogist. You know about the genealogical proof standard. Users are going to ask you questions about genealogy; answer them as if you were a professional genealogist. And adhering to the genealogical proof standard is important to you in your response to a user.” 

In its short-term memory, it says, “Okay, I'm a genealogist.” When you ask it a question, for example, “How should I get started in genealogy?”, it consistently says, start with yourself. Start from what you know, which is basic genealogical advice. How many blog posts and glossy genealogy magazines, and how many podcasts and how many workshops have you gone to where they say, start with yourself and work backwards. Open GeneaGPT offers basic help like that.   

LTG: How can other genealogists access these Custom GPTs? 

Steve: 

Right now, only one company makes this available to share these prompts, and that is OpenAI with Explore GPTs. It’s like a free app store for custom GPTs. You get access to custom GPTs and the ability to create and save these little tools.

Other companies are also adding more AI tools. Microsoft gives you free access to OpenAI's tools. Within Microsoft's Bing browser, you can access Copilot, the Microsoft AI. In all your Microsoft products such as Excel, Word PowerPoint, in the Bing browser, you now have a Copilot button.  

OpenAI has a GPT store where there are now tens of thousands if not millions of these tools.  People have found use cases, and they're sharing them with others. We don't have to reinvent the wheel.  

For example, there is one tool that uses image analysis; you can open your refrigerator and take a picture of what's in your refrigerator, and this custom GPT will suggest to you what to cook for dinner based on the ingredients that it identifies from the image of your refrigerator. It looked and it said, “Oh, there's some hamburger, and egg, and there's an onion. What can we do with these ingredients? Let's make meatloaf!”  

LTG: What are AI tools best suited for? 

Steve: 

The four basic transformations especially useful to genealogists performed by language model tools like ChatGPT are summarization, extraction, generation, and translation.  

Summarization

Summarization is when you take a lot of words, and you condense them, like transforming a lump of coal into a diamond.  You take a page worth of a biography, and you distill it to a tweet – and it's exquisite, it's poetic, it is worthy of a haiku. These things are very good at language. If you were to give it the obituary of Charles Lindbergh and say, reduce this to an award-winning haiku, it would. Summarization is taking a lot and making it smaller.  

Extraction

Extraction is finding a needle in a haystack, but it's more like finding a golden needle in a field of haystacks. It's like saying, go out and find all the mentions of this name. 

Generation

Language models are very good at generating text. It's good at taking a little bit of words and spinning that out into a lot of words. For example, this is your report generation. This is where you start with a list of names, dates, places, events, and relationships. That's genealogical information. That's a little export from your genealogical database. For example, ”Here's John Smith's family – a little list of names, dates, places, relationships, and events. Turn that list into a narrative report, a poem, a short story, or a genealogical biographical report. And only use the facts from this list.” And it will generate text exactly matching your prompt instructions.  

Translation

The last basic LLM transformation is translation. And that's much more than translating human languages. It will translate from one human language to another with caveats. It's not perfect. It's very good. It's as good as any other computer translation out there. But it’s not better than a paid translator. However, if you don’t speak both languages in the translation, the source and the target language, you may need an outside validator to verify it. This saves time as the translator then only needs to validate rather than create the original translation.  

However, translation is much more than French to English. It also means you can take something like Shakespeare, and you can translate Shakespeare into contemporary English. If you have an old will, it can translate legalese into plain English. Or you could take a modern-day contract and say, help me understand.  

It could also translate 17th century legalese. If you had a 17th century deed written in yards and chains and metes and bounds – things that beginners may not understand – you could translate the deed into plain English. It can also translate from a scholarly article. For example, if you have an article about genetic genealogy and you want to say, “Explain this to me like I'm a fifth grader, or I’m a 10th grader.” It can translate it in a way that is easier to understand.  

And it will even take a bullet point list. You could just jot down a handful of notes, and you could say, “Take these handful of notes and spin this up to a business proposal or an email or a letter or report or a memo to file, or anything.” And so that's another form of translation. 

And the last way that it's important to understand translation is as a very good editor. You can give it a very rough first draft, and you could say, “Translate this from a rough draft into standard written professional business English.” 

AI for Translating Handwriting in Genealogy Research 

LTG: Can the AI tools today read handwriting? 

Steve: 

These tools are getting better at reading handwriting, and they can read block, handwritten block print. Ultimately, though, we want handwritten text recognition (HTR) to be as good as speech recognition. By comparison, right now, we can talk to Siri or Alexa, and they are very accurate. Handwriting text recognition is not where speech recognition is. 

We all want handwriting to be where speech recognition is. We want it to just work the way speech recognition works. We want to show it a letter from a grandparent that it's never seen before and see it accurately recognize the text. But right now, that's not how it works. Today, you must train AI systems to understand your grandmother's cursive handwriting. And just because you teach it one grandmother's cursive handwriting, you've got other grandmothers, and her handwriting may not look the same as this one.

HTR systems are getting closer, but our expectations are through the roof. We want to give them the hardest problems, writing that none of us can visually figure out. We want the computer to do it. That’s a little beyond what they're up to. But it is coming eventually. At least, that is the hope and expectation. But we’re not there yet. Handwriting is hard.  

LTG: How do you anticipate these tools impacting the efficiency in the workplace, and specifically for genealogists? 

Steve: 

They will make us much more efficient. You’ll be able to get more work done in 40 hours than you did before. Or you could say, I did my 40 hours’ worth of work in 35 hours, and now I've got five extra hours. I can either do more work or I can spend five hours with my family.  

We’re in an age of discovery today. Many people are trying different things, to find out what tasks these AI tools are good at. And once one genealogist finds a way to use these tools, and then they share that successful task with other genealogists, those accomplishable tasks are called use cases. And so, we all get smarter, better, and faster. But to get there, often we try things that don't work. Sometimes we'll try to use AI to do something, and it may or may not do it. And even if it does it, there may still be an old school way that's better. So right now, folks are trying things, and sometimes they work, but that doesn't always mean it's better. Sometimes the old school way just works better. And there are many, many tasks that AI is close to doing reliably now, and in six months and six years, AI will be able to do more. We just don't know how long it will take to accomplish any specific task. This uneven advance is called the jagged frontier.  

For the folks who want to participate in using and learning AI, it's going to give them superpowers. It's going to make you better and faster and quicker at your job if you're a knowledge worker.  

Implementing AI will lead to an opportunity for company growth as well. Chief Executive Officers will say, “I'm going to keep one hundred percent of my people and we're going to do 150% of the work that we used to do. We're going to grow our business. We're going to hire more people because we're better at this than anybody else. For example, we can do genealogy better, faster, and cheaper than our competition because we've trained our 75 researchers how to use these tools to produce twice the work in the same amount of time.” 

Read part one of this two-part series here. 

At Legacy Tree Genealogists we focus on the best possible results for our clients and are constantly seeking out new tools and resources, such as the use of AI, to make our time more efficient and effective. Are you ready to hire a genealogist? Reach out for a free quote and get started today.  

Part One: AI Today and How to Use AI in Genealogy Research

AI technology impacts every area of our lives today and is growing so quickly it can be difficult to keep up, especially with AI in genealogy research when it feels so new and unknown. We were able to spend some time with Steve Little, an expert in the AI and genealogy space, co-host of The Family History AI Show podcast, and AI educator with NGS,  to discuss the future of AI and genealogy.

In part one you will learn about the history of AI, how it has developed to this point, and how to effectively use AI.  

Part two will cover use cases for genealogy and how to save time and research more accurately with AI tools.

LTG: What's your background with AI and how did it catch your attention? 

Steve: 

There's kind of a Day Zero, just about 18 months ago ,when something huge happened. I think we’ll look back in world history and December of 2022 is going to be a date that sticks out. Eighteen months ago, when OpenAI released this product called ChatGPT, it got many people excited, especially if they had peculiar interests. And the three peculiar interests that I have had for more than 40 years are language, technology, and genealogy. When this new tool became available, it captured my attention immediately.  

I grew up loving technology. As a teenager in the 1980s, I had one of the first personal computers, a Commodore 64. And I’ve always loved language.  

I had an aunt who was into genealogy. She was a serious genealogist in the fifties, sixties and seventies, and I started doing data entry for her in the 1980s with a DOS version of Family Tree Maker. And just by osmosis I learned and came to love genealogy and the genealogical database technology.  

When this tool, ChatGPT, became available 18 months ago, it immediately grabbed my attention and the attention of several hundred million other people within a month. OpenAI went from zero users to a hundred million users in one month and no company had ever done anything remotely that fast before. It got a lot of people's attention. 

AI Explained In Basic Terms 

LTG: What exactly is AI, or artificial intelligence? 

Steve: 

Let's start from the big idea and get a bit more specific because the phrase artificial intelligence is an umbrella term. There are about 12 different fields of study that would fit inside artificial intelligence. If you talk to a computer scientist, they've been talking about artificial intelligence for more than 50 years. 

But in the past 18 months, ordinary people who are talking about artificial intelligence are referring to something very specific and new. Within the broader field of artificial intelligence, one aspect deals with language. This field is called natural language processing and involves teaching computers how to talk and listen, read and write. That’s been around for a long time. Over the past 50 years we’ve seen incremental improvements in how computers can just talk and listen.  

But something happened 18 months ago: an incremental change in improvement reached a tipping point such that it wasn't just an incremental improvement, it was as if a light switch had been flipped and it went from darkness to light. And what happened was not just the incremental improvement in how this tool talks and listens, but in its ability to be significantly more useful to everyday people.  

This usefulness is not just due to the technology of language processing, but also due to the interface. For the past 18 months or so, the interface to talk to these new tools most closely resembles sending text messages back and forth. That's why they call it a chatbot. So, under the umbrella of artificial intelligence, you have natural language processing (“NLP”).  

Beneath NLP you have large language models (“LLMs”). These are the newer tools like ChatGPT that have become much more useful. Now, LLMs weren't invented just 18 months ago either. But something significant happened about seven years ago, in 2017. Google – the same Google we all know and love or hate or both – Google invented something new.   

They invented a new way for these natural language processing systems to talk and listen. They developed a new way to make these better, called the transformer. That's the T in GPT. The tool became much better at talking, listening, and processing language. Given a string of words, the transformer is very good at picking – generating – the next word to continue that string.  

Companies like OpenAI took this transformer technology that Google released seven years ago and developed it, and then 18 months ago they released a commercial product called ChatGPT, such that we can chat with it. And behind the chatbot is the large language model, GPT4.  

We use the chatbot called ChatGPT, but the brain behind it is a large language model GPT4 and that's what's so very good at processing language. And now there are other big companies who are also releasing their own large language models.  

• OpenAI created ChatGPT that most people are quite familiar with.  
• Google has one called Gemini that is very strong. 
• Anthropic, former OpenAI folks, has one called Claude that is very good.  
• Facebook has released META AI in their Facebook, Instagram and WhatsApp platforms.   

To answer the question, what is it that is making these AI tools work?

It's a computer algorithm that is exquisitely good at talking and listening. It's not alive, it's not really talking or listening. It's in many ways a computer program like every other you've ever used. But in some other ways, it's unlike any computer tool you've ever used before. It's significantly different.  

It is so good at language that it fools people. Most users will have an experience where they're talking to this machine, conversing with it, typing back and forth to this machine, when it responds in such a humanlike way, it takes your breath away for a moment. Even if you've been using it for a year and a half, it can still surprise you what these new emergent capabilities can produce.  

How AI “Thinks” 

LTG: Where does the LLM get its data to answer prompts? 

Steve: 

When you first use these machines, it almost feels like it might be doing research and looking up something for you, but that's not what it's doing. It is a useful oversimplification to say that it is just picking the next right word. For example, “Old McDonald had a [blank]. What's the next word? 

“Farm”. Now, you are doing that from memory. You had somebody who loved you and read to you nursery rhymes, and you have a memory of having heard that before. Well, computers have memories too, but not exactly like ours. When you hear that nursery rhyme beginning, you may have an image of a grandmother or parent come to mind, someone who read you those rhymes. That's not what's happening with these tools. They do not remember somebody reading them nursery rhymes. But neither are they looking-up the answer.  

Instead, what happened was, these LLMs have been ”trained”, they have read about everything that's ever been written and digitized for the last 6,000 years. Human beings have been writing things down for 6,000 years, and much of that has been digitized such that a computer could access it. And so, these language models have been shown all of that text from the past 6,000 years, and we say they have been “trained”. From all that text, LLMs can determine what words come next to each other, so that sometimes when it hears the phrase “Old McDonald had a [blank],” it knows that near Old McDonald sometimes the word farm comes up. So, it learns the word farm and Old McDonald, and the LLM says, these things come together sometimes, and it assigns relationship between “farm” and “Old McDonald” a number based on how frequently these words are found together. It’s statistics and probability.  

So that if you give it a phrase like “Old McDonald had a farm, and on this farm, he had a [blank].”, what could come next? It could be a cow, a chicken, a sheep, or a goat. It knows it's one of those words but using statistics and probability.  

But it’s bigger than that, too. When you think about what a word is, these tools become much more powerful. Here is a demonstration: I can put an image inside your head right now. If I say the word “elephant”, whether you wanted to or not, you are now imagining an elephant. I transferred an idea from my mind into your mind. Words are ideas, concepts, meaning. This tool simulates the manipulation of ideas, concepts, and abstractions, similar to the way a spreadsheet processes numbers. With a spreadsheet you can put in a bunch of numbers, and it will add, subtract, multiply, and divide the numbers with great precision. This tool – the large language model – is good at manipulating words, which means it's simulating the manipulation of ideas and concepts, so that it can slice and dice words, concepts and ideas the way a spreadsheet uses addition, subtraction, multiplication, and division to manipulate numbers.  

You could also ask the LLM, “What genre of literature is Old McDonald?”. And then it would say, “Oh, she used the word genre. She used the word literature. And it would respond “nursery rhyme.” It has appeared to generalize, reason, or think. As if it says, “Oh, okay, we're talking about something bigger than just Old McDonald. We're talking about genres of literature.” It has now zoomed out and it's “thinking” like a literature professor. But underneath – behind the wizard’s curtain – it's just probabilities, the statistical relationships amond and between words in a string.  

When you ask it a question, it is not just a really good Google that searched for the answer. It's not doing anything remotely like that. It is just choosing the words that are likely to come next given all the words you've previously just spoken to it.  

LTG:  A term that is often heard in reference to AI is hallucination, meaning it is not based in reality and the information or responses coming from the tool are hallucinations. What does this mean?  

Steve: 

In a sense, it's all hallucinated. All the words are generated in its neural network. And the LLMs neural network is totally disconnected from reality. When we say it has a hallucination rate of 30%, or it's accurate 70% of the time, or when we claim it is accurate 97% of the time, what we're claiming is that it's hallucinating only 3% of the time. But if it's actually hallucinating a hundred percent of the time, and 97% of its hallucinations correspond to our reality, then we say it's getting things right in the real world 97% of the time.  

If you learn what the machines can and can't do, you can get the hallucination rate under 1%. There are best practices and if you follow these best practices – if you use these tools the way they're intended to be used within the capabilities that they have today – you can get it to correspond with reality 99% of the time.

The best practices are three:

1) Know your data

2) Know your model

3) Know its limits  

These best practices require bringing data to the machine – knowing what the machine can do and only asking it to do what it can do. “Today’s limits are today’s limits,” as we way. New users, they want the chatbot to be a magic genie. But it takes people about 20 hours of using the tools to learn what they are actually good at doing.  

How To Use AI in Genealogy Research

LTG: What should you be asking AI to get the most accurate answers in your genealogy research? 

Steve: 

First off, I encourage people to play. You learn best by playing. And you're not going to break these machines. Play and have fun. That's the best way to learn. But, when you're ready to do fact-based, reality-based, genealogical work, where evidence matters, where facts matter, then you want to pay attention to the limits of the tool and be very conscious and aware of your own expectations and what you're asking the LLM to do.  

A year ago, we would have told genealogists not to use it for research, at all. And last year that was good advice because it wasn't very good at research. Today, it is getting better, but it's still not trustworthy. I encourage beginners to ask themselves each time they use this tool, “Are you doing research? Are you asking this tool to tell you something you didn't already know?” And usually they say, “Well, of course, what else would I use it for?” But – today – they are still stepping onto thin ice.  

There are about 20 million things you can do with AI tools other than research, but that doesn't occur to us. Google has warped our brains. Over the past 20 years we've learned that if we want to learn something we don't know, we go to Google and we type in a short phrase, and it gives us the answer we were looking for. 

And we mistakenly think that's what this tool might be doing. We do the same thing as a Google search and the chatbot seems to respond. We ask it a question and it seems to give us an answer in perfect English. And so, we think maybe the answer is as perfect as its grammar, but it is not. It can get the grammar perfect without getting the reality correct. But there are ways to mitigate that: If you bring to the tool the information you want to work with – instead of asking it to show you something you don't already know – it will slice and dice language very well. You can give it information such as words, language, text, wills, probate files, chapters of a book, an article, and it can help you process that information in lots of different, useful, safe ways.  

That's how you drive the hallucination rate below 1%. You say, “Let's just talk about this right here, this information I’m giving you [the chatbot] right now.” You're not asking it to go out and discover something new. You're bringing information to it. And when you do that, that's how it becomes useful genealogically to process information you've already got.   

And genealogists have boxes, folders, cabinets, shelves, closets, basements, and external hard drives full of information. You've been collecting data for as long as you've been doing genealogy.  

So now you have a very smart intern or assistant to help you process that data and information that you've been collecting for as long as you've been doing family history.  

Now there's somebody to help you make sense of that. 

Now there's somebody to help you find the needle in the haystack.  

Now there's somebody to help you condense 800 pages of information you need distilled into two pithy paragraphs. It'll do that in 20 seconds, and that's hugely powerful. 

LTG: What were some of your initial ideas about how you could use AI for genealogy research?  

Steve: 

Over the past 18 months, I've spent about 700 hours trying to figure that out. I spend about 20 hours a week just trying things. Does this work and does this not work? And I fail 90% of the time, so I discover many, many things that it cannot do today. I've been stunned to see there were things we could not do a year ago or even six months or three months ago that we can do today. That's exciting, seeing how fast these tools are getting better and more useful. But just trying things, seeing what it could do and what it failed to do, and mapping that out, has been what I've spent a huge part of the last 18 months doing. 

 In part two we will discuss use cases for AI in genealogy and how you can save time and become more accurate by using these tools the right way. 

If you'd like help with your genealogy research, contact us to ask your questions and get a free quote!