Nov 8, 20235 min read

The Editor’s Guide to Fact-Checking with AI

Updated: Jul 18

I used to say, “ChatGPT is not a fact-checker.” Period. Full stop. But not anymore. ChatGPT now has the capability to cite its sources and browse the live web so it can use current information that you can verify. Plus, it can gather the information you need—and only the information you need—which will save you heaps of time tracking down sources one by one.

This article demonstrates how to use ChatGPT for research and fact-checking and explores Perplexity AI, another highly useful AI tool for research.

Testing ChatGPT for Fact-Checking

To test ChatGPT’s research capabilities, I prompted it to give information about Ada Lovelace, an early pioneer of computer science. Remembering ChatGPT works best when given specific instructions, I wrote this prompt:

Please provide a two-paragraph biography of Ada Lovelace with citations in the text. Ensure sources and information are reputable, current, and accurate. Include Encyclopedia Britannica as a source. Do not use Wikipedia.

The reason I instructed it to use Encyclopedia Britannica as a source is because I’ve been waiting for ChatGPT to specifically be able to access Encyclopedia Britannica as a benchmark of its ability to seek out factual, quality, trustworthy information. Similarly, I asked it to avoid using Wikipedia to test this same ability but through eschewing less-reliable sources.

The Result

ChatGPT supplied the two-paragraph bio, as requested, with sources cited in the text, shown through superscript numbers. The sources it cited included Encyclopedia Britannica, Biography, the San Diego Supercomputer Center at the University of California, San Diego, and ThoughtCo. As requested, it did not use Wikipedia. I checked all of the links, and they were all valid and current.

Here’s a video showing the process and result.

Here’s a screenshot of the ChatGPT-created bio.

Ada Lovelace, born Augusta Ada Byron on December 10, 1815, in Middlesex (now part of London), England, was the only legitimate child of the renowned poet Lord Byron and Annabella Milbanke Byron. From a young age, Ada showed a profound aptitude for mathematics, a path encouraged by her mother to steer her away from her father's "dangerous" poetic disposition. Her mother, Annabella, emphasized the study of music, French, and mathematics in Ada's education. At the age of 17, Ada's life took a decisive turn when she met the inventor Charles Babbage in 1833, with whom she began a substantial correspondence on mathematics, logic, and various subjects123. Lovelace is celebrated as the world's first computer programmer due to her work on Babbage's early mechanical general-purpose computer, the Analytical Engine. In 1843, she developed a program for the Analytical Engine, demonstrating that it could be used not only for calculations but also to create any content, including music or art, given the right inputs and algorithm. This insight laid the foundation for the field of computer programming. Furthermore, Ada Lovelace's contributions extend to the concept of an operating system or software, embodying the idea that a computing machine could execute a program from a set of instructions. Her legacy continues to inspire, with Ada Lovelace Day, celebrated on the second Tuesday of October, recognizing the significant contributions of women in science and technology456.

Did ChatGPT work perfectly every time? No. When I prompted it to provide full citations and links to the sources it used, it sometimes included dead links and struggled to follow the requested style. Still, it can speed up your work greatly.

RSVP: AI for Editors

How to Use ChatGPT for Researching and Fact-Checking

To use ChatGPT to research, you can prompt it to supply information about the topic. Remember to be specific in your prompt. This includes telling ChatGPT how much information you want. If you want a quick overview, as with the first Ada Lovelace example, you could instruct ChatGPT to give you a two- or three-paragraph overview. If you want to dive deep into a topic and are looking for a lot of information, you could ask it to be comprehensive and tell it the subject areas it should include (if you know).

For In-Depth Researching

To do more in-depth research on Ada Lovelace, I used this prompt:

Please give a comprehensive biography of Ada Lovelace. Include information about her childhood and family, her personal life, her scientific research, her legacy, and other relevant information. Cite your sources. Include credible information. Do not use Wikipedia.

Here’s the response from ChatGPT:

Ada Lovelace: Comprehensive Biography Childhood and Family Born: Augusta Ada Byron on December 10, 1815, in London, England1. Parents: The only legitimate child of the poet Lord George Gordon Byron and Lady Anne Isabella Milbanke Byron2. Education: Her mother, Lady Byron, herself mathematically trained, insisted Ada study mathematics—a rare education for women at the time3. Personal Life Marriage: In 1835, Ada married William King, who later became the Earl of Lovelace, thereby Ada became the Countess of Lovelace4. Children: Ada had three children4. Nickname: Given by her father as his "Princess of Parallelograms" due to her mother's mathematical background3. Scientific Research Meeting Babbage: Ada met Charles Babbage in 1833 and was captivated by his work on the Analytical Engine5. Programming: She is credited with creating the first algorithm intended for processing on a machine, thus considered the first computer programmer6. Visionary Work: Ada's notes on Babbage's Analytical Engine included ideas that machines could manipulate symbols and create music, showing a vision of computing beyond mere calculation5. Legacy Recognition: Ada Lovelace Day, celebrated on the second Tuesday in October, honors women in science and technology6. Prophet of the Computer Age: She was the first to articulate the potential for computers beyond mathematics, seeing the possibility of a machine that could manipulate symbols according to rules, marking a fundamental transition from calculation to computation5.

It cited six sources, all credible, including Encyclopedia Britannica and the Computer History Museum. It did not use Wikipedia. And, it covered the requested subtopics. From here, I could ask a follow-up prompt to get more information about one of the details, or I could check out one of the sources to learn more.

For Checking Single Facts

If you are fact-checking an author or client’s writing about, say, Ada Lovelace and need to check one detail, you could try a prompt like this:

In a sentence, answer the following question: Was Ada Lovelace involved in the creation of the Analytical Engine? Cite your sources. Use multiple, credible sources.

Here’s how ChatGPT responded:

Yes, Ada Lovelace was involved in the creation of the Analytical Engine, for which she is celebrated as the first computer programmer, having developed a program for Charles Babbage's early computing prototype in 1843

It cited again Encyclopedia Britannica. This time it also cited IEEE Annals of the History of Computing. But, likely because I did not specifically instruct it to avoid it, it did include Wikipedia. Even so, this short response has given me two credible sources I can use to verify the information it has provided.

For a real-world example of using ChatGPT to track down a tricky fact, check out editor Kristen Tate's blog post, "How to Use AI Tools for Fact-Checking."

For Checking Multiple Facts at Once

If you’re fact-checking a passage or short article about a specific topic, where you know you’ll need to verify a number of related facts, you could incorporate ChatGPT into your workflow by prompting it to create a list of top facts about it.

Here’s the prompt I used in the Ada Lovelace experiment:

Please create a bullet-point list of top facts regarding Ada Lovelace's connection to the Analytical Engine. Cite your sources. Ensure sources are reputable, current, and accurate.

Here’s its response:

Here's a bullet-point list of key facts about Ada Lovelace's connection to the Analytical Engine, with citations from reputable sources: Ada Lovelace is renowned for her detailed account of Charles Babbage's Analytical Engine, which is acknowledged as an early model for a steam-powered programmable computer, though it was never actually constructed1. Lovelace was an English mathematician and is often celebrated as the first computer programmer because she developed a program for the Analytical Engine in 18432. Between 1842 and 1843, she expanded a translation with her own comprehensive "Notes," which were three times longer than the original document by Luigi Menabrea. These notes are considered the first detailed description of a digital computer's design and operation3. Her interest in the Analytical Engine began after meeting Charles Babbage in June 1833, facilitated by their mutual acquaintance and her tutor, Mary Somerville4. The work Lovelace did on the Analytical Engine included what we would now call a computer program, intended to calculate Bernoulli numbers, which was part of her translation of Menabrea's paper on the engine5. These points illustrate Lovelace's pivotal role in conceptualizing the operations and potential of early computing devices.

This time it cited the Mathematical Association of America, Oxford University, Patterns data science journal, and, yes, Wikipedia.

This is like a hyperspeed Google search, allowing you to check multiple details very quickly, cross-referencing facts between sources.

Easy-to-Use Option: Perplexity AI

While ChatGPT’s wide-ranging capabilities make it an editorial Swiss Army knife, fact-checkers and researchers may benefit from exploring Perplexity AI. This tool uses both the Claude and GPT AI models and the internet to create curated responses.

It works faster than ChatGPT. But, like ChatGPT, the paid version provides quite a bit better response (six sources with the free version vs. sixteen with the paid “copilot” enabled).

To test Perplexity AI, I used the same Ada Lovelace prompt as with the first experiment. Note that even though instructed not to use Wikipedia, Perplexity AI still did.

Here’s a video of Perplexity AI at work. Prepare to be amazed.

My New Motto

Now my motto about using ChatGPT to fact-check is the old adage: trust, but verify. While ChatGPT and Perplexity AI can reply with content that leans much farther toward Trust on the Trust-O-Meter, like any good fact-checker, it’s still important to double-check.

Key Insights & Takeaways

ChatGPT's Enhanced Capabilities:

When provided with specific and detailed prompts, ChatGPT can effectively pull information from reputable sources, including Encyclopedia Britannica.
ChatGPT did not use Wikipedia when instructed not to, indicating it can follow directions to exclude certain sources.
The tool provided valid and current links, demonstrating its ability to access and cite credible information from the live web.

Limitations and Issues

ChatGPT sometimes included dead links and had difficulty adhering to requested citation styles.
The AI occasionally produced “hallucinations,” or incorrect predictions, when not using the live internet browsing feature.

Perplexity AI's Performance:

It operates faster than ChatGPT and can provide more sources when using the paid version.
Perplexity AI still cited Wikipedia even when instructed not to, revealing a potential issue with source filtering.

Reliability of AI for Fact-Checking:

Both AI tools showed a high degree of compliance with user instructions, but they are not foolproof.
It’s vital to verify the information provided by AI and maintain a "trust but verify" approach.
The tests underscore the importance of using specific prompts and live internet features to guide AI tools towards more accurate and useful results for research and fact-checking.

Erin Servais helps editors upskill through AI. Her AI for Editors course is known worldwide as the #1 AI course for editors of all types, including medical editors, finance editors, education editors, corporate communications editors, and book editors.

Erin serves on the board of directors for ACES: The Society for Editing and has presented about editing, entrepreneurship, and artificial intelligence for the Professional Editors Network, Editors Canada, the Northwest Editors Guild, the Editorial Freelancers Association, and ACES.

Erin collaborated with artificial intelligence to write this article.

Email Erin: Erin@aiforeditors.com

AI for Editors