The Editor’s Guide to Fact-Checking with AI

I used to say, “ChatGPT is not a fact-checker.” Period. Full stop. But not anymore. ChatGPT now has the capability to cite its sources and browse the live web so it can use current information that you can verify. Plus, it can gather the information you need—and only the information you need—which will save you heaps of time tracking down sources one by one.

This article demonstrates how to use ChatGPT for research and fact-checking and explores Perplexity AI, another highly useful AI tool for research.

Testing ChatGPT for Fact-Checking

To test ChatGPT’s research capabilities, I prompted it to give information about Ada Lovelace, an early pioneer of computer science. Remembering ChatGPT works best when given specific instructions, I wrote this prompt:

Please provide a two-paragraph biography of Ada Lovelace with citations in the text. Ensure sources and information are reputable, current, and accurate. Include Encyclopedia Britannica as a source. Do not use Wikipedia.

The reason I instructed it to use Encyclopedia Britannica as a source is because I’ve been waiting for ChatGPT to specifically be able to access Encyclopedia Britannica as a benchmark of its ability to seek out factual, quality, trustworthy information. Similarly, I asked it to avoid using Wikipedia to test this same ability but through eschewing less-reliable sources.

The Result

ChatGPT supplied the two-paragraph bio, as requested, with sources cited in the text, shown through superscript numbers. The sources it cited included Encyclopedia Britannica, Biography, the San Diego Supercomputer Center at the University of California, San Diego, and ThoughtCo. As requested, it did not use Wikipedia. I checked all of the links, and they were all valid and current.

Here’s a video showing the process and result.

Here’s a screenshot of the ChatGPT-created bio.

Did ChatGPT work perfectly every time? No. When I prompted it to provide full citations and links to the sources it used, it sometimes included dead links and struggled to follow the requested style. Still, it can speed up your work greatly.

RSVP: AI for Editors

How to Use ChatGPT for Researching and Fact-Checking

To use ChatGPT to research, you can prompt it to supply information about the topic. Remember to be specific in your prompt. This includes telling ChatGPT how much information you want. If you want a quick overview, as with the first Ada Lovelace example, you could instruct ChatGPT to give you a two- or three-paragraph overview. If you want to dive deep into a topic and are looking for a lot of information, you could ask it to be comprehensive and tell it the subject areas it should include (if you know).

For In-Depth Researching

To do more in-depth research on Ada Lovelace, I used this prompt:

Please give a comprehensive biography of Ada Lovelace. Include information about her childhood and family, her personal life, her scientific research, her legacy, and other relevant information. Cite your sources. Include credible information. Do not use Wikipedia.

Here’s the response from ChatGPT:

It cited six sources, all credible, including Encyclopedia Britannica and the Computer History Museum. It did not use Wikipedia. And, it covered the requested subtopics. From here, I could ask a follow-up prompt to get more information about one of the details, or I could check out one of the sources to learn more.

For Checking Single Facts

If you are fact-checking an author or client’s writing about, say, Ada Lovelace and need to check one detail, you could try a prompt like this:

In a sentence, answer the following question: Was Ada Lovelace involved in the creation of the Analytical Engine? Cite your sources. Use multiple, credible sources.

Here’s how ChatGPT responded:

It cited again Encyclopedia Britannica. This time it also cited IEEE Annals of the History of Computing. But, likely because I did not specifically instruct it to avoid it, it did include Wikipedia. Even so, this short response has given me two credible sources I can use to verify the information it has provided.

For a real-world example of using ChatGPT to track down a tricky fact, check out editor Kristen Tate's blog post, "How to Use AI Tools for Fact-Checking."

For Checking Multiple Facts at Once

If you’re fact-checking a passage or short article about a specific topic, where you know you’ll need to verify a number of related facts, you could incorporate ChatGPT into your workflow by prompting it to create a list of top facts about it.

Here’s the prompt I used in the Ada Lovelace experiment:

Please create a bullet-point list of top facts regarding Ada Lovelace's connection to the Analytical Engine. Cite your sources. Ensure sources are reputable, current, and accurate.

Here’s its response:

This time it cited the Mathematical Association of America, Oxford University, Patterns data science journal, and, yes, Wikipedia.

This is like a hyperspeed Google search, allowing you to check multiple details very quickly, cross-referencing facts between sources.

Easy-to-Use Option: Perplexity AI

While ChatGPT’s wide-ranging capabilities make it an editorial Swiss Army knife, fact-checkers and researchers may benefit from exploring Perplexity AI. This tool uses both the Claude and GPT AI models and the internet to create curated responses.

It works faster than ChatGPT. But, like ChatGPT, the paid version provides quite a bit better response (six sources with the free version vs. sixteen with the paid “copilot” enabled).

To test Perplexity AI, I used the same Ada Lovelace prompt as with the first experiment. Note that even though instructed not to use Wikipedia, Perplexity AI still did.

Here’s a video of Perplexity AI at work. Prepare to be amazed.

My New Motto

Now my motto about using ChatGPT to fact-check is the old adage: trust, but verify. While ChatGPT and Perplexity AI can reply with content that leans much farther toward Trust on the Trust-O-Meter, like any good fact-checker, it’s still important to double-check.

Key Insights & Takeaways

ChatGPT's Enhanced Capabilities:

• When provided with specific and detailed prompts, ChatGPT can effectively pull information from reputable sources, including Encyclopedia Britannica.

• ChatGPT did not use Wikipedia when instructed not to, indicating it can follow directions to exclude certain sources.

• The tool provided valid and current links, demonstrating its ability to access and cite credible information from the live web.

Limitations and Issues

• ChatGPT sometimes included dead links and had difficulty adhering to requested citation styles.

• The AI occasionally produced “hallucinations,” or incorrect predictions, when not using the live internet browsing feature.

Perplexity AI's Performance:

• It operates faster than ChatGPT and can provide more sources when using the paid version.

• Perplexity AI still cited Wikipedia even when instructed not to, revealing a potential issue with source filtering.

Reliability of AI for Fact-Checking:

• Both AI tools showed a high degree of compliance with user instructions, but they are not foolproof.

• It’s vital to verify the information provided by AI and maintain a "trust but verify" approach.

• The tests underscore the importance of using specific prompts and live internet features to guide AI tools towards more accurate and useful results for research and fact-checking.

Register for AI for Editors

Erin Servais helps editors upskill through AI. Her AI for Editors course is known worldwide as the #1 AI course for editors of all types, including medical editors, finance editors, education editors, corporate communications editors, and book editors.

Erin serves on the board of directors for ACES: The Society for Editing and has presented about editing, entrepreneurship, and artificial intelligence for the Professional Editors Network, Editors Canada, the Northwest Editors Guild, the Editorial Freelancers Association, and ACES.

Erin collaborated with artificial intelligence to write this article.

Email Erin: Erin@aiforeditors.com

ChatGPT Role-Playing Prompts: A Guide for Editors

One ChatGPT prompting technique you'll want to try is role-playing. With role-playing, you instruct ChatGPT to take on a persona, such as editor, author, or reader. This approach fine-tunes the AI's responses and gives you more actionable insights and ideas for edits. Plus, it's a prompting method you can use whether you're a copyeditor, developmental editor, line editor, or author coach. Intrigued? Let's dive in!

What Is Prompting?

Before we delve into the role-playing method, let's revisit the basics of "prompting" with ChatGPT. A "prompt" is an instruction you give to ChatGPT to get it to generate a specific response.

For instance, you might use a direct prompt, such as "summarize the following text," followed by a few lengthy paragraphs you want ChatGPT to review. ChatGPT would then offer a concise overview, transforming your bulky text into an easily digestible snippet.

What Is Role-Playing Prompting?

Role-playing prompting is taking the art of question-asking one step further. You're not just requesting information; you're setting a scene and having ChatGPT adopt a role. Imagine writing, "ChatGPT, pretend you're an experienced copyeditor, and I'm a new copyeditor. What should I focus on during my first pass of a novel?"

Why Is Role-Playing Effective?

Contextual Clarity = Focused Feedback: Role-playing prompts provide a layer of context that generic and broad prompts can't. ChatGPT understands the scenario better and responds with more specific, targeted advice. You're essentially customizing the AI's expertise to suit your needs.

Enhanced Creativity: Stepping into different roles stimulates creativity and encourages you (and ChatGPT) to think about a problem from multiple perspectives.

RSVP: AI for Editors Course

Role-Playing Prompt Format

The formula for a role-playing prompt is super simple and can be broken down into two main pieces:

[Role] + [Direction]

Role: This is where you tell ChatGPT what role to take on. This sets the stage and lets the AI know what persona it should adopt.

Direction: After you've set the role, you'll want to give a task related to that role.

Format Examples

• Pretend you're a [role]. Provide your top advice on [topic].

• Act like you're a [role]. Explain how you would handle [situation].

• You are a [role]. Do this [task].

• Role-play as a [role]. Assess the following [text].

Role-Playing as Editor, Author, and Reader

Editors may find it most helpful to instruct ChatGPT to take on the personas of "editor," "author," and "reader." Here are starting ideas for using each role.

Role-Play the Editor

If you're dealing with a text that needs better organization, clarity, consistency, or technical refinement, the editor role is your best bet.

Examples

Prompt: Act like you're an editor. Explain how you would improve this paragraph for middle-grade readers.

ChatGPT's Answer: As an editor, I'd recommend breaking up the long sentences to improve readability and adding a topic sentence for clarity.

Prompt: You are an editor. Assess whether this argument is well-supported.

ChatGPT's Answer: The argument could be strengthened by adding more data or expert quotes.

Role-Playing the Author

With the author role, you can get more creative advice. This can be especially useful for author coaches and developmental editors. If you're helping an author flesh out an first draft, role-playing as the author can offer insights into how to build the narrative or argument effectively.

Examples

Prompt: Role-play as an author. Tell me what's a good plot twist here.

ChatGPT's Answer: How about revealing that the protagonist's closest ally is actually a double agent?

Prompt: Pretend you are an author. Tell me how I can make this dialogue more realistic.

ChatGPT's Answer: Try using contractions and colloquial language to make the dialogue sound more natural.

Prompt: You are a first-time author who needs gentle and encouraging advice. Write suggestions about the following text in the style and tone you'd want to receive them. Here is the text:

ChatGPT's Answer: First of all, congratulations on taking the brave step of putting your words on paper! Writing is a journey, and you've already taken the first important step. Now, let's take a look at your text together.

Role-Playing the Reader

If you want to understand how a text might be received by its intended audience, use the reader persona. You'll get insights into the text's readability and engagement factors, helping you make reasoned editorial decisions.

Prompt: Act as a new employee. Explain how clear and engaging this edition of our company newsletter is.

ChatGPT's Answer: The newsletter's writing clearly explained upcoming events and updates, but the layout could be more interesting. Consider adding visuals or quotes to break up the text and capture the reader's attention.

Prompt: As a reader, assess whether the lesson was effective.

ChatGPT's Answer: The lesson had a lot of facts. Using anecdotes and examples would help information retention.

The role-playing method with ChatGPT is like having a Swiss Army Knife for editing. It's versatile, it's fun, and it can make your work so much better. So the next time you're stuck on a tricky paragraph and are looking for answers or inspiration, remember—you've got roles to play!

Register for the Course

Erin Servais helps editors upskill through AI. Her AI for Editors course is known worldwide as the #1 AI course for editors of all types, including medical editors, finance editors, education editors, corporate communications editors, and book editors.

Erin serves on the board of directors for ACES: The Society for Editing and has presented about editing, entrepreneurship, and artificial intelligence for the Professional Editors Network, Editors Canada, the Northwest Editors Guild, the Editorial Freelancers Association, and ACES.

Erin collaborated with artificial intelligence to write this article.

Email Erin: Erin@aiforeditors.com

How to Write and Refine ChatGPT Prompts to Get Best Results

Using ChatGPT centers on the act of writing prompts, which are directions you give ChatGPT to guide its responses. The higher the quality of prompts you feed ChatGPT, the better the work it can do for you.

Editors who want to use ChatGPT to speed up and simplify their work may begin by simply asking ChatGPT to “edit this.” This broad prompt will give you a result of edited text, but the result may have more and different changes than you seek. Instead, giving the AI more directions will allow you to fine-tune the response to be more closely aligned with your goal.

Keep in mind that even with a well-phrased prompt, you will likely need to give ChatGPT follow-up prompts to help it dial in on your specific needs. This will become a routine part of your process.

ChatGPT Prompt-Writing Basics

There are four core best practices for writing an effective ChatGPT prompt. It all comes down to being clear and specific.

1. Use Statements

Format instructions as statements instead of questions. This helps ChatGPT determine what you want it to do.

❌ "Can you write a paragraph explaining first-person point of view?"

🟢 "Write a paragraph explaining first-person point of view."

2. Be Specific

Make your requests detailed. Tell ChatGPT its exact next steps.

❌ "Edit this."

🟢 "Edit this corporate report to improve clarity and consistency and to correct errors of grammar, spelling, and punctuation."

3. Define Format

Specify format requirements in your prompt.

❌ "Write a blog post about the history of singular they."

🟢 "Write a 1,500–word blog post about the history of singular they. Use subheadings and bullet points. Use The Chicago Manual of Style."

4. Provide Context

Include relevant background and nuance. This helps ChatGPT narrow its response.

❌ "Is this a good title?"

🟢 "Considering this is a thriller novel set in Victorian England, is The Shadow's Grasp a fitting title?"

Refining Initial Responses

Even with a well-crafted prompt, the first output from ChatGPT often won’t meet your exact needs. The skillful user engages ChatGPT in a back-and-forth to tweak the initial response.

Here are examples of how to ask ChatGPT to refine its output:

1. Specify Changes

Clearly explain how you want the initial response to be changed.

• "Refine this response to use title case in headings instead of sentence case."

• "Change the examples to be relevant to podiatrists."

2. Identify Errors

Being a new technology, ChatGPT requires a bit of patience. It does not respond with accuracy every time. When using it to edit text, this can look like the program not catching every error. When using it to write, this can look like ChatGPT replying with information that is factually incorrect.

Rather than manually correcting errors ChatGPT misses (the whole point is to save you time and wear on your hands), explain the errors and instruct it to fix them.

• "Refine this response to ensure all numbers 1 through 9 are written as numerals."

• "Revise by placing the commas before closing quotation marks, not after."

3. Adjust Length

If you prompt ChatGPT to write text and the first response is too long or short, ask it to adjust the length.

• "Add a section about how about developmental editors can use ChatGPT."

• "Condense your response into one paragraph."

Try and Try Again

If your attempt to refine ChatGPT’s response still does not give you what you want, keep trying. Rephrase your prompt and try to make your directions more specific. Or open up a new chat and try the original prompt again. ChatGPT won’t answer in the exact same way twice, and sometimes starting with this clean slate is quicker than refining and refining a response that went off the path.

Want to learn more about ChatGPT?

Sign up for the ChatGPT for Editors course.

RSVP: ChatGPT for Editors

Erin Servais is an editor, educator, and community builder. She founded the Editors Tea Club, an online space where editors gather for education and support, and she offers editor coaching through her company, Dot and Dash.

In her fifteen years in publishing, she has helped to bring hundreds of titles to print and has presented about editing and entrepreneurship across the United States and Canada. Erin serves on the board of directors for ACES: The Society for Editing.

She always tells ChatGPT please and thank you, just in case.

Email Erin: Erin@aiforeditors.com