Back to Course
Turn On Your Baloney Detector
30-40 minutesGetting comfortableBest for: People who use AI now and then and aren't sure when to trust what it says
What You'll Do
In class we talked about how AI predicts the "best" next word, and "best" doesn't always mean correct. In this task you'll ask AI about things you already know well, then use three follow-up questions to check its work. You'll leave knowing when an answer needs a second look and how to check it.
1Choose an AI tool
Any of these work, on a computer, tablet, or phone:
ChatGPT: https://chatgpt.com
Claude: https://claude.ai
Gemini: https://gemini.google.com
2Start a new chat and try the first prompt below
3Read the answer like a fact-checker
What do you already know is right? Does anything sound off?
4Ask the three follow-up questions (below) and see what changes
5Repeat with a topic you know well
That's where your baloney detector works best.
Prompt 1: Something Local
Prompt - Copy and paste this:
Tell me five facts about Flathead Lake, including how big it is, how deep it is, and one piece of its history.
What to notice:
- -Which facts do you already know are right?
- -Does anything sound too confident, or just a little off?
The Three Follow-Up Questions
After any answer, copy and paste these one at a time:
Follow-Up Question 1
Prompt - Copy and paste this:
How sure are you about that answer? Tell me which parts you're most and least confident about.
Follow-Up Question 2
Prompt - Copy and paste this:
What part of your answer might be wrong or out of date?
Follow-Up Question 3
Prompt - Copy and paste this:
How could I check this myself? Where would I look?
What to notice:
- -Did it admit to uncertainty it didn't mention the first time?
- -Did it change any of its facts when you pushed?
Prompt 2: Something You Know Well
Fill in the brackets, then copy and paste:
I know a lot about [your topic]. Explain [something specific about it] like you're talking to a beginner.
Examples
- •"I know a lot about knitting. Explain how to pick the right yarn for a baby blanket like you're talking to a beginner."
- •"I know a lot about gardening. Explain when to plant garlic in northwest Montana like you're talking to a beginner."
- •"I know a lot about trucks. Explain how to tell if my brakes need replacing like you're talking to a beginner."
Now read it as the expert. What did it get right? What did it miss or get wrong? Then try the three follow-up questions again.
Prompt 3 (Optional): Use a Photo
This one works great on a phone or tablet.
1. Take a photo of a plant, a tool, a part, or some yarn.
2. In your chat, tap the +, the paperclip, or the camera icon next to the message box and add the photo.
3. Copy and paste this with it:
Prompt for Your Photo
Prompt - Copy and paste this:
What is this? How sure are you? What else could it be?
What to notice:
- -Does it give you one answer, or a few possibilities? Does it tell you how to confirm?
What to Expect
- •AI usually sounds confident, even when it's wrong. That's how these tools work: they predict what an answer should sound like.
- •When you ask "How sure are you?", it often admits uncertainty it skipped the first time.
- •Some tools search the web and show links. Those are a good place to start checking, but links can be wrong too, or not say what the AI claims.
- •You might not catch any mistakes. That's a fine result, as long as you checked.
Tips
- •Pick topics where you're the expert. You'll spot mistakes faster than any fact-checking website could.
- •Ask the same question twice in a new chat. If the answer changes, that's a sign to double-check it.
- •AI is not Google. It isn't looking up a stored answer. It writes a new one each time.
- •Photo won't upload? Skip that step. Prompts 1 and 2 are the heart of this task.
Go Further
Ask the exact same question in a second chatbot (for example, Gemini and then ChatGPT). Do they agree? Where they disagree is usually where the baloney is.
General Tips for All Tasks
Saving Your Conversation (All Tasks):
- •If you're signed in to your account, your chats are saved automatically and show up in the history list (usually along the left side of the screen). You can come back to them any time.
- •ChatGPT: Your chat appears in the left sidebar. Click the three dots next to a chat to rename it or share it.
- •Claude: Your chat appears under your recent chats in the left sidebar. Click the chat's title at the top to rename it.
- •Gemini: Your chat appears in the left menu under recent chats. Click the three dots next to a chat to rename or pin it.
- •Used a Temporary or Incognito chat? Those aren't saved. Copy anything you want to keep before you close it.
- •Not signed in? The chat may disappear when you close the page. Copy what you want to keep and paste it into a document before you leave.
- •Remember: if you share a chat, anyone with the link can read the whole conversation.
Getting Unstuck:
- •If AI gives you an unhelpful response, try rephrasing your question
- •Add more details or context
- •Ask it to try again with specific changes
Technical Issues:
- •If the AI tool isn't loading, try refreshing the page or switching to a different tool
- •Make sure you're connected to wifi
- •Ask Connor for help! Working from home? Email hello@connormulvaney.com
Remember:
- •There's no "wrong" way to do this
- •Experimentation is encouraged
- •You can switch tasks if you want to try something different
- •AI makes mistakes. Keep your baloney detector on
What to Bring Next Week (Optional)
Next class is Monday, Oct 19, and Loga is running it while I'm out of town. There's no class Oct 26. I'll see you Nov 2.
- •Something AI got wrong, and how you caught it (Task 1)
- •Which model you'd use for what (Task 2)
- •Your plan and what step 2 looks like (Task 3)
- •Questions about AI that came up while you were experimenting