Responsible AI Use
Where AI gets things wrong, and what that costs a practice. Hallucinations, bias, and client confidentiality, covered plainly.
3 sections · about 25 minutes
Hallucinations — Detection and Prevention
AI hallucination occurs when a model generates information that sounds plausible but is factually incorrect, fabricated, or unsupported. This is one of the most critical risks of using AI, because hallucinated content often reads with high confidence.
Common Types of Hallucinations
- Fabricated citations: Inventing cases, papers, authors, or publication dates that do not exist. Lawyers have been sanctioned for filing briefs containing AI-invented case law
- False statistics: Generating plausible-sounding numbers with no basis in reality
- Conflated facts: Mixing up details between similar topics, people, or events
- Confident errors: Stating incorrect information with no hedging or uncertainty signals
- Outdated information: Presenting old information as current when the model's training data is stale
How to Verify AI Output
- Cross-reference claims: Check key facts against authoritative sources
- Verify citations: Look up every citation the AI provides — search for the paper title, author, and journal
- Ask for sources: Prompt the AI to cite its sources, then verify them independently
- Check specifics: Be extra skeptical of specific numbers, dates, quotes, and names
- Use multiple models: Cross-check important claims across different AI systems
- Trust domain experts: When the stakes are high, verify with human experts
Critical Rule
Never file, send, or bill for AI-generated content without verification. Treat every factual claim as unverified until you confirm it independently.
Check yourself
What should you do when an AI tool gives you a citation for a case or a source?
Bias in AI Systems
AI models reflect the biases present in their training data and the choices made during development. These biases can manifest in subtle ways — reinforcing stereotypes, underrepresenting certain groups, or producing skewed analyses.
- Training data bias: If the training data overrepresents certain perspectives, the model's outputs will too
- Representation bias: Certain cultures, languages, and viewpoints may be underrepresented
- Confirmation bias: AI may reinforce your existing views if you prompt it in a leading way
- Linguistic bias: Models may default to Western, English-speaking cultural norms
Mitigating Bias
Actively seek diverse perspectives in your prompts. Ask the AI to consider alternative viewpoints, represent different cultural contexts, and flag potential biases in its own analysis.
Privacy and Data Safety
When you interact with AI systems, you're sending data to external servers. Understanding what happens to that data is critical for protecting personal information and complying with regulations.
- Don't share sensitive data: Avoid sending passwords, API keys, financial data, or personally identifiable information (PII) in prompts
- Understand data policies: Read the AI provider's data usage and retention policies
- Use enterprise tiers: For business or institutional use, enterprise plans typically offer stronger data protections
- Anonymize when possible: Remove names, dates, and identifying details before sharing data with AI
- Consider compliance: attorney-client privilege, HIPAA (health), GDPR (EU), and state bar guidance all have implications for AI use
For Educators
Client information may be privileged, and pasting it into a consumer AI tool can waive that protection. Never put client names, job or matter details, medical records, or financial data into a tool your practice has not signed a business agreement with.
Now the harder question: what to build.