I’m not sure where I stand yet, but I was wondering the
other day what AI itself thinks about using AI for professional writing,
editing and research. Like any good journalist, I reached out to multiple sources
for comment. Here’s what I learned from the major AI platforms I consulted that
use large language models (LLMs).
ChatGPT and Perplexity kind of hemmed and hawed. Eventually
I got tired of waiting for them to formulate a response while “still thinking
and processing.” Gemini, sort of hid behind the Google Chrome search results. But
Claude to its credit answered my questions swiftly and confidently, but not
before telling me: “Hank, the irony of your request has not been lost on
me.”
According to Claude: “AI writing tools have swept into
newsrooms, law firms, academic institutions, and corporate communications
departments with remarkable speed. The pitch is seductive: faster drafts, fewer
bottlenecks, lower costs.” No surprise there, but then Claude confided: “For
professional writing — where accuracy, credibility, and sourcing are
non-negotiable — AI tools introduce a set of risks that should give any serious
writer or editor pause.”
Hmmm.
Unprompted by me, Claude then addressed the tendency of AI to come up with
make-believe-sources.
The Hallucination Problem
“The most damaging flaw of LLMs in professional writing
contexts is what researchers and engineers call ‘hallucination,’” asserted
Claude. “It’s the tendency of AI systems to generate false information with
complete confidence. This isn't a bug that will be patched in the next update.
It is, to a significant degree, structural,” said Claude.
If you’re in a situation or profession that expects you to document the sources
for your work, which should give you pause.
As Claude explained during our interview, “language models work by predicting
statistically likely sequences of text based on training data. They do not ‘know’
facts the way a database stores records. When asked to produce a citation — a
journal article, a court case, a news report — the model generates text that
looks like a citation. It follows the
correct format, includes plausible author names, and references a real-sounding
journal. But the underlying article may not exist at all,” Claude noted.
Personally, I’ve run into the source fabrication problem often and so have many
of our clients.
When I asked Claude for an example of AI hallucination, it
shared the following story: “This phenomenon gained widespread public attention
in 2023 when a New York attorney, Steven Schwartz, submitted a legal brief that
cited multiple court cases generated by ChatGPT — none of which were real. The
judge was not amused,” noted Claude. “Schwartz faced sanctions, and the case
became a widely cited cautionary tale about the dangers of using generative AI
in professional legal contexts,” Claude added.
Why Citations Are Especially Vulnerable
According to Claude, “citation requires something AI
fundamentally lacks: a reliable, verifiable link between a claim and a
specific, retrievable source.” A human researcher, Claude said, “finds a
source, reads it, and records it. The citation is downstream of an actual act
of retrieval and comprehension. By contrast, AI generates a citation because
citations are a pattern that appears in its training data. [AI] has
learned that academic sentences tend to be followed by parenthetical
author-year references, that legal briefs cite specific case names and dockets,
and that journalism includes attributed quotes. So, it produces those patterns
— whether or not the underlying source exists.”
To me, it’s amazing the human programmers have figured out a way to mimic
academic, legal and journalistic writing style, but you can see how dangerous
it can be when mis-used.
As Claude continued: “Even worse Hank, when AI tools do
have access to real sources (through retrieval-augmented generation or web
search), they can still misrepresent them — paraphrasing inaccurately,
attributing quotes to the wrong person, or presenting a source's argument as
the opposite of what it actually says.”
Personally, I’ve frequently been frustrated by inaccurate paraphrasing and misattribution
of quotes, and it’s something you should double check before you go live with
any material AI has generated for you.
The Problem of Confident Vagueness
If you’re still reading this post, or asking your AI
assistant to do so, I thank you for your time.
As Claude noted: “Professional writing demands precision. A medical writer
cannot say a drug ‘significantly reduces symptoms’ without citing a specific
study with a specific patient population and outcome measure,”
Claude said. “An investigative journalist cannot attribute a claim to ‘experts’
without naming them. AI prose tends toward what might be called confident
vagueness — it sounds authoritative while eliding the specific details that
give professional writing its credibility and accountability,” Claude noted.
Agreed. We’re constantly calling out our clients on that.
“This is partly why AI-generated text often passes a casual read but falls
apart under editorial scrutiny,” admitted Claude. “The sentences are
grammatical, the tone is appropriate, the structure is sound — but when a
fact-checker goes to verify the third paragraph, the sources either don't
exist or don't say what the text claims.”
In my experience, this is sadly all too accurate.
Accountability
“Beyond sourcing,” continued Claude, “professional writing
carries the voice, judgment, and accountability of a named author. A byline is
not merely a formality — it represents a human being who can be questioned, who
has a reputation at stake, and who made deliberate choices about what to include
and what to leave out. AI-generated content erodes that accountability layer
entirely,” asserted Claude.
“There is also the question of what gets left out. AI
models trained on existing text will reproduce the emphases, omissions, and
blind spots of that text,” said Claude. “A professional writer working a beat
develops the judgment to know what the data doesn't show, who hasn't been heard
from, and what a source has an interest in saying. AI has no such judgment — it
optimizes for plausibility, not truth,” stated Claude.
A Tool, But Not a Replacement for Professional Judgment
“None of this means AI has no place in a writer's workflow,”
asserted Claude. “Brainstorming, structural outlining, rough drafting, and
editing assistance are areas where the stakes of a fabricated citation are low
and the efficiency gains are real. But for any writing where sourcing,
accuracy, and accountability matter — journalism, legal writing, academic
research, medical communication — AI should be treated as a starting point at
best, and a liability at worst.”
Even Claude concedes: “The professional writer's value lies
precisely in what AI cannot reliably do: find real sources, read them
carefully, represent them accurately, and stake their name on the result.”
More tools and resources related to this post can be found
on our website.
Conclusion
AI is great for getting started, especially when you’re just staring at a blank
screen and can’t get out of first gear. It can be a great help when it comes to
brainstorming, structural outlining, rough drafting, and editing assistance.
But turning to your work over to AI blindly is not just lazy; it can
irreparably damage your reputation and your firm’s.
Posts
