Already in very simple scenarios, AI is
repeatedly failing and humiliating
anyone who's dumb or trusting enough uh
to totally rely on it. Recently, a
prosecutor in Georgia had to explain to
the state supreme court why she relied
on AI to create an important filing
which was uh riddled with fake case
citations. Watch.
>> If there are no more questions.
>> So, before you sit down, there's one
more thing I need to ask you about.
Unfortunately, um, in reviewing the
trial court's order denying the motion
for new trial,
there are at least five citations to
cases that don't exist. And there's at
least five more citations to cases that
do not support the proposition for which
they're cited, including three
quotations that don't exist.
My understanding is that you prepared
the order for new tri uh the denial
order for the trial court. Were those
citations in the version of the order
that you submitted to the trial court?
>> No, your honor. I do not believe so.
They were not. I did prepare an order.
That order was revised.
>> So those those non-existent cases were
cited in your initial brief opposing the
motion for new trial.
Your honor, I'm not aware
of that, but I would be glad to research
that.
>> That's a prosecutor in that clip.
someone who's responsible for sending
people to prison for a long time and her
office is relying on fake cases created
by an AI out of thin air. You have to
wonder how often this kind of thing
happens without being a detective. The
big law firm Sullivan and Cromwell, one
of the most famous and highric law enfor
law firms in the world, was just caught
doing the same thing, using AI to
deceive the court. Quoting from the New
York Times, uh, an elite Wall Street
Journal, uh, rather elite Wall Street
law firm has apologized to a federal
judge for submitting a court filing
replete with errors created by
artificial intelligence, including
hallucinations that fabricated case
citations. The AI generated errors came
in a recent motion in US Bankruptcy
Court in Manhattan. The firm provided a
ledger of the errors which spanned three
pages and totaled around three dozen.
Number of them involve the citation of
seemingly imagined passages from real
cases. So again, it's happening all over
the country. This is uh in Minnesota.
Watch.
In a recent court filing in Henipin
County Court, attorney Frederick Kak
wrote that prior Minnesota cases support
his argument, citing a 1992 case called
State by Sunquist versus Provos. The
problem, that case doesn't exist.
Neither did the other case connect cited
right before it or another case cited
later. When Judge Lorie Miller caught
it, she wrote, "The court wonders if
this citation may be the result of an AI
generated hallucination.
>> It's something that created to give you
a convincing answer of what you asked
for, but it's not necessarily an
accurate answer."
>> David Larson is a law professor at
Mitchell Hamlin, teaching a course next
semester on artificial intelligence and
the law.
>> Is this happening everywhere?
>> I think it's fair to say it's happening
everywhere. Okay. Um, one thing I
believe is that the temptation to use AI
is so strong that people just can't
resist it.
>> According to a database tracking AI
hallucinations in court filings, there
are 134 cases across the country in
which an attorney cited fabricated case
law. Carol 11 News found one other case
in Minnesota in July when Judge
Christian Sandy caught attorney David
Lutz citing a phony case. Lutz
confessed, admitting he used AI and
forgot to cross reference the cases.
Now,
>> all these lawyers should be arrested and
disparred immediately, but that's not
happening. Uh, the Georgia DA got a slap
on the wrist, and as you just heard,
these Minnesota attorneys simply have to
pay a fine and maybe take a scolding
from the state bar.
It's enough to make you wonder when
we're going to get full-fledged AI
attorneys who make a complete mockery of
the courtroom and get every case wrong.
Um, the more Sam Alman tells us that AI
super intelligence has arrived, the more
tempted these attorneys will be.
Actually, in one case in New York, this
has already happened. An elderly man
representing himself deployed an AI to
speak to the court on his behalf and uh
it it didn't really go well. Watch.
>> May it please the court. I come here
today a humble pro before a panel of
five distinguished justices.
>> Is this Hold on. Is that council for the
case?
>> That I generated that.
>> I'm sorry.
>> I generated that. That That is not a
real person.
>> Okay. It would have been nice to know
that when you made your application.
>> You did not tell me that, sir. I
received the application and you have
appeared before this court and been able
to testify verbally.
In the past, you have gone to my clerk's
office and held verbal conversations
with our staff for over 30 minutes.
Okay? I don't appreciate
being misled.
So either you are suffering from an
ailment that prevents you from being
able to articulate
or you don't you are not going to use
this courtroom as a launch for your
business sir. So if you are able to to
shut that off
uh well the AI lawyer's having a good
time. You can tell that it's safe to say
the AI did not in fact please the court.
Uh and and yet on and on it goes across
every industry. Uh this is from the
Atlantic in an article published just
the other day. Quote, "Earlier this
week, the New York Times reported that
the future of truth, Steven Rosenbomb's
much discussed book about how AI shapes
reality, contains more than half a dozen
fake or misattributed quotes. Rosenbomb
pinned some of them on his use of AI.
and he claimed responsibility for the
errors said he was investigating what
went wrong by the time I spoke with him
on Thursday though he was pointing his
finger elsewhere chat GPT effed up the
book Rosenbalm said it's been a rough
week for human authorship all around on
Monday a viral post showed a Nobel
novelist seemingly admitting to using AI
to sharpen her story ideas before later
claiming she had been misunderstood on
Tuesday allegations mounted that the
Trinidadian author Jamir Nazir had used
AI to write the serpent in the grove
which won the Commonwealth Short Story
Prize. By Wednesday, two of the other
five uh uh five prize winners had come
under similar scrutiny.
Now, seeing all these cases, you have to
ask yourself, when exactly are we
getting these superhuman intelligence
from these AI chatbots?
They can't write novels or legal briefs
without failing in spectacular fashion
and destroying careers in the process.
As I've always said, the technology is
extremely impressive and very useful in
many ways.
But there's a clear ceiling to the
knowledge base of these AI chatbots.
There's only so many books and websites
you can rip off before the AI has to
start thinking for itself and before
people stop voluntarily training these
models for free. If every leading AI is
already failing in significant ways,
then how much improvement can we really
expect? Exactly. This time of year is
always awkward because nobody's wardrobe
really matches the weather anymore. You
leave the house in the morning and it's
sometimes still chilly. By the
afternoon, it feels like summer and then
somehow by night need another layer
again. What most people really need are
clothes that are versatile enough to
handle all of the weather changes. And
that's part of the appeal of Mac Weldon.
Mac Weldon makes clothes that are
comfortable enough for everyday life,
but still look sharp and put together.
Things you can wear while traveling,
working, going out to dinner, running
errands, whatever the day turns into.
They offer updated basics like stretch
twill uh chinos, button-downs,
pullovers, and uh pieces from the Ace
collection. The Ace collection is
especially built for travel and long
days when comfort matters, but you still
want to look reasonably presentable.
Lightweight materials, flexible fit,
classic styles, clothes that move easily
between different parts of your day, and
the overall style is simple in a good
way. Nothing flashy or overly trendy,
just comfortable, well-made clothes that
work. Get moving with Mac Weldon. Go to
macweldon.com and get 20% off your first
order of $125 or more with promo code
walsh. That's m a ck weld d.com code
walsh. Now, as I mentioned when we
discussed AI data centers a couple weeks
ago,
uh I'm not suggesting that AI has no
valuable uses or that it can't improve.
AI can obviously benefit the country. It
can launch businesses and perform
valuable services. It can give certain
kinds of useful feedback. It can even I
mean it could it could diagnose medical
conditions sometimes more reliably than
many human doctors who are also using AI
anyway. Um it it can save lives
potentially given the right
circumstances.
But if we keep pretending that AI has
humanlike capabilities it never actually
has. If we keep threatening to destroy
millions of jobs and prevent young
people from even starting a life of
their own, then before long no one will
be able to use AI for any reason. There
will be a revolt that forecloses the
possibility and probably does a lot of
damage in the process.