LLMs are bad at big numbers

[ergophobe]

I was trying to find the difference between 2^99 and 1.8^99. I thought, I'll let an AI do it.

First try, Claude.

According to Claude,
2^99 = 6.338 × 10^29
1.8^99 = 1.6 x 10^84

You'll notice that not only is 1.8^99 larger than 2^99, it is massively larger. It is 55 orders of magnitude larger.

So I asked why it thinks 1.8^99 is larger than 2^99. Claude explains it to me like I'm seven, explaining that it is correct, but didn't explain it well, then does some nonsensical math with logarithms.

[ergophobe]

Maxed out at four attachments.

So in the end, Claude asked me to teach it math, but I was out of credits on the free plan because I did NOT get my lifetime subscription to 1min.ai yet

So I switched to ChatGPT which also gave me an answer that is WAY off

According to ChatGPT, it was 6.33 x 10^17

That would be convenient since 2^99 = 6.34 x 10^25 and I don't care about the third significant digit, so it should make the math easy.

Except that the Windows calculator, the iPhone calculator and the Google search box and the Bing search box all tell me the right answer is 1.87 x 10^25

But for fun, I asked ChatGPT to notate 6.338 x 10^29 in terms of 10^17 rather than 10^29, the idea being that the numbers would then be directly comparable. ChatGPT helpfully told me that to express 6.338 x 10^29 in terms of 10^17, you would write it a 6.338 x 10^12 x 10^17. Not exactly what I was hoping for.

In any case, the bigger point is that both LLMs sem to do okay with integer math with large numbers, but floating point math is way way way off.