"I’m not going to publish the results of this query here – but I did ask NotebookLM to calculate the average and median attainment numbers for the different teams, and it did not calculate them correctly. Neither did Claude or ChatGPT’s Advanced Data Analysis. These are large LANGUAGE models – not large MATH models. ChatGPT’s ADA can do incredible analysis if you upload a spreadsheet, and Claude can also do incredible calculations in certain situations (as can Bard) – but if you are extracting numerical data from a long text-based document – I strongly urge you to be careful because you will likely get or very questionable answers."
I see so many folks DUNKING on the AI systems, when in reality it was the:
- prompt
- context set-up
- fact that LLMs are calculators for words NOT for numbers (as you said)
Just subscribed look forward to catching up with the back catalog in the coming weeks.
Steve really enjoyed this!
In particular:
"I’m not going to publish the results of this query here – but I did ask NotebookLM to calculate the average and median attainment numbers for the different teams, and it did not calculate them correctly. Neither did Claude or ChatGPT’s Advanced Data Analysis. These are large LANGUAGE models – not large MATH models. ChatGPT’s ADA can do incredible analysis if you upload a spreadsheet, and Claude can also do incredible calculations in certain situations (as can Bard) – but if you are extracting numerical data from a long text-based document – I strongly urge you to be careful because you will likely get or very questionable answers."
I see so many folks DUNKING on the AI systems, when in reality it was the:
- prompt
- context set-up
- fact that LLMs are calculators for words NOT for numbers (as you said)
Just subscribed look forward to catching up with the back catalog in the coming weeks.