• 0 Posts
  • 99 Comments
Joined 3 years ago
cake
Cake day: August 22nd, 2023

help-circle
  • FYI, whether fair or unfair, since you don’t mention having an error-catching mechanism, the rest of us suspect that Lumo is hallucinating things regularly, and you’re not catching the lies, because you lack familiarity with the original source material.

    I desperately hope that AI makes drastically fewer mistakes in some roles than others, because most people do not report the error rates that I see in my areas of expertise. So either the errors aren’t happening evenly everywhere (probably true) or some people are simply not catching the many mistakes (also probably true, though?).


















  • For simple math that could work, and as long as the question is close enough to an exact match with plenty of published examples to copy.

    A good rule of thumb is that the script it will come up with is about as likely to be correct as blindly taking the highest voted answer to the most similar question on Stack Overflow.

    If the question is simple and common, the odds are quite good. If the question is nuanced or rare, the odds of a correct result drop off aggressively.

    Edit: Your mileage may vary - providing an API to these LLMs that can do math correctly for them is pretty easy. Getting the LLM to consistently detect when to use that API is more challenging.