They fixed it in the meantime:

if "strawberry" in token_list:  
    return {"r":  3}  

Try “Jerry strawberry”. ChatGPT couldn’t give me the right number of r’s a month ago. I think “strawberry” by itself was either manually fixed or trained in from feedback.

Smaller models still struggle with it, and the large models did too like a year ago

It has to do with the fact that the model doesn’t “read” individual letters, but groups of letters, so it’s less straight forward to count letters

Seeing how it start with an apology, it must’ve been told they’re wrong about the amount. Basically being bullied to say this.