RT @ylecun: I have claimed that Auto-Regressive LLMs are exponentially diverging diffusion processes.
Here is the argument:
Let e be the probability that any generated token exits the tree of "correct" answers.
Then the probability that an answer of length n is correct is (1-e)^n
1/

🐦🔗: https://n.respublicae.eu/lugaricano/status/1640137670956396545