🚨Stunning document, leaked today supposedly from GOOGLE, on whether there is a "moat" (Barrier to entry) in the LLM space.
The author argues neither Google nor OpenAI have a moat, and open source wins. Thread with critical comment at end:
https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194403509149696
The question of whether one-three players dominate the industry (like Operating Systems or Search) or it is perfectly competitive is hugely important:
1) For consumer welfare: more competition is better.
2) For control of direction/ethics: more competition makes it harder.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194407003009024
To answer the question requires answering: what are the barriers to entry protecting the incumbents? What can their competitive advantages be?
If there are huge barriers, then the winner (Google in search) takes it all in AI, like Google did in search or MSFT in Office.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194409578307584
There was an excellent New Yorker article on Whisper (transcription software) which suggested there are no barriers. The model was fine tuned and ported to a phone app in no time. https://www.newyorker.com/tech/annals-of-technology/whispers-of-ais-modular-future
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194411960684544
The argument in the blog post is brutally simple and extends the above point:
1) No technology advantage: intellectual property is impossible to protect, as the knowledge is in the head of the researchers, who are leaving Google and spreading the knowledge around.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194414628270081
This is particularly the case after Meta's model *with the weights* was leaked.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194417488785415
2) No scale advantage (this is key) since finetuning a model with cheap hardware (through "LoRA") dominates training from scratch a gigantic model (which would be the advantage of Google or OpenAI).
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194420294774784
And you can use better data!!!
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194426623967232
3) The document finishes with a lot of mush and BS about controlling the narrative through open source. But there is no clear path here to monetization here!
Google has made money in open source Android because THE MOAT ON SEARCH WAS HUGE. No longer the case.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194433095786496
Plus you can customize the fine tune , since each user has a clear use case and can build on top of the leaked model.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194429958426624