🚨Stunning document, leaked today supposedly from GOOGLE, on whether there is a "moat" (Barrier to entry) in the LLM space.
The author argues neither Google nor OpenAI have a moat, and open source wins. Thread with critical comment at end:
https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194403509149696
The question of whether one-three players dominate the industry (like Operating Systems or Search) or it is perfectly competitive is hugely important:
1) For consumer welfare: more competition is better.
2) For control of direction/ethics: more competition makes it harder.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194407003009024
To answer the question requires answering: what are the barriers to entry protecting the incumbents? What can their competitive advantages be?
If there are huge barriers, then the winner (Google in search) takes it all in AI, like Google did in search or MSFT in Office.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194409578307584
There was an excellent New Yorker article on Whisper (transcription software) which suggested there are no barriers. The model was fine tuned and ported to a phone app in no time. https://www.newyorker.com/tech/annals-of-technology/whispers-of-ais-modular-future
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194411960684544
This is particularly the case after Meta's model *with the weights* was leaked.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194417488785415
The cost of fine tuning an existing model with LoRA is $100
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194423797026816
Plus you can customize the fine tune , since each user has a clear use case and can build on top of the leaked model.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194429958426624
3) The document finishes with a lot of mush and BS about controlling the narrative through open source. But there is no clear path here to monetization here!
Google has made money in open source Android because THE MOAT ON SEARCH WAS HUGE. No longer the case.
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194433095786496
2) No scale advantage (this is key) since finetuning a model with cheap hardware (through "LoRA") dominates training from scratch a gigantic model (which would be the advantage of Google or OpenAI).
🐦🔗: https://n.respublicae.eu/lugaricano/status/1654194420294774784