Yeah whatever. I know its a been a minute, but I have been stuck for 5 hours on a Google OAuth problem and I have had several glasses of wine on a Friday night and its very late. So some musings for you since my tongue and fingers are feeling in conjunction atm.
Why does Meta want to open source their LLM?
The typical answer everyone in Tech seems to have shrugreed upon (I just made that up, that’s where everyone just shrugs and agrees) is that that it is analogous to what Google did in open sourcing Android OS.
Google’s main revenue has always come from search, but in order to maintain market share they need to be the default search option on everyone’s browser. But Apple comes standard with Safari and could tell Google to f right off, and create their own search tool. So Google open sourced the complementary software, being the browser, and OS, so that phone developers could use Chrome and Android as the default on their hardware, for free. Paid off big time, as the majority of the global phone market uses Android OS, and thus Google defaulted itself into the default internet search tool which is why we all use the word Google synonymously with Search.
Meta’s main revenue is advertising, and it probably always will be. Apple will eventually outcompete them in VR hardware, and everything everywhere is driven by marketing spend, and things are only becoming more digital, not less, over time. Network effects + Lindy effects compound = why would you ever pivot into another main source of cash? You wouldn’t.
So is Meta saying that the complementary piece of software to their ad business is an LLM? And that by commoditizing LLMs they will hold other big tech companies at arms length from their precious ad revenue? That’s what everyone is collectively shrugreeing upon.
Let’s examine that closer. But first, a product idea I thought of that made me think “this is very meta” (lowercase m), and thus brought the idea full circle. I am straight up musing all of this here so bear with me.
I’m stuck on a coding problem. The entire world has converged on a React / Next JS / Typescript / Tailwind / Supabase stack, and GPT4 doesn’t know App Router is. Hint: it’s a new app structure that was released mid-23 in Next JS 14, but GP4’s training data only goes to early-23. So for GPT4 to be helpful with my very current and relevant problem, it needs me to feed it a bunch of context, mainly pages of documentation straight from Next JS. So not cool. Release GPT5!
Claude Opus is the same way. Gemini, etc. Each training data set only goes up to <(TODAY) date, upon which they know nothing of the world we live in.
What if there was a Google for LLMs? Where you could query a problem, it would ping all LLMs in existence that had APIs, and index their respective training data sets. Perplexity kinda does this, but not really. It would be ridiculously expensive to index all LLMs. The only limitations are time, money, and electricity.
Anyways, this LLM to index all LLMs idea struck me as very self-referential, ie meta. I selfishly want it to solve coding problems, but think of the value creation if you could force LLMs (the way Google forced websites) to conform to a structure to make them more easily index-able (which would have huge copyright/attribution benefits). Open source would certainly comply, and big tech LLMs would probably not want to at first, until the collection of OSS LLMs became more valuable than the big tech foundational LLMs. Maybe I will take a stab at that once I figure out this OAuth issue.
Is this what Meta is thinking? Prob not quite, but on the same tangent and order of magnitude I would suppose.
What else do we know right now? Zuck likes open source, Zuck likes decentralization (super weird for him, I know), Zuck named his kids after Roman emperors, Zuck is going to give away all his money when he dies to charity, Zuck is playing a long ass game and he loves to steal and he loves to win. He is prob pro-transhumanist, very pro-VR, and 100% has a little bit of an ego. My point is that he is not skating where the puck is going, he is building a new rink with a new game and building a new puck. Or rather, you’ll all collectively do that for him, and he will scoop it up as it comes along.
What do I mean here?
Meta is the world’s most valuable resource because it connects all the people. (Remember Meta owns WhatsApp, that thing every single one of your non-US friends tries to annoyingly text you on). Obvi fb/ig too. Monetization comes from people consuming data, not computers. When you work with an LLM, you have to give it specific instructions, talk to it like a computer, you can’t just drop a pile of work and walk away. You have to be extremely articulate and explain to it what exactly what you mean. Meta has the most data on how people communicate. It’s not that everyone has to rush to learn how to talk to LLMs, the LLMs have to learn how to talk to us. And Meta is best positioned to focus in that language, and accept all the API calls. But it will take sooo much variance in language that will require hundreds of thousands of LLMs.
“Youu get an LLM, and youuu get an LLM!” - Oprah, (in Zuck’s fantasy).
Because, all of that personalization, isn’t so that my personal LLM can find the best sock on the internet. It’s so that my LLM can sell me the best sock on Meta. Meta doesn’t need to have an LLM, or plug into one to improve its ad recommendation AI, all it has to do is open its kimono and let the user plug in. Any type of user. Big companies trying to sell you shit. Little companies trying to sell you shit. Your grandma’s grandma’s digital replica trying to sell you shit. When the quantity of content creation goes up, so does ad revenue. So why even create, and then open source its own LLM?
“All the better to see you with, my dear, replied the Wolf”. - Little Red Riding Zuck.
Actually, its more about giving the ability for more users to create more content to push it back onto some Meta network. LLMs do a lot of stuff. Mainly they increase productivity. Increasing productivity increases company formation, content creation, baby formation (maybe). Zuck loves connecting people, the more nodes in the network, the more things zing. More LLMs = More zinging, you don’t really have to think too hard about it.
But does commoditizing LLMs really give Meta the advertising edge advantage? Not necessarily. One could make the case that close sourcing, and using their proprietary data on how humans communicate could give them a leg up on creating an LLM that would be better at selling us shit than the other big tech companies. Advertisers would rush to use Llama and sell on Meta, and scoff at the other platforms as they shift their marketing spends and watch their ROAS’s soar.
Which is why they are also on the side creating their own closed source one as well. Hedging their bets of course. Which is why I don’t think the collective shugreement is apt. It’s creating an analogy for analogies sake because there isn’t anything like this that has happened before. Nassim Taleb’s narrative fallacy at work.
Here are a couple other scenarios of why Meta would open source their LLM that refute the complementary software hypothesis:
Sheer egotism/scoreboarding: in order to remain big tech supreme, and not fall off mount Zuckmore into a paltry Fortune-500 resting place, we need to compete where the game is, which is compute, tokens, energy, inference. All derived from AI. For all we know, the future could just be us drilling for oil to burn to run the servers in some Mad Max scenario where the AI runs all our shit because we don’t know how to use screwdrivers anymore. Surely we can’t let Amazon’s Anthropic’s Claude Opus or Google’s Gemini, or Microsoft’s OpenAI’s GPT99 take all that sweet milk money in the future? Therefore since all the big boys are closing source, Meta opens source, and LLaMa simply does nothing except take a little wind out of its competitors for bragging rights.
Decentravision: the take rate of social media advertising is something like 98% when you add up TT/Meta/YT (paraphrasing Chris Dixon’s latest book, a prophetic must read). YT pays a creator fund of roughly 50%, TT tried to match that but probably has scaled back close to 0, Meta pays essentially 0. That means all of that content that gets created, that drives the eyeballs, and attention, and consumes your god damn lives, 98% of the ad revenue goes back to the corporations. If you haven’t heard, its an awful lot of money. Decentralization is coming for that big, fat juicy ad revenue, and Zuck knows it. He wants to be on the side of devs, and release a token, windsurf or whatever, get a cool haircut! and yadda yadda. But Bezo’s Law is always looming (your take rate is my opportunity), and is quite literally the foundational principle of decentralized finance. Why let the bank take 10 when I can build an open protocol that will do it for 5? Once social media becomes decentralized, and I truly believe it is only a matter of time, all of that ad revenue gets pushed to the edges of the network, going back to the creators. Who better to lead the charge than the man who took all the money to begin with! Even the Buddha gave up all his possessions to achieve enlightenment. Open source it all! Decentralize my own damn shit! (just make sure to wear this headset and use my META 0.00%↑ token to make all of your micropayments (wink).
In short, I don’t even think Zuck knows which direction he is going in yet. The Tech world always seems to get caught up in a convergence, which is good for software stacks, but bad for groupthink. Also, who cares! We’ll all be dead and Zuck’s robot body will be ruling our digital replica brains anyways!