- Roko's Basilisk
- Posts
- Can Open Source Keep AI Honest?
Can Open Source Keep AI Honest?
Plus: Wrongful arrest by AI, OpenAI cuts, and Alexa’s new moods.
Here’s what’s on our plate today:
🧪 Hugging Face, open models, and who really steers AI.
📰 Facial-rec error, OpenAI cuts spending, Alexa gets personas.
🎯 Weekend To-Do: three hands-on tools to actually ship.
📊 Friday Poll: Is open-source AI a moat or a mirage?
Let’s dive in. No floaties needed…

Want to get the most out of ChatGPT?
ChatGPT is a superpower if you know how to use it correctly.
Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.
Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.
*This is sponsored content

The Laboratory
How Hugging Face is making the case for open-source AI
The very foundation of modern civilizations rest on the ability of individuals with specialized skills to come together to achieve stated goals. A mason may know how to build a wall; however, if they want to scale that wall into a durable structure, they need the assistance of a civil engineer who has spent their life developing the skills needed to bridge the gap between the architect’s vision and the mason’s skills.
In the tech space, bridging the gap between cutting-edge research and consumer-facing products is especially challenging given the rapid pace of technological change and the complexity of translating research into real-world use cases. Often, this complexity enables a handful of companies to dominate industries by being the few that can bridge the gap.
How technologies become industries
Look at the example of the internet. The idea and research behind the technology began in the 1960s; however, it would take decades before it could be packaged into consumer-facing products like computers and mobile phones, and a couple of decades more before it would change the way the world communicated.
When seen through this lens, artificial intelligence has already reached the second phase of its development. The earliest phase was when the technology was limited to the elite research labs and hidden behind corporate structures. Today, access to technology is funneled through a handful of enterprises that package it into products for mass consumption.
AI’s second phase
Companies like OpenAI, Anthropic, Meta, and Google dominate the AI landscape with their products and infrastructure. And if there were no counterweight, the AI industry would be limited by its ability and willingness to develop products beyond what makes economic sense for them.
In 2026, the counterweight to the multi-billion-dollar companies is being shaped by platforms like Hugging Face.
Enter Hugging Face
For many, the name Hugging Face may not ring a bell; however, for those who track hyperscaler investments, it will stand out. According to a Financial Times report, Hugging Face is among the few that declined NVIDIA’s $500M investment offer.
The platform did so to ensure that no single investor can sway the company’s decisions. Thereby ensuring it could preserve its identity as an open-source haven for developers.
Hugging Face is best understood as both a company and a vast open-source community. Together, they develop tools, machine learning models, and infrastructure that make it easier to build and work with AI, particularly in data science, machine learning, and natural language processing.
The company is widely known for its Transformers library and for a platform where researchers and developers can openly share models and datasets.
Valued at $4.5B in 2023, the platform provides access to open-source AI tools and is reportedly used by over 13M developers to build AI applications.
Hugging Face is designed with developers in mind, and its value lies in how it makes it easier for everyone to work with AI. The platform provides tools and shared space for researchers, developers, and companies to create, exchange, and run AI systems.
Rather than being a traditional technology company, Hugging Face operates more like a central hub for the AI ecosystem. It works alongside the basic infrastructure people build on: a marketplace for sharing models and datasets, a public library of AI knowledge, and a community space for collaboration.
Beyond serving as a counterweight to companies using closed-source AI models, the platform also participates in conversations on AI safety and regulation, given its role as a cornerstone of AI development.
From chatbots to AI infrastructure
Started in 2016 by three Frenchmen, Clément Delangue, Julien Chaumond, and Thomas Wolf, as a chatbot app aimed at teenagers, the company rose to prominence when it open-sourced its Transformers library. This framework made cutting-edge natural language processing models accessible through a standardized interface.
At the time, Google had just released BERT, and the research community was hungry for tools to experiment with transformer architectures. By packaging these models in a developer-friendly library, Hugging Face began its transformation from a consumer app into the connective tissue of the AI research world.
By 2020, the Hugging Face Hub had launched as a model-sharing platform conceptually modeled on GitHub. The comparison to GitHub is not just a convenient shorthand. Like GitHub before it, Hugging Face bet that the real value in a technology ecosystem lies not in owning the code, but in hosting the collaboration around it.
The business model also follows GitHub’s playbook: while the platform is free and open-source, revenue comes from enterprise subscriptions (starting at $50 per user per month), inference endpoints, and consulting contracts with companies such as NVIDIA, Amazon, and Microsoft.
However, unlike GitHub, Hugging Face integrates with every major cloud and chip provider: AWS, Azure, Google Cloud, NVIDIA, AMD, Intel, and Qualcomm. This is precisely why the NVIDIA rejection matters. If Hugging Face had taken the $500M, it would risk becoming an extension of a single hardware ecosystem, potentially alienating developers who depend on the platform remaining agnostic.
This unique position of neutrality enjoyed by Hugging Face has made it a venue where the geopolitical contest over AI plays out publicly.
In the words of Clem Delangue, co-founder and chief executive of Hugging Face, open models “contribute to democratising AI, to fighting concentration,” which, in their opinion, is the biggest risk in AI.
Hugging Face hosts access to models across the spectrum, including Meta’s Llama models, which have surpassed 1B downloads, and China’s DeepSeek-R1, released under the MIT License in January 2025, which claims performance on par with OpenAI’s o1 at a fraction of the training cost. Then there are European models from companies like Mistral valued at €5.8B. This makes the platform a platform for AI labs to share their models and see how developers will use them.
The platform also presents an interesting area within the regulatory framework. Since it works like GitHub for AI developers, the platform has become central to conversations around open-sourcing AI models.
The open-source tension
Yet Hugging Face’s vision for open AI does not exist without friction. The same openness that Delangue champions is also the source of growing unease among policymakers and security researchers. For every argument in favor of open-source AI, including transparency, accessibility, and collective innovation, there is a counterargument rooted in tangible risk.
Regulators have repeatedly warned that widely distributing model weights could lower the barrier for misuse. The concern is straightforward. Once weights are public, technical safeguards can be removed, modified, or entirely bypassed. The NTIA’s examination of dual-use foundation models underscored this dilemma, cautioning that unrestricted access could enable harms ranging from synthetic biological threats to scalable misinformation, while leaving few practical accountability mechanisms in place.
These anxieties have only intensified as open models grow more capable. Independent evaluations of recently released systems have suggested that safety controls are often brittle, particularly under adversarial prompting. Security researchers have also demonstrated how generative models, especially those that can be freely modified, may be repurposed for malicious automation, including malware development. In an ecosystem defined by openness, guardrails are not always permanent features. They can be stripped away.
Delangue, however, frames the debate differently. In his view, open science and open-source development act as a counterweight rather than a vulnerability. Openness, he argues, prevents the emergence of inscrutable black-box systems, distributes power more evenly across the industry, and equips civil society with the tools needed to scrutinise and challenge dominant AI actors. From this perspective, the concentration of advanced AI capabilities behind closed corporate boundaries presents a greater systemic risk than the potential misuse of open models.
Whether that reasoning will hold as models become more powerful remains a defining question for AI governance. For now, neither regulators nor markets appear inclined to halt the open-source momentum. Instead, the emerging consensus favours guardrails over prohibitions. This approach allows the open AI experiment and platforms like Hugging Face to continue shaping how intelligence is built, shared, and controlled.
Hugging Face’s structural role
Hugging Face today is the bridge between researchers pushing the boundaries of what was thought possible in AI and the public-facing form of these advancements. The platform represents more than a community of developers; it has become an important contributor to the story of AI, and whether the building blocks of the future will be controlled by a handful of companies or by the masses, who can harness the power of LLMs to fulfil their needs.
TL;DR
Hugging Face is positioning open source as a counterweight to closed AI giants.
It evolved from a teen chatbot into core infrastructure for models and datasets.
Staying investor-neutral, including rejecting Nvidia's money, keeps it cloud-agnostic.
Openness boosts innovation while intensifying regulatory fights over safety and misuse.


Friday Poll
🗳️ For your own AI roadmap, where do you want to lean harder in 2026? |

The AI Talent Bottleneck Ends Here
If you're building applied AI, the hard part is rarely the first prototype. You need engineers who can design and deploy models that hold up in production, then keep improving them once they're live.
Deep learning and LLM expertise
Production deployment experience
40–60% cost savings
This is the kind of talent you get with Athyna Intelligence—vetted LATAM PhDs and Masters working in U.S.-aligned time zones.
*This is sponsored content

Weekend To-Do
Hugging Face Hub: Browse and fork one open model, then test it in the hosted playground.
Ollama: Install it and run a local LLM or coder model on your own machine.
Replicate: Pick a community model, deploy it behind their API, and hit it from a simple script.
Headlines You Actually Need
Facial recognition misfire: UK police wrongly arrest an Asian software engineer 100 miles away after facial recognition flags him as a burglary suspect, sharpening bias concerns.
OpenAI trims ambitions: OpenAI reportedly scales back its aggressive chip and data center spending plan, a sign of tighter economics for frontier AI.
Alexa gets moods: Amazon launches Alexa Plus with new personality styles and a paid tier, nudging voice assistants further into subscription business models.
The Toolkit
AssemblyAI: API-first platform for speech recognition and audio intelligence. Try it to auto-transcribe calls and pull out topics, sentiment, and speakers from long-form audio, so your team is not stuck taking notes.
Chroma: Open source vector database built for AI apps. Use it to store and query embeddings so your agents or RAG systems can actually remember context instead of re-reading everything on every call.
Continue: Free, open source AI copilot that runs inside VS Code and JetBrains. Use it to get inline code suggestions, refactors, and local RAG on your own repos without handing everything to a cloud IDE.

Rate This Edition
What did you think of today's email? |





