Here’s what’s on our plate today:

• 🧪 Who really manages the data behind AI platforms?

• 🧵 SoftBank tumbles, Grok idolizes Ohtani, Google’s Banana Pro leaks.

• 📌 How to vet your data infrastructure before trusting it with AI.

• 🗳️ Should data platforms be allowed to run AI agents autonomously?

Let’s dive in. No floaties needed…

The Laboratory

Who really manages the data behind the AI?

Every major shift in civilization is built on the backs of countless nameless and faceless entities that nurture and enable progress. Take the example of the Renaissance, while today many understand it through sculptures, paintings, and revolutionary works in different fields of thinking.

The Renaissance was largely the outcome of the fall of Constantinople to the Ottomans in 1453. As a result of the fall, Greek scholars fled to Italy with long-lost classical texts and knowledge, sparking a renewed interest in ancient learning.

Sure, there were many other aspects that contributed to the flourishing of arts and the ideas of humanism, but without the building blocks of a resurgence of classical knowledge, it would not have been possible.

Today, many overlook the role of classical knowledge in the establishment of the modern world, akin to how many overlook the role of data in the emergence of modern AI systems.

A close look at the current valuations of companies that are part of the AI infrastructure boom reflects two simple truths. Investors have placed their bets on companies that build the computing power for AI systems, like Nvidia, SK Hynix, AMD, and companies that develop and train AI models like OpenAI, Google, and Anthropic.

However, these valuations often overlook the importance and the role of companies like Databricks, Snowflake, and others that provide unified environments to consolidate data for data engineering, data science, and AI development workflows.

Why data platforms are surging

A recent report from Reuters shared that Databricks is in talks to raise funds at a valuation of more than $130 billion. This is reported to be around 30% higher than its last financing round two months ago.

Just months ago, in September, the company reportedly closed a $1-billion-funding round that valued it at $100 billion, making it one of the world's most valuable private companies.

The company claims it is on its way to hit $4 billion in annualized revenue, which it said would be used to accelerate its AI strategy, expand products, launch an operational database category, and pursue AI acquisitions and research.

It is important to note that Databricks was founded in 2013 and has grown at an exponential rate since the AI boom.

So what exactly is it that data companies like Databricks bring to the table?

What these platforms do

Databricks describes itself as a data intelligence platform, which democratizes insights to everyone in an organization.

The company provides a single system that brings all of an organization’s data and governance together and pairs it with AI models tailored to that organization.

This lets anyone use natural language and automation to find and use data easily, while technical teams can quickly build and deploy secure data and AI tools.

Essentially, what companies like Databricks and Snowflake do is to provide organizations with a place to store, manage, and use their data and to build and run AI systems. Instead of relying on many separate tools, businesses can do everything in a single platform that handles data processing, analytics, and AI development from start to finish.

The success and importance of their business models can be understood by the scale of Snowflake’s IPO in 2020, which raised $3.36 billion after it was priced above the target range in the biggest U.S. listing for that year.

The lakehouse advantage

Modern data and AI platforms became successful because companies were struggling with too many disconnected tools for storing, processing, and analyzing data.

By the mid-2010s, most organizations were juggling separate systems for data warehouses, data lakes, analytics, and machine learning, which created silos and made operations hard to manage.

Databricks helped solve this by creating the lakehouse model that unifies these functions in one place, while Snowflake grew quickly with a cloud data warehouse that scales easily.

Other companies like Palantir, Scale AI, and tools from AWS, Google, and Microsoft expanded the space as demand for AI-ready infrastructure increased. By 2024, most enterprises were moving from experimenting with AI to using it in real business operations, and a large share of their budgets shifted toward platforms that could support AI at scale.

What primed them for this position was the idea of a lakehouse. This is essentially a data system that combines the flexibility of data lakes with the speed and structure of data warehouses.

Many companies spend a large amount of their data engineering time just moving information from one system to another, and lakehouses fix this by keeping all data in open formats in cloud storage while adding the tools needed for quality, security, and fast queries.

This is important because modern AI models need huge amounts of different types of data and work best when everything is stored together and kept up to date. Databricks says customers often cut their infrastructure costs in half when they consolidate their workloads, although the savings depend on the situation.

However, while their business models appear sound on paper, in reality, there is more to managing enterprise data.

Where data platforms struggle

Despite massive investment, AI platform adoption faces friction.

A 2024 Gartner survey found that just 54% of AI projects make it from pilot to production. Often, the biggest challenge they encounter is data quality issues, skill gaps, and unclear ROI.

Enterprises also face challenges in ensuring models don't train on sensitive or biased data. Add to this regulatory pressure of providing explainability and auditability, the provision of the EU’s AI Act, and the work of data companies becomes more challenging.

Pricing is another pain point. Snowflake users often complain of high billings way beyond expectations due to under-optimized queries, oversized warehouses, and poor visibility into usage.

Another major challenge is how to ensure that costs can be explained and limited to a level that does not push enterprise clients to build their own standards for data warehousing.

A splintering market

Currently, the market is splitting into three groups. Databricks and Snowflake want to be all-in-one platforms. Specialist tools like Weights and Biases, Pinecone, and LangChain believe focused products work better than large platforms.

Meanwhile, cloud giants such as AWS, Google Cloud, and Azure use their reach to push their own AI services, although customers worry about being tied to one cloud.

Some companies like Uber, Netflix, and Airbnb question the need for external AI platforms because they can build their own systems that better fit their performance and integration needs.

For them, the reasoning is simple: why rely on an external source when you can fine-tune your AI offerings and clean internal data?

Whatever the end decision, it impacts data analysis firms.

What comes next

According to Deloitte, 55% of organizations surveyed for its Tech Trends 2024 report said they avoided certain AI use cases due to data-related issues, and an equal proportion are working to enhance their data security.

Data companies can fix this gap and lower the cost for enterprises to build and manage their AI workflows.

Around the same time, new opportunities keep presenting themself. The need for AI observability (monitoring model behavior), AI security (detecting adversarial inputs), and synthetic data generation.

Another shift is the rise of AI agents, which can take actions rather than simply answer questions.

Databricks and Salesforce are already building agent frameworks. Supporting agents require new tools such as workflow managers, integrations, oversight controls, and long-term memory systems. If agents become the primary means by which people utilize AI, platforms will shift from focusing on model building to concentrating on agent deployment and management. All of which benefits data companies.

Data is AI’s fuel

If AI is the future, then data will be the fuel that powers it. Data companies then become reservoirs that keep the engine of neural networks running.

However, whether they will be the sculptures that future generations admire or be relegated to events like the fall of Constantinople, a precursor to something even bigger, remains to be seen.

Bite-Sized Brains

• Asia’s chip panic: SoftBank, SK Hynix, and Samsung stocks plunge after Nvidia’s sell-off sparks fears of a semiconductor downturn in Asia.

• Grok’s Ohtani obsession: Elon’s AI chatbot, Grok, says “Musk is better than basically everyone”, except Japanese baseball star Shohei Ohtani. Yes, really.

• Google’s mystery gadget: New device called Banana Pro spotted in FCC filings. It might be the next-gen Pixel Nano, or something even weirder.

Roko Pro Tip

Monday Poll

Meme Of The Day

Rate This Edition