AI/ML vs. Traditional Software

Build Once, Sell Many Times

Software as a Service (SaaS) companies have come to dominate the technology space, and there is one metric that sums up just why: gross margin. Gross margin is the difference between the company’s revenue and its cost of goods sold. The higher the gross margin, the more of the revenue the company retains for each and every sale it makes. SaaS companies have very high gross margins - typically 50% on the low end and as much as 80% or more on the high end.

SaaS companies are not easy to build. Typically, the initial product development cycle is lengthy and expensive. This is why software startups raise as much money as they do; founders either (a) cannot fund the substantial product development costs out of pocket or (b) do not want to risk substantial sums of personal capital on a new venture by themselves. However, once the product is up and running and if it is well-received by the market (i.e., has good product-market fit), the cost of delivering an incremental copy of the software is very low. This is why software companies enjoy such great gross margins and have such high multipliers in their valuation.

SaaS companies typically build once and sell many times over; they have high initial fixed costs but low variable costs over time. This results in high gross margins as the businesses scale.

SaaS companies provide off-the-shelf products that deliver good return on investment (ROI) to their customers; their products do not require a lot of ongoing support. If you run a SaaS business and your product requires a lot of services to provide a good ROI to your customer, you are in trouble because the cost of delivering an incremental copy of software will NOT be low, and the services cost will eat into your gross margins. This is one reason most successful SaaS businesses do not have a services team in-house - if at all. Some SaaS businesses, of course, do rely on external partners to provide the services component to complement the core product. For example, Amazon relies on its AWS Partner Network to help AWS clients that need additional support.

A byproduct of low variable costs is that SaaS companies are able to scale very quickly. This is also why SaaS companies are more likely than other businesses to develop monopolies in their space. Just look at the top companies in the world today - many of them are SaaS businesses. SaaS companies operate in a world of winner-take-all. If a SaaS company has software that is better and cheaper than their competitors, they have the potential to take nearly all the business in their space. There is not much room for second-place winners.

AI/ML Companies Are Different

Unlike traditional SaaS companies, AI/ML and data companies have lower gross margins because the cost of delivering an incremental copy of an AI/ML application is NOT low.

I agree with Andreessen Horowitz; AI companies are different from traditional software companies.

First, AI/ML companies rely on data, and real world data is very dirty and varied. Even if the AI/ML application is in a very narrowly defined domain, data from one client may vary meaningfully from data from another client OR a single client’s data may change over time (i.e., it may exhibit data drift).

Dirty and varied data requires data scientists, data engineers, and data analysts to clean and prepare the data before the data is ready for use in the AI/ML application. Yes, part of this can be automated, especially using some of the AutoML software by companies such as Paxata, Tamr, DataRobot, Dataiku, and H2O.ai. But, not all of the data preparation work can be automated.

After data preparation, the companies will need to train their AI/ML model, and this will require lots of annotations. Since the model will need to be re-trained over time, the companies will need humans to re-annotate data as data drift occurs. Or, these companies may need to support multiple models simultaneously if data among their clients varies considerably.

Even once the data is prepared and the model has been trained, these companies will encounter edge cases that their model(s) will not be well-equipped to handle. At this point, these companies have a choice. They could choose not to solve these edge cases and instead surface them to the client as exceptions (e.g., create a fault tolerant UX). Or, they could surface these edge cases to humans they have on staff (i.e., the human in the loop) to solve.

Humans will be necessary for data preparation, model training and retraining, and edge case resolution. This means a non-negligible portion of the AI/ML company will require humans, resembling a quasi-services business, not a pure software business. In other words, the variable costs of an AI/ML company are higher than those for a traditional SaaS company, resulting in lower gross margins.

In addition to these human costs, AI/ML cloud infrastructure is much more expensive than the cloud infrastructure traditional software companies rely on. Training a computer vision model or a natural language processing model on large, complex data is an expensive endeavor and even model interference costs are not insubstantial. Yes, these costs are falling as AI/ML hardware infrastructure improves, but these costs are not falling fast enough, especially as the amount of data continues to explode.

Software + Services

AI/ML companies will require a combination of machines plus humans to solve many of the problems they are tackling. There is no way around it. That means software + services is the new working model, and startups will not be able to eschew the services component like traditional SaaS businesses have.

Fortunately, a rich ecosystem of partners has begun to emerge to support these AI/ML companies - partners to source data, to clean data, to annotate data, to train models, to deploy and maintain models in production, and to perform human in the loop services.

AI/ML companies have also matured quite remarkably over the past few years. Instead of trying to tackle highly complex, broadly scoped problems, they are now opting for low complexity, narrowly scoped problems that scale more similarly to traditional SaaS companies.

Trying to solve highly complex problems requires more of everything (more data, more data preparation, more annotations, more complex models, and more humans to deal with the wide variety of edge cases that will come up). Opting for low complexity problems results in higher gross margins, more closely resembling the gross margins of traditional SaaS companies.

That being said, if customers demand a full solution to their problems not just narrowly scoped software, AI/ML companies may have to run a hybrid software + services business to compete, even if that means lower gross margins.

My personal take: these hybrid businesses will win in the longer run, not because they will have higher gross margins (they won’t), but because they will be able to solve more broadly scoped problems, handling more of the customers’ pain points.