I was staring at my AWS bill last Tuesday, trying to figure out how a simple image classification microservice managed to rack up $840 in three weeks. The answer was annoying but obvious. I was relying on a massive, expensive cloud endpoint for a job a local model could do practically for free.

We are seeing this exact problem play out across the entire tech industry right now. Huge, heavily funded AI products are quietly shutting down or scaling back. The math simply doesn’t work. You can’t spend millions on server compute when user downloads fall off a cliff after the initial novelty fades. The economics of running heavy generative models for everyday tasks are completely broken.

Which brings me back to the Fast.ai community.

Jeremy Howard and the Fast.ai crowd have been right about this for years. While everyone else was scrambling to hook their apps into the largest language and vision models available, Fast.ai kept teaching people how to train small, highly specific models on cheap hardware. Now that the venture capital subsidies for those big API calls are drying up, the Fast.ai approach looks less like a learning exercise and more like a survival strategy.

The Local Compute Reality Check

I decided to rip out the expensive third-party calls in my project. I spent maybe 45 minutes writing a script to handle the quality control image sorting we needed. I didn’t need a multi-billion parameter beast to tell me if a manufactured part had a scratch on it.

I just needed a basic convolutional neural network. So I set up a fresh environment with fastai 2.7.14 and PyTorch 2.3.0.

from fastai.vision.all import *

# Pointing to my local folder of good/defective part images
path = Path('./manufacturing_dataset')

# Setting up the data loaders with basic augmentation
dls = ImageDataLoaders.from_folder(
    path, 
    valid_pct=0.2, 
    item_tfms=Resize(224),
    batch_tfms=aug_transforms()
)

# Using a lightweight pre-trained model
learn = vision_learner(dls, resnet34, metrics=error_rate)

# Finding a good learning rate before training
learn.lr_find()

# Fine-tuning the top layers
learn.fine_tune(4, base_lr=2e-3)

# Exporting for production
learn.export('qc_model.pkl')

I ran this on my local workstation—an RTX 4090 with 64GB RAM. It chewed through the data instantly.

The Benchmark That Made Me Feel Stupid

Here is the specific breakdown that made me regret not doing this months ago.

I ran our standard 12,500 image dataset through the commercial API we were using. It took 4.2 hours due to rate limits and cost about $60 just for that one batch. When I ran that same dataset through the local Fast.ai model I just trained? It took 8 minutes to train from scratch. Inference now takes roughly 14 milliseconds per image.