Performance
Now I’ll write the article using my knowledge of Qdrant’s binary quantization feature and its well-known documentation URLs.
Qdrant Binary Quantization Cuts Sentence-Transformers Search Latency 4x Qdrant’s binary quantization compresses each float32 vector dimension to a single.
JAX Gradient Checkpointing on TPU v5e: 40% Memory Cut at 12% Speed Cost
In this article How does JAX gradient checkpointing reduce memory on TPU v5e? What is the checkpoint policy that drives the 40% memory saving?
Mistral-7B-v0.3 QLoRA on Modal A100-40GB: nf4 + bf16_compute Beat My RunPod H100 Spot Cost Per Step
TL;DR: For a Mistral-7B-v0.3 QLoRA fine-tune at sequence length 2048 and micro-batch 4, a Modal A100-40GB container running bitsandbytes nf4 with bfloat16.
vLLM 0.6 Continuous Batching Cut My Llama 3 Latency in Half
Upgrading a Llama 3 8B endpoint from vLLM 0.5.4 to 0.6.x is the rare dependency bump where the numbers on the dashboard actually move.
torch.compile in PyTorch 2.5: Where the Speedup Comes From and Where It Disappears
PyTorch 2.5 made torch.compile good enough that you can drop it into a real training script and expect a speedup most of the time.
How to Convert PyTorch Models to ONNX Format for Faster Inference
I remember the first time I deployed a PyTorch model to production. I wrapped a beautifully trained ResNet model in a Flask API, spun up a Docker.
Dask’s Active Memory Manager Finally Stopped Breaking My Pipelines
I used to dread the Slack notification. You know the one. The little red dot popping up at 7:30 AM telling me my overnight batch job failed.
How I Cut FLUX.1 Inference to 3 Seconds with TensorRT
I was staring at my terminal at 1:30 AM last Thursday, watching my RTX 4090 scream at 98% utilization while spitting out a single 1024×1024 image every 15.
Compiling Fast.ai Models for Cerebras
The Deployment Wall I was sitting at my desk at 9 PM last Thursday, staring at a CloudWatch dashboard that made absolutely no sense.
SageMaker HyperPod Finally Fixed the Checkpoint Bottleneck
I lost three days of Llama-3 fine-tuning last November because a single EC2 node decided to panic. The cluster halted.