Model deployment
Fit useful models onto available hardware without hiding memory, latency or access constraints.
What we currently think
A model fitting into memory is only the first gate. Batch behavior, kernels, model format and switching costs determine whether it is usable.
Start with your question
- Will this fit the hardware I have?
- What changes between a smoke test and sustained use?
- Which constraint should determine the deployment route?
Start here
FP8 on RTX 3090 Ti - What Actually Works on Consumer GPUs
FP8 on an RTX 3090 Ti is useful, but not for the reason most examples imply. Ampere SM 8.6 can use FP8 as a storage dtype for VRAM savings, while native FP8 tensor-core compute starts with newer GPU architectures. Here is the practical guide for FLUX-style image and video pipelines on 24 GB cards.
On-Demand GPU Embeddings - Testing SIE Model Switching and VRAM Eviction
We tested SIE with embedding and reranking models on an RTX 3090 Ti. Here is what model switching, cold loading, idle eviction, VRAM release, and the untested pressure-driven LRU path mean for an on-demand embedding service.
China AI Model Access Guide (2026): Requirements, Compliance, and Risks
A neutral, operations-first guide to accessing popular China AI platforms in 2026: when +86 numbers are required, when email login works, and what legal, privacy, and policy risks teams should evaluate before setup.