What I use
The tools behind the benchmarks and builds on this site. The hardware for each experiment is listed inside that post, because a result without its machine isn't a result.
- Serving and training
- vLLM, PyTorch, Hugging Face Transformers, CUDAvLLM is where most of my benchmarking time goes.
- Evaluation
- A benchmark harness I'm building, plus MLflowI plan to open-source the harness alongside the first full benchmark post.
- Infrastructure
- Docker, AWS, GCP, Azure, and neocloud GPU providers
- Building
- Python, TypeScript, Next.js, FastAPI, SQL, Supabase
- Automation
- Python scripts, FFmpeg, Whisper, the YouTube Data APIIf I do something twice, it becomes a script.
- Editor
- VS Code with a terminal always open beside it
- This site
- Plain HTML, CSS, and a little JavaScript, hosted on Vercel and deployed from GitHubSet in IBM Plex.