SlmBench

Model Setup - Download preset models from Hugging Face - Download custom models from Hugging Face - Select installed models stored on the device Local Inference - Fast on-device inference accelerated by Apple Metal - Real-time streaming of generated responses - Save conversation history and switch between models for comparison Performance Benchmarking - Latency: Time to First Token (TTFT), end-to-end latency - Inference Speed: Prefill speed, token generation speed (tokens/s) - Execution Details: Input/output token counts, model and inference settings, device information