When you click
Run
, model will be first downloaded and cached in browser.
Single Thread
Model:
Qwen/Qwen1.5-0.5B-Chat Q3_K_M (350 MB)
tinymistral-248m-sft-v4 q8_0 (265.26 MB)
TinyLlama/TinyLlama-1.1B-Chat-v1.0 Q4_K_M (669 MB)
Qwen/Qwen1.5-1.8B-Chat Q3_K_M (1.02 GB)
stabilityai/stablelm-2-zephyr-1_6b Q4_1 (1.07 GB)
microsoft/phi-1_5 Q4_K_M (918 MB)
microsoft/phi-2 Q3_K_M (1.48 GB)
Prompt:
Good morning how are you doing
Result:
Run
Loading model...
Loaded model
Generating...