README update
This commit is contained in:
parent
64b79ee953
commit
76457818a8
1 changed files with 25 additions and 0 deletions
25
README.md
25
README.md
|
|
@ -0,0 +1,25 @@
|
||||||
|
# Qwen3-0.6B
|
||||||
|
|
||||||
|
Run [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B-GGUF) locally with ollama or llama.cpp.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```sh
|
||||||
|
nix develop
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### ollama
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ollama serve &
|
||||||
|
ollama run qwen3:0.6b
|
||||||
|
```
|
||||||
|
|
||||||
|
### llama.cpp
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Download a GGUF from https://huggingface.co/Qwen/Qwen3-0.6B-GGUF
|
||||||
|
llama-cli -m Qwen3-0.6B-Q4_K_M.gguf -p "Hello" -cnv
|
||||||
|
```
|
||||||
Loading…
Add table
Add a link
Reference in a new issue