asxpi/nixOS
asxpi/nixOS
1
0
Fork 0
Code Issues Pull requests Activity

ollama: local LLM serving with ROCm on Krackan Point (gfx1151)

Browse source 
Enable ollama-rocm bound to 127.0.0.1:11434 with gfx1151 (RDNA 3.5)
overrides, flash attention, q8_0 KV cache, and preload qwen3:14b.
This commit is contained in:
Sergei Poljanski 2026-06-02 14:07:29 +03:00
commit f31ecbf6e6
Signed by: asxpi
GPG key ID: 4F8851660FA4121B
2 changed files with 18 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
configuration.nix
View file

@ -17,6 +17,7 @@
./modules/security.nix
./modules/power.nix
./modules/xray.nix
./modules/ollama.nix
./modules/gaming.nix
];

17
modules/ollama.nix Normal file
View file

@ -0,0 +1,17 @@
{ config, pkgs, lib, ... }:
{
services.ollama = {
enable = true;
package = pkgs.ollama-rocm;
rocmOverrideGfx = "11.5.1"; # gfx1151, Krackan Point RDNA 3.5
host = "127.0.0.1";
port = 11434;
environmentVariables = {
HSA_OVERRIDE_GFX_VERSION = "11.5.1";
OLLAMA_FLASH_ATTENTION = "1";
OLLAMA_KV_CACHE_TYPE = "q8_0";
};
loadModels = [ "qwen3:14b" ];
};
}