cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

updated benchmarks

Browse Source
This commit is contained in:
Donato Capitella
2025-12-06 09:24:07 +00:00
parent 5fe1510981
commit cef614d895
200 changed files with 4594 additions and 6468 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7-nightly-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1622.34 ± 128.86 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.53 ± 0.07 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 818.04 ± 30.69 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.16 ± 0.50 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7-nightly-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 637.77 ± 13.61 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.05 ± 0.03 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 644.36 ± 8.83 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 51.88 ± 0.17 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7-nightly__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1544.19 ± 224.65 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.28 ± 0.07 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 800.50 ± 43.62 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.29 ± 0.03 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7-nightly__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 629.39 ± 20.64 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 51.87 ± 0.05 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 638.77 ± 8.84 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 51.65 ± 0.42 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7.9-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1538.72 ± 208.00 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 58.02 ± 0.11 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1577.01 ± 83.25 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.72 ± 0.50 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7.9-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 664.86 ± 54.89 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.84 ± 0.06 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 670.79 ± 24.36 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.75 ± 0.50 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7.9__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1599.94 ± 126.30 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.50 ± 0.05 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1640.28 ± 17.48 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 55.41 ± 0.06 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm-7.9__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 682.09 ± 7.42 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 58.06 ± 0.43 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 676.65 ± 20.88 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.63 ± 0.04 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm6_4_4-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1704.02 ± 12.95 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.95 ± 0.52 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1686.79 ± 32.81 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 58.26 ± 0.15 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm6_4_4-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 664.27 ± 16.59 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.80 ± 0.02 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 661.24 ± 12.20 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.64 ± 0.01 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm6_4_4__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1731.74 ± 16.06 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.64 ± 0.54 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1730.61 ± 34.04 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.20 ± 0.77 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm6_4_4__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 687.91 ± 6.98 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.27 ± 0.43 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 688.64 ± 11.04 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.51 ± 0.01 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1553.68 ± 160.23 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.81 ± 0.07 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1588.08 ± 46.01 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.30 ± 0.67 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 672.84 ± 8.78 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.86 ± 0.40 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 676.12 ± 15.33 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.72 ± 0.04 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1__fa1__dual.log
+18 -4
View File
@@ -5,7 +5,21 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1629.33 ± 67.83 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.58 ± 0.05 |
build: fa0465954 (7205)
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fa4cecdb565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fa4cecdb92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fa4cecdbaaf]
/usr/local/lib64/libggml-hip.so.0(+0x2fd1ef2) [0x7fa4d1d69ef2]
/usr/local/lib64/libggml-hip.so.0(+0x2fdca9a) [0x7fa4d1d74a9a]
/usr/local/lib64/libggml-hip.so.0(+0x2fd716f) [0x7fa4d1d6f16f]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x7f3) [0x7fa4cecf5f63]
/usr/local/lib64/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa0) [0x7fa4d2437a90]
/usr/local/lib64/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0xe2) [0x7fa4d2439722]
/usr/local/lib64/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x3bf) [0x7fa4d243e5df]
/usr/local/lib64/libllama.so.0(llama_decode+0xe) [0x7fa4d243f3fe]
/usr/local/bin/llama-bench() [0x40acdb]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7fa4ce6715b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fa4ce671668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm7.1] Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__fa1 failed (exit 0)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 684.37 ± 2.61 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.54 ± 0.08 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 683.86 ± 15.60 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 56.94 ± 0.52 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__vulkan_amdvlk__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 286.55 ± 12.36 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 16.21 ± 0.15 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 286.15 ± 12.06 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 15.81 ± 0.17 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__vulkan_radv__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1593.95 ± 58.52 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 30.17 ± 0.36 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1886.73 ± 107.96 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 26.98 ± 1.90 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7-nightly-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3020.37 ± 43.13 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.80 ± 0.58 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1187.71 ± 4.64 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.60 ± 0.57 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7-nightly-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2997.38 ± 60.33 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.79 ± 0.46 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2914.78 ± 233.63 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.46 ± 0.48 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7-nightly__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2913.43 ± 23.64 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.01 ± 0.69 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1205.03 ± 11.21 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 107.14 ± 0.69 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7-nightly__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2919.46 ± 42.12 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 106.96 ± 0.69 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2873.06 ± 13.41 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 106.98 ± 0.64 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7.9-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2819.16 ± 36.53 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.55 ± 0.36 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2803.16 ± 16.89 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.98 ± 0.55 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7.9-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2798.79 ± 24.28 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.01 ± 0.29 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2795.19 ± 24.52 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.55 ± 0.40 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7.9__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2855.40 ± 20.98 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.88 ± 0.52 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2887.22 ± 42.71 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 103.24 ± 0.49 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm-7.9__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2857.35 ± 21.82 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 103.00 ± 0.52 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2856.67 ± 13.72 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 100.53 ± 0.62 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm6_4_4-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2839.13 ± 17.80 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 106.43 ± 0.52 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2813.43 ± 19.17 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.57 ± 0.35 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm6_4_4-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2805.58 ± 24.28 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 106.42 ± 0.50 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2805.82 ± 12.56 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.84 ± 0.41 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm6_4_4__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2948.99 ± 46.59 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.97 ± 0.24 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2898.68 ± 108.51 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.66 ± 0.45 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm6_4_4__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2921.36 ± 33.18 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.79 ± 0.44 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2934.64 ± 33.57 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 105.81 ± 0.41 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2827.41 ± 23.48 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.05 ± 1.22 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2733.56 ± 171.50 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.21 ± 0.55 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2783.87 ± 35.60 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.30 ± 0.29 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2779.82 ± 39.08 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 100.99 ± 0.24 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2867.39 ± 32.96 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.97 ± 0.51 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2661.39 ± 352.85 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 103.15 ± 0.67 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2846.47 ± 25.01 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.24 ± 1.48 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2860.07 ± 36.46 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 101.60 ± 0.58 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__vulkan_amdvlk__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1450.58 ± 57.46 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 112.62 ± 0.49 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1213.41 ± 52.49 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 110.66 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__vulkan_radv__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1278.02 ± 94.48 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 102.18 ± 0.36 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1186.84 ± 47.80 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 102.53 ± 0.36 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7-nightly-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1888.53 ± 18.64 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.77 ± 0.12 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1124.96 ± 18.06 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.36 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7-nightly-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1757.56 ± 2.78 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.62 ± 0.12 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1741.88 ± 9.34 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.41 ± 0.11 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7-nightly__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1823.40 ± 79.46 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.05 ± 0.21 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1161.10 ± 14.16 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.94 ± 0.66 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7-nightly__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1713.47 ± 20.05 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.21 ± 0.13 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1716.43 ± 20.79 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.09 ± 0.15 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7.9-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2995.60 ± 21.52 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.06 ± 0.31 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2964.37 ± 19.03 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.10 ± 0.22 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7.9-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2692.40 ± 14.09 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.29 ± 0.23 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2683.21 ± 17.21 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.97 ± 0.22 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7.9__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3048.39 ± 23.80 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.22 ± 0.23 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3047.17 ± 30.17 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.95 ± 0.22 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm-7.9__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2737.79 ± 9.63 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.02 ± 0.21 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2739.55 ± 21.33 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.92 ± 0.24 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm6_4_4-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3042.89 ± 19.22 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 78.56 ± 0.28 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3040.27 ± 8.51 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 78.34 ± 0.26 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm6_4_4-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2692.73 ± 7.68 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 78.04 ± 0.99 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2686.46 ± 14.34 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 78.42 ± 0.27 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm6_4_4__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3141.98 ± 27.97 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.74 ± 0.29 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3078.21 ± 160.21 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.63 ± 0.31 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm6_4_4__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2795.57 ± 9.79 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.54 ± 0.34 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2773.21 ± 10.68 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.53 ± 0.28 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2973.42 ± 43.94 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.22 ± 0.23 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2965.21 ± 17.70 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.09 ± 0.21 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2689.18 ± 17.39 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.83 ± 1.01 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2670.23 ± 11.73 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.22 ± 0.21 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3050.31 ± 19.73 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.79 ± 0.44 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3036.68 ± 20.10 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.98 ± 0.24 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2752.40 ± 19.12 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.00 ± 0.23 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2742.93 ± 16.52 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.97 ± 0.28 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__vulkan_amdvlk__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1716.73 ± 53.79 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 82.29 ± 0.10 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1713.67 ± 53.31 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 82.76 ± 0.08 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__vulkan_radv__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1437.77 ± 62.07 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 78.40 ± 0.13 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | pp512 | 1383.56 ± 58.73 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | Vulkan | 99 | 1 | tg128 | 77.98 ± 0.13 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7-nightly-rocwmma__fa1__dual.log
+4 -1
View File
@@ -5,4 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 35.87 ± 0.10 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 35.15 ± 0.36 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 0.79 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7-nightly-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 925.06 ± 3.97 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.47 ± 0.01 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 809.06 ± 33.23 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.09 ± 0.65 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7-nightly__fa1__dual.log
+4 -1
View File
@@ -5,4 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 35.58 ± 0.09 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 36.04 ± 1.53 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 0.79 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7-nightly__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 920.43 ± 4.69 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.54 ± 0.01 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 915.57 ± 4.70 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.42 ± 0.05 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7.9-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 930.23 ± 3.66 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.72 ± 0.03 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 921.00 ± 6.15 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.13 ± 0.90 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7.9-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 967.56 ± 1.62 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.73 ± 0.02 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 963.42 ± 6.03 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.59 ± 0.07 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7.9__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 928.76 ± 5.89 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.94 ± 0.05 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 928.02 ± 4.53 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.70 ± 0.04 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm-7.9__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 971.69 ± 3.83 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.79 ± 0.03 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 965.17 ± 4.88 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.81 ± 0.06 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm6_4_4-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 922.99 ± 11.27 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 38.77 ± 0.05 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 922.52 ± 7.87 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 38.59 ± 0.07 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm6_4_4-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 897.76 ± 4.81 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.50 ± 0.03 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 904.88 ± 3.87 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.42 ± 0.04 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm6_4_4__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 938.18 ± 10.04 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 38.93 ± 0.09 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 936.74 ± 8.89 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 38.87 ± 0.06 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm6_4_4__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 900.84 ± 3.43 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.54 ± 0.07 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 906.89 ± 5.40 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.53 ± 0.10 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1-rocwmma__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 908.21 ± 2.86 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.63 ± 0.07 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 895.25 ± 9.23 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.64 ± 0.07 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 935.46 ± 5.91 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.84 ± 0.03 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 932.20 ± 5.85 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.80 ± 0.04 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 911.83 ± 4.32 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.69 ± 0.12 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 755.54 ± 61.49 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.00 ± 1.56 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 943.82 ± 7.26 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.81 ± 0.09 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 934.42 ± 4.61 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.67 ± 0.15 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__vulkan_amdvlk__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | pp512 | 613.50 ± 47.21 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | tg128 | 53.23 ± 0.27 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | pp512 | 510.76 ± 1.54 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | tg128 | 52.54 ± 0.26 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__vulkan_radv__fa1__dual.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | pp512 | 478.78 ± 58.68 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | tg128 | 25.92 ± 0.52 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | pp512 | 413.79 ± 31.74 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | Vulkan | 99 | 1 | tg128 | 29.61 ± 0.27 |
build: 03d9a77b8 (7278)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7-nightly-rocwmma__fa1__single.log
+14 -4
View File
@@ -4,7 +4,17 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1467.59 ± 33.66 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.98 ± 0.00 |
build: fa0465954 (7205)
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fc62634a565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fc62634a92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fc62634aaaf]
/usr/local/lib64/libggml-hip.so.0(+0x2cb5eb2) [0x7fc6290bceb2]
/usr/local/lib64/libggml-hip.so.0(+0x2cbb07e) [0x7fc6290c207e]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7fc626361a1e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7fc629785d70]
/usr/local/bin/llama-bench() [0x40acfc]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7fc625ce05b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fc625ce0668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm-7-nightly-rocwmma] gemma-3-12b-it-UD-Q8_K_XL__fa1 failed (exit 0)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7-nightly-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1209.48 ± 4.18 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.94 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1204.93 ± 0.73 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.82 ± 0.27 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7-nightly__fa1__single.log
+2 -2
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1471.76 ± 6.40 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1459.81 ± 7.69 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.28 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7-nightly__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1193.30 ± 0.92 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1188.76 ± 0.80 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.18 ± 0.18 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7.9-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2620.27 ± 58.52 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.99 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2607.80 ± 28.74 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.98 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7.9-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1838.59 ± 5.99 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.96 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1831.32 ± 5.25 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.88 ± 0.19 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7.9__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2777.85 ± 3.51 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.25 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2602.75 ± 353.11 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm-7.9__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1906.35 ± 7.11 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.31 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1901.78 ± 7.17 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.09 ± 0.25 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm6_4_4-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2759.96 ± 14.76 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.02 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2707.18 ± 15.76 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.92 ± 0.06 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm6_4_4-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1925.96 ± 5.08 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.00 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1894.60 ± 6.55 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.87 ± 0.23 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm6_4_4__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2885.85 ± 24.14 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.28 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2860.62 ± 21.62 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.03 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm6_4_4__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1976.74 ± 6.47 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.26 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1960.79 ± 6.80 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.35 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1-rocwmma__fa1__single.log
+2 -2
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2626.70 ± 13.60 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2465.10 ± 257.60 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.97 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1833.53 ± 6.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.94 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1828.85 ± 6.68 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.83 ± 0.29 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2780.99 ± 4.35 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.26 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2754.63 ± 44.08 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.25 ± 0.02 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1906.89 ± 6.13 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.02 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1900.55 ± 8.55 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.12 ± 0.20 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__vulkan_amdvlk__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | pp512 | 1437.05 ± 134.24 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | tg128 | 30.69 ± 0.30 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | pp512 | 1234.73 ± 28.51 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | tg128 | 31.58 ± 0.04 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__vulkan_radv__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | pp512 | 1362.70 ± 91.17 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | tg128 | 27.07 ± 0.13 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | pp512 | 1319.14 ± 103.83 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | Vulkan | 99 | 1 | tg128 | 28.04 ± 0.09 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7-nightly-rocwmma__fa1__dual.log
+4 -14
View File
@@ -5,17 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:90: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7f2fca3af565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f2fca3af92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f2fca3afaaf]
/usr/local/lib64/libggml-hip.so.0(+0x2c7b152) [0x7f2fcd0e7152]
/usr/local/lib64/libggml-hip.so.0(+0x2c8033e) [0x7f2fcd0ec33e]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7f2fca3c694e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7f2fcd7a1c70]
/usr/local/bin/llama-bench() [0x40abfc]
/usr/local/bin/llama-bench() [0x40898d]
/lib64/libc.so.6(+0x35b5) [0x7f2fc9d455b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f2fc9d45668]
/usr/local/bin/llama-bench() [0x409a75]
✖ ! [rocm-7-nightly-rocwmma] gemma-3-27b-it-BF16-00001-of-00002__fa1 failed (exit 0)
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1585.99 ± 4.29 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.44 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7-nightly-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 185.29 ± 0.29 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.30 ± 0.01 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 182.83 ± 0.05 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.30 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7-nightly__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1512.53 ± 2.86 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.45 ± 0.00 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1493.04 ± 2.86 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.44 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7-nightly__hblt0__fa1__dual.log
+2 -2
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 181.36 ± 0.17 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 180.46 ± 0.12 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.31 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7.9-rocwmma__fa1__dual.log
+17 -4
View File
@@ -5,7 +5,20 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1569.01 ± 8.41 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.49 ± 0.00 |
build: fa0465954 (7205)
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fb2437ed565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fb2437ed92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fb2437edaaf]
/usr/local/lib64/libggml-hip.so.0(+0x2efaf92) [0x7fb2467a4f92]
/usr/local/lib64/libggml-hip.so.0(+0x2f000e9) [0x7fb2467aa0e9]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x36d) [0x7fb243807add]
/usr/local/lib64/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa0) [0x7fb246e72a90]
/usr/local/lib64/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0xe2) [0x7fb246e74722]
/usr/local/lib64/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x3bf) [0x7fb246e795df]
/usr/local/lib64/libllama.so.0(llama_decode+0xe) [0x7fb246e7a3fe]
/usr/local/bin/llama-bench() [0x40acdb]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7fb2431835b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fb243183668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm-7.9-rocwmma] gemma-3-27b-it-BF16-00001-of-00002__fa1 failed (exit 0)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7.9-rocwmma__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 175.20 ± 0.23 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.49 ± 0.00 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 175.25 ± 0.34 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.48 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7.9__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1542.16 ± 90.21 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.47 ± 0.01 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1576.23 ± 3.21 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.48 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm-7.9__hblt0__fa1__dual.log
+3 -3
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 174.65 ± 0.15 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.47 ± 0.02 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 174.12 ± 0.14 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.48 ± 0.00 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm6_4_4-rocwmma__fa1__dual.log
+2 -2
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1267.81 ± 2.66 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1276.65 ± 2.16 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.48 ± 0.01 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm6_4_4-rocwmma__hblt0__fa1__dual.log
+2 -2
View File
@@ -5,7 +5,7 @@ ggml_cuda_init: found 2 ROCm devices:
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 178.11 ± 0.12 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 178.55 ± 0.20 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.36 ± 0.01 |
build: fa0465954 (7205)
build: 6016d0bd4 (7285)

Some files were not shown because too many files have changed in this diff Show More