diff --git a/Dockerfile b/Dockerfile index fa70850..ac76c9f 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,4 @@ -# FROM docker.1ms.run/vllm/vllm-openai-rocm:latest -FROM vllm/vllm-openai-rocm:nightly +FROM vllm/vllm-openai-rocm:latest WORKDIR /workspace diff --git a/README.md b/README.md index 2dfc4c7..23ef15c 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ curl -X POST "http://localhost:8001/v1/chat/completions" \ ## 常见故障排查 -- 报错 `Model architectures ['Qwen3_5MoeForConditionalGeneration'] are not supported for now` 时,在对应 profile 增加 `model_impl=transformers`,并使用当前 Dockerfile 重建镜像后再启动。 +- 报错 `Model architectures ['Qwen3_5MoeForConditionalGeneration'] are not supported for now` 或 `The Transformers implementation ... is not compatible with vLLM` 时,说明当前 vLLM 栈与该模型架构不兼容,需切换到兼容模型或改用其他推理后端。 - 报错 `model config (gptq) does not match quantization argument (gptq_marlin)` 时,将该模型配置改为 `dtype=float16` 且 `quantization=gptq`。 - 报错 `RPC call to sample_tokens timed out` 或出现 `GPU core dump` 时,先下调模型配置为更稳参数:`ctx=32768`、`max_num_seqs=4`、`max_tokens=2048`、`gpu_util=0.90`,并开启 `enforce_eager=true`。 - 若模型目录存在但仍加载失败,检查挂载路径是否为 `/opt/model:/opt/model:ro`,并确认容器内可见模型文件。 diff --git a/config.json b/config.json index 1f828c0..f3c6e72 100644 --- a/config.json +++ b/config.json @@ -73,7 +73,6 @@ "local_path": "Qwen3.5-35B-A3B-GPTQ-Int4", "dtype": "float16", "quantization": "moe_wna16", - "model_impl": "transformers", "ctx": "32768", "trust_remote": true, "valid_tp": [1, 2],