Browse Source

Merge pull request #3990 from myhloli/dev

Dev
Xiaomeng Zhao 2 weeks ago
parent
commit
79f2d03d32

+ 4 - 3
README.md

@@ -650,14 +650,14 @@ A WebUI developed based on Gradio, with a simple interface and only core parsing
             <td>Faster than transformers</td>
             <td>Fast, compatible with the vLLM ecosystem</td>
             <td>Fast, compatible with the LMDeploy ecosystem</td>
-            <td>Suitable for OpenAI-compatible servers<sup>5</sup></td>
+            <td>Suitable for OpenAI-compatible servers<sup>6</sup></td>
         </tr>
         <tr>
             <th>Operating System</th>
             <td colspan="2" style="text-align:center;">Linux<sup>2</sup> / Windows / macOS</td>
             <td style="text-align:center;">macOS<sup>3</sup></td>
             <td style="text-align:center;">Linux<sup>2</sup> / Windows<sup>4</sup> </td>
-            <td style="text-align:center;">Linux<sup>2</sup> / Windows </td>
+            <td style="text-align:center;">Linux<sup>2</sup> / Windows<sup>5</sup> </td>
             <td>Any</td>
         </tr>
         <tr>
@@ -693,7 +693,8 @@ A WebUI developed based on Gradio, with a simple interface and only core parsing
 <sup>2</sup> Linux supports only distributions released in 2019 or later.  
 <sup>3</sup> MLX requires macOS 13.5 or later, recommended for use with version 14.0 or higher.  
 <sup>4</sup> Windows vLLM support via WSL2(Windows Subsystem for Linux).  
-<sup>5</sup> Servers compatible with the OpenAI API, such as local or remote model services deployed via inference frameworks like `vLLM`, `SGLang`, or `LMDeploy`.
+<sup>5</sup> Windows LMDeploy can only use the `turbomind` backend, which is slightly slower than the `pytorch` backend. If performance is critical, it is recommended to run it via WSL2.  
+<sup>6</sup> Servers compatible with the OpenAI API, such as local or remote model services deployed via inference frameworks like `vLLM`, `SGLang`, or `LMDeploy`.
 
 
 ### Install MinerU

+ 4 - 3
README_zh-CN.md

@@ -637,14 +637,14 @@ https://github.com/user-attachments/assets/4bea02c9-6d54-4cd6-97ed-dff14340982c
             <td>比transformers快</td>
             <td>速度快, 兼容vllm生态</td>
             <td>速度快, 兼容lmdeploy生态</td>
-            <td>适用于OpenAI兼容服务器<sup>5</sup></td>
+            <td>适用于OpenAI兼容服务器<sup>6</sup></td>
         </tr>
         <tr>
             <th>操作系统</th>
             <td colspan="2" style="text-align:center;">Linux<sup>2</sup> / Windows / macOS</td>
             <td style="text-align:center;">macOS<sup>3</sup></td>
             <td style="text-align:center;">Linux<sup>2</sup> / Windows<sup>4</sup> </td>
-            <td style="text-align:center;">Linux<sup>2</sup> / Windows </td>
+            <td style="text-align:center;">Linux<sup>2</sup> / Windows<sup>5</sup> </td>
             <td>不限</td>
         </tr>
         <tr>
@@ -680,7 +680,8 @@ https://github.com/user-attachments/assets/4bea02c9-6d54-4cd6-97ed-dff14340982c
 <sup>2</sup> Linux仅支持2019年及以后发行版  
 <sup>3</sup> MLX需macOS 13.5及以上版本支持,推荐14.0以上版本使用  
 <sup>4</sup> Windows vLLM通过WSL2(适用于 Linux 的 Windows 子系统)实现支持  
-<sup>5</sup> 兼容OpenAI API的服务器,如通过`vLLM`/`SGLang`/`LMDeploy`等推理框架部署的本地模型服务器或远程模型服务
+<sup>5</sup> Windows LMDeploy只能使用`turbomind`后端,速度比`pytorch`后端稍慢,如对速度有要求建议通过WSL2运行  
+<sup>6</sup> 兼容OpenAI API的服务器,如通过`vLLM`/`SGLang`/`LMDeploy`等推理框架部署的本地模型服务器或远程模型服务
 
 > [!TIP]
 > 除以上主流环境与平台外,我们也收录了一些社区用户反馈的其他平台支持情况,详情请参考[其他加速卡适配](https://opendatalab.github.io/MinerU/zh/usage/)。  

+ 23 - 0
docker/china/camb.Dockerfile

@@ -0,0 +1,23 @@
+# Base image containing the LMDeploy inference environment, requiring amd64 CPU + cambricon MLU.
+FROM
+
+# Install libgl for opencv support & Noto fonts for Chinese characters
+RUN apt-get update && \
+    apt-get install -y \
+        fonts-noto-core \
+        fonts-noto-cjk \
+        fontconfig \
+        libgl1 && \
+    fc-cache -fv && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Install mineru latest
+RUN python3 -m pip install -U 'mineru[core]' -i https://mirrors.aliyun.com/pypi/simple --break-system-packages && \
+    python3 -m pip cache purge
+
+# Download models and update the configuration file
+RUN /bin/bash -c "mineru-models-download -s modelscope -m all"
+
+# Set the entry point to activate the virtual environment and run the command line tool
+ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]

+ 23 - 0
docker/china/maca.Dockerfile

@@ -0,0 +1,23 @@
+# Base image containing the LMDeploy inference environment, requiring amd64 CPU + metax GPU.
+FROM
+
+# Install libgl for opencv support & Noto fonts for Chinese characters
+RUN apt-get update && \
+    apt-get install -y \
+        fonts-noto-core \
+        fonts-noto-cjk \
+        fontconfig \
+        libgl1 && \
+    fc-cache -fv && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Install mineru latest
+RUN python3 -m pip install -U 'mineru[core]' -i https://mirrors.aliyun.com/pypi/simple --break-system-packages && \
+    python3 -m pip cache purge
+
+# Download models and update the configuration file
+RUN /bin/bash -c "mineru-models-download -s modelscope -m all"
+
+# Set the entry point to activate the virtual environment and run the command line tool
+ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]

+ 23 - 0
docker/china/npu.Dockerfile

@@ -0,0 +1,23 @@
+# Base image containing the LMDeploy inference environment, requiring ARM CPU + Ascend NPU.
+FROM crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ascend:mineru-a2
+
+# Install libgl for opencv support & Noto fonts for Chinese characters
+RUN apt-get update && \
+    apt-get install -y \
+        fonts-noto-core \
+        fonts-noto-cjk \
+        fontconfig \
+        libgl1 && \
+    fc-cache -fv && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Install mineru latest
+RUN python3 -m pip install -U 'mineru[core]' -i https://mirrors.aliyun.com/pypi/simple --break-system-packages && \
+    python3 -m pip cache purge
+
+# Download models and update the configuration file
+RUN /bin/bash -c "mineru-models-download -s modelscope -m all"
+
+# Set the entry point to activate the virtual environment and run the command line tool
+ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]

+ 23 - 0
docker/china/ppu.Dockerfile

@@ -0,0 +1,23 @@
+# Base image containing the LMDeploy inference environment, requiring amd64 CPU + t-head PPU.
+FROM crpi-4crprmm5baj1v8iv.cn-hangzhou.personal.cr.aliyuncs.com/lmdeploy_dlinfer/ppu:mineru-ppu
+
+# Install libgl for opencv support & Noto fonts for Chinese characters
+RUN apt-get update && \
+    apt-get install -y \
+        fonts-noto-core \
+        fonts-noto-cjk \
+        fontconfig \
+        libgl1 && \
+    fc-cache -fv && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+# Install mineru latest
+RUN python3 -m pip install -U 'mineru[core]' -i https://mirrors.aliyun.com/pypi/simple --break-system-packages && \
+    python3 -m pip cache purge
+
+# Download models and update the configuration file
+RUN /bin/bash -c "mineru-models-download -s modelscope -m all"
+
+# Set the entry point to activate the virtual environment and run the command line tool
+ENTRYPOINT ["/bin/bash", "-c", "export MINERU_MODEL_SOURCE=local && exec \"$@\"", "--"]

+ 32 - 3
docker/compose.yaml

@@ -1,6 +1,6 @@
 services:
   mineru-vllm-server:
-    image: mineru-vllm:latest
+    image: mineru:latest
     container_name: mineru-vllm-server
     restart: always
     profiles: ["vllm-server"]
@@ -28,8 +28,37 @@ services:
               device_ids: ["0"]
               capabilities: [gpu]
 
+  mineru-lmdeploy-server:
+    image: mineru:latest
+    container_name: mineru-lmdeploy-server
+    restart: always
+    profiles: [ "lmdeploy-server" ]
+    ports:
+      - 30000:30000
+    environment:
+      MINERU_MODEL_SOURCE: local
+    entrypoint: mineru-lmdeploy-server
+    command:
+      --host 0.0.0.0
+      --port 30000
+    # --dp 2  # If using multiple GPUs, increase throughput using lmdeploy's multi-GPU parallel mode
+    # --cache-max-entry-count 0.5  # If running on a single GPU and encountering VRAM shortage, reduce the KV cache size by this parameter, if VRAM issues persist, try lowering it further to `0.4` or below.
+    ulimits:
+      memlock: -1
+      stack: 67108864
+    ipc: host
+    healthcheck:
+      test: [ "CMD-SHELL", "curl -f http://localhost:30000/health || exit 1" ]
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              device_ids: [ "0" ]
+              capabilities: [ gpu ]
+
   mineru-api:
-    image: mineru-vllm:latest
+    image: mineru:latest
     container_name: mineru-api
     restart: always
     profiles: ["api"]
@@ -57,7 +86,7 @@ services:
               capabilities: [ gpu ]
 
   mineru-gradio:
-    image: mineru-vllm:latest
+    image: mineru:latest
     container_name: mineru-gradio
     restart: always
     profiles: ["gradio"]

+ 2 - 2
docs/en/quick_start/docker_deployment.md

@@ -6,7 +6,7 @@ MinerU provides a convenient Docker deployment method, which helps quickly set u
 
 ```bash
 wget https://gcore.jsdelivr.net/gh/opendatalab/MinerU@master/docker/global/Dockerfile
-docker build -t mineru-vllm:latest -f Dockerfile .
+docker build -t mineru:latest -f Dockerfile .
 ```
 
 > [!TIP]
@@ -31,7 +31,7 @@ docker run --gpus all \
   --shm-size 32g \
   -p 30000:30000 -p 7860:7860 -p 8000:8000 \
   --ipc=host \
-  -it mineru-vllm:latest \
+  -it mineru:latest \
   /bin/bash
 ```
 

+ 2 - 2
docs/zh/quick_start/docker_deployment.md

@@ -6,7 +6,7 @@ MinerU提供了便捷的docker部署方式,这有助于快速搭建环境并
 
 ```bash
 wget https://gcore.jsdelivr.net/gh/opendatalab/MinerU@master/docker/china/Dockerfile
-docker build -t mineru-vllm:latest -f Dockerfile .
+docker build -t mineru:latest -f Dockerfile .
 ```
 
 > [!TIP]
@@ -31,7 +31,7 @@ docker run --gpus all \
   --shm-size 32g \
   -p 30000:30000 -p 7860:7860 -p 8000:8000 \
   --ipc=host \
-  -it mineru-vllm:latest \
+  -it mineru:latest \
   /bin/bash
 ```
 

+ 1 - 4
mineru/backend/vlm/utils.py

@@ -46,7 +46,6 @@ def enable_custom_logits_processors() -> bool:
 
 
 def set_lmdeploy_backend(device_type:str) -> str:
-    lmdeploy_backend = ""
     if device_type.lower() in ["ascend", "maca", "camb"]:
         lmdeploy_backend = "pytorch"
     elif device_type.lower() in ["cuda"]:
@@ -65,12 +64,10 @@ def set_lmdeploy_backend(device_type:str) -> str:
         else:
             raise ValueError("Unsupported operating system.")
     else:
-        raise ValueError(f"Unsupported device type: {device_type}")
+        raise ValueError(f"Unsupported lmdeploy device type: {device_type}")
     return lmdeploy_backend
 
 
-
-
 def set_default_gpu_memory_utilization() -> float:
     from vllm import __version__ as vllm_version
     if version.parse(vllm_version) >= version.parse("0.11.0"):

+ 106 - 102
mineru/backend/vlm/vlm_analyze.py

@@ -50,116 +50,120 @@ class ModelSingleton:
             for param in ["batch_size", "max_concurrency", "http_timeout"]:
                 if param in kwargs:
                     del kwargs[param]
-            if backend in ['transformers', 'vllm-engine', "vllm-async-engine", "mlx-engine", "lmdeploy-engine"] and not model_path:
+            if backend not in ["http-client"] and not model_path:
                 model_path = auto_download_and_get_model_root_path("/","vlm")
-                if backend == "transformers":
+            if backend == "transformers":
+                try:
+                    from transformers import (
+                        AutoProcessor,
+                        Qwen2VLForConditionalGeneration,
+                    )
+                    from transformers import __version__ as transformers_version
+                except ImportError:
+                    raise ImportError("Please install transformers to use the transformers backend.")
+
+                if version.parse(transformers_version) >= version.parse("4.56.0"):
+                    dtype_key = "dtype"
+                else:
+                    dtype_key = "torch_dtype"
+                device = get_device()
+                model = Qwen2VLForConditionalGeneration.from_pretrained(
+                    model_path,
+                    device_map={"": device},
+                    **{dtype_key: "auto"},  # type: ignore
+                )
+                processor = AutoProcessor.from_pretrained(
+                    model_path,
+                    use_fast=True,
+                )
+                if batch_size == 0:
+                    batch_size = set_default_batch_size()
+            elif backend == "mlx-engine":
+                mlx_supported = is_mac_os_version_supported()
+                if not mlx_supported:
+                    raise EnvironmentError("mlx-engine backend is only supported on macOS 13.5+ with Apple Silicon.")
+                try:
+                    from mlx_vlm import load as mlx_load
+                except ImportError:
+                    raise ImportError("Please install mlx-vlm to use the mlx-engine backend.")
+                model, processor = mlx_load(model_path)
+            else:
+                if os.getenv('OMP_NUM_THREADS') is None:
+                    os.environ["OMP_NUM_THREADS"] = "1"
+
+                if backend == "vllm-engine":
                     try:
-                        from transformers import (
-                            AutoProcessor,
-                            Qwen2VLForConditionalGeneration,
-                        )
-                        from transformers import __version__ as transformers_version
+                        import vllm
+                        from mineru_vl_utils import MinerULogitsProcessor
                     except ImportError:
-                        raise ImportError("Please install transformers to use the transformers backend.")
-
-                    if version.parse(transformers_version) >= version.parse("4.56.0"):
-                        dtype_key = "dtype"
-                    else:
-                        dtype_key = "torch_dtype"
-                    device = get_device()
-                    model = Qwen2VLForConditionalGeneration.from_pretrained(
-                        model_path,
-                        device_map={"": device},
-                        **{dtype_key: "auto"},  # type: ignore
-                    )
-                    processor = AutoProcessor.from_pretrained(
-                        model_path,
-                        use_fast=True,
-                    )
-                    if batch_size == 0:
-                        batch_size = set_default_batch_size()
-                elif backend == "mlx-engine":
-                    mlx_supported = is_mac_os_version_supported()
-                    if not mlx_supported:
-                        raise EnvironmentError("mlx-engine backend is only supported on macOS 13.5+ with Apple Silicon.")
+                        raise ImportError("Please install vllm to use the vllm-engine backend.")
+                    if "gpu_memory_utilization" not in kwargs:
+                        kwargs["gpu_memory_utilization"] = set_default_gpu_memory_utilization()
+                    if "model" not in kwargs:
+                        kwargs["model"] = model_path
+                    if enable_custom_logits_processors() and ("logits_processors" not in kwargs):
+                        kwargs["logits_processors"] = [MinerULogitsProcessor]
+                    # 使用kwargs为 vllm初始化参数
+                    vllm_llm = vllm.LLM(**kwargs)
+                elif backend == "vllm-async-engine":
                     try:
-                        from mlx_vlm import load as mlx_load
+                        from vllm.engine.arg_utils import AsyncEngineArgs
+                        from vllm.v1.engine.async_llm import AsyncLLM
+                        from mineru_vl_utils import MinerULogitsProcessor
                     except ImportError:
-                        raise ImportError("Please install mlx-vlm to use the mlx-engine backend.")
-                    model, processor = mlx_load(model_path)
-                else:
-                    if os.getenv('OMP_NUM_THREADS') is None:
-                        os.environ["OMP_NUM_THREADS"] = "1"
-
-                    if backend == "vllm-engine":
-                        try:
-                            import vllm
-                            from mineru_vl_utils import MinerULogitsProcessor
-                        except ImportError:
-                            raise ImportError("Please install vllm to use the vllm-engine backend.")
-                        if "gpu_memory_utilization" not in kwargs:
-                            kwargs["gpu_memory_utilization"] = set_default_gpu_memory_utilization()
-                        if "model" not in kwargs:
-                            kwargs["model"] = model_path
-                        if enable_custom_logits_processors() and ("logits_processors" not in kwargs):
-                            kwargs["logits_processors"] = [MinerULogitsProcessor]
-                        # 使用kwargs为 vllm初始化参数
-                        vllm_llm = vllm.LLM(**kwargs)
-                    elif backend == "vllm-async-engine":
-                        try:
-                            from vllm.engine.arg_utils import AsyncEngineArgs
-                            from vllm.v1.engine.async_llm import AsyncLLM
-                            from mineru_vl_utils import MinerULogitsProcessor
-                        except ImportError:
-                            raise ImportError("Please install vllm to use the vllm-async-engine backend.")
-                        if "gpu_memory_utilization" not in kwargs:
-                            kwargs["gpu_memory_utilization"] = set_default_gpu_memory_utilization()
-                        if "model" not in kwargs:
-                            kwargs["model"] = model_path
-                        if enable_custom_logits_processors() and ("logits_processors" not in kwargs):
-                            kwargs["logits_processors"] = [MinerULogitsProcessor]
-                        # 使用kwargs为 vllm初始化参数
-                        vllm_async_llm = AsyncLLM.from_engine_args(AsyncEngineArgs(**kwargs))
-                    elif backend == "lmdeploy-engine":
-                        try:
-                            from lmdeploy import PytorchEngineConfig, TurbomindEngineConfig
-                            from lmdeploy.serve.vl_async_engine import VLAsyncEngine
-                        except ImportError:
-                            raise ImportError("Please install lmdeploy to use the lmdeploy-engine backend.")
-                        if "cache_max_entry_count" not in kwargs:
-                            kwargs["cache_max_entry_count"] = 0.5
-
-                        if "device" in kwargs:
-                            device_type = kwargs.pop("device")
-                        else:
-                            device_type = os.getenv('MINERU_DEVICE_MODE', "cuda").lower()
-                        # device_type 如果有则去除":"
-                        if ":" in device_type:
-                            device_type = device_type.split(":")[0]
+                        raise ImportError("Please install vllm to use the vllm-async-engine backend.")
+                    if "gpu_memory_utilization" not in kwargs:
+                        kwargs["gpu_memory_utilization"] = set_default_gpu_memory_utilization()
+                    if "model" not in kwargs:
+                        kwargs["model"] = model_path
+                    if enable_custom_logits_processors() and ("logits_processors" not in kwargs):
+                        kwargs["logits_processors"] = [MinerULogitsProcessor]
+                    # 使用kwargs为 vllm初始化参数
+                    vllm_async_llm = AsyncLLM.from_engine_args(AsyncEngineArgs(**kwargs))
+                elif backend == "lmdeploy-engine":
+                    try:
+                        from lmdeploy import PytorchEngineConfig, TurbomindEngineConfig
+                        from lmdeploy.serve.vl_async_engine import VLAsyncEngine
+                    except ImportError:
+                        raise ImportError("Please install lmdeploy to use the lmdeploy-engine backend.")
+                    if "cache_max_entry_count" not in kwargs:
+                        kwargs["cache_max_entry_count"] = 0.5
+
+                    if "lmdeploy_device" in kwargs:
+                        device_type = kwargs.pop("lmdeploy_device")
+                        if device_type not in ["cuda", "ascend", "maca", "camb"]:
+                            raise ValueError(f"Unsupported lmdeploy device type: {device_type}")
+                    else:
+                        device_type = "cuda"
 
-                        lm_backend = set_lmdeploy_backend(device_type)
-                        logger.info(f"Set lmdeploy_backend to: {lm_backend}")
-
-                        if lm_backend == "pytorch":
-                            kwargs["device_type"] = device_type
-                            backend_config = PytorchEngineConfig(**kwargs)
-                        elif lm_backend == "turbomind":
-                            backend_config = TurbomindEngineConfig(**kwargs)
-                        else:
+                    if "lmdeploy_backend" in kwargs:
+                        lm_backend = kwargs.pop("lmdeploy_backend")
+                        if lm_backend not in ["pytorch", "turbomind"]:
                             raise ValueError(f"Unsupported lmdeploy backend: {lm_backend}")
+                    else:
+                        lm_backend = set_lmdeploy_backend(device_type)
+                    logger.info(f"lmdeploy device is: {device_type}, lmdeploy backend is: {lm_backend}")
 
-                        log_level = 'ERROR'
-                        from lmdeploy.utils import get_logger
-                        lm_logger = get_logger('lmdeploy')
-                        lm_logger.setLevel(log_level)
-                        if os.getenv('TM_LOG_LEVEL') is None:
-                            os.environ['TM_LOG_LEVEL'] = log_level
-
-                        lmdeploy_engine = VLAsyncEngine(
-                            model_path,
-                            backend=lm_backend,
-                            backend_config=backend_config,
-                        )
+                    if lm_backend == "pytorch":
+                        kwargs["device_type"] = device_type
+                        backend_config = PytorchEngineConfig(**kwargs)
+                    elif lm_backend == "turbomind":
+                        backend_config = TurbomindEngineConfig(**kwargs)
+                    else:
+                        raise ValueError(f"Unsupported lmdeploy backend: {lm_backend}")
+
+                    log_level = 'ERROR'
+                    from lmdeploy.utils import get_logger
+                    lm_logger = get_logger('lmdeploy')
+                    lm_logger.setLevel(log_level)
+                    if os.getenv('TM_LOG_LEVEL') is None:
+                        os.environ['TM_LOG_LEVEL'] = log_level
+
+                    lmdeploy_engine = VLAsyncEngine(
+                        model_path,
+                        backend=lm_backend,
+                        backend_config=backend_config,
+                    )
             self._models[key] = MinerUClient(
                 backend=backend,
                 model=model,

+ 27 - 14
mineru/model/vlm/lmdeploy_server.py

@@ -13,11 +13,12 @@ def main():
     has_port_arg = False
     has_gpu_memory_utilization_arg = False
     has_log_level_arg = False
-    has_backend_arg = False
-    device_type = "cuda"
+    device_type = ""
     lm_backend = ""
 
     # 检查现有参数
+    indices_to_remove = []
+
     for i, arg in enumerate(args):
         if arg == "--server-port" or arg.startswith("--server-port="):
             has_port_arg = True
@@ -25,18 +26,24 @@ def main():
             has_gpu_memory_utilization_arg = True
         if arg == "--log-level" or arg.startswith("--log-level="):
             has_log_level_arg = True
-        if arg == "--backend":
-            has_backend_arg = True
+        if arg == "--backend" or arg == "--lmdeploy-backend":
             if i + 1 < len(args):
                 lm_backend = args[i + 1]
-        if arg.startswith("--backend="):
-            has_backend_arg = True
+                indices_to_remove.extend([i, i + 1])
+        elif arg.startswith("--backend=") or arg.startswith("--lmdeploy-backend="):
             lm_backend = arg.split("=", 1)[1]
-        if arg == "--device":
+            indices_to_remove.append(i)
+        if arg == "--device" or arg == "--lmdeploy-device":
             if i + 1 < len(args):
                 device_type = args[i + 1]
-        if arg.startswith("--device="):
+                indices_to_remove.extend([i, i + 1])
+        elif arg.startswith("--device=") or arg.startswith("--lmdeploy-device="):
             device_type = arg.split("=", 1)[1]
+            indices_to_remove.append(i)
+
+    # 从后往前删除,避免索引错位
+    for i in sorted(set(indices_to_remove), reverse=True):
+        args.pop(i)
 
     # 添加默认参数
     if not has_port_arg:
@@ -46,21 +53,27 @@ def main():
     if not has_log_level_arg:
         args.extend(["--log-level", "ERROR"])
 
-    if ":" in device_type:
-        device_type = device_type.split(":")[0]
+    if device_type == "":
+        device_type = "cuda"
+    elif device_type not in ["cuda", "ascend", "maca", "camb"]:
+        raise ValueError(f"Unsupported lmdeploy device type: {device_type}")
     if lm_backend == "":
         lm_backend = set_lmdeploy_backend(device_type)
-    logger.info(f"Set lmdeploy_backend to: {lm_backend}")
+    elif lm_backend not in ["pytorch", "turbomind"]:
+        raise ValueError(f"Unsupported lmdeploy backend: {lm_backend}")
+    logger.info(f"lmdeploy device is: {device_type}, lmdeploy backend is: {lm_backend}")
 
     if lm_backend == "pytorch":
         os.environ["TOKENIZERS_PARALLELISM"] = "false"
 
-    # args中如果有--backend参数,则不设置
-    if not has_backend_arg:
-        args.extend(["--backend", lm_backend])
+
+    args.extend(["--device", device_type])
+    args.extend(["--backend", lm_backend])
 
     model_path = auto_download_and_get_model_root_path("/", "vlm")
 
+    # logger.debug(args)
+
     # 重构参数,将模型路径作为位置参数
     sys.argv = [sys.argv[0]] + ["serve", "api_server", model_path] + args