|
@@ -0,0 +1,184 @@
|
|
|
|
|
+# MinerU 输出格式说明
|
|
|
|
|
+
|
|
|
|
|
+## 概述
|
|
|
|
|
+
|
|
|
|
|
+`MinerUOutputFormatter` 严格遵循 MinerU 的输出格式,与 `mineru_vllm_results_cell_bbox` 目录下的格式完全一致。
|
|
|
|
|
+
|
|
|
|
|
+## 输出格式
|
|
|
|
|
+
|
|
|
|
|
+### 1. JSON 格式
|
|
|
|
|
+
|
|
|
|
|
+每页一个 JSON 文件,包含元素列表:
|
|
|
|
|
+
|
|
|
|
|
+```json
|
|
|
|
|
+[
|
|
|
|
|
+ {
|
|
|
|
|
+ "type": "header",
|
|
|
|
|
+ "text": "页眉内容",
|
|
|
|
|
+ "bbox": [160, 126, 590, 161],
|
|
|
|
|
+ "page_idx": 0
|
|
|
|
|
+ },
|
|
|
|
|
+ {
|
|
|
|
|
+ "type": "text",
|
|
|
|
|
+ "text": "正文内容...",
|
|
|
|
|
+ "bbox": [158, 226, 1463, 322],
|
|
|
|
|
+ "page_idx": 0
|
|
|
|
|
+ },
|
|
|
|
|
+ {
|
|
|
|
|
+ "type": "table",
|
|
|
|
|
+ "img_path": "images/xxx.jpg",
|
|
|
|
|
+ "table_caption": ["表格标题"],
|
|
|
|
|
+ "table_footnote": [],
|
|
|
|
|
+ "table_body": "<table>...</table>",
|
|
|
|
|
+ "bbox": [251, 264, 1404, 2111],
|
|
|
|
|
+ "page_idx": 0,
|
|
|
|
|
+ "table_cells": [...],
|
|
|
|
|
+ "image_rotation_angle": 270.0,
|
|
|
|
|
+ "skew_angle": -0.26
|
|
|
|
|
+ }
|
|
|
|
|
+]
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 2. 表格单元格格式 (table_cells)
|
|
|
|
|
+
|
|
|
|
|
+```json
|
|
|
|
|
+{
|
|
|
|
|
+ "type": "table_cell",
|
|
|
|
|
+ "text": "单元格内容",
|
|
|
|
|
+ "matched_text": "OCR匹配的文本",
|
|
|
|
|
+ "bbox": [273, 1653, 302, 2106],
|
|
|
|
|
+ "row": 2,
|
|
|
|
|
+ "col": 1,
|
|
|
|
|
+ "score": 100.0,
|
|
|
|
|
+ "paddle_bbox_indices": [11, 12]
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 3. 表格 HTML 格式 (table_body)
|
|
|
|
|
+
|
|
|
|
|
+HTML 表格带有 `data-bbox` 属性:
|
|
|
|
|
+
|
|
|
|
|
+```html
|
|
|
|
|
+<table>
|
|
|
|
|
+ <tr>
|
|
|
|
|
+ <td data-bbox="[232,273,685,302]" data-paddle-indices="[11, 12]" data-score="100.0000">流动资产:</td>
|
|
|
|
|
+ <td></td>
|
|
|
|
|
+ <td></td>
|
|
|
|
|
+ </tr>
|
|
|
|
|
+</table>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 4. Markdown 格式
|
|
|
|
|
+
|
|
|
|
|
+带 bbox 注释的 Markdown:
|
|
|
|
|
+
|
|
|
|
|
+```markdown
|
|
|
|
|
+<!-- bbox: [160, 126, 590, 161] -->
|
|
|
|
|
+<!-- 页眉: 广东荣德会计师事务所有限公司 -->
|
|
|
|
|
+
|
|
|
|
|
+<!-- bbox: [158, 226, 1463, 322] -->
|
|
|
|
|
+在按照审计准则执行审计工作的过程中...
|
|
|
|
|
+
|
|
|
|
|
+<!-- bbox: [251, 264, 1404, 2111] -->
|
|
|
|
|
+**资产负债表**
|
|
|
|
|
+
|
|
|
|
|
+<table>...</table>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## 使用方法
|
|
|
|
|
+
|
|
|
|
|
+### 方法1: 直接使用格式化器
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+from utils.mineru_output_formatter import MinerUOutputFormatter
|
|
|
|
|
+
|
|
|
|
|
+# 创建格式化器
|
|
|
|
|
+formatter = MinerUOutputFormatter(
|
|
|
|
|
+ output_dir="./output",
|
|
|
|
|
+ save_images=True
|
|
|
|
|
+)
|
|
|
|
|
+
|
|
|
|
|
+# 格式化并保存
|
|
|
|
|
+output_paths = formatter.format_and_save(
|
|
|
|
|
+ results=pipeline_results,
|
|
|
|
|
+ doc_name="my_document"
|
|
|
|
|
+)
|
|
|
|
|
+
|
|
|
|
|
+print(output_paths)
|
|
|
|
|
+# {
|
|
|
|
|
+# 'json': ['./output/my_document_page_001.json', ...],
|
|
|
|
|
+# 'markdown': ['./output/my_document_page_001.md', ...],
|
|
|
|
|
+# 'images': ['./output/images/xxx.png', ...]
|
|
|
|
|
+# }
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 方法2: 使用便捷函数
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+from utils.mineru_output_formatter import save_mineru_format
|
|
|
|
|
+
|
|
|
|
|
+output_paths = save_mineru_format(
|
|
|
|
|
+ results=pipeline_results,
|
|
|
|
|
+ output_dir="./output",
|
|
|
|
|
+ doc_name="my_document",
|
|
|
|
|
+ save_images=True
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 方法3: 仅转换格式(不保存)
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+from utils.mineru_output_formatter import MinerUFormatConverter
|
|
|
|
|
+
|
|
|
|
|
+# 转换为 MinerU content_list 格式
|
|
|
|
|
+content_list = MinerUFormatConverter.convert_pipeline_result_to_mineru(
|
|
|
|
|
+ pipeline_results
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## 与 MinerU 原生输出的对比
|
|
|
|
|
+
|
|
|
|
|
+| 特性 | MinerU 原生 | MinerUOutputFormatter |
|
|
|
|
|
+|------|------------|----------------------|
|
|
|
|
|
+| JSON 格式 | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| Markdown | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| bbox 注释 | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| table_body HTML | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| table_cells | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| data-bbox 属性 | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| image_rotation_angle | ✅ | ✅ 完全一致 |
|
|
|
|
|
+| skew_angle | ✅ | ✅ 完全一致 |
|
|
|
|
|
+
|
|
|
|
|
+## 坐标系统
|
|
|
|
|
+
|
|
|
|
|
+### 输出坐标
|
|
|
|
|
+
|
|
|
|
|
+- **bbox**: `[x1, y1, x2, y2]` 格式,基于**原始未旋转图片**的像素坐标
|
|
|
|
|
+- **table_cells.bbox**: 同上,基于原始图片坐标系
|
|
|
|
|
+
|
|
|
|
|
+### 旋转信息
|
|
|
|
|
+
|
|
|
|
|
+- **image_rotation_angle**: 图片旋转角度(0, 90, 180, 270)
|
|
|
|
|
+- **skew_angle**: 倾斜校正角度
|
|
|
|
|
+
|
|
|
|
|
+## 目录结构
|
|
|
|
|
+
|
|
|
|
|
+```
|
|
|
|
|
+output/
|
|
|
|
|
+├── document_page_001.json
|
|
|
|
|
+├── document_page_001.md
|
|
|
|
|
+├── document_page_002.json
|
|
|
|
|
+├── document_page_002.md
|
|
|
|
|
+└── images/
|
|
|
|
|
+ ├── table_0_251_264_xxx.jpg
|
|
|
|
|
+ ├── image_0_133_1582_xxx.jpg
|
|
|
|
|
+ └── ...
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## 注意事项
|
|
|
|
|
+
|
|
|
|
|
+1. **坐标系**: 所有坐标都是基于原始未旋转图片的坐标系
|
|
|
|
|
+2. **图片**: 保存的是原始未旋转的图片
|
|
|
|
|
+3. **表格 HTML**: 包含 `data-bbox` 属性,可用于前端渲染和坐标定位
|
|
|
|
|
+4. **单元格匹配**: `paddle_bbox_indices` 记录了匹配的 OCR 框索引
|
|
|
|
|
+
|