Spring Boot实现分片上传、断点续传与进度条功能_Java

spring boot 实现分片上传、断点续传与进度条

# spring boot 实现分片上传、断点续传与进度条  
## —— 支持 minio / rustfs / seaweedfs 可配置切换
在大文件上传场景下，传统单次上传存在明显问题：
- 文件过大，失败后需要整体重传
- 网络不稳定，用户体验差
- 无法展示上传进度
本文基于 **spring boot**，实现一套**生产可用**的大文件上传方案，支持：
- 分片上传
- 断点续传
- 上传进度查询
- minio / rustfs / seaweedfs / 本地存储
- 通过 yml 配置文件切换存储类型
---
## 一、整体架构设计
系统整体采用「接口隔离 + 策略模式」设计：

controller ↓ uploadservice ↓ storageservice（统一抽象） ↓ minio / rustfs / seaweedfs / local

**核心思想：上传逻辑与底层存储解耦。**
---
## 二、统一配置设计
### 1. 存储类型切换
```yaml
file:
  path: file/
  prefix: pre
  domain: domain/
  storage:
    type: minio   # minio / rustfs / seaweedfs / local

只需修改 storage.type，即可切换存储实现。

2. minio 配置

minio:
  url: http://localhost:9000
  accesskey: minioadmin
  secretkey: minioadmin123
  bucketname: xxx

3. rustfs 配置（s3 协议）

rustfs:
  url: http://localhost:9000
  accesskey: rustfsadmin
  secretkey: rustfsadmin
  bucketname: xxx

rustfs 兼容 s3 协议，可直接复用 minio sdk。

4. seaweedfs 配置

seaweedfs:
  url: http://127.0.0.1:8333
  accesskey: weed
  secretkey: weed
  bucketname: xxx

三、分片上传核心流程

1. 前端切片思路

前端将大文件切割为多个分片（如 5mb）：

file
├── chunk_0
├── chunk_1
├── chunk_2
└── chunk_n

每个分片上传时携带：

文件唯一标识（guid / md5）
分片索引（chunkindex）
总分片数（totalchunk）

2. 分片上传接口

post /file/chunk/upload

四、断点续传实现

1. 查询已上传分片

get /file/chunk/uploaded?guid=xxx

返回示例：

[0,1,3,5]

前端只上传缺失分片即可。

2. 实现原则

是否已上传以「存储层」为准
不依赖内存或 redis
服务重启不影响续传

五、上传进度计算

进度 = 已上传分片数 / 总分片数 × 100%

示例返回：

{
  "uploaded": 6,
  "total": 10,
  "percent": 60
}

六、存储层抽象设计

1. 统一接口定义

public interface storageservice {
    void uploadchunk(string path, inputstream inputstream);
    boolean exists(string path);
    list<integer> listchunks(string prefix);
    void mergechunks(string chunkprefix, string targetpath, int totalchunk);
    void deletechunks(string chunkprefix);
}

2. minio / rustfs 实现（s3 通用）

@slf4j
public class miniostorageservice implements storageservice {
    private final minioclient minioclient;
    private final string bucket;
    public miniostorageservice(minioclient client, string bucket) {
        this.minioclient = client;
        this.bucket = bucket;
    }
    @override
    public void uploadchunk(string path, inputstream inputstream) {
        try {
            minioclient.putobject(
                putobjectargs.builder()
                    .bucket(bucket)
                    .object(path)
                    .stream(inputstream, -1, 5 * 1024 * 1024)
                    .build()
            );
        } catch (exception e) {
            throw new runtimeexception("分片上传失败", e);
        }
    }
    @override
    public boolean exists(string path) {
        try {
            minioclient.statobject(
                statobjectargs.builder()
                    .bucket(bucket)
                    .object(path)
                    .build()
            );
            return true;
        } catch (exception e) {
            return false;
        }
    }
    @override
    public list<integer> listchunks(string prefix) {
        list<integer> chunks = new arraylist<>();
        iterable<result<item>> results =
            minioclient.listobjects(
                listobjectsargs.builder()
                    .bucket(bucket)
                    .prefix(prefix)
                    .build()
            );
        for (result<item> r : results) {
            string name = r.get().objectname();
            chunks.add(integer.parseint(
                name.substring(name.lastindexof("_") + 1)));
        }
        return chunks;
    }
    @override
    public void mergechunks(string chunkprefix,
                            string targetpath,
                            int totalchunk) {
        try {
            bytearrayoutputstream out = new bytearrayoutputstream();
            for (int i = 0; i < totalchunk; i++) {
                inputstream in = minioclient.getobject(
                    getobjectargs.builder()
                        .bucket(bucket)
                        .object(chunkprefix + "/chunk_" + i)
                        .build()
                );
                ioutils.copy(in, out);
            }
            uploadchunk(
                targetpath,
                new bytearrayinputstream(out.tobytearray()));
        } catch (exception e) {
            throw new runtimeexception("分片合并失败", e);
        }
    }
    @override
    public void deletechunks(string chunkprefix) {
        // 可按需实现批量删除
    }
}

七、业务 service 实现

@service
public class fileuploadserviceimpl implements fileuploadservice {
    @autowired
    private storageservice storageservice;
    @override
    public void uploadchunk(chunkuploaddto dto) throws ioexception {
        string path = dto.getguid() + "/chunk_" + dto.getchunkindex();
        if (storageservice.exists(path)) {
            return;
        }
        storageservice.uploadchunk(
            path, dto.getfile().getinputstream());
    }
    @override
    public list<integer> uploadedchunks(string guid) {
        return storageservice.listchunks(guid + "/");
    }
    @override
    public void merge(string guid, int totalchunk) {
        storageservice.mergechunks(
            guid, guid + ".final", totalchunk);
        storageservice.deletechunks(guid + "/");
    }
}

八、controller 接口

@restcontroller
@requestmapping("/file/chunk")
public class fileuploadcontroller {
    @autowired
    private fileuploadservice fileuploadservice;
    @postmapping("/upload")
    public void upload(chunkuploaddto dto) throws ioexception {
        fileuploadservice.uploadchunk(dto);
    }
    @getmapping("/uploaded")
    public list<integer> uploaded(@requestparam string guid) {
        return fileuploadservice.uploadedchunks(guid);
    }
    @postmapping("/merge")
    public void merge(@requestparam string guid,
                      @requestparam integer totalchunk) {
        fileuploadservice.merge(guid, totalchunk);
    }
}