第03讲：MinIO分布式文件服务器

编程入门行业动态更新时间:2024-10-27 01:23:22

第03讲：MinIO<a href=https://www.elefans.com/category/jswz/34/1770120.html style= 分布式文件服务器"/>

第03讲：MinIO分布式文件服务器

一、什么是MinIO

Minio 是个基于 Golang 编写的开源对象存储套件，虽然轻量，却拥有着不错的性能。

官网地址：MinIO | High Performance, Kubernetes Native Object Storageopen in new window
官网文档地址：MinIO | The MinIO Quickstart Guideopen in new window
官网文档（中文）地址：官网中文网址open in new window 中文文档对应的是上个版本，新版本中有些内容已发生了变化，所以最好是看英文文档。
JAVA SDK API：minio java sdk api 文档

何为对象存储？

对象存储服务（ Object Storage Service，OSS ）是一种海量、安全、低成本、高可靠的云存储服务，适合存放任意类型的文件。容量和处理能力弹性扩展，多种存储类型供选择，全面优化存储成本。

对于中小型企业，如果不选择存储上云，那么 Minio 是个不错的选择，麻雀虽小，五脏俱全

二、CentOS7下安装MinIO

2.1、下载MinIO

在usr/local下创建minio文件夹，并在minio文件里面创建bin和data目录，把下载好的minio文件拷贝到bin目录里面

创建目录，命令：

mkdir /usr/local/minio && cd /usr/local/minio && mkdir bin data

效果：

下载MinIO，命令：

wget  bin
mv minio bin/

效果：

2.2、运行MinIO

赋予可执行的权限，命令：

chmod +x bin/minio

运行MinIO，命令：

./bin/minio server ./data

2.3、将 minio 添加成 Linux 的服务

cat > /etc/systemd/system/minio.service << EOF
[Unit]
Description=Minio
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/minio/bin/minio[Service]
WorkingDirectory=/usr/local/minio/
PermissionsStartOnly=true
ExecStart=/usr/local/minio/bin/minio server /usr/local/minio/data
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true[Install]
WantedBy=multi-user.target
EOF

这样就可以使用 systemctl 启停 minio，命令：

systemctl start minio   # 启动
systemctl stop minio    # 停止

运行效果：

2.4、测试

MinIO Server 成功启动后访问 http://10.211.55.9:9000，你会看到类似如下界面：

Ps：要关闭防火墙或开放9000端口

systemctl stop firewalld

输入用户名/密码 minioadmin/minioadmin 可以进入 web 管理系统：

刚打开的时候，是没有bucket桶，可以手动或者通过java代码来创建一个桶。

创建的桶默认的权限时private私有的，外部不能访问，你可以修改桶的属性，点击manage，找到Access Policy，修改权限为public即可。

三、SpringBoot整合MinIO

第1步：pom.xml中添加依赖

<dependency><groupId>com.squareup.okhttp3</groupId><artifactId>okhttp</artifactId><version>4.8.1</version>
</dependency>
<dependency><groupId>io.minio</groupId><artifactId>minio</artifactId><version>8.3.9</version>
</dependency>
<dependency><groupId>com.alibaba</groupId><artifactId>fastjson</artifactId><version>1.2.70</version>
</dependency>

第2步：application.yml

server:port: 8070
spring:servlet:multipart:max-file-size: 200MB  #设置单个文件的大小  因为springboot内置tomact的的文件传输默认为10MBmax-request-size: 500MB   #设置单次请求的文件总大小enabled: true    #千万注意要设置该参数，否则不生效
# minio 文件存储配置信息
minio:endpoint: http://10.211.55.9:9000accesskey: minioadminsecretKey: minioadmin

第3步：minio配置类和配置属性

配置属性MinioProp.java

package demo.config;import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;@Data
@Component
@ConfigurationProperties(prefix = "minio")
public class MinioProp {private String endpoint;private String accesskey;private String secretKey;
}

配置类MinioConfig.java

package demo.config;import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;@Configuration
@EnableConfigurationProperties(MinioProp.class)
public class MinioConfig {@Autowiredprivate MinioProp minioProp;@Beanpublic MinioClient minioClient() throws Exception {return MinioClient.builder().endpoint(minioProp.getEndpoint()).credentials(minioProp.getAccesskey(), minioProp.getSecretKey()).build();}
}

第4步：文件上传工具类

package demo.utils;import com.alibaba.fastjson.JSONObject;
import demo.config.MinioProp;
import io.minio.BucketExistsArgs;
import io.minio.MakeBucketArgs;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;import java.io.InputStream;@Slf4j
@Component
public class MinioUtils {@Autowiredprivate MinioClient client;@Autowiredprivate MinioProp minioProp;/*** 创建bucket*/public void createBucket(String bucketName) {BucketExistsArgs bucketExistsArgs = BucketExistsArgs.builder().bucket(bucketName).build();MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(bucketName).build();try {if (client.bucketExists(bucketExistsArgs))return;client.makeBucket(makeBucketArgs);} catch (Exception e) {log.error("创建桶失败：{}", e.getMessage());throw new RuntimeException(e);}}/*** @param file       文件* @param bucketName 存储桶* @return*/public JSONObject uploadFile(MultipartFile file, String bucketName) throws Exception {JSONObject res = new JSONObject();res.put("code", 0);// 判断上传文件是否为空if (null == file || 0 == file.getSize()) {res.put("msg", "上传文件不能为空");return res;}// 判断存储桶是否存在createBucket(bucketName);// 文件名String originalFilename = file.getOriginalFilename();// 新的文件名 = 存储桶名称_时间戳.后缀名String fileName = bucketName + "_" + System.currentTimeMillis() + originalFilename.substring(originalFilename.lastIndexOf("."));// 开始上传InputStream inputStream = file.getInputStream();PutObjectArgs args = PutObjectArgs.builder().bucket(bucketName).object(fileName).stream(inputStream,inputStream.available(),-1).build();client.putObject(args);res.put("code", 1);res.put("msg", minioProp.getEndpoint() + "/" + bucketName + "/" + fileName);return res;}
}

第5步：测试Controller

package demo.controller;import com.alibaba.fastjson.JSONObject;
import demo.utils.MinioUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;import javax.servlet.http.HttpServletRequest;@RestController
public class TestController {@Autowiredprivate MinioUtils minioUtils;@PostMapping("/uploadimg")@ResponseBodypublic String uploadimg(@RequestParam(name = "file", required = false) MultipartFile file,HttpServletRequest request) {JSONObject res = null;try {res = minioUtils.uploadFile(file, "qin");} catch (Exception e) {e.printStackTrace();res.put("code", 0);res.put("msg", "上传失败");}return res.toJSONString();}
}

第6步：前端页面upload.html

<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><title>Title</title><script src="assets/vue.min-v2.5.16.js"></script><script src="assets/axios.min.js"></script>
</head>
<body>
<div id="app"><input type="file" @change="Upload" />
</div>
<script>new Vue({el: '#app',data: {},methods: {Upload(event){const flie = event.target.files[0];// 在这里进行一系列的校验const formData = new FormData();formData.append("file",flie);axios.post('http://localhost:8070/uploadimg',formData,{'Content-type' : 'multipart/form-data'}).then(res=>{console.log(res.data);},err=>{console.log(err)})}}});
</script>
</body>
</html>

运行效果：

浏览器访问upload.html，选择文件上传

访问console响应的url，发现文件已经成功上传

<body><imgsrc="http://10.211.55.9:9000/qin/qin_1680766685084.jpg"style="width: 500px;">
</body>

四、附录

4.1、JavaSDK

@SpringBootTest
class DemoApplicationTests {@Testpublic void demo() throws Exception {// 使用 MinIO 服务的 URL 和端口，Access key 和 Secret key 创建一个 MinioClient 对象。MinioClient minioClient = MinioClient.builder().endpoint("http://127.0.0.1:9000").credentials("minioadmin", "minioadmin").build();// 检查存储桶是否已经存在boolean isExist = minioClient.bucketExists(BucketExistsArgs.builder().bucket("kongming").build());if (isExist) {System.out.println("Bucket already exists.");} else {// 创建一个名为 asiatrip 的存储桶，用于存储文件。minioClient.makeBucket(MakeBucketArgs.builder().bucket("kongming").build());}// 使用 putObject 上传一个文件到存储桶中。File file = new File("e:/BluceLee/1.jpg");InputStream inputStream = new FileInputStream(file);PutObjectArgs args = PutObjectArgs.builder().bucket("kongming").object("xiaolong.jpg").contentType("image/jpg").stream(inputStream, inputStream.available(), -1).build();minioClient.putObject(args);System.out.println("  successfully uploaded as xiaolong.png to `kongming` bucket.");}
}

4.2、完整工具类

package com.woniu.util;import io.minio.*;
import io.minio.errors.*;
import io.minio.http.Method;
import io.minio.messages.*;
// import net.minidev.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.multipart.MultipartFile;import java.io.*;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;
import java.util.concurrent.TimeUnit;public class MinioUtils1 {private static final Logger log = LoggerFactory.getLogger(MinioUtils1.class);private final String endpoint;private final String accessKey;private final String secretKey;private MinioClient minioClient;public MinioUtils1(String endpoint, String accessKey, String secretKey) {this.endpoint = endpoint;this.accessKey = accessKey;this.secretKey = secretKey;this.minioClient = MinioClient.builder().endpoint(endpoint).credentials(accessKey, secretKey).build();}/*@PostConstructprivate MinioClient client() {}
*/public boolean doesBucketExists(String bucketName) {BucketExistsArgs args = BucketExistsArgs.builder().bucket(bucketName).build();try {return minioClient.bucketExists(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 创建 bucket** @param bucketName 桶名*/public void createBucket(String bucketName) {BucketExistsArgs bucketExistsArgs = BucketExistsArgs.builder().bucket(bucketName).build();MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(bucketName).build();try {if (minioClient.bucketExists(bucketExistsArgs))return;minioClient.makeBucket(makeBucketArgs);} catch (Exception e) {log.error("创建桶失败：{}", e.getMessage());throw new RuntimeException(e);}}/*** 判断文件是否存在** @param bucketName 存储桶* @param objectName 对象* @return true：存在*/public boolean doesObjectExist(String bucketName, String objectName) {StatObjectArgs args = StatObjectArgs.builder().bucket(bucketName).object(objectName).build();try {minioClient.statObject(args);} catch (Exception e) {return false;}return true;}/*** 判断文件夹是否存在** @param bucketName 存储桶* @param objectName 文件夹名称（去掉/）* @return true：存在*/public boolean doesFolderExist(String bucketName, String objectName) {ListObjectsArgs args = ListObjectsArgs.builder().bucket(bucketName).prefix(objectName).recursive(false).build();boolean exist = false;try {Iterable<Result<Item>> results = minioClient.listObjects(args);for (Result<Item> result : results) {Item item = result.get();if (!item.isDir())continue;if (objectName.equals(item.objectName())) {exist = true;}}} catch (Exception e) {exist = false;}return exist;}/*** 通过 MultipartFile ，上传文件** @param bucketName 存储桶* @param file       文件* @param objectName 对象名*/public ObjectWriteResponse putObject(String bucketName, MultipartFile file, String objectName, String contentType) {try {InputStream inputStream = file.getInputStream();PutObjectArgs args = PutObjectArgs.builder().bucket(bucketName).object(objectName).contentType(contentType).stream(inputStream, inputStream.available(), -1).build();return minioClient.putObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 上传本地文件** @param bucketName 存储桶* @param objectName 对象名称* @param fileName   本地文件路径*/public ObjectWriteResponse putObject(String bucketName, String objectName, String fileName) {try {UploadObjectArgs args = UploadObjectArgs.builder().bucket(bucketName).object(objectName).filename(fileName).build();return minioClient.uploadObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 通过流上传文件** @param bucketName  存储桶* @param objectName  文件对象* @param inputStream 文件流*/public ObjectWriteResponse putObjectByStream(String bucketName, String objectName, InputStream inputStream) {try {PutObjectArgs args = PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(inputStream, inputStream.available(), -1).build();return minioClient.putObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 创建文件夹或目录** @param bucketName 存储桶* @param objectName 目录路径*/public ObjectWriteResponse putDirObject(String bucketName, String objectName) {PutObjectArgs args = PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(new ByteArrayInputStream(new byte[]{}), 0, -1).build();try {return minioClient.putObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 获取全部 bucket*/public List<Bucket> getAllBuckets() throws Exception {return minioClient.listBuckets();}/*** 根据 bucketName 删除信息** @param bucketName 桶名*/public void removeBucket(String bucketName) {try {minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {e.printStackTrace();}}/*** 获取⽂件外链** @param bucketName bucket名称* @param objectName ⽂件名称* @param expires    过期时间 <=7*/public String getObjectUrl(String bucketName, String objectName, Integer expires) {GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder().method(Method.GET).bucket(bucketName).object(objectName).expiry(expires)    // 单位：秒.build();try {return minioClient.getPresignedObjectUrl(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidResponseException | InvalidKeyException | NoSuchAlgorithmException | IOException | XmlParserException | ServerException e) {throw new RuntimeException(e);}}/*** 获取⽂件外链** @param bucketName bucket名称* @param objectName ⽂件名称* @param duration 过期时间* @param unit  过期时间的单位*/public String getObjectUrl(String bucketName, String objectName, int duration, TimeUnit unit) {GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder().method(Method.GET).bucket(bucketName).object(objectName).expiry(duration, unit).build();try {return minioClient.getPresignedObjectUrl(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidResponseException | InvalidKeyException | NoSuchAlgorithmException | IOException | XmlParserException | ServerException e) {throw new RuntimeException(e);}}/*** 获取文件** @param bucketName bucket名称* @param objectName ⽂件名称* @return ⼆进制流*/public InputStream getObject(String bucketName, String objectName) throws Exception {GetObjectArgs args = GetObjectArgs.builder().bucket(bucketName).object(objectName).build();return minioClient.getObject(args);}/*** 上传文件** @param bucketName bucket名称* @param objectName ⽂件名称* @param stream     ⽂件流* @throws Exception .html#putObject*/public void putObject(String bucketName, String objectName, InputStream stream) {putObjectByStream(bucketName, objectName, stream);}/*** 文件流上传文件** @param bucketName  bucket名称* @param objectName  ⽂件名称* @param stream      ⽂件流* @param size        ⼤⼩* @param contextType 类型*/public void putObject(String bucketName, String objectName, InputStream stream, long size, String contextType) {putObjectByStream(bucketName, objectName, stream);}/*** 获取文件信息** @param bucketName bucket名称* @param objectName ⽂件名称* @throws Exception .html#statObject*/public StatObjectResponse getObjectInfo(String bucketName, String objectName) {StatObjectArgs args = StatObjectArgs.builder().bucket(bucketName).object(objectName).build();try {return minioClient.statObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {throw new RuntimeException(e);}}/*** 删除文件** @param bucketName bucket名称* @param objectName ⽂件名称* @throws Exception .html#removeObject*/public void removeObject(String bucketName, String objectName) {RemoveObjectArgs args = RemoveObjectArgs.builder().bucket(bucketName).object(objectName).build();try {minioClient.removeObject(args);} catch (ErrorResponseException | InsufficientDataException | InternalException | InvalidKeyException | InvalidResponseException | IOException | NoSuchAlgorithmException | ServerException | XmlParserException e) {e.printStackTrace();}}
}