【分布式云储存】Springboot微服务接入MinIO实现文件服务

文章目录 前言技术回顾准备工作申请accessKey\secretKey创建数据存储桶公共资源直接访问测试 接入springboot实现文件服务依赖引入配置文件MinIO配置MinIO工具类 OkHttpSSLSocketClient兼容ssl静态资源预览解决方案资源上传预览测试测试结果

前言

上篇博客我们介绍了分布式云存储MinIO作业环境的搭建，以及分布式云储存MinIO在实际的文件服务中的优势。那么，今天我们就小试牛刀来将MinIO接入我们的微服务项目，实现一个分布式的文件服务器。

技术回顾

MinIO 提供高性能、与S3 兼容的对象存储系统，让你自己能够构建自己的私有云储存服务。 MinIO原生支持 Kubernetes，它可用于每个独立的公共云、每个 Kubernetes 发行版、私有云和边缘的对象存储套件。 MinIO是软件定义的，不需要购买其他任何硬件，在 GNU AGPL v3 下是 100% 开源的。

准备工作 申请accessKey\secretKey

MinIO控制台申请accessKey\secretKey http://your_hostname:18001~18004/login minio/minio123

创建数据存储桶

创建数据存储桶

点击创建的桶我们进行访问策略配置

访问策略有private\custom\public public 公共的桶，任何人都可以访问资源，直接映射为静态资源，可直接提供预览和下载 custom 自定义桶，用户根据自身需求定义访问规则 private 私有的桶，需要授权才能访问

根据一般的生产需求，我们定义一个private，一个custom桶。private保存私有资源，custom保存公共资源并禁止目录访问。

private 规则：

custom 规则：

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "*" ] }, "Action": [ "s3:GetBucketLocation", "s3:ListBucketMultipartUploads" ], "Resource": [ "arn:aws:s3:::sacpublic" ] }, { "Effect": "Allow", "Principal": { "AWS": [ "*" ] }, "Action": [ "s3:AbortMultipartUpload", "s3:DeleteObject", "s3:GetObject", "s3:ListMultipartUploadParts", "s3:PutObject" ], "Resource": [ "arn:aws:s3:::sacpublic/*" ] } ] } 公共资源直接访问测试

私有资源代码博文最后测试 公共资源直接访问测试：

接入springboot实现文件服务 依赖引入

maven引入依赖

<!--minio--> <dependency> <groupId>io.minio</groupId> <artifactId>minio</artifactId> <version>8.2.0</version> </dependency> <!--minio--> 配置文件

配置文件

minio.url = 10.10.22.91:9100 minio.accessKey = fUIXbkBZ9UQTHOOZXNGW minio.secretKey = sy1RAgItAOk9pk1gE7FbrPYzsZI87CfpGkuoY0KW minio.buckets.public = sacpublic minio.buckets.private = sacprivate

注意：接口调用minio url 为nginx映射出来的9000端口 http://your_hostname:9000

MinIO配置

MinIO配置

/** * MinioConfig * @author senfel * @version 1.0 * @date 2023/9/14 11:37 */ @Configuration @ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true) public class MinioConfig { @Value("${minio.url}") private String minioUrl; @Value("${minio.accessKey}") private String accessKey; @Value("${minio.secretKey}") private String secretKey; @Bean public MinioClient getMinioClient() { return MinioClient.builder().endpoint(url) .credentials(accessKey, secretKey).build(); } } MinIO工具类

MinIO工具类

/** * MinioUtil * @author senfel * @version 1.0 * @date 2023/9/14 10:26 */ @Component @ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true) public class MinioUtil { @Resource private MinioClient minioClient; /** * 创建一个桶 */ public void createBucket(String bucket) throws Exception { boolean found = minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build()); if (!found) { minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build()); } } /** * 上传一个文件 */ public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception { String prefix = objectName.substring(objectName.lastIndexOf(".") + 1); //静态资源预览解决方案 //String contentType = ViewContentType.getContentType(prefix); ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName) .contentType(contentType) .stream(stream, -1, 10485760).build()); return objectWriteResponse; } /** * 列出所有的桶 */ public List<String> listBuckets() throws Exception { List<Bucket> list = minioClient.listBuckets(); List<String> names = new ArrayList<>(); list.forEach(b -> { names.add(b.name()); }); return names; } /** * 下载一个文件 */ public InputStream download(String bucket, String objectName) throws Exception { InputStream stream = minioClient.getObject( GetObjectArgs.builder().bucket(bucket).object(objectName).build()); return stream; } /** * 删除一个桶 */ public void deleteBucket(String bucket) throws Exception { minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucket).build()); } /** * 删除一个对象 */ public void deleteObject(String bucket, String objectName) throws Exception { minioClient.removeObject(RemoveObjectArgs.builder().bucket(bucket).object(objectName).build()); } /** * 复制文件 * * @Param: [sourceBucket, sourceObject, targetBucket, targetObject] * @return: void * @Date: 2021/11/15 */ public void copyObject(String sourceBucket, String sourceObject, String targetBucket, String targetObject) throws Exception { this.createBucket(targetBucket); minioClient.copyObject(CopyObjectArgs.builder().bucket(targetBucket).object(targetObject) .source(CopySource.builder().bucket(sourceBucket).object(sourceObject).build()).build()); } /** * 获取文件信息 * * @Param: [bucket, objectName] * @return: java.lang.String */ public String getObjectInfo(String bucket, String objectName) throws Exception { return minioClient.statObject(StatObjectArgs.builder().bucket(bucket).object(objectName).build()).toString(); } /** * 生成一个给HTTP GET请求用的presigned URL。浏览器/移动端的客户端可以用这个URL进行下载，即使其所在的存储桶是私有的。 * @Param: [bucketName, objectName, expires] * @return: java.lang.String */ public String getPresignedObjectUrl(String bucketName, String objectName, Integer expires) throws Exception { GetPresignedObjectUrlArgs build = GetPresignedObjectUrlArgs .builder().bucket(bucketName).object(objectName).expiry(expires).method(Method.GET).build(); return minioClient.getPresignedObjectUrl(build); } } OkHttpSSLSocketClient兼容ssl

对于改解决方案一般生产环境都有固定的域名和匹配的ssl证书，如果也不用https我们代码则不用兼容ssl。但是如果我们配置了不可信任的ssl，这里我们则需要进行ssl兼容方案。

/** * MinioConfig * @author senfel * @version 1.0 * @date 2023/9/14 11:37 */ @Configuration @ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true) public class MinioConfig { @Value("${minio.url}") private String minioUrl; @Value("${minio.accessKey}") private String accessKey; @Value("${minio.secretKey}") private String secretKey; @Bean public MinioClient getMinioClient() { OkHttpClient okHttpClient = new OkHttpClient.Builder() .sslSocketFactory(OkHttpSSLSocketClient.getSSLSocketFactory(),OkHttpSSLSocketClient.getX509TrustManager()) // //通过sslSocketFactory方法设置https证书 .hostnameVerifier(OkHttpSSLSocketClient.getHostnameVerifier()) .build(); return MinioClient.builder().endpoint(minioUrl).httpClient(okHttpClient) .credentials(accessKey, secretKey).build(); } } /** * OkHttpSSLSocketClient * @author senfel * @version 1.0 * @date 2023/9/15 10:07 */ public class OkHttpSSLSocketClient{ //获取SSLSocketFactory public static SSLSocketFactory getSSLSocketFactory() { try { SSLContext sslContext = SSLContext.getInstance("SSL"); sslContext.init(null, getTrustManager(), new SecureRandom()); return sslContext.getSocketFactory(); } catch (Exception e) { throw new RuntimeException(e); } } //获取TrustManager private static TrustManager[] getTrustManager() { TrustManager[] trustAllCerts = new TrustManager[]{ new X509TrustManager() { @Override public void checkClientTrusted(X509Certificate[] chain, String authType) { } @Override public void checkServerTrusted(X509Certificate[] chain, String authType) { } @Override public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[]{}; } } }; return trustAllCerts; } //获取HostnameVerifier，验证主机名 public static HostnameVerifier getHostnameVerifier() { HostnameVerifier hostnameVerifier = (s, sslSession) -> true; return hostnameVerifier; } //X509TrustManager：证书信任器管理类 public static X509TrustManager getX509TrustManager() { X509TrustManager x509TrustManager = new X509TrustManager() { //检查客户端的证书是否可信 @Override public void checkClientTrusted(X509Certificate[] chain, String authType) { } //检查服务器端的证书是否可信 @Override public void checkServerTrusted(X509Certificate[] chain, String authType) { } @Override public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } }; return x509TrustManager; } } 静态资源预览解决方案

由于我们使用API上传，minio不能区分我们资源的类型，如果资源类型不对则不能正确提供预览功能，都直接变成为访问资源链接就下载了。

所以，在实际的项目集成中我们在上传资源前需要将资源类型写入请求中，方便minio解析并提供预览和下载功能。

/** * 上传一个文件 */ public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception { String prefix = objectName.substring(objectName.lastIndexOf(".") + 1); //静态资源预览解决方案 String contentType = ViewContentType.getContentType(prefix); ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName) .contentType(contentType) .stream(stream, -1, 10485760).build()); return objectWriteResponse; } /** * ViewContentType * @author senfel * @version 1.0 * @date 2023/9/14 17:27 */ public enum ViewContentType { DEFAULT("default","application/octet-stream"), JPG("jpg", "image/jpeg"), TIFF("tiff", "image/tiff"), GIF("gif", "image/gif"), JFIF("jfif", "image/jpeg"), PNG("png", "image/png"), TIF("tif", "image/tiff"), ICO("ico", "image/x-icon"), JPEG("jpeg", "image/jpeg"), WBMP("wbmp", "image/vnd.wap.wbmp"), FAX("fax", "image/fax"), NET("net", "image/pnetvue"), JPE("jpe", "image/jpeg"), RP("rp", "image/vnd.rn-realpix"); private String prefix; private String type; public static String getContentType(String prefix){ if(StringUtils.isEmpty(prefix)){ return DEFAULT.getType(); } prefix = prefix.substring(prefix.lastIndexOf(".") + 1); for (ViewContentType value : ViewContentType.values()) { if(prefix.equalsIgnoreCase(value.getPrefix())){ return value.getType(); } } return DEFAULT.getType(); } ViewContentType(String prefix, String type) { this.prefix = prefix; this.type = type; } public String getPrefix() { return prefix; } public String getType() { return type; } } 资源上传预览测试

MinIO对于图片等资源可以直接预览，对于excel文档等直接提供下载功能

@SpringBootTest @RunWith(SpringJUnit4ClassRunner.class) public class CodeDbInfoServiceTests { @Resource @Lazy private MinioUtil minioUtil; /** * upload * @author senfel * @date 2023/9/27 10:00 * @return void */ @Test public void upload() throws Exception { FileInputStream fileInputStream = new FileInputStream("C:/Users/dev/Desktop/minio.png"); ObjectWriteResponse sacprivate = minioUtil.uploadFile(fileInputStream, "sacprivate", "test/minio.png"); System.err.println(sacprivate.bucket()); } /** * 获取私库连接 * @author senfel * @date 2023/9/27 10:01 * @return void */ @Test public void getPrivateUrl() throws Exception{ String url = minioUtil.getPresignedObjectUrl("sacprivate", "test/minio.png", 60); System.err.println(url); } } 测试结果

图片成功上传

私库直接预览访问失败

获取私库链接过期时间为60s

访问成功

60s后再次访问该链接，提示链接已经过期

至此minio接入并测试完成，对于公开桶的资源直接可任意访问不用获取有效期链接。

写在最后 Springboot微服务接入MinIO实现文件服务较为简单，我们只要按照官方文档引入依赖调用即可。值得注意的是我们根据需求选择接入ssl兼容方案和静态资源预览功能。当然，minio的private\custom\public可以完全实现我们各种需求，可以完全替代市场上付费和笨重的分布式服务。