背景:

原方案:关于文件上传下载接口,服务端生成对象存储的请求临时链返回给客户端,客户端使用临时链直接请求对象存储服务,进行上传、下载文件。
image.png
弊端:无法控制上传文件大小
同时客户场地的网络环境禁止办公网直接访问对象存储服务域名。

改造后:用户携带临时链参数请求服务端,服务端使用这些参数访问对象存储服务,将文件返回给用户。在服务端做代理请求。
问题:有一道代理,文件下载耗时问题

image.png

第一版下载接口代码

使用临时链下载文件,再返回响应。整个下载文件过程在服务内部静默,客户端无法感知。同时文件流处理耗时较长。

@GetMapping("/**")
    public ResponseEntity<byte[]> downloadFile(HttpServletRequest request) {
        log.info("url:{}, queryString:{}", request.getRequestURL(), request.getQueryString());
        String url = request.getRequestURL().toString()+"?"+request.getQueryString();
        log.info("url:{}",url);
        String downloadUrl = url.replace(url.substring(0,url.indexOf(minioProperties.getBucketName())-1), minioProperties.getEndpoint());
        log.info("downloadUrl:{}",downloadUrl);
        Date startTime = new Date();
        byte[] responseBytes = restTemplate.execute(
                URI.create(downloadUrl),
                HttpMethod.GET,
                new RequestCallback() {
                    @Override
                    public void doWithRequest(ClientHttpRequest request) throws IOException {
                        // 清除默认的 Content-Type 请求头
                        request.getHeaders().remove("Content-Type");
                    }
                },
                new ResponseExtractor<byte[]>() {
                    @Override
                    public byte[] extractData(ClientHttpResponse response) throws IOException {
                        // 从响应中读取字节数组
                        InputStream inputStream = response.getBody();
                        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
                        byte[] buffer = new byte[10240];
                        int bytesRead;
                        while ((bytesRead = inputStream.read(buffer)) != -1) {
                            byteArrayOutputStream.write(buffer, 0, bytesRead);
                        }
                        return byteArrayOutputStream.toByteArray();
                    }
                }
        );
        Date downTime = new Date();
        log.info("downloadTime:{}",downTime.getTime()-startTime.getTime());
        HttpHeaders responseHeaders = new HttpHeaders();
        responseHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        responseHeaders.setContentDisposition(
                ContentDisposition.builder("attachment")
                        .filename(UrlUtils.getFileNameFromUrl(request.getRequestURL().toString()))  // 设置下载文件的名称
                        .build()
        );
        Date returnTime = new Date();
        log.info("responseTime:{}",returnTime.getTime()-downTime.getTime());
        return new ResponseEntity<>(responseBytes, responseHeaders, HttpStatus.OK);
    }

第二版下载接口代码

直接将下载的响应返回给客户端,消除了文件流处理的耗时,但是等待服务端下载完成的耗时仍然没有解决,且下载过程客户端没有感知。

@GetMapping("/**")
    public ResponseEntity<Resource> downloadFile(HttpServletRequest request) {
        log.info("url:{}, queryString:{}", request.getRequestURL(), request.getQueryString());
        String url = request.getRequestURL().toString() + "?" + request.getQueryString();
        log.info("url:{}", url);
        String downloadUrl = url.replace(url.substring(0, url.indexOf(minioProperties.getBucketName()) - 1), minioProperties.getEndpoint());
        log.info("downloadUrl:{}", downloadUrl);
        HttpHeaders responseHeaders = new HttpHeaders();
        responseHeaders.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        responseHeaders.setContentDisposition(
                ContentDisposition.builder("attachment")
                        .filename(UrlUtils.getFileNameFromUrl(request.getRequestURL().toString()))  // 设置下载文件的名称
                        .build()
        );
        // 获取文件大小,如果不可用,则设置为未知长度
        long contentLength = getContentLength(request.getRequestURL().toString());
        log.info("contentLength:{}", contentLength);
        responseHeaders.setContentLength(contentLength);

        Date startTime = new Date();
        ResponseEntity<Resource> responseEntity = storageRestTemplate.execute(
                URI.create(downloadUrl),
                HttpMethod.GET,
                requestCallback -> requestCallback.getHeaders().remove("Content-Type"),
                response -> {
                    InputStream inputStream = response.getBody();
                    byte[] bytes = IOUtils.toByteArray(inputStream);
                    return ResponseEntity.ok()
                            .headers(responseHeaders)
                            .body(new InputStreamResource(new ByteArrayInputStream(bytes)));
                }
        );
        Date downTime = new Date();
        log.info("downloadTime:{}", downTime.getTime() - startTime.getTime());

        log.info("responseTime:{}", downTime.getTime() - startTime.getTime());
        return responseEntity;
    }

第三版下载接口代码

使用minioClient获取对象的GetObjectResponse,这是一个inputSream,使用这个inputStream返回ResponseEntity。响应体是一个包装了文件流的 InputStreamResource。这样客户端的响应和直接访问对象存储服务是相同的,直接就会有文件流返回,无需再等服务端完全下载文件完成。

@GetMapping("/**")
    public ResponseEntity<InputStreamResource> downloadFile(HttpServletRequest request) throws MinioException {
        try {
            log.info("url:{}, queryString:{}", request.getRequestURL(), request.getQueryString());
            String url = request.getRequestURL().toString() + "?" + request.getQueryString();
            log.info("url:{}", url);
            String downloadUrl = url.replace(url.substring(0, url.indexOf(minioProperties.getBucketName()) - 1), minioProperties.getEndpoint());
            log.info("downloadUrl:{}", downloadUrl);
            //尝试用下载链接下载文件
            if (!checkUrlSafety(downloadUrl)) {
                log.error("下载链接{}不合法", downloadUrl);
                return ResponseEntity.status(403).build();
            }
            //从链接中获取对象存储信息
            String[] urlArr = request.getRequestURL().toString().split("/");
            if (urlArr.length < 3) {
                //未知长度
                return ResponseEntity.badRequest().build();
            }
            String bucketName = urlArr[urlArr.length - 3];
            String objectId = urlArr[urlArr.length - 2] + "/" + urlArr[urlArr.length - 1];
            //通过minioClient获取响应流
            GetObjectResponse getObjectResponse = minioUtil.getObject(bucketName, objectId);
            InputStream inputStream = (InputStream) getObjectResponse;
            //设置下载文件名
            HttpHeaders headers = new HttpHeaders();
            headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" + UrlUtils.getFileNameFromUrl(request.getRequestURL().toString()));
            //获取文件大小
            long contentLength = minioUtil.statObject(bucketName, objectId).size();
            return ResponseEntity.ok()
                    .headers(headers)
                    .contentType(MediaType.APPLICATION_OCTET_STREAM)
                    .contentLength(contentLength)
                    .body(new InputStreamResource(inputStream));
        } catch (Exception e) {
            return ResponseEntity.status(500).body(null);
        }
    }

    /**
     * 校验链接安全
     * @param urlToCheck
     * @return
     */
    private boolean checkUrlSafety(String urlToCheck) {
        try {
            URL url = new URL(urlToCheck);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setRequestMethod("GET");
            int responseCode = connection.getResponseCode();
            if (responseCode == HttpURLConnection.HTTP_OK) {
                //响应code为200
                return true;
            }
        } catch (IOException e) {
            System.out.println("Error checking URL: " + e.getMessage());
        }
        return false;
    }

客户端感知:

当返回为文件流输出时播放器会直接有加载进度条,不会需要等较长时间一次性加载完成。可以直接进行播放,不需要等文件下