springcloud api调用链路监控设计思路

前言

目前springcloud生态圈中有zipkin系统用于调用链路获取展示。
Zipkin分布式跟踪系统;它可以帮助收集时间数据,解决在microservice架构下的延迟问题;它管理这些数据的收集和查找;Zipkin的设计是基于谷歌的Google Dapper论文。
每个应用程序向Zipkin报告定时数据,Zipkin UI呈现了一个依赖图表来展示多少跟踪请求经过了每个应用程序;如果想解决延迟问题,可以过滤或者排序所有的跟踪请求,并且可以查看每个跟踪请求占总跟踪时间的百分比。
zipkin提供rest api为其他服务使用,rest api文档

由于zipkin提供的接口信息不能满足我们所需,这里我们选择自己设计开发以获取更多监控api的信息。

监控目标:

  • 1.调用链路获取(服务名称 接口名称 调用顺序)

  • 2.各接口耗费时长,所占总时长百分比

设计思路:

通过zuul过滤器和各服务的切面编程监听controller方法

服务调用方式:

15064F3C-9898-4AD2-ACC7-AC0FF5046C10.png
前提:此处服务之间调用基于eureka提供的服务信息列表,使用feign直接调用远程服务,不再经过zuul转发调用。

探针思路:

DCE31B49-F4A0-4767-B940-B58AA4F15FD4.png

  • 我们在zuul PRE过滤器中设置一个随机数root-id以标识这些请求是属于同一个请求,zuul经过负载均衡算法将请求转发到服务B1;

  • 在B1的controller @Aspect切面的@Around方法中将root-id设为root-id和parent-id,设置一个随机数为request.attributekey为request-id的值(由于不能直接设置request header,需要通过attribute键值做一次转发),在B1的RequestInterceptor 请求拦截中将parent-id request-id属性值放置在请求头中,在@Around方法完成前将root-id、parent-id、request-id、服务名称、调用开始时间、调用结束时间发送到rabbitmq;

  • 当请求调用serviceA1中时,将请求头中的request-id的值设置为parent-id,设置一个随机数为request.attribute key为request-id的值,同样在@Around方法完成前将root-id、parent-id、request-id、服务名称、、调用开始时间、调用结束时间发送到rabbitmq;

  • 当请求调用serviceC2中时,处理如A1。

  • 当各服务调用完成时会进入zuul POST过滤器中,我们将root-id、服务开始时间、服务结束时间、服务耗时发送到rabbitmq中。当一个请求完成时,我们接收到rabbitmq消息,其中包含了root-id(一次请求唯一标示)、请求开始时间、请求结束时间、请求耗时,使用parent-id request-id将此请求的一系列服务调用关系画成树状调用结构,其中包含了各服务的调用链路、每个服务所占用时长。

备注:

此处root-id parent-id request-id是一个不可重复的随机数,设计思路可参考 时间戳+redis自增 实现,每天设置一个redis自增的key(redis是单线程 不会出现并发重复的问题)
为了避免处理耗时影响api请求响应时间,使用rabbitmq进行异步统计处理

样例数据

B8783DDC-3423-44F7-A5EF-A222D6AA537D.png

服务调用时间图谱

AD8F789D-27F9-4D8E-A25C-9AEB0B5A4D56.png

代码实现:

zuul过滤器

PreFilter

@Component
public class PreFilter extends ZuulFilter {
    @Override
    public String filterType() {
        return "pre";
    }
    @Override
    public int filterOrder() {
        return 0;
    }
    @Override
    public boolean shouldFilter() {
        return true;
    }
    @Override
    public Object run() {
        RequestContext ctx = RequestContext.getCurrentContext();
        //设置初始时间
        ctx.set("startTime",System.currentTimeMillis());
        //设置请求id

        String reqId = System.currentTimeMillis()+String.format("%0" + 3 + "d", new Random().nextInt(1000) );
        ctx.addZuulRequestHeader("root-id",reqId);
        ctx.addZuulRequestHeader("request-id",reqId);
        ctx.set("root-id",reqId);
        return null;
    }
}

PostFilter

package com.kaicom.mes.monitor.filter;

import com.kaicom.mes.monitor.mq.ApiInfo;
import com.kaicom.mes.monitor.mq.MqClient;
import com.kaicom.mes.util.GsonUtil;
import com.netflix.zuul.ZuulFilter;
import com.netflix.zuul.context.RequestContext;
import org.apache.commons.io.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.netflix.zuul.filters.support.FilterConstants;
import org.springframework.stereotype.Component;

import javax.servlet.http.HttpServletRequest;
import java.io.InputStream;
import java.net.URL;
import java.util.Map;

/**
 * @Author: BillYu
 * @Description:
 * @Date: Created in 下午3:57 2018/11/20.
 */
@Component
public class PostFilter extends ZuulFilter {
    @Autowired
    private MqClient mqClient;

    @Override
    public String filterType() {
        return FilterConstants.POST_TYPE;
    }

    @Override
    public int filterOrder() {
        return FilterConstants.SEND_RESPONSE_FILTER_ORDER - 1;
    }

    @Override
    public boolean shouldFilter() {
        return true;
    }

    @Override
    public Object run() {
        RequestContext context = RequestContext.getCurrentContext();
        HttpServletRequest request = context.getRequest();

        Map<String, String[]> params = request.getParameterMap();
        //请求开始时间
        long startTime = (long) context.get("startTime");
        //请求的uri
        String uri = request.getServletPath();
        //请求的状态
        int statusCode = context.getResponseStatusCode();
        ApiInfo apiInfo = new ApiInfo();
        apiInfo.setEndTime(System.currentTimeMillis());
        apiInfo.setStartTime(startTime);
        apiInfo.setUri(uri);
        apiInfo.setStatus(statusCode);

        apiInfo.setRootId(Long.decode(context.get("root-id").toString()));
        mqClient.send(GsonUtil.toJson(apiInfo));

        return null;
    }
}

rabbitmq配置和客户端方法参考另一篇博客

服务请求请求头转发配置

package com.kaicom.mes.equipment.config;

import feign.RequestInterceptor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.StringUtils;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;

import javax.servlet.http.HttpServletRequest;
import java.util.Enumeration;

/**
 * @Author: BillYu
 * @Description: 转发请求头
 * @Date: Created in 下午3:03 2018/7/10.
 */
@Configuration
public class FeignConfig {
    private Logger logger = LoggerFactory.getLogger(FeignConfig.class);

    @Bean
    public RequestInterceptor headerInterceptor() {
        return template -> {
            ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
            HttpServletRequest request = attributes.getRequest();
            Enumeration<String> headerNames = request.getHeaderNames();
            if (headerNames != null) {
                while (headerNames.hasMoreElements()) {
                    String name = headerNames.nextElement();
                    String values = request.getHeader(name);
                    template.header(name, values);
                }
            }
            System.err.println("feign===> requestId:"+request.getAttribute("request-id"));
            //转发attribute中设置的requestId
            if(!StringUtils.isEmpty(request.getAttribute("request-id"))){
                template.header("request-id",String.valueOf(request.getAttribute("request-id")));
            }
        };
    }
}

controller切面编程

package com.kaicom.mes.equipment.monitor.aspect;

import com.kaicom.mes.equipment.monitor.mq.ApiLink;
import com.kaicom.mes.equipment.monitor.mq.MqClient;
import com.kaicom.mes.util.GsonUtil;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;

import javax.servlet.http.HttpServletRequest;
import java.util.Date;
import java.util.Random;

/**
 * @Author: BillYu
 * @Description:
 * @Date: Created in 下午2:41 2018/11/27.
 */
@Aspect
@Component
public class ControllerAspect {
    private Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private Environment env;

    @Autowired
    private MqClient mqClient;

    @Pointcut("execution(public * com.kaicom.mes.equipment.controller..*.*(..))")
    public void apiLink(){

    }

    @Around("apiLink()")
    public Object doAround(ProceedingJoinPoint point)  throws Throwable{
        ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
        HttpServletRequest request = attributes.getRequest();
        String reqId = System.currentTimeMillis()+String.format("%0" + 3 + "d", new Random().nextInt(1000) );
        request.setAttribute("request-id",reqId);
        ApiLink apiLink = new ApiLink();
        apiLink.setAppName(env.getProperty("spring.application.name"));
        apiLink.setRequestId(Long.decode(reqId));
        apiLink.setRootId(Long.decode(request.getHeader("root-id").toString()));
        apiLink.setParentId(Long.decode(request.getHeader("request-id").toString()));
        apiLink.setStartTime(new Date());
        apiLink.setUri(request.getRequestURI());
        //方法内部处理
        Object object = point.proceed();
        apiLink.setStatus(attributes.getResponse().getStatus());

        apiLink.setEndTime(new Date());
        apiLink.setConsumeTime(apiLink.getEndTime().getTime()-apiLink.getStartTime().getTime());
        
        mqClient.send(GsonUtil.toJson(apiLink));
        return object;


    }





}

消息处理方法

    @Override
    @RabbitListener(queues= MqLinkConfig.queue)
    public void processApiLinkMessage(String message) {
        //具体业务处理
    }

调用链路接口返回数据格式

{
  "code": 200,
  "msg": "请求成功",
  "data": {
    "root": {
      "id": 1,
      "apiId": 1,
      "uri": "/mes-equipment/sys/api/user/{param}",
      "startTime": "2018-11-28T09:13:27.000+0000",
      "endTime": "2018-11-28T09:13:28.000+0000",
      "consumeTime": 1909,
      "status": 200,
      "rootId": 1543396406507510
    },
    "links": [
      {
        "id": 3,
        "requestId": 1543396406789549,
        "rootId": 1543396406507510,
        "parentId": 1543396406507510,
        "startTime": "2018-11-28T09:13:26.000+0000",
        "endTime": "2018-11-28T09:13:28.000+0000",
        "consumeTime": 1276,
        "appName": "mes-equipment",
        "uri": "/sys/api/user/15",
        "status": 200
      },
      {
        "id": 1,
        "requestId": 1543396407827065,
        "rootId": 1543396406507510,
        "parentId": 1543396406789549,
        "startTime": "2018-11-28T09:13:27.000+0000",
        "endTime": "2018-11-28T09:13:27.000+0000",
        "consumeTime": 56,
        "appName": "mes-user",
        "uri": "/sys/user/getUserById/15",
        "status": 200
      },
      {
        "id": 2,
        "requestId": 1543396407991217,
        "rootId": 1543396406507510,
        "parentId": 1543396406789549,
        "startTime": "2018-11-28T09:13:27.000+0000",
        "endTime": "2018-11-28T09:13:28.000+0000",
        "consumeTime": 37,
        "appName": "mes-user",
        "uri": "/sys/user/getCurrentUser",
        "status": 200
      }
    ]
  }
}

root:整个请求的耗时记录,root-id串联整个请求中各服务调用记录
links:各服务调用记录,通过parent-id request-id将各服务调用间的父子关系串联起来,形成一个调用链路

已有 13 条评论
  1. cialis buy online

    WOW just what I was searching for. Came here by searching for cialis online

    cialis buy online November 17th, 2019 at 09:30 am回复
  2. KelSuefab

    Cat Keflex Dose Mail Order Macrobid 100mg Mastercard Cephalexin Same As Ceftin Buy Cialis Cialis One Day Funziona Priligy Informacion Cialis Que Es

    KelSuefab January 16th, 2020 at 08:15 pm回复
  3. MatInfoge

    Viagra Et Mode D'Emploi http://cheapcialisir.com - Buy Cialis Levitra Wirkdauer Cialis Buy Nolvadex For Lab Rats

    MatInfoge January 27th, 2020 at 04:43 am回复
  4. LesAstona

    Viagra A Quoi Ca Sert http://buyciaonlinex.com - Buy Cialis Is Keflex Like Penicellin Buy Cialis Isotretinoin Roaccutane How To Buy

    LesAstona January 28th, 2020 at 06:12 pm回复
  5. StevWews

    Zithromax Stomach Pain http://abuycialisb.com - cialis Cialis Viagra Dosage tadalafil cialis from india Come Prendere Il Cialis

    StevWews February 5th, 2020 at 12:23 am回复
  6. LesAstona

    Falange A Propecia http://buycialisuss.com - cialis from canada Levitra How Long To Work Buy Cialis Cialis Es Bueno

    LesAstona February 8th, 2020 at 03:51 am回复
  7. LesAstona

    Propecia Scalp Fitch http://abuycialisb.com - cialis online Stendra On Sale cialis 5 mg best price usa isotretinoin 10mg in germany discount store

    LesAstona March 2nd, 2020 at 08:01 pm回复
  8. StevWews

    Stendra 100mg Where To Purchase https://buyciallisonline.com/# - Cialis Real Viagra Pills Buy Cialis Nuit Priligy

    StevWews March 4th, 2020 at 05:38 am回复
  9. Stepsok

    Amoxicillin Used In Abortions cialis Dosage Of Cephalexin cheapest cialis Evista

    Stepsok March 8th, 2020 at 11:48 am回复
  10. Stepsok

    Amoxicillin Dosage For Cats Capsule Buy Cialis Comprar Cialis De 10 Buy Cialis Viagra Alternativen Test

    Stepsok March 14th, 2020 at 02:48 pm回复
  11. Stepsok

    Amoxicillin Dosing In Infants Cialis Levitra 10 Mg Usato Cialis Looking For Viagra

    Stepsok March 16th, 2020 at 11:59 am回复
  12. Stepsok

    Pfizer Vgr 100 buy cialis generic online cheap Macrobid Next Day On Line Shop buy cialis professional What Does Lasix Do To Blood Pressure

    Stepsok March 23rd, 2020 at 09:16 pm回复
  13. Janmit

    Cialis Bruxelles Cialis Viagra Without A Presabana cialis pills Acheter Priligy Dapoxetine Etats

    Janmit March 25th, 2020 at 12:26 am回复
发表新评论