thrift服务化改造原理分析

╰+攻爆jí腚メ 2021-09-14 12:24 361阅读 0赞

上面文章（[http://blog.csdn.net/liuxiao723846/article/details/78846007][http_blog.csdn.net_liuxiao723846_article_details_78846007]）介绍了一个thrift的服务框架，本文介绍如何使用，以及背后的一些原理。

将上文中的项目打包成jar，引入到一个工程中，然后通过spring的配置方式模拟一个服务端和客户端。

一、thrift model：

用来定义thrift的消息和服务，然后通过maven命令生成java文件。

1）service.thrift

namespace java com.abc.ttbrain.recommend.thrift
    
    include "../model/select_feeds.thrift"
    
    	
    	service IRecallService {
    		 	list<select_feeds.FInfoModel> reCall2(1:select_feeds.InnerRequest request,2:i32 num)
    	}
    	
    	service INewRecallService extends IRecallService{
    	}

2）select\_feeds.thrift：

namespace java com.abc.ttbrain.recommend.model
    
    include "user_feature.thrift"
    
    struct SelectFeedsModel {
    	1:list<FeedModel> feedIds
    }
    
    struct FeedModel {
    	1:i64 id,
    	2:double score,
    	3:string tag
    }
    
    struct InnerRequest {
    			1:user_feature.UserLongPrefModel userLongPerf,
    			2:user_feature.UserShortPrefModel userShortPerf,
    			3:user_feature.UserStaticPrefModel userStaticPerf,
    			4:set<i64> recHistory,
    			5:i32 feedNum,
    			6:string channelId,
    			7:PolicyModel policy,
    			8:string deviceId,
    			9:string userId,
    			10:string debug,
    			11:map<string,set<i64>> sameTag,
    			12:UserBucketModel userBucketModel,
    			13:map<string,string> innerParam,
    			14:ServiceStrategy serviceStrategy,
    			15:user_feature.UserLongPrefModel userLongQuery,
    			16:user_feature.UserShortPrefModel userShortQuery,
    			17:set<string> reverse, //负反馈
    			18:user_feature.UserLongPrefModel userLongCategory,
    			19:user_feature.UserShortPrefModel userShortCategory,
    			20:user_feature.UserLongPrefModel userLongChannel,
    			21:user_feature.UserShortPrefModel userShortChannel,
    			22:user_feature.UserLongPrefModel userLongDig,
    			23:user_feature.UserShortPrefModel userShortDig,
    			24:map<string,user_feature.UserShortPrefModel> userPrefMap,
    			25:set<string> reversePref //负兴趣,只浏览不点
    }

3）pom.xml

<?xml version="1.0"?>
    <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
      <modelVersion>4.0.0</modelVersion>
      <parent>
        <groupId>com.abc</groupId>
        <artifactId>ttbrain-recommend</artifactId>
        <version>0.0.1-SNAPSHOT</version>
      </parent>
      
      <groupId>com.abc</groupId>
      <artifactId>ttbrain-recommend-thrift</artifactId>
      <version>0.0.1-SNAPSHOT</version>
      <name>ttbrain-recommend-thrift</name>
      
      <properties>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
     <compiler-plugin.version>2.3.2</compiler-plugin.version>
     <thrift.version>0.9.2</thrift.version>
     <thrift.executable>thrift</thrift.executable>
     <thrift.outputDirectory>target/generated-sources</thrift.outputDirectory>
     </properties>
    
    
     <dependencies>
     <dependency>
     <groupId>junit</groupId>
     <artifactId>junit</artifactId>
     <scope>test</scope>
     </dependency>
     <dependency>
     <groupId>org.apache.thrift</groupId>
     <artifactId>libthrift</artifactId>
     <version>${thrift.version}</version>
     </dependency>
     </dependencies>
    
    
     <build>
     <plugins>
     <plugin>
     <artifactId>maven-antrun-plugin</artifactId>
     <version>1.7</version>
     <executions>
     <execution>
     <id>setup</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <mkdir dir="${thrift.outputDirectory}" />
     <delete includeemptydirs="true" failοnerrοr="false">
     <fileset dir="src/main/java/com/abc/ttbrain/recommend/thrift" includes="*.java" />
     <fileset dir="src/main/java/com/abc/ttbrain/recommend/model" includes="*.java" />
     </delete>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>personal-recommend-model</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <exec executable="${thrift.executable}">
     <arg value="-v" />
     <arg value="--gen" />
     <arg value="java:beans" />
     <arg value="-I" />
     <arg value="src/main/thrift/model" />
     <arg value="-o" />
     <arg value="${thrift.outputDirectory}" />
     <arg value="src/main/thrift/model/user_feature.thrift" />
     </exec>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>personal-recommend-model1</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <exec executable="${thrift.executable}">
     <arg value="-v" />
     <arg value="--gen" />
     <arg value="java:beans" />
     <arg value="-I" />
     <arg value="src/main/thrift/model" />
     <arg value="-o" />
     <arg value="${thrift.outputDirectory}" />
     <arg value="src/main/thrift/model/select_feeds.thrift" />
     </exec>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>personal-recommend-service</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <exec executable="${thrift.executable}">
     <arg value="-v" />
     <arg value="--gen" />
     <arg value="java:beans" />
     <arg value="-I" />
     <arg value="src/main/thrift/model" />
     <arg value="-I" />
     <arg value="src/main/thrift/service" />
     <arg value="-o" />
     <arg value="${thrift.outputDirectory}" />
     <arg value="src/main/thrift/service/personal_recommend_service.thrift" />
     </exec>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>feed-plugin-model</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <exec executable="${thrift.executable}">
     <arg value="-v" />
     <arg value="--gen" />
     <arg value="java:beans" />
     <arg value="-I" />
     <arg value="src/main/thrift/model" />
     <arg value="-I" />
     <arg value="src/main/thrift/service" />
     <arg value="-o" />
     <arg value="${thrift.outputDirectory}" />
     <arg value="src/main/thrift/model/feed_plugins.thrift" />
     </exec>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>feed-plugin-service</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <exec executable="${thrift.executable}">
     <arg value="-v" />
     <arg value="--gen" />
     <arg value="java:beans" />
     <arg value="-I" />
     <arg value="src/main/thrift/model" />
     <arg value="-I" />
     <arg value="src/main/thrift/service" />
     <arg value="-o" />
     <arg value="${thrift.outputDirectory}" />
     <arg value="src/main/thrift/service/feed_plugin_service.thrift" />
     </exec>
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     <execution>
     <id>clean</id>
     <phase>generate-sources</phase>
     <configuration>
     <tasks>
     <copy todir="src/main/java">
     <fileset dir="${thrift.outputDirectory}/gen-javabean"
     includes="**/*" />
     </copy>
     <delete dir="${thrift.outputDirectory}/gen-javabean" />
     </tasks>
     </configuration>
     <goals>
     <goal>run</goal>
     </goals>
     </execution>
     </executions>
     </plugin>
     </plugins>
     <pluginManagement>
     <plugins>
     
     <plugin>
     <groupId>org.eclipse.m2e</groupId>
     <artifactId>lifecycle-mapping</artifactId>
     <version>1.0.0</version>
     <configuration>
     <lifecycleMappingMetadata>
     <pluginExecutions>
     <pluginExecution>
     <pluginExecutionFilter>
     <groupId>
     org.apache.maven.plugins
     </groupId>
     <artifactId>
     maven-antrun-plugin
     </artifactId>
     <versionRange>
     [1.7,)
     </versionRange>
     <goals>
     <goal>run</goal>
     </goals>
     </pluginExecutionFilter>
     <action>
     <ignore></ignore>
     </action>
     </pluginExecution>
     </pluginExecutions>
     </lifecycleMappingMetadata>
     </configuration>
     </plugin>
     </plugins>
     </pluginManagement>
     </build>
    
    
     <distributionManagement>
     <repository>
     <id>abc</id>
     <url>http://maven.abc.virtual/a/abc-internal-repo</url>
     </repository>
     </distributionManagement>
    </project>

运行maven命令，将该model打包成jar，引入到下面的服务端和客户端工程中。

二、服务端model：

实现thrift定义的iface接口，然后通过main函数启动进程对外提供服务。

1、java代码：

package com.abc.ttbrain.recommend.server.services.impl;
    
    @Service
    public class PersonalNewRecommend implements INewRecallService.Iface {
    	private static final Logger logger = LoggerFactory.getLogger(PersonalNewRecommend.class);  
    	
    	@Override
    	public List<FInfoModel> reCall2(InnerRequest request,int num) throws TException {
    		Long a = System.currentTimeMillis();
    		List<FInfoModel> reCallList = new LinkedList<>();
    		try {
    			//业务逻辑
    			
    		} catch (Exception e) {
    			logger.error("new recall is error...",e);
    		}
    		return reCallList;
    	}
    }

2、 spring配置文件：

1）application-thrift.xml

<?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
        xmlns:context="http://www.springframework.org/schema/context"
        xmlns:tx="http://www.springframework.org/schema/tx"
        xmlns:jaxws="http://cxf.apache.org/jaxws"
        xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/aop
        http://www.springframework.org/schema/aop/spring-aop-3.2.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.2.xsd
        http://www.springframework.org/schema/tx
        http://www.springframework.org/schema/tx/spring-tx-3.2.xsd">
        
       
     <context:property-placeholder location="classpath*:config.properties"/>
          
        <bean id="thriftZookeeper" class="com.abc.ttbrain.thriftrpc.zookeeper.ZookeeperFactory" destroy-method="close">  
            <property name="zkHosts"  value="${thrift.zkHosts}" />  
            <property name="namespace" value="${thrift.namespace}" />  
            <property name="ROOT" value="ttengine" />  
            <property name="connectionTimeout" value="300000" />  
            <property name="sessionTimeout" value="300000" />  
            <property name="singleton" value="true" />  
        </bean>  
        <bean id="sericeAddressRegister"  class="com.abc.ttbrain.thriftrpc.zookeeper.ThriftServerAddressRegisterZookeeper"  destroy-method="close">  
            <property name="zkClient" ref="thriftZookeeper" />  
        </bean>  
          
      
        <bean id="hotRecThriftServiceServerFactory" class="com.abc.ttbrain.thriftrpc.ThriftServiceServerFactory"  destroy-method="close">  
            <property name="service" ref="personalNewRecommend" />  
            <property name="port" value="9011" />  
            <property name="version" value="${thrift.new.version}" />  
            <property name="weight" value="1" />
            <property name="selectorThreads" value="10" /> 
            <property name="workerThreads" value="500" /> 
            <property name="acceptQueueSizePerThread" value="150" />   
            <property name="thriftServerAddressRegister" ref="sericeAddressRegister" />  
        </bean>
        
    </beans>

2)application.xml

3、启动 代码：

public class NewServer{
    	private static final Logger logger = LoggerFactory.getLogger(NewServer.class);
    
    	private static GenericXmlApplicationContext ac;
    	
    	@Override
    	public static void main() {
    		/*ac = new ClassPathXmlApplicationContext(
    				new String[] { "classpath:application.xml"});*/
    		ac = new GenericXmlApplicationContext();
    		ac.getEnvironment().setActiveProfiles(flag);
    		//ac.load("classpath:application.xml");
    		ac.load("classpath:application.xml","application-thrift-new.xml","application-quartz-new.xml");
    		ac.refresh();
    	}
    }

运行这个main函数后，会将服务注册到zk上，同时提供一个thrift远程服务。

三、客户端：

1、java代码：

package com.abc.ttbrain.recommend.server.services.impl;
    
    import com.abc.ttbrain.recommend.thrift.INewRecallService;
    
    @Service
    public class RecallMasterService implements IRecallMasterService {
    	private static final Logger logger = LoggerFactory.getLogger(RecallMasterService.class);
    
    	@Autowired
    	private INewRecallService.Iface newRecallService;
    	
    	@Override
    	public RecallMasterResponse recallMaster(InnerRequest innerRequest)
    			throws TException {
    		long a = System.currentTimeMillis();
    		
    		return newRecallService.reCall2(innerRequest,num);
    	}
    }

2、spring配置文件：

1）application-thrift.xml:

<?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    	   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
    	   xmlns:context="http://www.springframework.org/schema/context"
    	   xmlns:tx="http://www.springframework.org/schema/tx"
    	   xmlns:jaxws="http://cxf.apache.org/jaxws"
    	   xsi:schemaLocation="http://www.springframework.org/schema/beans
    	   http://www.springframework.org/schema/beans/spring-beans.xsd
    	   http://www.springframework.org/schema/aop
    	   http://www.springframework.org/schema/aop/spring-aop-3.2.xsd
    	   http://www.springframework.org/schema/context
    	   http://www.springframework.org/schema/context/spring-context-3.2.xsd
    	   http://www.springframework.org/schema/tx
    	   http://www.springframework.org/schema/tx/spring-tx-3.2.xsd">
    	   
    	  
    	<context:property-placeholder location="classpath*:config.properties"/>
          
        <bean id="thriftZookeeper" class="com.abc.ttbrain.thriftrpc.zookeeper.ZookeeperFactory" destroy-method="close">  
            <property name="zkHosts"  value="${thrift.zkHosts}" />  
            <property name="namespace" value="${thrift.namespace}" />  
            <property name="ROOT" value="ttengine" />  
            <property name="connectionTimeout" value="300000" />  
            <property name="sessionTimeout" value="300000" />  
            <property name="singleton" value="true" />  
        </bean>  
        
          
        
        <bean id="newFactory" class="com.abc.ttbrain.thriftrpc.ThriftServiceClientProxyFactory" destroy-method="close">  
            <property name="maxActive" value="500" />  
            <property name="maxIdle" value="500" />  
            <property name="idleTime" value="1800000" />  
            <property name="serverAddressProvider">  
                <bean class="com.abc.ttbrain.thriftrpc.zookeeper.ThriftServerAddressProviderZookeeper">  
                    <property name="service" value="com.abc.ttbrain.recommend.thrift.INewRecallService" />  
                    <property name="version" value="${thrift.new.slave.version}" />  
                    <property name="zkClient" ref="thriftZookeeper" />  
                </bean>  
            </property>  
        </bean>
    </beans>

2)application.xml

<?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    	   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
    	   xmlns:context="http://www.springframework.org/schema/context"
    	   xmlns:mvc="http://www.springframework.org/schema/mvc"
    	   xmlns:tx="http://www.springframework.org/schema/tx"
    	   xmlns:jaxws="http://cxf.apache.org/jaxws"
    	   xmlns:task="http://www.springframework.org/schema/task"
    	   xsi:schemaLocation="http://www.springframework.org/schema/beans
    	   http://www.springframework.org/schema/beans/spring-beans.xsd
    	   http://www.springframework.org/schema/aop
    	   http://www.springframework.org/schema/aop/spring-aop-3.2.xsd
    	   http://www.springframework.org/schema/context
    	   http://www.springframework.org/schema/context/spring-context-3.2.xsd
    	   http://www.springframework.org/schema/mvc
    	   http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd
    	   http://www.springframework.org/schema/tx
    	   http://www.springframework.org/schema/tx/spring-tx-3.2.xsd
    	   http://www.springframework.org/schema/task
           http://www.springframework.org/schema/task/spring-task-3.2.xsd">
    	   
    	
    	<task:annotation-driven/>
        
        <context:component-scan base-package="com.abc.ttbrain.recommend.server" />
    	   
    	
    	<context:component-scan base-package="com.abc.ttbrain" >
    		<context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/> 
    		<context:exclude-filter type="regex" expression="com.abc.ttbrain.base.dao.db.impl.*"/> 
    		<context:exclude-filter type="regex" expression="com.abc.ttbrain.base.dao.couchbase.impl.*"/> 
    		<context:exclude-filter type="regex" expression="com.abc.ttbrain.base.dao.hbase.impl.*"/>
    	</context:component-scan>
    </beans>

3、客户端启动：

public class RecallMasterServer {
    	private static final Logger logger = LoggerFactory.getLogger(RecallMasterServer.class);
    	private static ApplicationContext ac;
    	
    	public static void main(String[] args)  {
    		ac = new ClassPathXmlApplicationContext(
    				new String[] { "classpath:application.xml","application-thrift.xml"});
    	}
    }

4、客户端代码分析：

1、依赖注入：

在RecallMasterService 中，有如下代码：

@Autowired  
private INewRecallService.Iface newRecallService;  
按照正常spring的思维来分析，spring会找到一个实现了INewRecallService.Iface接口的实现类来注入到RecallMasterService 中，我们通过源码可以知道INewRecallService.Iface就是thrift生成的一个接口，具体实现在远端Server上。那么，上面提到的自己封装的thrift服务框架起到了很大的作用。具体做法是：main函数启动后，依靠spring拉起了application-thrift.xml中com.abc.ttbrain.thriftrpc.ThriftServiceClientProxyFactory框架代码，我们进去可以看到，该类实现了spring的FactoryBean, InitializingBean两个接口，在afterPropertiesSet()方法中，通过反射加载了INewRecallService.Iface类，以及对其进行了动态代理，所以这里代理类很自然的实现了INewRecallService.Iface接口，又因为com.abc.ttbrain.thriftrpc.ThriftServiceClientProxyFactory实现了FactoryBean接口，所以可以返回代理类，这样spring就很自然的能够找到INewRecallService.Iface接口的实现类了，并进行注入。

2、代理实现了远程方法的调用；

根据上面分析可以知道，我们在RecallMasterService的方法中，调用注入进来的接口INewRecallService.Iface的任何方法，都会被代理到指定的代理中。这样，我们就可以在代理方法中使用thrift api来调用远程的Server了，而且很好的屏蔽了这些复杂的细节，在RecallMasterService中就像调用本地方法一样使用。

[http_blog.csdn.net_liuxiao723846_article_details_78846007]: http://blog.csdn.net/liuxiao723846/article/details/78846007