1. 程式人生 > >rpc框架--grpc-java

rpc框架--grpc-java

grpc-java 初探

環境準備

  1. 下載protobuf
  2. 新建maven專案

使用protoc命令列生成類檔案

安裝protocbuf

安裝步驟略

編譯 protoc-gen-grpc-java外掛

下載grpc-java後,執行一下命令

cd grpc-java/compile
# 編譯外掛
../gradlew java_pluginExecutable
# 測試外掛
../gradlew test

若看到BUILD SUCCESSFUL字樣,則編譯成功

編輯proto檔案

syntax = "proto3";

option java_multiple_files = true
; option java_package = "io.grpc.examples.helloworld"; # java_package 用來指定生成的java類所在的包,如果該proto檔案被指定編譯為其他語言時,java引數失效,預設包為package option java_outer_classname = "HelloWorldProto"; option objc_class_prefix = "HLW"; package helloworld; # 定義服務 service Greeter { # 定義grpc方法,並且制訂訊息的請求型別和反回型別 rpc SayHello (HelloRequest) returns (HelloReply) {} } # 訊息型別都是protocol buffer的訊息格式
message HelloRequest { string name = 1; } message HelloReply { string message = 1; }

grpc可以有四種服務型別
1. 簡單的rcp模式,client端使用stub傳送請求到服務端,然後進入等待模式,直到響應傳送回client端

rpc GetFeature(Point) returns (Feature) {}
2. 服務端流式rpc,client端傳送請求到服務端,返回訊息序列的流資料(gets a stream to read a sequence of messages back),client從返回的流資料讀取訊息,直至不再有資料,如果指定服務端返回流資料型別,需使用stream關鍵字
rpc ListFeatures(Rectangle) returns (stream Feature) {}


3. 客戶端流式rpc,client使用指定的stream方式寫訊息序列併發送給服務端,一旦客戶端寫訊息完成,會等待服務端全部讀取並做出響應
rpc RecordRoute(stream Point) returns (RouteSummary) {}
4. 雙向流式rpc,雙向流式操作獨立,客戶端和服務端可以以任何序列隨意讀寫,比如,服務端可以在等待client端傳送所有訊息之前寫響應訊息,或者服務端可以選擇讀一個訊息寫一個訊息或者其他的讀寫組合。訊息的順序都是被儲存的
rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}

生成client端和server端程式碼

使用protocol編譯器protoc和指定的grpc的一個java外掛。

使用maven的protobuf外掛生成相關程式碼
命令列:
protoc --java_out=./java/ ./proto/hello.proto

生成grpc通訊程式碼

protoc --plugin=protoc-gen-grpc-java=/Users/liuyu9/Documents/personal/golang/src/github.com/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java --grpc-java_out=./java ./proto/hello.proto   

檔案列表如下:

# GreeterGrpc包含服務端和客戶端要實現的一些基本類
-rw-r--r--  1 root  staff   7352 Oct 25 18:48 GreeterGrpc.java

# 包含了protocol buffer傳送、序列化、反序列化我們請求和響應程式碼
-rw-r--r--  1 root  staff  16207 Oct 25 18:53 HelloReply.java
-rw-r--r--  1 root  staff    501 Oct 25 18:53 HelloReplyOrBuilder.java
-rw-r--r--  1 root  staff  16192 Oct 25 18:53 HelloRequest.java
-rw-r--r--  1 root  staff    493 Oct 25 18:53 HelloRequestOrBuilder.java
-rw-r--r--  1 root  staff   3011 Oct 25 18:53 HelloWorldProto.java

編寫服務端檔案

編寫服務端檔案的工作有兩部分:
1. 重寫proto檔案定義的服務生成的服務基類,確定服務的工作內容
2. 執行grpc服務,監聽客戶端請求並做出響應

package com.weibo.dorteam.grpc;

import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;

import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;

import java.io.IOException;
import java.util.logging.Logger;

public class helloServer {
    private static final Logger logger = Logger.getLogger(helloServer.class.getName());
    private int port = 50051;
    private Server server;

    private void start() throws IOException {
      // 使用ServerBuilder來構建和啟動服務,通過使用forPort方法來指定監聽的地址和埠
      // 建立一個實現方法的服務GreeterImpl的例項,並通過addService方法將該例項納入
      // 呼叫build() start()方法構建和啟動rpcserver
      server = ServerBuilder.forPort(port)
          .addService(new GreeterImpl())
          .build()
          .start();
      logger.info("Server started, listening on " + port);

        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
              // Use stderr here since the logger may have been reset by its JVM shutdown hook.
              System.err.println("*** shutting down gRPC server since JVM is shutting down");
              helloServer.this.stop();
              System.err.println("*** server shut down");
            }
          });
        }

        private void stop() {
          if (server != null) {
            server.shutdown();
          }
        }

        /**
         * Await termination on the main thread since the grpc library uses daemon threads.
         */
        private void blockUntilShutdown() throws InterruptedException {
          if (server != null) {
            server.awaitTermination();
          }
        }

        /**
         * Main launches the server from the command line.
         */
        public static void main(String[] args) throws IOException, InterruptedException {
          final helloServer server = new helloServer();
          server.start();
          server.blockUntilShutdown();
        }

    // 我們的服務GreeterImpl繼承了生成抽象類GreeterGrpc.GreeterImplBase,實現了服務的所有方法
    private class GreeterImpl extends GreeterGrpc.GreeterImplBase {

        @Override
        public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) {
          HelloReply reply = HelloReply.newBuilder().setMessage("Hello " + req.getName()).build();
          // 使用響應監視器的onNext方法返回HelloReply
          responseObserver.onNext(reply);
          // 使用onCompleted方法指定本次呼叫已經完成
          responseObserver.onCompleted();
        }
      }
}

訊息型別,StreamObserver一個響應監視器,是伺服器調生成響應結果使用的特定介面

編寫客戶端檔案

為了呼叫服務端的方法,首先需要建立一個或兩個stub:
阻塞式stub:rpc呼叫會阻塞等待服務端響應後返回訊息或異常資訊
非阻塞式stub:非阻塞呼叫方式,響應非同步返回

package com.weibo.dorteam.grpc;

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;

import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

public class helloClient {


    private static final Logger logger = Logger.getLogger(helloClient.class.getName());

    private final ManagedChannel channel;
    private final GreeterGrpc.GreeterBlockingStub blockingStub;

    /** Construct client connecting to HelloWorld server at {@code host:port}. */
    // 首先,我們需要為stub建立一個grpc的channel,指定我們連線服務端的地址和埠
    // 使用ManagedChannelBuilder方法來建立channel
    public helloClient(String host, int port) {
        channel = ManagedChannelBuilder.forAddress(host, port)
        // Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
        // needing certificates.
        .usePlaintext(true)
        .build();
        // 使用我們從proto檔案生成的GreeterGrpc類提供的newBlockingStub方法指定channel建立stubs
        blockingStub = GreeterGrpc.newBlockingStub(channel);
    }

    public void shutdown() throws InterruptedException {
        channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
    }
    // 呼叫服務端方法
    /** Say hello to server. */
    public void greet(String name) {
        logger.info("Will try to greet " + name + " ...");
        // 建立並定製protocol buffer物件,使用該物件呼叫服務端的sayHello方法,獲得response
        HelloRequest request = HelloRequest.newBuilder().setName(name).build();
        HelloReply response;
        try {
            response = blockingStub.sayHello(request);
        // 如果有異常發生,則異常被編碼成Status,可以從StatusRuntimeException異常中捕獲
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
        logger.info("Greeting: " + response.getMessage());
    }

    /**
     * Greet server. If provided, the first element of {@code args} is the name to use in the
     * greeting.
     */
    public static void main(String[] args) throws Exception {
        helloClient client = new helloClient("localhost", 50051);
        try {
            /* Access a service running on the local machine on port 50051 */
            String user = "world";
            if (args.length > 0) {
                user = args[0]; /* Use the arg as the name to greet if provided */
            }
            client.greet(user);
        } finally {
            client.shutdown();
        }
    }
}

mvn編譯執行

服務端執行:

#  不需要引數
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer"

# 需要引數
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.args="arg0 arg1 arg2"

# 傳遞classpath
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.classpathScope=runtime

客戶端執行:

mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloClient"

使用maven依賴生成類檔案

新增pom依賴

<dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-all</artifactId>
      <version>1.0.1</version>
 </dependency>
<build> 
    <extensions> 
        <extension> 
            <groupId>kr.motd.maven</groupId> 
            <artifactId>os-maven-plugin</artifactId> 
            <version>1.4.1.Final</version> 
        </extension> 
    </extensions> 
    <plugins> 
        <plugin> 
            <groupId>org.xolstice.maven.plugins</groupId> 
            <artifactId>protobuf-maven-plugin</artifactId> 
            <version>0.5.0</version> 
            <configuration> 
                <!-- 
                  The version of protoc must match protobuf-java. If you don't depend on 
                  protobuf-java directly, you will be transitively depending on the 
                  protobuf-java version that grpc depends on. 
                --> 
                <protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}</protocArtifact> 
                <pluginId>grpc-java</pluginId> 
                <pluginArtifact>io.grpc:protoc-gen-grpc-java:0.13.2:exe:${os.detected.classifier}</pluginArtifact> 
                <protocExecutable>/Users/peng/protoc-3.0.0-beta-2-osx-x86_64/protoc</protocExecutable>
            </configuration> 
            <executions> 
                <execution> 
                    <goals> 
                        <goal>compile</goal> 
                        <goal>compile-custom</goal> 
                    </goals> 
                </execution> 
            </executions> 
        </plugin> 
    </plugins> 
</build>

注:protoc的版本需要匹配protobuf-java,如果不是直接依賴protobuf-java,則依賴grpc時,間接依賴protobuf-java,版本為3.0.0

編輯proto檔案

同上

生成相關類檔案

mvn generated-sources

預設生成路徑為${basedir}/target/generated-sources

編輯服務端檔案

同上

編輯客戶端檔案

同上

使用python編寫客戶端檔案訪問java服務端

環境準備

  1. python版本2.7及以上
  2. 安裝pip管理軟體
  3. pip install grpc
  4. python,檢視importgrpc是否成功,缺啥裝啥
    這裡裝環境太麻煩,下載了個docker映象,直接使用了,映象名grpc/python,用著很方便

編輯proto檔案

準備一份proto檔案

syntax = "proto3";
service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

使用protoc命令生成相關類檔案

docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python -m grpc.tools.protoc -I /python-grpc --python_out=/python-grpc --grpc_python_out=/python-grpc /python-grpc/hello.proto

指定protos檔案的路徑,以及輸出路徑,並指定proto 檔案

建立客戶端

from __future__ import print_function

import grpc

import hello_pb2


def run():
  channel = grpc.insecure_channel('**.**.**.**:50051')
  stub = hello_pb2.GreeterStub(channel)
  response = stub.SayHello(hello_pb2.HelloRequest(name='you'))
  print("Greeter client received: " + response.message)

if __name__ == '__main__':
  run()

使用python的client端去訪問java的server端

docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python /python-grpc/helloClient.py
輸出結果為:hello you

若不適用grpc/python的映象,物理機的python需要安裝grpc,安裝過程中,版本及各種依賴包的衝突非常麻煩。。。沒有耐心了

遇到問題

I. pom execution報錯:
感覺像是包依賴衝突的問題。。。未解

Multiple annotations found at this line:
    - Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom failed: An API 
     incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom: 
     java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V 
     ----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
     0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
     repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
     liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
     repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
     google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
     maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
     plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
     sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental 
     from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
     SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] ----------------------------------------------------- 
     (org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom:default:generate-sources)
    - Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile failed: An API 
     incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile: 
     java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V 
     ----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
     0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
     repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
     liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
     repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
     google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
     maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
     plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
     sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental 
     from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
     SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] ----------------------------------------------------- 
     (org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile:default:generate-sources)

原因:M2E的bug eclipse版本過低

更新eclipse為最新版,刪除所有.project資料夾,重新更新project

II. 編譯protoc-gen-grpc-java外掛時報錯:

fatal error: ‘google/protobuf/io/zero_copy_stream.h’ file not found

原因:找不到標頭檔案,因為macos系統沒有將/usr/local作為搜尋標頭檔案和庫的預設路徑
解決:export CXXFLAGS=”-I/usr/local/include” LDFLAGS=”-L/usr/local/lib”

III. 生成的grpc通訊程式碼GreeterGrpc編譯報錯

報錯內容:

Multiple markers at this line
    - Method breakpoint:GreeterGrpc$GreeterImplBase [entry] - bindService()
    - The method bindService() of type GreeterGrpc.GreeterImplBase must override a superclass 
     method
    - implements io.grpc.BindableService.bindService

Multiple markers at this line
    - The method invoke(Req, StreamObserver<Resp>) of type GreeterGrpc.MethodHandlers<Req,Resp> 
     must override a superclass method
    - implements io.grpc.stub.ServerCalls.UnaryRequestMethod<Req,Resp>.invoke

Multiple markers at this line
    - implements io.grpc.stub.ServerCalls.StreamingRequestMethod<Req,Resp>.invoke
    - The method invoke(StreamObserver<Resp>) of type GreeterGrpc.MethodHandlers<Req,Resp> must 
     override a superclass method

原因:同問題I,由於M2E外掛版本過低有bug
解決辦法:更新M2E,或者更新eclipse為最新版

參考文獻: