rpc框架--grpc-java
grpc-java 初探
環境準備
- 下載protobuf
- 新建maven專案
使用protoc命令列生成類檔案
安裝protocbuf
安裝步驟略
編譯 protoc-gen-grpc-java外掛
下載grpc-java後,執行一下命令
cd grpc-java/compile
# 編譯外掛
../gradlew java_pluginExecutable
# 測試外掛
../gradlew test
若看到BUILD SUCCESSFUL字樣,則編譯成功
編輯proto檔案
syntax = "proto3";
option java_multiple_files = true ;
option java_package = "io.grpc.examples.helloworld";
# java_package 用來指定生成的java類所在的包,如果該proto檔案被指定編譯為其他語言時,java引數失效,預設包為package
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";
package helloworld;
# 定義服務
service Greeter {
# 定義grpc方法,並且制訂訊息的請求型別和反回型別
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
# 訊息型別都是protocol buffer的訊息格式
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
grpc可以有四種服務型別
1. 簡單的rcp模式,client端使用stub傳送請求到服務端,然後進入等待模式,直到響應傳送回client端
rpc GetFeature(Point) returns (Feature) {}
2. 服務端流式rpc,client端傳送請求到服務端,返回訊息序列的流資料(gets a stream to read a sequence of messages back),client從返回的流資料讀取訊息,直至不再有資料,如果指定服務端返回流資料型別,需使用stream關鍵字
rpc ListFeatures(Rectangle) returns (stream Feature) {}
3. 客戶端流式rpc,client使用指定的stream方式寫訊息序列併發送給服務端,一旦客戶端寫訊息完成,會等待服務端全部讀取並做出響應
rpc RecordRoute(stream Point) returns (RouteSummary) {}
4. 雙向流式rpc,雙向流式操作獨立,客戶端和服務端可以以任何序列隨意讀寫,比如,服務端可以在等待client端傳送所有訊息之前寫響應訊息,或者服務端可以選擇讀一個訊息寫一個訊息或者其他的讀寫組合。訊息的順序都是被儲存的
rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
生成client端和server端程式碼
使用protocol編譯器protoc和指定的grpc的一個java外掛。
使用maven的protobuf外掛生成相關程式碼
命令列:
protoc --java_out=./java/ ./proto/hello.proto
生成grpc通訊程式碼
protoc --plugin=protoc-gen-grpc-java=/Users/liuyu9/Documents/personal/golang/src/github.com/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java --grpc-java_out=./java ./proto/hello.proto
檔案列表如下:
# GreeterGrpc包含服務端和客戶端要實現的一些基本類
-rw-r--r-- 1 root staff 7352 Oct 25 18:48 GreeterGrpc.java
# 包含了protocol buffer傳送、序列化、反序列化我們請求和響應程式碼
-rw-r--r-- 1 root staff 16207 Oct 25 18:53 HelloReply.java
-rw-r--r-- 1 root staff 501 Oct 25 18:53 HelloReplyOrBuilder.java
-rw-r--r-- 1 root staff 16192 Oct 25 18:53 HelloRequest.java
-rw-r--r-- 1 root staff 493 Oct 25 18:53 HelloRequestOrBuilder.java
-rw-r--r-- 1 root staff 3011 Oct 25 18:53 HelloWorldProto.java
編寫服務端檔案
編寫服務端檔案的工作有兩部分:
1. 重寫proto檔案定義的服務生成的服務基類,確定服務的工作內容
2. 執行grpc服務,監聽客戶端請求並做出響應
package com.weibo.dorteam.grpc;
import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;
import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;
import java.io.IOException;
import java.util.logging.Logger;
public class helloServer {
private static final Logger logger = Logger.getLogger(helloServer.class.getName());
private int port = 50051;
private Server server;
private void start() throws IOException {
// 使用ServerBuilder來構建和啟動服務,通過使用forPort方法來指定監聽的地址和埠
// 建立一個實現方法的服務GreeterImpl的例項,並通過addService方法將該例項納入
// 呼叫build() start()方法構建和啟動rpcserver
server = ServerBuilder.forPort(port)
.addService(new GreeterImpl())
.build()
.start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
// Use stderr here since the logger may have been reset by its JVM shutdown hook.
System.err.println("*** shutting down gRPC server since JVM is shutting down");
helloServer.this.stop();
System.err.println("*** server shut down");
}
});
}
private void stop() {
if (server != null) {
server.shutdown();
}
}
/**
* Await termination on the main thread since the grpc library uses daemon threads.
*/
private void blockUntilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
/**
* Main launches the server from the command line.
*/
public static void main(String[] args) throws IOException, InterruptedException {
final helloServer server = new helloServer();
server.start();
server.blockUntilShutdown();
}
// 我們的服務GreeterImpl繼承了生成抽象類GreeterGrpc.GreeterImplBase,實現了服務的所有方法
private class GreeterImpl extends GreeterGrpc.GreeterImplBase {
@Override
public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) {
HelloReply reply = HelloReply.newBuilder().setMessage("Hello " + req.getName()).build();
// 使用響應監視器的onNext方法返回HelloReply
responseObserver.onNext(reply);
// 使用onCompleted方法指定本次呼叫已經完成
responseObserver.onCompleted();
}
}
}
訊息型別,StreamObserver一個響應監視器,是伺服器調生成響應結果使用的特定介面
編寫客戶端檔案
為了呼叫服務端的方法,首先需要建立一個或兩個stub:
阻塞式stub:rpc呼叫會阻塞等待服務端響應後返回訊息或異常資訊
非阻塞式stub:非阻塞呼叫方式,響應非同步返回
package com.weibo.dorteam.grpc;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
public class helloClient {
private static final Logger logger = Logger.getLogger(helloClient.class.getName());
private final ManagedChannel channel;
private final GreeterGrpc.GreeterBlockingStub blockingStub;
/** Construct client connecting to HelloWorld server at {@code host:port}. */
// 首先,我們需要為stub建立一個grpc的channel,指定我們連線服務端的地址和埠
// 使用ManagedChannelBuilder方法來建立channel
public helloClient(String host, int port) {
channel = ManagedChannelBuilder.forAddress(host, port)
// Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
// needing certificates.
.usePlaintext(true)
.build();
// 使用我們從proto檔案生成的GreeterGrpc類提供的newBlockingStub方法指定channel建立stubs
blockingStub = GreeterGrpc.newBlockingStub(channel);
}
public void shutdown() throws InterruptedException {
channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
}
// 呼叫服務端方法
/** Say hello to server. */
public void greet(String name) {
logger.info("Will try to greet " + name + " ...");
// 建立並定製protocol buffer物件,使用該物件呼叫服務端的sayHello方法,獲得response
HelloRequest request = HelloRequest.newBuilder().setName(name).build();
HelloReply response;
try {
response = blockingStub.sayHello(request);
// 如果有異常發生,則異常被編碼成Status,可以從StatusRuntimeException異常中捕獲
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
return;
}
logger.info("Greeting: " + response.getMessage());
}
/**
* Greet server. If provided, the first element of {@code args} is the name to use in the
* greeting.
*/
public static void main(String[] args) throws Exception {
helloClient client = new helloClient("localhost", 50051);
try {
/* Access a service running on the local machine on port 50051 */
String user = "world";
if (args.length > 0) {
user = args[0]; /* Use the arg as the name to greet if provided */
}
client.greet(user);
} finally {
client.shutdown();
}
}
}
mvn編譯執行
服務端執行:
# 不需要引數
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer"
# 需要引數
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.args="arg0 arg1 arg2"
# 傳遞classpath
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.classpathScope=runtime
客戶端執行:
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloClient"
使用maven依賴生成類檔案
新增pom依賴
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-all</artifactId>
<version>1.0.1</version>
</dependency>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.4.1.Final</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.0</version>
<configuration>
<!--
The version of protoc must match protobuf-java. If you don't depend on
protobuf-java directly, you will be transitively depending on the
protobuf-java version that grpc depends on.
-->
<protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:0.13.2:exe:${os.detected.classifier}</pluginArtifact>
<protocExecutable>/Users/peng/protoc-3.0.0-beta-2-osx-x86_64/protoc</protocExecutable>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
注:protoc的版本需要匹配protobuf-java,如果不是直接依賴protobuf-java,則依賴grpc時,間接依賴protobuf-java,版本為3.0.0
編輯proto檔案
同上
生成相關類檔案
mvn generated-sources
預設生成路徑為${basedir}/target/generated-sources
編輯服務端檔案
同上
編輯客戶端檔案
同上
使用python編寫客戶端檔案訪問java服務端
環境準備
- python版本2.7及以上
- 安裝pip管理軟體
- pip install grpc
- python,檢視importgrpc是否成功,缺啥裝啥
這裡裝環境太麻煩,下載了個docker映象,直接使用了,映象名grpc/python,用著很方便
編輯proto檔案
準備一份proto檔案
syntax = "proto3";
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
使用protoc命令生成相關類檔案
docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python -m grpc.tools.protoc -I /python-grpc --python_out=/python-grpc --grpc_python_out=/python-grpc /python-grpc/hello.proto
指定protos檔案的路徑,以及輸出路徑,並指定proto 檔案
建立客戶端
from __future__ import print_function
import grpc
import hello_pb2
def run():
channel = grpc.insecure_channel('**.**.**.**:50051')
stub = hello_pb2.GreeterStub(channel)
response = stub.SayHello(hello_pb2.HelloRequest(name='you'))
print("Greeter client received: " + response.message)
if __name__ == '__main__':
run()
使用python的client端去訪問java的server端
docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python /python-grpc/helloClient.py
輸出結果為:hello you
若不適用grpc/python的映象,物理機的python需要安裝grpc,安裝過程中,版本及各種依賴包的衝突非常麻煩。。。沒有耐心了
遇到問題
I. pom execution報錯:
感覺像是包依賴衝突的問題。。。未解
Multiple annotations found at this line:
- Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom failed: An API
incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom:
java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V
----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental
from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm
ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm
ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] -----------------------------------------------------
(org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom:default:generate-sources)
- Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile failed: An API
incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile:
java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V
----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental
from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm
ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm
ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] -----------------------------------------------------
(org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile:default:generate-sources)
原因:M2E的bug eclipse版本過低
更新eclipse為最新版,刪除所有.project資料夾,重新更新project
II. 編譯protoc-gen-grpc-java外掛時報錯:
fatal error: ‘google/protobuf/io/zero_copy_stream.h’ file not found
原因:找不到標頭檔案,因為macos系統沒有將/usr/local作為搜尋標頭檔案和庫的預設路徑
解決:export CXXFLAGS=”-I/usr/local/include” LDFLAGS=”-L/usr/local/lib”
III. 生成的grpc通訊程式碼GreeterGrpc編譯報錯
報錯內容:
Multiple markers at this line
- Method breakpoint:GreeterGrpc$GreeterImplBase [entry] - bindService()
- The method bindService() of type GreeterGrpc.GreeterImplBase must override a superclass
method
- implements io.grpc.BindableService.bindService
Multiple markers at this line
- The method invoke(Req, StreamObserver<Resp>) of type GreeterGrpc.MethodHandlers<Req,Resp>
must override a superclass method
- implements io.grpc.stub.ServerCalls.UnaryRequestMethod<Req,Resp>.invoke
Multiple markers at this line
- implements io.grpc.stub.ServerCalls.StreamingRequestMethod<Req,Resp>.invoke
- The method invoke(StreamObserver<Resp>) of type GreeterGrpc.MethodHandlers<Req,Resp> must
override a superclass method
原因:同問題I,由於M2E外掛版本過低有bug
解決辦法:更新M2E,或者更新eclipse為最新版