avro序列化例項
阿新 • • 發佈:2019-01-09
hadoop中,就是用的avro作為序列化的,現在按照官方文件,我們按照如下步驟來做這個例項。
- 構建maven工程,加入apache-avro依賴庫,以及外掛依賴。
- 編寫avro檔案
- 生成java實體類
- 編寫測試程式
pom.xml配置檔案
<dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.9</version> <scope>test</scope> </dependency> <dependency> <groupId>org.apache.avro</groupId> <artifactId>avro</artifactId> <version>1.8.2</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.7.21</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.avro</groupId> <artifactId>avro-maven-plugin</artifactId> <version>1.8.2</version> <executions> <execution> <phase>generate-sources</phase> <goals> <goal>schema</goal> </goals> <configuration> <sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory> <outputDirectory>${project.basedir}/src/main/java/</outputDirectory> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> </plugins> </build>
src/main/avro目錄下編寫user.avsc,注意字尾是avsc
{ "namespace":"com.xxx.avro.entity", "type":"record", "name":"User", "fields":[ {"name":"id","type":"int"}, {"name":"name","type":"string"}, {"name":"age","type":["int","null"]}, {"name":"mobile","type":["string","null"]} ] }
這是一個用json格式表示的avro物件
因為我們的pom.xml配置檔案中配置了avro生成java的外掛,所以這裡直接執行Run As ->Maven install,構建成功之後,會在com.xxx.avro.entity目錄下生成User.java的檔案。
User.java檔案部分內容截圖:
接下來,編寫測試驗證:
執行結果:
eclipse下,使用avro-maven-plugin外掛,工程中的pom.xml會報錯: Plugin execution not covered by lifecycle configuration...這個錯誤提示上有解決辦法,要麼在pom.xml配置檔案中增加配置,忽略phase,要麼在lifecycle-mapping-metadata.xml配置檔案中增加配置忽略phase,然後重新載入lifecycle-mapping-metadata.xml檔案。這裡兩種解決辦法都貼出來。
1、在pom.xml中增加配置:
<pluginManagement>
<plugins>
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<versionRange>[1.8.2,)</versionRange>
<goals>
<goal>schema</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore></ignore>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
2、在lifecycle-mapping-metadata.xml中增加配置:
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<versionRange>1.8.2</versionRange>
<goals>
<goal>schema</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>