[2] Hive3.x 查詢流程原始碼-Cli端-01
阿新 • • 發佈:2019-01-11
Hive架構簡圖
Hive架構簡圖 hive原理與原始碼分析-hive原始碼架構與理論
Hive3.x安裝準備工作
詳細參見:Hive3.x 安裝與debug
1 Hive命令列提交查詢
SELECT deptno, count(deptname) as deptno_cnt from hive3_test.depts group by deptno;
執行棧
下面簡要分析關鍵路徑程式碼
2 CliDriver接收查詢
public int processCmd(String cmd) {
...
processLocalCmd(String cmd, CommandProcessor proc, CliSessionState ss) {
ret = (ReExecDriver)qp.run(cmd).getResponseCode();
這裡run呼叫Driver::run
//public CommandProcessorResponse run(String command) //見下文
ret = processLocalCmd(cmd, proc, ss);//待單獨分析
}
...
}
3 ReExecDriver compileAndRespond查詢
public CommandProcessorResponse run(String command) {
CommandProcessorResponse r0 = compileAndRespond(command);
private void runInternal(String command, boolean alreadyCompiled){
}
}
compileAndRespond中呼叫Driver compile查詢
public CommandProcessorResponse compileAndRespond(String command, boolean cleanupTxnList){
private void compileInternal(String command, boolean deferClose){
Driver::compile(command, true, deferClose);
}
}
4 Driver compile請求
private void compile(String command, boolean resetTaskIds, boolean deferClose) throws CommandProcessorResponse {
......
//ParseUtils.parse 見command sql解析為AST
try {
tree = ParseUtils.parse(command, ctx);
} catch (ParseException e) {
parseError = true;
throw e;
} finally {
hookRunner.runAfterParseHook(command, parseError);
}
......
//語法分析 Do semantic analysis and plan generation
BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(queryState, tree);
sem.analyze(tree, ctx);
//檢測語法合法性 validate the plan
sem.validate();
.....
// 邏輯執行計劃
plan = new QueryPlan(queryStr, sem, perfLogger.getStartTime(PerfLogger.DRIVER_RUN), queryId,
queryState.getHiveOperation(), schema);
}
4.1 ParseUitls SQL AST解析
ParseUitls
/** Parses the Hive query. */
public static ASTNode parse(
String command, Context ctx, String viewFullyQualifiedName) throws ParseException {
ParseDriver pd = new ParseDriver();
ASTNode tree = pd.parse(command, ctx, viewFullyQualifiedName);
tree = findRootNonNullToken(tree);
handleSetColRefs(tree);
return tree;
}
ParseUitls 調起HiveParser
ParseDriver
public ASTNode parse(String command, Context ctx, String viewFullyQualifiedName){
HiveParser parser = new HiveParser(tokens);
...
ASTNode tree = (ASTNode) r.getTree();
}
ASTNode的結構如下
4.2 BaseSemanticAnalyzer語法分析
接上文sem.analyze(tree, ctx);語法分析
會先進入calciteplanner進行查詢優化
CalcitePlanner待續
public void analyzeInternal(ASTNode ast) throws SemanticException {
if (runCBO) {
super.analyzeInternal(ast, new PlannerContextFactory() {
@Override
public PlannerContext create() {
return new PreCboCtx();
}
});
} else {
super.analyzeInternal(ast);
}
}