ALINK(三十八):模型評估(三)多分類評估 (EvalMultiClassBatchOp)
Java 類名:com.alibaba.alink.operator.batch.evaluation.EvalMultiClassBatchOp
Python 類名:EvalMultiClassBatchOp
功能介紹
多分類評估是對多分類演算法的預測結果進行效果評估。
支援Roc曲線,LiftChart曲線,K-S曲線,Recall-Precision曲線繪製。
流式的實驗支援累計統計和視窗統計,除卻上述四條曲線外,還給出Auc/Kappa/Accuracy/Logloss隨時間的變化曲線。
給出整體的評估指標包括:AUC、K-S、PRC, 不同閾值下的Precision、Recall、F-Measure、Sensitivity、Accuracy、Specificity和Kappa。
混淆矩陣
Precision
Precision = \dfrac{TP}{TP + FP}
Recall
Recall = \dfrac{TP}{TP + FN}
F-Measure
F1=\dfrac{2TP}{2TP+FP+FN}=\dfrac{2\cdot Precision \cdot Recall}{Precision+Recall}
Sensitivity
Sensitivity=\dfrac{TP}{TP+FN}
Accuracy
Accuray=\dfrac{TP + TN}{TP + TN + FP + FN}
Specificity
Specificity=\dfrac{TN}{FP+T}
Kappa
p_a =\dfrac{TP + TN}{TP + TN + FP + FN}
p_e = \dfrac{(TN + FP) * (TN + FN) + (FN + TP) * (FP + TP)}{(TP + TN + FP + FN) * (TP + TN + FP + FN)}
kappa = \dfrac{p_a - p_e}{1 - p_e}
Logloss
logloss=- \dfrac{1}{N}\sum_{i=1}^N \sum_{j=1}^My_{i,j}log(p_{i,j})
引數說明
名稱 |
中文名稱 |
描述 |
型別 |
是否必須? |
預設值 |
labelCol |
標籤列名 |
輸入表中的標籤列名 |
String |
✓ |
|
predictionCol |
預測結果列名 |
預測結果列名 |
String |
||
predictionDetailCol |
預測詳細資訊列名 |
預測詳細資訊列名 |
String |
程式碼示例
Python 程式碼
from pyalink.alink import * import pandas as pd useLocalEnv(1) df = pd.DataFrame([ ["prefix1", "{\"prefix1\": 0.9, \"prefix0\": 0.1}"], ["prefix1", "{\"prefix1\": 0.8, \"prefix0\": 0.2}"], ["prefix1", "{\"prefix1\": 0.7, \"prefix0\": 0.3}"], ["prefix0", "{\"prefix1\": 0.75, \"prefix0\": 0.25}"], ["prefix0", "{\"prefix1\": 0.6, \"prefix0\": 0.4}"]]) inOp = BatchOperator.fromDataframe(df, schemaStr='label string, detailInput string') metrics = EvalMultiClassBatchOp().setLabelCol("label").setPredictionDetailCol("detailInput").linkFrom(inOp).collectMetrics() print("Prefix0 accuracy:", metrics.getAccuracy("prefix0")) print("Prefix1 recall:", metrics.getRecall("prefix1")) print("Macro Precision:", metrics.getMacroPrecision()) print("Micro Recall:", metrics.getMicroRecall()) print("Weighted Sensitivity:", metrics.getWeightedSensitivity())
Java 程式碼
import org.apache.flink.types.Row; import com.alibaba.alink.operator.batch.BatchOperator; import com.alibaba.alink.operator.batch.evaluation.EvalMultiClassBatchOp; import com.alibaba.alink.operator.batch.source.MemSourceBatchOp; import com.alibaba.alink.operator.common.evaluation.MultiClassMetrics; import org.junit.Test; import java.util.Arrays; import java.util.List; public class EvalMultiClassBatchOpTest { @Test public void testEvalMultiClassBatchOp() throws Exception { List <Row> df = Arrays.asList( Row.of("prefix1", "{\"prefix1\": 0.9, \"prefix0\": 0.1}"), Row.of("prefix1", "{\"prefix1\": 0.8, \"prefix0\": 0.2}"), Row.of("prefix1", "{\"prefix1\": 0.7, \"prefix0\": 0.3}"), Row.of("prefix0", "{\"prefix1\": 0.75, \"prefix0\": 0.25}") ); BatchOperator <?> inOp = new MemSourceBatchOp(df, "label string, detailInput string"); MultiClassMetrics metrics = new EvalMultiClassBatchOp().setLabelCol("label").setPredictionDetailCol( "detailInput").linkFrom(inOp).collectMetrics(); System.out.println("Prefix0 accuracy:" + metrics.getAccuracy("prefix0")); System.out.println("Prefix1 recall:" + metrics.getRecall("prefix1")); System.out.println("Macro Precision:" + metrics.getMacroPrecision()); System.out.println("Micro Recall:" + metrics.getMicroRecall()); System.out.println("Weighted Sensitivity:" + metrics.getWeightedSensitivity()); } }
執行結果
Prefix0 accuracy: 0.6 Prefix1 recall: 1.0 Macro Precision: 0.8 Micro Recall: 0.6 Weighted Sensitivity: 0.6