算法工程師筆/面試總結
1. 有哪些生成式模型，有哪些判決式模型？
判別式模型與生成式模型的區別
產生式模型(Generative Model)與判別式模型(Discrimitive Model)是分類器常遇到的概念，它們的區別在於：
對於輸入x，類別標簽y：
產生式模型估計它們的聯合概率分布P(x,y)
判別式模型估計條件概率分布P(y|x)
產生式模型可以根據貝葉斯公式得到判別式模型，但反過來不行。
Andrew Ng在NIPS2001年有一篇專門比較判別模型和產生式模型的文章：
On Discrimitive vs. Generative classifiers: A comparision of logistic regression and naive Bayes
(http://robotics.stanford.edu/~ang/papers/nips01-discriminativegenerative.pdf)
判別式模型常見的主要有：
Linear Regression、Logistic Regression、SVM、Traditional Neural Networks、Linear Discriminant Analysis、Nearest Neighbor、CRF、Boosting、
產生式模型常見的主要有：
Gaussians、 Naive Bayes、 Mixtures of Multinomials、Mixtures of Gaussians、 Mixtures of Experts、HMMs、Sigmoidal Belief Networks, Bayesian Networks、Markov Random Fields、Latent Dirichlet Allocation
一個通俗易懂的解釋
Let‘s say you have input data x and you want to classify the data into labels y. A generative model learns the joint probability distribution p(x,y) and a discriminative model learns the conditional probability distribution p(y|x) – which you should read as ‘the probability of y given x‘.
Here‘s a really simple example. Suppose you have the following data in the form (x,y):
(1,0), (1,0), (2,0), (2, 1)
p(x,y) is
| y=0 | y=1 | |
| x=1 | 1/2 | 0 |
| x=2 | 1/4 | 1/4 |
p(y|x) is
| y=0 | y=1 | |
| x=1 | 1 | 0 |
| x=2 | 1/2 | 1/2 |
If you take a few minutes to stare at those two matrices, you will understand the difference between the two probability distributions.
The distribution p(y|x) is the natural distribution for classifying a given example x into a class y, which is why algorithms that model this directly are called discriminative algorithms. Generative algorithms model p(x,y), which can be tranformed into p(y|x) by applying Bayes rule and then used for classification. However, the distribution p(x,y) can also be used for other purposes. For example you could use p(x,y) to generate likely (x,y) pairs.
From the description above you might be thinking that generative models are more generally useful and therefore better, but it‘s not as simple as that. This paper is a very popular reference on the subject of discriminative vs. generative classifiers, but it‘s pretty heavy going. The overall gist is that discriminative models generally outperform generative models in classification tasks.
兩個模型的對比
算法工程師筆/面試總結