該系列僅在原課程基礎上部分知識點添加個人學習筆記，或相關推導補充等。如有錯誤，還請批評指教。在學習了 Andrew Ng 課程的基礎上，為了更方便的查閱複習，將其整理成文字。因本人一直在學習英語，所以該系列以英文為主，同時也建議讀者以英文為主，中文輔助，以便後期進階時，為學習相關領域的學術論文做鋪墊。- ZJ

轉載請註明作者和出處：微信公眾號-「SelfImprovementLab」

Supervised Learning with Neural Networks

Supervised Learning

There’s been a lot of hype about neural networks.And perhaps some of that type is justified,given how well they’re working.But it turns out that so far,almost all the economic value created by neural networks has been through one type of machine learning

,called supervised learning. Let’s see what that means, and let’s go over some examples.In supervised learning, you have some input x,and you want to learn a function mapping to some output y.

神經網路有時媒體 炒作 得很厲害。考慮到它們的使用效果，有些說法還是靠譜的。事實上到目前為止，幾乎所有由神經網路創造的經濟價值都基於其中一種 機器學習 ，我們稱之“監督學習”。那是什麼意思呢？ 我們來看一些例子。在監督學習中，你有一些輸入 X,然後你想學習到一個函式，可以對映到輸出 y。

So for example, just now we saw the housing price prediction application where you input some features of a home and try to output or estimate the price y.Here are some other examples that neural networks have been applied to very effectively.Possibly the single most lucrative application of deep learning today is online advertising

,maybe not the most inspiring, but certainly very lucrative, in which,by inputting an ad, the information of an ad to the website it’s thinking of showing you,and some information about the user,neural networks have gotten very good at predicting whether or not you click on an ad.

比如我們之前看到的，應用於房價預測的例子，輸入房屋的一些特徵就能輸出或者預測價格 y。下面是一些其它例子，這些例子中神經網路效果拔群。很可能今天通過深度學習 獲利最大 的，就是 線上廣告，這也許不是最鼓舞人心的，但真的很賺錢，給網站輸入廣告資訊，網站會考慮是否給你看這個廣告，有時還需要輸入一些使用者資訊，神經網路在預測你是否會點選這個廣告方面已經表現得很好。

And by showing you and showing users the ads that you are most likely to click on,this has been an incredibly lucrative application of neural networks at multiple companies.Because the ability to show you ads that you’re more likely to click on has a direct impact on the bottom line of some of the very large online advertising companies.

通過向你展示，向用戶展示，最有可能點開的廣告，這就是神經網路在很多家公司 ，賺取無法想象的高額利潤的應用方式。因為有了這種向你展示最有可能點選開的廣告的能力，直接影響到了不少大型線上廣告公司的收入。

Computer vision has also made huge strides in the last several years,mostly due to deep learning.So you might input an image and want to output an index,say from 1 to 1,000 trying to tell you if this picture,it might be any one of, say a 1000 different images.So, you might use that for photo tagging.I think the recent progress in speech recognition has also been very exciting,where you can now input an audio clip to a neural network,and have it output a text transcript.

過去的幾年裡，計算機視覺 也有很大進展，這要感謝深度學習。你輸入一個影象，然後想輸出一個指數，可以是從1到1000 來表明這張照片，是1000個不同的影象中的某一個，可以用來給照片打標籤。深度學習最近在 語音識別 方面的進展，也是非常令人興奮的。你可以將 音訊片段 輸入神經網路，它可以輸出 文字。

Machine translation has also made huge strides thanks to deep learning where now you can have a neural network input an English sentence and directly output say,a Chinese sentence. And in autonomous driving, you might input an image,say a picture of what’s in front of your car as well as some information from a radar, and based on that, maybe a neural network can be trained to tell you the position of the other cars on the road.So this becomes a key component in autonomous driving systems.So a lot of the value creation through neural networks has been through cleverly selecting what should be x and what should be y for your particular problem, and then fitting this supervised learning component into often a bigger system such as an autonomous vehicle.

機器翻譯 也進步很大，這得感謝深度學習，讓你有一個神經網路能實現輸入英語句子，它直接輸出 一箇中文句子。在無人駕駛技術中，你輸入一幅影象，汽車前方的一個快照，還有一些雷達資訊，基於這個訓練過的神經網路，能告訴你路上其他汽車的位置，這是無人駕駛系統的關鍵元件。神經網路創造這麼多價值的案例中，你要 機智地選擇 x 和 y，才能解決特定問題，然後把這個監督學習過的元件，嵌入到更大型的系統中，比如無人駕駛。

It turns out that slightly different types of neural networks are useful for different applications. For example, in the real estate application that we saw in the previous video,we use a universally standard neural network architecture, right? Maybe for real estate and online advertising might be a relatively standard neural network,like the one that we saw.

可以看出稍微不同的神經網路應用到不同的地方，也都行之有效。比如說 應用到房地產上，我們上節課看過了，我們用了通用標準的神經網路架構 ，是吧？ 對於 房地產 和 線上廣告，用的都是相對標準的神經網路，正如我們之前見到的。

For image applications we’ll often use convolutional neural networks,often abbreviated CNN.And for sequence data.So for example, audio has a temporal component, right?Audio is played out over time, so audio is most naturally represented as a one-dimensional time series or as a one-dimensional temporal sequence.And so for sequence data, you often use an RNN,a recurrent neural network.Language, English and Chinese, the alphabets or the words come one at a time.

影象領域裡，我們經常應用的是 卷積神經網路，通常縮寫為 CNN。對於序列資料，例如，音訊中含有時間成分，對吧？音訊是隨著時間播放的 所以音訊很自然地被表示為一維時間序列， 一維的時間序列 。對於序列資料，你經常使用 RNN 迴圈神經網路，語言、 英語和漢語 、字母或單詞， 都是逐個出現的。

So language is also most naturally represented as**sequence data**.And so more complex versions of RNNs are often used for these applications.And then, for more complex applications, like autonomous driving,where you have an image, that might suggest more of**a CNN convolution neural network structur**e and radar info which is something quite different.You might end up with a more custom, or some more complex,hybrid neural network architecture.So, just to be a bit more concrete about what are the standard CNN and RNN architectures.So in the literature you might have seen pictures like this.So that’s a standard neural net.

所以語言最自然的表示方式也是序列資料。更復雜的 RNNs，經常會用於這些應用，對於更復雜的應用，比如無人駕駛。你有一張圖片，可能需要 CNN “卷積神經網路結構” 架構去處理，雷達資訊會更不一樣，你需要一些更復雜的，混合的神經網路結構。所以，為了更具體地說明，標準的 CNN 和 RNN 結構是什麼， 在文獻中，你可能見過這樣的圖片，這是一個標準的神經網路。

Neural Network examples

You might have seen pictures like this.Well this is an example of a Convolutional Neural Network ,and we’ll see in a later course exactly what this picture means and how can you implement this.But convolutional networks are often use for image data.And you might also have seen pictures like this.And you’ll learn how to implement this in a later course.

你可能見過這樣的圖片，這是 一個卷積神經網路，在後續的課程，我們會去了解這幅圖的含義和如何實現它。卷積網路通常用於影象資料，你可能也會看到這樣的圖片 後續的課程也會去實現它。

Structured Data and Unstructured Data

Recurrent neural networks are very good for this type of one-dimensional sequence data that has maybe a temporal component.You might also have heard about applications of machine learning to both Structured Data and Unstructured Data .Here’s what the terms mean.Structured Data means basically databases of data.So, for example, in housing price prediction,you might have a database or the column that tells you the size and the number of bedrooms.So, this is structured data,or in predicting whether or not a user will click on an ad,you might have information about the user, such as the age,some information about the ad, and then labels y that you’re trying to predict.

迴圈神經網路非常適合處理一維序列資料，其中包含時間成分，你可能也聽說過機器學習被應用於結構化資料 和 非結構化資料，下面是這些術語的含義，結構化資料 是資料的資料庫。例如，在房價預測中，你可能有一個數據庫或者資料列，告訴你房間的大小和臥室數量，這就是結構化資料在預測使用者是否會點選廣告的例子中，你可能會有使用者資訊：比如年齡，還有廣告資訊，還有你要預測的標籤 y。

So that’s structured data, meaning that each of the features,such as size of the house, the number of bedrooms,or the age of a user, has a very well defined meaning.In contrast, unstructured data refers to things like audio, raw audio, or images where you might want to recognize what’s in the image or text.Here the features might be the pixel values in an image or the individual words in a piece of text.

這就是結構化資料，意味著每個特徵，比如說房屋大小，臥房數量，使用者的年齡，都有著清晰的定義。相反，非結構化資料指的是，比如音訊、原始音訊、影象，你想要識別影象或文字中的內容，這裡的特徵可能是影象中的畫素值，或者是文字中的單個單詞。

Historically, it has been much harder for computers to make sense of unstructured data compared to structured data.And the fact the human race has evolved to be very good at understanding audio cues as well as images.And then text was a more recent invention,but people are just really good at interpreting unstructured data.And so one of the most exciting things about the rise of neural networks is that,thanks to deep learning, thanks to neural networks,computers are now much better at interpreting unstructured data as well compared to just a few years ago.

從歷史角度看，非結構化資料與結構化資料比較 ，讓計算機理解起來更難。但人類進化到現在，很擅長理解音訊訊號和影象， 文字是一個更近代的發明， 但人們真的很擅長解讀非結構化資料，神經網路的興起過程中，最令人興奮的事情之一就是 多虧了深度學習，多虧了神經網路， 計算機現在能更好地解釋非結構化資料 ，和幾年前對比的話 。

And this creates opportunities for many new exciting applications that use
**speech recognition, image recognition,natural language processing on text,**much more than was possible even just two or three years ago.I think because people have a natural empathy to understanding unstructured data,you might hear about neural network successes on unstructured data more in the media because it’s just cool when the neural network recognizes a cat.

這給我們創造了很多令人興奮的應用機會，語音識別、影象識別、自然語言文書處理 ，現在能做的事情比兩三年前要豐富多了。我認為，因為人們生來就有能力 理解非結構化資料，你可能在媒體上聽到了更多神經網路在非結構化資料上的成功，尤其是當神經網路識別了一隻貓時。

We all like that, and we all know what that means.But it turns out that a lot of short term economic value that neural networks are creating has also been on structured data,such as much better advertising systems,much better profit recommendations,and just a much better ability to process the giant databases that many companies have to make accurate predictions from them.

那真的很酷，我們都知道那意味著什麼。神經網路在很多短期經濟價值的創造 是基於結構化資料的，比如更好的廣告系統，更好的獲利建議，有更好的能力去處理很多公司擁有的海量資料庫，並用這些資料準確預測未來趨勢。

So in this course, a lot of the techniques we’ll go over will apply to both structured data and to unstructured data.For the purposes of explaining the
algorithms,we will draw a little bit more on examples that use unstructured data.But as you think through applications of neural networks within your own team I hope you find both uses for them in both structured and unstructured data.

在這門課中，我們會學到很多技巧可以應用到結構化資料，也可以應用到非結構化資料 。為了更清楚地解釋演算法原理，我們會多用非結構化資料的例子，但當你自己的團隊評估了各種神經網路的應用之後，希望你的演算法能夠同時學習結構化和非結構化資料。

So neural networks have transformed supervised learning and are creating tremendous economic value.It turns out though, that the basic technical ideas behind neural networks have mostly been around, sometimes for many decades.So why is it, then, that they’re only just now taking off and working so well?In the next video, we’ll talk about why it’s only quite recently that neural networks have become this incredibly powerful tool that you can use.

神經網路徹底改變了監督學習正創造著巨大的經濟價值。其實呢 ，基本的神經網路背後的技術理念，大部分都不是新概念，有些甚至有幾十年歷史了。那麼 為什麼它們現在才流行，才行之有效呢？下一集視訊中，我們將討論為什麼是最近神經網路才成為你可以使用的強大工具。

PS: 歡迎掃碼關注公眾號：「SelfImprovementLab」！專注「深度學習」，「機器學習」，「人工智慧」。以及「早起」，「閱讀」，「運動」，「英語」「其他」不定期建群 打卡互助活動。