寫在前面

上次梳理了10個經典排序演算法，今天在看《演算法導論》這本書時，看到第7章快速排序。為了彌補假期沒學習的遺憾，把第7章學習一下，特此筆記。我在學習的時候越學越吃驚，這本書也太深奧了吧，課後思考題擴充套件好多，我這個理科生都受不了。原本以為一杯茶，一個演算法，整一天。發現不是一天的問題啊，這得整一週才能弄明白啊！怪不得說這本書是神書，忽然感覺頭皮有點涼。。。。。。

參考答案來自於https://walkccc.github.io/CLRS/Chap07/Problems/7-3/，感謝大神助力。

快速排序(Quick Sort)

對於包含\(n\)個數的輸入陣列來說，快速排序是一種最壞情況時間複雜度為\(Q(n^2)\)

快速排序的分治過程

快速排序使用了分治的思想。下面是對一個典型的子陣列\(A[p,r]\)進行快速排序的三步分治過程：

分解：

陣列A[p,r]被劃分為兩個（可能為空）子陣列\(A[p..q-1]\)和\(A[q+1..r]\)，使得\(A[p..q-1]\)中的每一個元素都小於等於\(A[q]\)，而\(A[q]\)也小於等於\(A[q+1..r]\)中的每一個元素。其中計算下標\(q\)也是劃分過程的一部分。

解決：

通過遞迴呼叫快速排序，對子陣列\(A[p..q-1]\)和\(A[q+1..r]\)進行排序。

合併：

因為子陣列都是原址排序的，所以不需要排序操作：陣列\(A[p..r]\)已經有序。

虛擬碼 QuickSort（A,p,r)

QuickSort（A,p,r)
if	p < r
    q = Partition(A,p,r)
    QuickSort(A,p,q-1)
    QuickSort(A,q+1,r)

為了排序一個數組\(A\)的全部元素，初始呼叫是 QuickSort(A,1,A.length)

陣列的劃分(Partition)

演算法的關鍵部分是Partition過程，它實現了對子陣列\(A[p..r]\)

虛擬碼 Partition(A,p,r)

Partition(A,p,r)
x = A[r]
i = p-1
for j = p to r-1
	if A[j] <= x
		i = i + 1
		exchange A[i] with A[j]
exchange A[i+1] with A[r]
return i + 1

在第4-7行迴圈體的每一輪迭代開始時，對於任意陣列下標\(k\)有

1. 若\(p \leq k \leq i\)，則\(A[k] \leq x\)。
2. 若\(i+1 \leq k \leq j-1\)，則\(A[k]>x\)。
3. 若\(k=r\)，則\(A[k]=x\)。

在子陣列\(A[p..r]\)中，Partition維護了四個區域。\(A[p..i]\)區間內所有值都小於等於\(x\)，\(A[i+1..j-1]\)區間內的所有值都大於\(x\)，\(A[r]=x\)。子陣列\(A[j..r-1]\)中的值可能屬於任何一種情況。

迴圈不變數的證明

初始化：

在迴圈的第一輪迭代開始之前，\(i=p-1,j=p\)，並且在\(p和i\)之間，\(i+1和j-1\)之間都不存在值，所以迴圈不變數的前兩個條件顯然滿足。第一行的賦值操作滿足了第三個條件。

保持：

終止：

當終止時，\(j=r\)。我們將陣列中的所有元素劃分為三個集合：包含了所有小於等於\(x\)元素的集合、包含了所有大於\(x\)元素的集合和只有一個元素\(x\)的集合。

執行過程

這裡選取了第一個結點作為分隔樞紐。

快速排序的效能

快速排序的執行時間依賴於劃分是否平衡，而平衡與否又依賴於用於劃分的元素。

最壞情況劃分

當劃分的兩個子問題分別包含了\(n-1\)個元素和\(0\)個元素時，快速排序的最壞情況發生了。

不妨假設演算法的每一次遞迴呼叫中都出現了這種不平衡劃分。

劃分操作的時間複雜度為\(\Theta(n)\)。

由於對一個大小為0的陣列進行遞迴呼叫會直接返回，因此\(T(0)=\Theta(1)\)。

於是演算法執行的遞迴式可以表示為:

\[T(n)=T(n-1)+T(0)+\Theta(n)=T(n-1)+\Theta(n) \]

從直觀上，每一層遞迴累加，結果為\(\Theta(n^2)\)。實際上，利用代入法也可以得到解為\(T(n)=\Theta(n^2)\)。

因此，在最壞情況下，快速排序演算法的執行時間並不比插入排序好。

此外，當輸入陣列已經完全有序時，快速排序的時間複雜度仍然為\(\Theta(n^2)\)，而在同樣情況下，插入排序時間複雜度為\(O(n)\)。

代入法證明：

假設\(T(n)\)是最壞情況下QuickSort在輸入規模為\(n\)的資料集合上花費的時間，則有遞迴式：

\[T(n)=max_{0 \leq q \leq n-1}(T(q)+T(n-q-1))+\Theta(n) \]

因為Partition函式生成的兩個子問題的規模總和為\(n-1\)，所以引數\(q\)的變化範圍是$0 \backsim {n-1} $。

我們不妨假設\(T(n) \leq cn^2\)成立，其中\(c\)為常數。將此式帶入上遞迴式，得：

\[\begin{aligned} T(n) & \leq max_{0 \leq q \leq n-1}(cq^2+c(n-q+1)^2+\Theta(n)) \\ &= c\cdotp max_{0 \leq q \leq n-1}(q^2+(n-q-1)^2)+\Theta(n) \end{aligned} \]

表示式\(q^2+(n-q+1)^2\)在引數取值區間\(0 \leq q \leq n-1\)的端點上取得最大值。

由於該表示式對\(q\)的二階導數是正的，我們可以得到表示式的上界

\[max_{0 \leq q \leq n-1}(q^2+(n-q+1)^2) \leq (n-1)^2 =n^2-2n+1 \]

將其帶入上式中的\(T(n)\)中，我們得到：

\[T(n) \leq cn^2-c(2n-1)+\Theta(n) \leq cn^2 \]

因為我們可以選擇一個足夠大的常數\(c\)，使得\(c(2n-1)\)項能顯著大於\(\Theta(n)\)項，所以有\(T(n)=O(n^2)\)。

最好情況劃分

在可能的最平衡劃分中，Partition得到的兩個子問題的規模都不大於\(n/2\)。這是因為其中一個子問題的規模為\(\lfloor n/2 \rfloor\)，而另一個子問題的規模為\(\lfloor n/2 \rfloor-1\)。在這種情況下，快速排序的效能非常好。

此時，演算法執行時間的遞迴式為：\(T(n)=2T(n/2)+\Theta(n)\) （我們忽略了一些餘項及減1操作的影響）

解得 \(T(n)=\Theta(nlogn)\)。

平衡的劃分

快速排序的平均執行時間更接近於其最好情況，而非最壞情況。

事實上，任何一種常數比例的劃分都會產生深度為\(\Theta(logn)\)的遞迴樹，其中每一層的時間代價都是\(O(n)\)。因此只要劃分是常數比例的，演算法的執行時間總是\(O(nlogn)\)。

快速排序的隨機化版本

在討論快速排序的平均情況時，我們的假設前提是：輸入資料的所有排列都是等概率的。但是在實際中，這個假設不會總成立。

採用隨機抽樣(random sampling)的隨機化技術。隨機抽樣是從子陣列\(A[p..r]\)中隨機選擇一個元素作為主元。為達到這一目的，首先將\(A[r]\)與從\(A[p..r]\)中隨機選擇的一個元素交換位置。通過對序列\(p,\cdots，r\)的隨機抽樣，我們可以保證主元元素\(x=A[r]\)是等概率地從子陣列的\(r-p+1\)個元素中選取的。因為主元元素是隨機選取的，我們期望在平均情況下，對輸入陣列的劃分是比較均衡的。

虛擬碼Randomized_QuickSort(A,p,r)

Randomized_QuickSort(A,p,r)
if	p < r
	q = Randomized_Partition(A,p,r)
	Randomized_Partition(A,p,q-1)
	Randomized_Partition(A,q+1,r)

Randomized_Partition(A,p,r)

Randomized_QuickSort(A,p,r)
i = Random(p,r)
exchange A[r] with A[i]
return Partition(A,p,r)

隨機化快速排序的期望執行時間

假設待排序的元素始終是互異的

*引理

當在一個包含\(n\)個元素的陣列上執行QuickSort時，假設在Partition的第5行中所作的比較次數為\(X\),那麼QuickSort的執行時間為\(O(n+X)\)

證明：QuickSort的執行時間是在Partition操作所花費的時間決定的。每次對Partition的呼叫時，都會選擇一個主元元素，而且該元素不會被後續的對QuickSort和Partition的遞迴呼叫中。因此，在快速排序演算法的整個執行階段，至多隻可能呼叫Partition操作\(n\)次。每次呼叫都包含一個固定的工作量和執行若干次\(for\)迴圈，在每一次\(for\)迴圈中，都要執行第5行。

我們的目標是計算出\(X\),即所有對Partition的呼叫中，所執行的總的比較次數。我們並不打算分析在每一次Partition呼叫中進行了多少次比較，而是希望能夠推匯出關於總的比較次數的一個界。

為此我們必須瞭解演算法在什麼時候對陣列中的兩個元素進行比較，什麼時候不會進行比較。

為了便於分析，我們將陣列A的各個元素重新命名為\(z_i,z_{i+1},\cdots ,z_j\)，其中\(z_i\)是陣列\(A\)中的第\(i\)小的元素。

我們還定義\(Z_{ij}=\{z_i,z_{i+1},\cdots ,z_j\}\)為\(z_i\)與\(z_j\)之間（含\(i\)和\(j\)）的元素集合。

演算法什麼時候會比較\(z_i\)和\(z_j\)呢？

我們首先注意到每一對元素至多比較一次。因為各個元素只與主元元素進行比較，並且在某一次的Partition呼叫結束之後，該元素就再也不會與其他元素進行比較了。

我們的分析要用到指示器隨機變數。定義

\[X_{ij}=I\{z_i與z_j進行比較\}=\begin{cases}1,如果z_i與z_j進行比較發生 \\ 0,如果z_i與z_j進行比較不發生 \end{cases} \]

由於每一對元素至多比較一次，所以我們可以計算出演算法的總比較次數：

\[X=\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}X_{ij} \]

對上式兩邊取期望，再由期望值的線性特徵和引理5.1，可以得到：

\[\begin{aligned} E(X) &=E[\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}] \\ &=\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}E[X_{ij}] \\ &=\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}Pr\{z_i與z_j進行比較\} \end {aligned} \]

假設Randomized_Partition隨機且獨立選擇主元。假設每一個元素是互異的。

一旦一個滿足\(z_i<x<z_j\)的主元\(x\)被選擇後，我們就知道\(z_i\)和\(z_j\)以後再也不可能進行比較了。

另一種情況，如果\(z_i\)在\(Z_{ij}\)中的所有其他元素之前被選擇為主元，那麼\(z_i\)就將與除了它自身以外的所有元素進行比較。類似的，如果\(z_j\)在\(Z_{ij}\)中的所有其他元素之前被選擇為主元，那麼\(z_j\)就將與除了它自身以外的所有元素進行比較。

因此，\(z_i\)與\(z_j\)會進行比較，當且僅當\(Z_{ij}\)中被選為主元的第一個元素是\(z_i\)或者\(z_j\)。

在\(Z_{ij}\)的某個元素被選為主元之前，整個集合\(Z_{ij}\)的元素都屬於某一劃分的同一分割槽。因此，\(Z_{ij}\)中的任何元素都會等可能首先被選為主元。因為集合\(Z_{ij}\)中有\(j-i+1\)個元素，並且主元的選取是隨機獨立的。所以任何元素被首先選為主元的概率是\(\frac{1}{j-i+1}\)。

\[\begin{aligned} Pr\{z_i與z_j進行比較\} & =Pr\{z_i或z_j是集合Z_{ij}中選出的第一個主元 \} \\ &=Pr\{z_i是集合Z_{ij}中選出的第一個主元\}+Pr\{z_j是集合Z_{ij}中選出的第一個主元\} \\ &=\frac{1}{j=i+1}+\frac{1}{j=i+1} \\ &=\frac{2}{j-i+1} \end{aligned} \]

上式中第二行成立的原因在於其中涉及的兩個事件是互斥的。

於是綜上兩式我們有:

\[E[X]=\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}\frac{2}{j-i+1} \]

在求這個累加和時。可以將變數做個變換\((k=j-i)\)，並利用有關調和級數的界，得到：

\[\begin{aligned} E[X] &=\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}\frac{2}{j-i+1} \\ &=\sum_{i=1}^{n-1}\sum_{k=1}^{n-i}\frac{2}{k+1}\\ &<\sum_{i=1}^{n-1}\sum_{k=1}^{n}\frac{2}{k} \\ &=\sum_{i=1}^{n-1}O(logn) \\ &=O(nogn) \end{aligned} \]

於是我們可以得到結論：

使用Randomized_Partition，在輸入元素互異的情況下，快速排序的期望執行時間為\(O(nlogn)\)。

7-3另一種快速排序的分析方法

An alternative analysis of the running time of randomized quicksort focuses on the expected running time of each individual recursive call to \(\text{RANDOMIZED-QUICKSORT}\), rather than on the number of comparisons performed.

這一方法關注於每一次單獨遞迴呼叫的期望執行時間，而不是比較的次數。

a. 給定一個大小為\(n\)的陣列，特定元素選為主元的概率為\(1/n\)。利用這一點來定義指示器隨機變數：

\[X_{i}=I\{第i小的元素被選為主元\}=\begin{cases}1,如果第i小的元素被選為主元發生 \\ 0,如果第i小的元素被選為主元不發生 \end{cases} \]

\(E[X_i]\)是什麼？

b. 設\(T(n)\)是一個表示快速排序在一個大小為\(n\)的陣列上的執行時間的隨機變數，試證明：

\[E[T(n)]=E[\sum_{q=1}^nX_q(T(q-1)+T(n-q)+\Theta(n))] \ \ \ (7.5) \]

c. 證明公式\(（7.5）\)可以重寫為：

\[E[T(n)] = \frac{2}{n} \sum_{q=2}^{n-1}E[T(q)]+\Theta(n) \ \ \ (7.6) \]

d. 證明：

\[\sum_{k=2}^{n-1}klogk\leq \frac{1}{2}n^2logn-\frac{1}{8}n^2 \ \ (7.7) \]

（提示：可以將該累加式分成兩個部分，一部分是\(k=2,3,\cdots,\lceil n/2 \rceil-1\)，另一部分是\(k=\lceil n/2 \rceil ,\cdots,n-1\)。）

e. 利用公式\((7.7)\)給出的界，證明：公式\((7.6)\)中的遞迴式有解\(E[T(n)]=\Theta(nlog(n))\)。

（提示：使用代入法，證明對某個正常數\(a\)和足夠大的\(n\)，有\(E[T(n)]\leq anlog(n)\)）。

1. 給定一個大小為\(n\)的陣列，特定元素選為主元的概率為\(1/n\)。

   定義指示器隨機變數：

   \[X_{i}=I\{第i小的元素被選為主元\}=\begin{cases}1,如果第i小的元素被選為主元發生 \\ 0,如果第i小的元素被選為主元不發生 \end{cases} \]

   Since the pivot is selected as a random element in the array, which has size \(n\), the probabilities of any particular element being selected are all equal, and add to one, so, are all$ \frac{1}{n}$. .

   \[\begin{aligned}E[X_i] &=Pr\{第i小的元素被選為主元\} \\ &=\frac{1}{n}\end{aligned} \]

   b.證明：

   We can apply linearity of expectation over all of the events \(X_i\). Suppose we have a particular \(X_i\) be true.

   Then, we will have one of the sub arrays be length \(i - 1\) and the other be \(n - i\),

   and will of course still need linear time to run the partition procedure.

   This corresponds exactly to the summand in equation \((7.5)\)。

   被隨機選中的元素	$$z_1$$	$$z_2$$	$$\dots$$	\(z_{n-1}\)	\(z_n\)
   執行時間	\((T(0)+T(n-1)+O(n))\)	\((T(1)+T(n-2)+O(n))\)	\(\dots\)	\((T(n-2)+T(1)+O(n))\)	\((T(n-1)+T(0)+O(n))\)
   概率	$$X_1$$	$$X_2$$	\(\dots\)	\(X_{n-1}\)	\(X_{n}\)

   \[T(n)=\sum_{q=1}^nX_q(T(q-1)+T(n-q)+\Theta(n)) \\ E[T(n)]=E[\sum_{q=1}^nX_q(T(q-1)+T(n-q)+\Theta(n))] \]

2. 證明公式\(（7.5）\)可以重寫為：

   \[E[T(n)] = \frac{2}{n} \sum_{q=2}^{n-1}E[T(q)]+\Theta(n) \ \ \ (7.6) \]

   證明：

   \[\begin{aligned} E[T(n)]& =E[\sum_{q=1}^nX_q(T(q-1)+T(n-q)+\Theta(n))] \ \ \ (7.5)\\ & =\sum_{q=1}^nE[X_q(T(q-1)+T(n-q)+\Theta(n))] \\ & =\sum_{q=1}^nE[X_q]*E[(T(q-1)+T(n-q)+\Theta(n))] \\ & =\sum_{q=1}^n\frac{1}{n}*E[T(q-1)+T(n-q)+\Theta(n)] \\ &= \Theta(n)+\frac{1}{n} * \sum_{q=1}^{n-1}(E[T(q-1)+T(n-q)]) \\ \end {aligned} \]

   選定\(X_q\)並完成一次Partition操作後，需要排序的陣列剩餘長度為n-1,

   被劃分的左端和右端繼續進行相同的處理，不管怎樣劃分，由於選取特定元素的概率相同，可以認為在接下來的操作中，左端和右段進行快速排序期望的執行時間相同，不妨令左端等於左端，即\(E[T(q-1)]=E[T(n-q)]\),於是我們有

   \[\begin{aligned} E[T(n)] &= \Theta(n)+\frac{1}{n} * \sum_{q=1}^{n-1}(E[T(q-1)+T(n-q)]) \\ &=\Theta(n)+\frac{1}{n} * \sum_{q=1}^{n-1}(E[T(q-1)+T(q-1)]) \\ &=\Theta(n)+\frac{2}{n} * \sum_{q=1}^{n-1}(E[T(q-1)]) \\ &=\Theta(n)+\frac{2}{n} * \sum_{q=2}^{n}(E[T(q)]) \\ \end {aligned} \]

   實際上，當只有一個元素時可以認為已經排序完成，不需要再次排序。

   4.證明：

   \[\sum_{k=2}^{n-1}klogk\leq \frac{1}{2}n^2logn-\frac{1}{8}n^2 \ \ (7.7) \]

   （提示：可以將該累加式分成兩個部分，一部分是\(k=2,3,\cdots,\lceil n/2 \rceil-1\)，另一部分是\(k=\lceil n/2 \rceil ,\cdots,n-1\)。）

   證明：

   \[\begin{aligned} let: \ \ &f(k) =k log(k) \\ &f'=log(k)+1 \\ & if \ k>2, f(k) \nearrow \\ F &=\int f(k) \\ &=\frac{x^2}{2}(log(x)-\frac{1}{2})+C \\ & because \ f(k)>0 , F \nearrow \\ \int_2^{n-1} f(k)&=F_{n-1}-F_2 \\ &\leq F_n -F_2 \\ &=\frac{n^2}{2}log(n)-\frac{n^2}{4}-2(log(2)- \frac{1}{2}) \\ &\leq\frac{n^2}{2}log(n)-\frac{n^2}{4} \\ &\leq\frac{n^2}{2}log(n)-\frac{n^2}{8} \\ \sum_{k=2}^{n-1}klogk&\leq \int_2^{n-1} f(k) \\ &\leq \frac{n^2}{2}log(n)-\frac{n^2}{8}\\ \end {aligned} \]

   5.利用公式\((7.7)\)給出的界，證明：公式\((7.6)\)中的遞迴式有解\(E[T(n)]=\Theta(nlog(n))\)。

   （提示：使用代入法，證明對某個正常數\(a\)和足夠大的\(n\)，有\(E[T(n)]\leq anlog(n)\)）。

   證明：

   \[\begin{aligned} &let: \ \ T(q)\leq qlog(q)+ \Theta(n) \\ E[T(n)] &= \frac{2}{n} \sum_{q=2}^{n-1}E[T(q)]+\Theta(n) \ \ \ (7.6) \\ &\leq \frac{2}{n} \sum_{q=2}^{n-1} (qlog(q)+ \Theta(n) )+ \Theta(n)\\ &=\frac{2}{n} \sum_{q=2}^{n-1} qlog(q)+\frac{2}{n}(n-3)\Theta(n)+\Theta(n) \\ &\leq \frac{2}{n} (\frac{n^2}{2}log(n)-\frac{n^2}{8}) +\Theta(n) \\ &=nlog(n)-\frac{n}{4}+\Theta(n) \\ &=nlog(n)+\Theta(n) \end{aligned} \]

7-4快速排序的棧深度

The \(\text{QUICKSORT}\) algorithm of Section 7.1 contains two recursive calls to itself. After \(\text{QUICKSORT}\) calls \(\text{PARTITION}\), it recursively sorts the left subarray and then it recursively sorts the right subarray. The second recursive call in \(\text{QUICKSORT}\) is not really necessary; we can avoid it by using an iterative control structure. This technique, called *tail recursion*, is provided automatically by good compilers. Consider the following version of quicksort, which simulates tail recursion:

TAIL-RECURSIVE-QUICKSORT(A, p, r)
 while p < r
     // Partition and sort left subarray.
     q = PARTITION(A, p, r)
     TAIL-RECURSIVE-QUICKSORT(A, p, q - 1)
     p = q + 1

a. Argue that \(\text{TAIL-RECURSIVE-QUICKSORT}(A, 1, A.length)\) correctly sorts the array \(A\).

Compilers usually execute recursive procedures by using a *stack* that contains pertinent information, including the parameter values, for each recursive call. The information for the most recent call is at the top of the stack, and the information for the initial call is at the bottom. Upon calling a procedure, its information is *pushed* onto the stack; when it terminates, its information is *popped*. Since we assume that array parameters are represented by pointers, the information for each procedure call on the stack requires \(O(1)\) stack space. The *stack depth* is the maximum amount of stack space used at any time during a computation.

b. Describe a scenario in which \(\text{TAIL-RECURSIVE-QUICKSORT}\)'s stack depth is \(\Theta(n)\) on an \(n\)-element input array.

c. Modify the code for \(\text{TAIL-RECURSIVE-QUICKSORT}\) so that the worst-case stack depth is \(\Theta(\lg n)\). Maintain the \(O(n\lg n)\) expected running time of the algorithm.

實際上，在\(C++ STL\)庫中就使用了尾遞迴，這樣可以減少棧空間的使用。

它並不是沒有管左子序列。注意看，在分割原始區域之後，對左子序列\([p,q-1]\)進行了遞迴，接下來的\(p=q+1\)將起點位置調整到了分割點，那麼此時的區間就是右子序列了。又因為這是一個迴圈結構，那麼在下一次的迴圈中，右子序列便得到了處理。只是並未以遞迴來呼叫。

我們來比較一下兩者的區別，試想，如果一個序列只需要遞迴兩次便可結束，即它可以分成四個子序列。原始的方式需要兩個遞迴函式呼叫，接著兩者各自呼叫一次，也就是說進行了7次函式呼叫，如下圖左邊所示。但是STL這種寫法每次劃分子序列之後僅對右子序列進行函式呼叫，左邊子序列進行正常的迴圈呼叫，如下圖右邊所示。

兩者區別就在於STL節省了接近一半的函式呼叫，由於每次的函式呼叫有一定的開銷，因此對於資料量非常龐大時，這一半的函式呼叫可能能夠省下相當可觀的時間。

a. The book proved that \(\text{QUICKSORT}\) correctly sorts the array \(A\). \(\text{TAIL-RECURSIVE-QUICKSORT}\) differs from \(\text{QUICKSORT}\) in only the last line of the loop.

It is clear that the conditions starting the second iteration of the while loop in \(\text{TAIL-RECURSIVE-QUICKSORT}\) are identical to the conditions starting the second recursive call in \(\text{QUICKSORT}\). Therefore, \(\text{TAIL-RECURSIVE-QUICKSORT}\) effectively performs the sort in the same manner as \(\text{QUICKSORT}\). Therefore, \(\text{TAIL-RECURSIVE-QUICKSORT}\) must correctly sort the array \(A\).

b. The stack depth will be \(\Theta(n)\) if the input array is already sorted. The right subarray will always have size \(0\) so there will be \(n − 1\) recursive calls before the while-condition \(p < r\) is violated.\

c.

MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, p, r)
while p < r
  q = PARTITION(A, p, r)
  if q < floor((p + r) / 2)
      MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, p, q - 1)
      p = q + 1
  else
      MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, q + 1, r)
      r = q - 1

7-5三數取中劃分

One way to improve the \(\text{RANDOMIZED-QUICKSORT}\) procedure is to partition around a pivot that is chosen more carefully than by picking a random element from the subarray. One common approach is the *median-of-3*method: choose the pivot as the median (middle element) of a set of 3 elements randomly selected from the subarray. (See exercise 7.4-6.) For this problem, let us assume that the elements of the input array \(A[1..n]\) are distinct and that \(n \ge 3\). We denote the sorted output array by \(A'[1..n]\). Using the median-of-3 method to choose the pivot element \(x\), define \(p_i = \Pr\{x = A'[i]\}\).

a. Give an exact formula for \(p_i\) as a function of \(n\) and \(i\) for \(i = 2, 3, \ldots, n - 1\). (Note that \(p_1 = p_n = 0\).)

b. By what amount have we increased the likelihood of choosing the pivot as \(x = A'[\lfloor (n + 1) / 2 \rfloor]\), the median of \(A[1..n]\), compared with the ordinary implementation? Assume that \(n \to \infty\), and give the limiting ratio of these probabilities.

c. If we define a "good" split to mean choosing the pivot as \(x = A'[i]\), where \(n / 3 \le i \le 2n / 3\), by what amount have we increased the likelihood of getting a good split compared with the ordinary implementation? (\(\textit{Hint:}\) Approximate the sum by an integral.)

d. Argue that in the \(\Omega(n\lg n)\) running time of quicksort, the median-of-3 method affects only the constant factor.

答案

a. \(p_i\) is the probability that a randomly selected subset of size three has the \(A'[i]\) as it's middle element. There are 6 possible orderings of the three elements selected. So, suppose that \(S'\) is the set of three elements selected. We will compute the probability that the second element of \(S'\) is \(A'[i]\) among all possible \(3\)-sets we can pick, since there are exactly six ordered \(3\)-sets corresponding to each \(3\)-set, these probabilities will be equal. We will compute the probability that \(S'[2] = A[i]\).

For any such \(S'\), we would need to select the first element from \([i - 1]\) and the third from \({i + 1, \ldots , n}\). So, there are \((i - 1)(n - i)\) such \(3\)-sets.

The total number of \(3\)-sets is \(\binom{n}{3} = \frac{n(n - 1)(n - 2)}{6}\).

So,$$p_i = \frac{6(n - i)(i - 1)}{n(n - 1)(n - 2)}.$$

b. If we let \(i = \lfloor \frac{n + 1}{2} \rfloor\), the previous result gets us an increase of

\[\frac{6(\lfloor\frac{n - 1}{2}\rfloor)(n - \lfloor\frac{n + 1}{2}\rfloor)}{n(n - 1)(n - 2)} - \frac{1}{n} \]

in the limit \(n\) going to infinity, we get $$\lim_{n \to \infty} \frac{\frac{6(\lfloor \frac{n - 1}{2} \rfloor)(n - \lfloor \frac{n + 1}{2} \rfloor)}{n(n - 1)(n - 2)}}{\frac{1}{n}} = \frac{3}{2}.$$

c. To save the messiness, suppose \(n\) is a multiple of \(3\). We will approximate the sum as an integral, so,

\[\begin{aligned} \sum_{i = n / 3}^{2n / 3} & \approx \int_{n / 3}^{2n / 3} \frac{6(-x^2 + nx + x - n)}{n(n - 1)(n - 2)}dx \\ & = \frac{6(-7n^3 / 81 + 3n^3 / 18 + 3n^2 / 18 - n^2 / 3)}{n(n - 1)(n - 2)}, \end{aligned} \]

which, in the limit \(n\) goes to infinity, is \(\frac{13}{27}\) which is a constant that \(>\frac{1}{3}\) as it was in the original randomized quicksort implementation.

d. Even though we always choose the middle element as the pivot (which is the best case), the height of the recursion tree will be \(\Theta(\lg n)\). Therefore, the running time is still \(\Omega(n\lg n)\).

7-6對區間的模糊排序

Consider the problem in which we do not know the numbers exactly. Instead, for each number, we know an interval on the real line to which it belongs. That is, we are given \(n\) closed intervals of the form \([a_i, b_i]\), where \(a_i \le b_i\). We wish to *fuzzy-sort* these intervals, i.e., to produce a permutation \(\langle i_1, i_2, \ldots, i_n \rangle\) of the intervals such that for \(j = 1, 2, \ldots, n\), there exists \(c_j \in [a_{i_j}, b_{i_j}]\) satisfying \(c_1 \le c_2 \le \cdots \le c_n\).

a. Design a randomized algorithm for fuzzy-sorting \(n\) intervals. Your algorithm should have the general structure of an algorithm that quicksorts the left endpoints (the \(a_i\) values), but it should take advantage of overlapping intervals to improve the running time. (As the intervals overlap more and more, the problem of fuzzy-sorting the intervals becoes progressively easier. Your algorithm should take advantage of such overlapping, to the extend that it exists.)

b. Argue that your algorithm runs in expected time \(\Theta(n\lg n)\) in general, but runs in expected time \(\Theta(n)\) when all of the intervals overlap (i.e., when there exists a value \(x\) such that \(x \in [a_i, b_i]\) for all \(i\)). Your algorithm should not be checking for this case explicitly; rather, its performance should naturally improve as the amount of overlap increases.

a. With randomly selected left endpoint for the pivot, we could trivially perform fuzzy sorting by quicksorting the left endpoints, \(a_i\)'s. This would achieve the worst-case expected running time of \(\Theta(n\lg n)\). We definitely can do better by exploit the characteristic that we don't have to sort overlapping intervals. That is, for two overlapping intervals, \([a_i, b_i]\) and \([a_j, b_j]\). In such situations, we can always choose \(\{c_i, c_j\}\) (within the intersection of these intervals) such that \(c_i \le c_j\) or \(c_j \le c_i\).

Since overlapping intervals do not require sorting, we can improve the expected running time by modifying quicksort to identify overlaps:

FIND-INTERSECTION(A, p, r)
rand = RANDOM(p, r)
exchange A[rand] with A[r]
a = A[r].a
b = A[r].b
for i = p to r - 1
  if A[i].a ≤ b and A[i].b ≥ a
      if A[i].a > a
          a = A[i].a
      if A[i].b < b
          b = A[i].b
return (a, b)

On lines 2 through 3 of \(\text{FIND-INTERSECTION}\), we select a random pivot interval as the initial region of overlap \([a ,b]\). There are two situations:

• If the intervals are all disjoint, then the estimated region of overlap will be this randomly-selected interval;
• otherwise, on lines 6 through 11, we loop through all intervals in arrays \(A\) (except the endpoint which is the initial pivot interval). At each iteration, we determine if the current interval overlaps the current estimated region of overlap. If it does, we update the estimated region of overlap as \([a, b] = [a_i, b_i] \cap [a, b]\).

\(\text{FIND-INTERSECTION}\) has a worst-case running time \(\Theta(n)\) since we evaluate the intersection from index \(1\) to \(A.length\) of the array.

We can extend the \(\text{QUICKSORT}\) to allow fuzzy sorting using \(\text{FIND-INTERSECTION}\).

First, partition the input array into "left", "middle", and "right" subarrays. The "middle" subarray elements overlap the interval \([a, b]\) found by \(\text{FIND-INTERSECTION}\). As a result, they can appear in any order in the output.

We recursively call \(\text{FUZZY-SORT}\) on the "left" and "right" subarrays to produce a fuzzy sorted array in-place. The following pseudocode implements these basic operations. One can run \(\text{FUZZY-SORT}(A, 1, A.length)\) to fuzzy-sort an array.

The first and last elements in a subarray are indexed by \(p\) and \(r\), respectively. The index of the first and last intervals in the "middle" region are indexed by \(q\) and \(t\), respectively.

FUZZY-SORT(A, p, r)
  if p < r
      (a, b) = FIND-INTERSECTION(A, p, r)
      t = PARTITION-RIGHT(A, a, p, r)
      q = PARTITION-LEFT(A, b, p, t)
      FUZZY-SORT(A, p, q - 1)
      FUZZY-SORT(A, t + 1, r)

We need to determine how to partition the input arrays into "left", "middle", and "right" subarrays in-place.

First, we \(\text{PARTITION-RIGHT}\) the entire array from \(p\) to \(r\) using a pivot value equal to the left endpoint \(a\) found by \(\text{FIND-INTERSECTION}\), such that \(a_i \le a\).

Then, we \(\text{PARTITION-LEFT}\) the subarray from \(p\) to \(t\) using a pivot value equal to the right endpoint \(b\) found by \(\text{FIND-INTERSECTION}\), such that \(b_i < b\).

PARTITION-RIGHT(A, a, p, r)
  i = p - 1
  for j = p to r - 1
      if A[j].a ≤ a
          i = i + 1
          exchange A[i] with A[j]
  exchange A[i + 1] with A[r]
  return i + 1
PARTITION-LEFT(A, b, p, t)
  i = p - 1
  for j = p to t - 1
      if A[j].b < b
          i = i + 1
          exchange A[i] with A[j]
  exchange A[i + 1] with A[t]
  return i + 1

The \(\text{FUZZY-SORT}\) is similar to the randomized quicksort presented in the textbook. In fact, \(\text{PARTITION-RIGHT}\) and \(\text{PARTITION-LEFT}\) are nearly identical to the \(\text{PARTITION}\) procedure on page 171. The primary difference is the value of the pivot used to sort the intervals.

b. We expect \(\text{FUZZY-SORT}\) to have a worst-case running time \(\Theta(n\lg n)\) for a set of input intervals which do not overlap each other. First, notice that lines 2 through 3 of \(\text{FIND-INTERSECTION}\) select a random interval as the initial pivot interval. Recall that if the intervals are disjoint, then \([a, b]\) will simply be this initial interval.

Since for this example there are no overlaps, the "middle" region created by lines 4 and 5 of \(\text{FUZZY-SORT}\) will only contain the initially-selected interval. In general, line 3 is \(\Theta(n)\). Fortunately, since the pivot interval \([a, b]\) is randomly-selected, the expected sizes of the "left" and "right" subarrays are both \(\left\lfloor \frac{n}{2} \right\rfloor\). In conclusion, the recurrence of the running time is

\[\begin{aligned} T(n) & \le 2T(n / 2) + \Theta(n) \\ & = \Theta(n\lg n). \end{aligned} \]

The \(\text{FIND-INTERSECTION}\) will always return a non-empty region of overlap \([a, b]\) containing \(x\) if the intervals all overlap at \(x\). For this situation, every interval will be within the "middle" region since the "left" and "right" subarrays will be empty, lines 6 and 7 of \(\text{FUZZY-SORT}\) are \(\Theta(1)\). As a result, there is no recursion and the running time of \(\text{FUZZY-SORT}\) is determined by the \(\Theta(n)\) running time required to find the region of overlap. Therefor, if the input intervals all overlap at a point, then the expected worst-case running time is \(\Theta(n)\).