Solving Divide-and-Conquer Recurrences
Victor Adamchik
A divide-and-conquer algorithm consists of three steps:
•
dividing a problem into smaller subproblems
•
solving (recursively) each subproblem
•
then combining solutions to subproblems to get solution to original problem
We use recurrences to analyze the running time of such algorithms. Suppose Tn is the number of steps in the worst case needed to solve the problem of size n. Let us split a n problem into a ¥ 1 subproblems, each of which is of the input size b where b > 1.
Observe, that the number of subproblems a is not necessarily equal to b. The total number of steps Tn is obtained by all steps needed to solve smaller subproblems Tnêb plus the number needed to combine solutions into a final one. The following equation is called divide-and-conquer recurrence relation
Tn = a Tnêb + f HnL
As an example, consider the mergesort:
-divide the input in half
-recursively sort the two halves
-combine the two sorted subsequences by merging them.
Let THnL be worst-case runtime on a sequence of n keys:
If n = 1, then THnL = QH1L constant time
If n > 1, then THnL = 2 THn ê 2L + QHnL here Q(n) is time to do the merge. Then
15-451: Algorithm Design and Analysis
2
Tn = 2 Tnê2 + QHnL
Other examples of divide and conquer algorithms: quicksort, integer multiplication, matrix multiplication, fast Fourier trnsform, finding conver hull and more.
There are several techniques of solving such recurrence equations:
•
the iteration method
•
the tree method
•
the master-theorem method
•
guess-and-verify ü Tree method
We could visualize the recursion as a tree, where each node represents a recursive call.
The root is the initial call. Leaves correspond to the exit condition. We can often solve the recurrence by looking at the structure of the tree. To illustrate, we take this example n THnL = 2 TK O + n2
2
TH1L = 1
Here is a recursion tree that diagrams the recursive function calls
T(n)
T(n/2)
T(n/4)
T(n/2)
T(n/4)
T(n/4)
T(n/4)
………………
T(1)
T(1)
Using a recursion tree we can model the time of a recursive execution by writing the size of the problem in each node.
3
Using a recursion tree we can model the time of a recursive execution by writing the size of the problem in each node.
The last level corresponds to the initial condition of the recurrence. Since the work at each leaf is constant, the total work at all leaves is equal to the number of leaves, which is
2h = 2log2 n = n
To find the total time (for the whole tree), we must add up all the terms
2
THnL = n + n 1 +
1
2
+
1
4
+
1
8
-1+log2 n
2
+ ... = n + n
k=0
The sum is easily computed by means of the geometric series h ‚ xk = k=0 This yeilds
xh+1 - 1 x-1 ‚
, x∫1
1
2
k
15-451: Algorithm Design and Analysis
4
THnL = 2 n2 - 2 n + n = 2 n2 - n
Check with Mathematica
RSolveA9T@nD == 2 T@n ê 2D + n2 , T@1D ä 1=, T@nD, nE
88T@nD → n H−1 + 2 nL<<
Example. Solve the recurrence n THnL = 3 TK O + n
4
The work at all levels is n 1+
3
4
9
+
16
+ ...
Since the height is log4 n, the tree has 3log4 n leaves. Hence, the total work is given by
-1+log4 n
THnL = n
‚ k=0 3
4
k
+ 3log4 n TH1L
By means of the geometric series and taking into account
3log4 n = nlog4 3 the above sum yields
THnL = 4 n - 4 nlog4 3 + nlog4 3 TH1L = OHnL ü The Master Theorem
The master theorem solves recurrences of the form n THnL = a T
+ f HnL b for a wide variety of function f HnL and a ¥ 1, b > 1. In this section we will outline the
5
H L main idea. Here is the recursive tree for the above equation
It is easy to see that the tree has alogb n leaves. Indeed, since the height is logb n, and the tree branching factor is a, the number of leaves is loga n
1
ah = alogb n = a log b = n log b = nlogb a a a
Summing up values at each level, gives
THnL = f HnL + a f
n b + a2 f
n
2
b
+ ... + nlogb a TH1L
Therefore, the solution is
-1+logb n logb a
THnL = n
TH1L +
‚ ak f k=0 n bk Now we need to compare the asymptotic behavior of
