Transcript Document 7192426

Design and Analysis of Algorithm
Lecture on
Divide and Conquer
Pradondet Nilagupta [email protected]
Department of Computer Engineering
Kasetsart University
Acknowledgement
 This lecture note has been summarized from lecture note
on Data Structure and Algorithm, Design and Analysis of
Computer Algorithm all over the world. I can’t remember
where those slide come from. However, I’d like to thank
all professors who create such a good work on those
lecture notes. Without those lectures, this slide can’t be
finished.
Divide and Conquer Technique
Friday, 20 May 2016
2
Review: What is an algorithm?
 A step by step procedure for performing some
task in a finite amount of time
 A precise set of instructions for achieving a
particular goal
Divide and Conquer Technique
Friday, 20 May 2016
3
Review: General Concepts
 Algorithm strategy


Approach to solving a problem
May combine several approaches
 Algorithm structure


Iterative
Recursive
 execute action in loop
 reapply action to subproblem(s)
 Problem type


Satisfying
 find any satisfactory solution
Optimization  find best solutions (vs. cost
metric)
Divide and Conquer Technique
Friday, 20 May 2016
4
Review: What you need to know about
algorithms?
 Running time
 Memory requirements
 Understand how the complexity of an algorithm
is measured
 Develop techniques to measure these
complexities experimentally
 In other words, measure the efficiency of an
algorithm!
Divide and Conquer Technique
Friday, 20 May 2016
5
Review: Measures of efficiency
 Three main measures



Best
Worst
Average
 E.g. consider a list



Best: first item in the list
Worst: last in the list, or not in the list
Average: in the middle of the list
Divide and Conquer Technique
Friday, 20 May 2016
6
Review: An example of an algorithm
(pseudocode)
BEGIN
Get WAGES
IF WAGES <=100
Tax = wages*0.25
ELSE
Tax = 100*0.25 + (wages-100)*0.4
END IF
Display Tax
END
It calculates tax at 25% on the first £100 of
earnings then at 40% on subsequent earnings.
Divide and Conquer Technique
Friday, 20 May 2016
7
Some Algorithm Strategies
 Recursive algorithms
 Backtracking algorithms
 Divide and conquer algorithms
 Dynamic programming algorithms
 Greedy algorithms
 Brute force algorithms
 Branch and bound algorithms
 Heuristic algorithms
Divide and Conquer Technique
Friday, 20 May 2016
8
Divide-and-Conquer
 A general methodology for using recursion to
design efficient algorithms
 It solves a problem by:



Diving the data into parts
Finding sub solutions for each of the parts
Constructing the final answer from the sub
solutions
Divide and Conquer Technique
Friday, 20 May 2016
9
Divide and Conquer
 Based on dividing problem into subproblems
 Approach
1.
Divide problem into smaller subproblems
Subproblems must be of same type
Subproblems do not need to overlap
2.
3.
Solve each subproblem recursively
Combine solutions to solve original problem
 Usually contains two or more recursive calls
Divide and Conquer Technique
Friday, 20 May 2016
10
Divide-and-conquer technique
a problem of size n
subproblem 1
of size n/2
subproblem 2
of size n/2
a solution to
subproblem 1
a solution to
subproblem 2
a solution to
the original problem
Divide and Conquer Technique
Friday, 20 May 2016
11
Divide and Conquer Algorithms
 Based on dividing problem into subproblems
Divide problem into sub-problems
Subproblems must be of same type
Subproblems do not need to overlap
 Conquer by solving sub-problems recursively.
If the sub-problems are small enough, solve
them in brute force fashion
 Combine the solutions of sub-problems into a
solution of the original problem (tricky part)

Divide and Conquer Technique
Friday, 20 May 2016
12
D-A-C
 For Divide-and-Conquer algorithms the running
time is mainly affected by 3 criteria:
 The number of sub-instances into which a
problem is split.
 The ratio of initial problem size to subproblem size.
 The number of steps required to divide the
initial instance and to combine sub-solutions.
Divide and Conquer Technique
Friday, 20 May 2016
13
An example of D-A-C
 Given an array of n values find, in order, the lowest two
values efficiently.
 To make life easier we assume n is at least 2 and that it
is always divisible by 2
 Have you understood the problem??
2
10
9
6
4
5
 The solution should be 2 followed by 4
What if??
Divide and Conquer Technique
2
8
2
7
Friday, 20 May 2016
9
3
14
Possible Solutions
 Sort the elements and take the two first

We get more information than we require (not
very efficient!!)
 Find the lowest, then find the second lowest


Better than the first but still we have to go through
the array two times
Be careful not to count the lowest value twice
 Search through the array keeping track of the
lowest two found so far
 Lets see how the divide and conquer technique
works!
Divide and Conquer Technique
Friday, 20 May 2016
15
Divide and conquer approach
 There are many ways to divide the array
 It seems reasonable that the division should be
related to the problem
 Division based on 2!!


Divide the array into two parts?
Divide the array into pairs?
 Go ahead with the second option!
Divide and Conquer Technique
Friday, 20 May 2016
16
The algorithm
 Lets assume the following array
2 6 7 3 5 6 9 2 4 1
 We divide the values into pairs
2 6 7 3 5 6 9 2 4 1
 We sort each pair
2 6 3 7 5 6 2 9 1 4
 Get the first pair (both lowest values!)
Divide and Conquer Technique
Friday, 20 May 2016
17
The algorithm (2)
 We compare these values (2 and 6) with the values of
the next pair (3 and 7)
2 6 3 7 5 6 2 9 1 4

Lowest 2,3
 The next one (5 and 6)
 Lowest 2,3
 The next one (2 and 9)
 Lowest 2,2
 The next one (1 and 4)
 Lowest 1,2
Divide and Conquer Technique
Friday, 20 May 2016
18
Example: Divide and Conquer
 Binary Search
 Heap Construction
 Tower of Hanoi
 Exponentiation
Fibonnacci Sequence
 Quick Sort
 Merge Sort
 Multiplying large Integers
 Matrix Multiplications
 Closest Pairs

Divide and Conquer Technique
Friday, 20 May 2016
19
Exponentiation
Power can be easily computed in O(logn) time as follows:
long Power (int x, int n) {
if (n==0) return 1;
else {
y = Power (x, n/2);
if (n % 2 == 0) return(y*y)
else return (y*y*x);
}
Divide and Conquer Technique
Friday, 20 May 2016
// n even
20
Time Complexity Example 1:
Fibonnacci Sequence
Fibonnacci sequence F(n) = F(n-1) + F(n-2)
Approach 1:
Approach 2:
int Fib(n)
{
if (n < 2) return (n);
else {
return (Fib (n-1) + Fib (n-2));
}
}
Divide and Conquer Technique
int Fib(n) {
if (n < 2) return (n);
else {
int F1 = 1; F2 = 0;
for (i = 2; i <= n; i++)
{ F = F1 + F2;
F2 = F1;
F1 = F;
}
return (F)
}
Friday, 20 May 2016
21
Fibonnacci Sequence
Approach 3: the complexity is O(logn)
Note that we can rewrite
| F(n) |
|
|
| F(n-1)|
Thus,
|
|
|
F(n)
=
|
|
F(n-1)|
=
| 1
|
| 1
| 1 1 |
| F(n-1) |
|
| * |
|
| 1 0 |
| F(n-2) |
| 1
= |
| 1
1 |
|
0 |
1 |
| F(n-1) |
| 1
| * |
| = |
0 |
| F(n-2) |
| 1
2
1 |
| F(n-2) |
| * |
|
0 |
| F(n-3) |
(n-1)
| F(1) |
* |
|
| F(0) |
Therefore, computing F(n) is reduced to computing
| 1
|
| 1
Divide and Conquer Technique
1 |
|
0 |
(n-1)
Friday, 20 May 2016
22
Heap Construction
Construct heap of T =>
Construct heap T1,
Construct heap T2
Adjust heap with root T
T
T1
Divide and Conquer Technique
T2
Friday, 20 May 2016
23
Divide and Conquer – Examples
 Quicksort



Partition array into two parts around pivot
Recursively quicksort each part of array
Concatenate solutions
 Mergesort



Partition array into two parts
Recursively mergesort each half
Merge two sorted arrays into single sorted array
Divide and Conquer Technique
Friday, 20 May 2016
24
Quick Sort
Sorting Problem Revisited
Given: an unsorted array
5 2 4 7 1 3 2 6
 Goal: sort it
1 2 2 3 4 5 6 7
Divide and Conquer Technique
Friday, 20 May 2016
26
Quick Sort
 Approach
1.
2.
3.
4.

Select pivot value (near median of list)
Partition elements (into 2 lists) using pivot value
Recursively sort both resulting lists
Concatenate resulting lists
For efficiency pivot needs to partition list evenly
 Performance


O( n log(n) ) average case
O( n2 ) worst case
Divide and Conquer Technique
Friday, 20 May 2016
27
Quick Sort Algorithm
1. If list below size K

Sort w/ other algorithm
x
2. Else pick pivot x and
partition S into



L elements < x
E elements = x
G elements > x
x
L
E
G
3. Quicksort L & G
4. Concatenate L, E & G

x
If not sorting in place
Divide and Conquer Technique
Friday, 20 May 2016
28
Quick Sort Code
void quickSort(int[] a, int x, int y) {
int pivotIndex;
if ((y – x) > 0) {
pivotIndex = partionList(a, x, y);
quickSort(a, x, pivotIndex – 1);
quickSort(a, pivotIndex+1, y);
}
}
Lower Upper
end of end of
array
array
region region
to be
to be
sorted sorted
int partionList(int[] a, int x, int y) {
… // partitions list and returns index of pivot
}
Divide and Conquer Technique
Friday, 20 May 2016
29
Quick Sort Example
7 2 8 5 4
2 5 4
2
4
7
2 4 5 7 8
8
2 4 5
5 4
2
5
4
Partition & Sort
Divide and Conquer Technique
7
8
4 5
5
Result
Friday, 20 May 2016
30
Quick Sort Code
int partitionList(int[] a, int x, int y) {
int pivot = a[x];
Use first
element
int left = x;
as pivot
int right = y;
while (left < right) {
while ((a[left] < pivot) && (left < right))
left++;
Partition elements
while (a[right] > pivot)
in array relative to
right--;
value of pivot
if (left < right)
swap(a, left, right);
Place pivot in middle
}
of partitioned array,
swap(a, x, right);
return index of pivot
return right;
Divide and Conquer Technique
Friday, 20 May 2016
31
Merge Sort
Merge Sort
 Approach
1.
2.
3.
Partition list of elements into 2 lists
Recursively sort both lists
Given 2 sorted lists, merge into 1 sorted list
a)
b)
Examine head of both lists
Move smaller to end of new list
 Performance

O( n log(n) ) average / worst case
Divide and Conquer Technique
Friday, 20 May 2016
33
Merge Example
2 4 5
2 7
4 5 8
4 5 8
Divide and Conquer Technique
8
2 4 5 7 8
2 4
7
8
2 4 5 7
2
7
7
5 8
Friday, 20 May 2016
34
Merge Sort Example
2 4 5 7 8
7 2 8 5 4
8 5 4
7 2
7
2
8
5 4
5
7
4
Split
Divide and Conquer Technique
4 5 8
2 7
2
8
4 5
5
4
Merge
Friday, 20 May 2016
35
Merge Sort Code
void mergeSort(int[] a, int x, int y) {
int mid = (x + y) / 2;
Lower
if (y == x) return;
end of
mergeSort(a, x, mid);
array
mergeSort(a, mid+1, y);
region
merge(a, x, y, mid);
to be
}
sorted
void merge(int[] a, int x, int y, int mid) {
… // merges 2 adjacent sorted lists in array
}
Divide and Conquer Technique
Friday, 20 May 2016
Upper
end of
array
region
to be
sorted
36
Merge Sort Code
Upper
end of
1st array
region
void merge (int[] a, int x, int y, int mid) {
int size = y – x;
Upper
Lower
int left = x;
end of
end of 2nd array
int right = mid+1;
st array
1
region
int[] tmp; int j;
region
for (j = 0; j < size; j++) {
if (left > mid) tmp[j] = a[right++];
else if (right > y) || (a[left] < a[right])
tmp[j] = a[left++];
Copy smaller of two
else tmp[j] = a[right++];
elements at head of 2
}
array regions to tmp
for (j = 0; j < size; j++)
buffer, then move on
a[x+j] = tmp[j];
Copy merged
}
array back
Divide and Conquer Technique
Friday, 20 May 2016
37
Mergesort: Divide Step
Step 1 – Divide
5 2 4 7 1 3 2 6
5 2 4 7
5 2
5
2
1 3 2 6
4 7
4
7
1 3
1
2 6
3
2
6
log(n) divisions to split an array of size n into single elements
Divide and Conquer Technique
Friday, 20 May 2016
38
Mergesort: Conquer Step
Step 2 – Conquer
5
2
4
2 5
7
4 7
2 4 5 7
1
1 3
3
2
2 6
1 2 3 6
1 2 2 3 4 5 6 7
6
O(n)
O(n)
O(n)
O(n)
logn iterations, each iteration takes O(n) time. Total Time: O(n logn)
Divide and Conquer Technique
Friday, 20 May 2016
39
Mergesort: Combine Step
Step 3 – Combine
5
2
2 5
• 2 arrays of size 1 can be easily merged to
form a sorted array of size 2
• 2 sorted arrays of size n and m can be
merged in O(n+m) time to form a sorted
array of size n+m
Divide and Conquer Technique
Friday, 20 May 2016
40
Mergesort: Combine Step
Combining 2 arrays of size 4
2 4 5 7
1 2 3 6
4 5 7
2 3 6
4 5 7
6
2 4 5 7
1
2 3 6
4 5 7
1 2 2
1 2 2 3 4
Divide and Conquer Technique
3 6
1 2
1 2 2 3
Etcetera…
1 2 2 3 4 5 6 7
Friday, 20 May 2016
41
Mergesort: Example
20
Divide
20
20
4
7
4
7
4
4
20
6
1
3
9
6
4
20
7
1
6
7
1
7
3
9
9
3
1
5
3
1
6
6
5
5
9
3
5
5
9
Conquer
4
6
7
1
Divide and Conquer Technique
20
3
1
4
5
Friday, 20 May 2016
6
7
9
3
5
9
20
42
MergeSort: Running Time
 The problem is simplified to baby steps
the i’th merging iteration, the
complexity of the problem is O(n)
 number of iterations is O(log n)
 running time: O(n logn)
 for
Divide and Conquer Technique
Friday, 20 May 2016
43
Integer Multiplication
Integer Arithmetic
 Add. Given two n-digit integers a and b, compute a + b.
O(n) bit operations.
 Multiply. Given two n-digit integers a and b, compute a × b.
 Brute force solution: (n2) bit operations.
11010101
* 01111101
1 1 1 1 1 1 0 1
110101010
1 1 0 1 0 1 0 1
000000000
Multiply
+ 0 1 1 1 1 1 0 1
110101010
1 0 1 0 1 0 0 1 0
110101010
110101010
Add
110101010
110101010
000000000
01101000000000010

Divide and Conquer Technique
Friday, 20 May 2016
45

Divide-and-Conquer Multiplication
 To multiply two n-digit integers:


Multiply four ½n-digit integers.
Add two ½n-digit integers, and shift to obtain
result.
x
 2n / 2  x 1  x 0
y
xy
 2n / 2  y 1  y 0
 2 n / 2  x 1  x 0 2 n / 2  y 1  y 0  2 n  x 1 y 1  2 n / 2  x 1 y 0  x 0 y 1   x 0 y 0



T(n )  4T n /2   (n )
recursive calls
 T(n )  (n 2 )
add, shift
assumes n is a power of 2
Divide and Conquer Technique
Friday, 20 May 2016
46
Karatsuba Multiplication
 To multiply two n-digit integers:
Add two ½n digit integers.
 Multiply three ½n-digit integers.
 Add, subtract, and shift ½n-digit integers to
obtain result.

x
y
xy




2n / 2  x 1
2n / 2  y 1
2n  x 1 y 1
2n  x 1 y 1




x0
y0
2 n / 2  x 1 y 0  x 0 y 1   x 0 y 0
2 n / 2   (x 1  x 0 ) (y 1  y 0 )  x 1 y 1  x 0 y 0   x 0 y 0
B
A
Divide and Conquer Technique
Friday, 20 May 2016
A
C
C
47
Karatsuba Multiplication (cont’)
 Theorem. [Karatsuba-Ofman, 1962] Can
multiply two n-digit integers in O(n1.585) bit
operations.
T(n )  T
 n / 2   T  n / 2   T 1 n / 2 
recursive calls
 T(n )  O (n
log 2 3

(n )
add, subtract, shift
)  O (n 1.585 )

Divide and Conquer Technique
Friday, 20 May 2016
48
Karatsuba: Recursion Tree
 0
T(n )  
 3T (n /2)  n
if n  1
otherwise
log 2 n
T(n )   n
k 0

3 k
2
23
1 log 2 n

3 1
2
 3n log 2 3  2
n
T(n)

1

T(n/2)
T(n/2)
T(n/2)
3(n/2)
T(n/4) T(n/4) T(n/4)
T(n/4) T(n/4) T(n/4)
T(n/4) T(n/4) T(n/4)
9(n/4)
...
...
3k (n / 2k)
T(n / 2k)
...
T(2)
T(2)
Divide and Conquer Technique
T(2)
T(2)
...
T(2)
Friday, 20 May 2016
T(2)
T(2)
T(2)
3 lg n (2)
49
Multiplying Large Integers
High school multiplications
A = an-1 an-2 … a1 a0
 B = bn-1 bn-2 … b1 b0
Time O(N2)
Divide and Conquer Technique
Friday, 20 May 2016
eg)
110011
101001
110011
000000
000000
110011
000000
110011
50
Multiplying Large Integers (cont’)
A1 = an-1 an-2 …an/2
A0 = an/2-1 … a1 …a0
B1 = bn-1 bn-2 …bn/2
B0 = bn/2-1 …b1 …b0
Define
X = A1 + A0
Y = B1 + B0
AB = A1B12n + (XY – A1B1 – A0B0) 2n/2 + A0B0
Complexity? An integer multiplication of two n bit integer is
reduced to
=> three integer multiplications of n/2 bits, and 6 additions of n bits
T(n) = 3T(n/2) + O(n) = O(nlog23) = O(n1.59)
Divide and Conquer Technique
Friday, 20 May 2016
51
Multiplying integers
(Decimal numbers)
A = 2 3 6 7 8 1 8 3 A1 = 2 3 6 7
B = 1 3 0 4 6 4 9 1 B1 = 1 3 0 4
A0 = 8 1 8 3
B0 = 6 4 9 1
AB = ( A1104 + A0 ) ( B1104 + B0 )
= A1B1108 + (A1B0+A0B1)104 + A0B0
4 multiplications
3 additions
2 shiftings
T(n) = 4T(n/2) + O(n)
T(1) = O(1)
T(n) = O(n2)
Divide and Conquer Technique
Friday, 20 May 2016
52
Multiplying integers(Decimal numbers)
D-A-C Approach
Let X = ( A1 + A0 ) , Y = ( B1 + B0 )
XY =A1B1 + A1B0 + A0B1 + A0B0
XY - A1B1 - A0B0
= A1B0+A0B1
AB = ( A1104 + A0 ) ( B1104 + B0 )
= A1B1108 + (A1B0+A0B1)104 +A0B0
= A1B1108 + (XY - A1B1 - A0B0)104 + A0B0
Recursively compute XY, A1B1 , A0B0,
3 multiplications
6 additions
2 shiftings
Divide and Conquer Technique
T(n) = 3T(n/2) + O(n)
T(1) = O(1)
T(n) = nlog23 = O(n1.59)
Friday, 20 May 2016
53
Matrix Multiplication
Matrix Multiplication
 Matrix multiplication. Given two n-by-n matrices A and B,
compute C = AB.
n
c ij   a ik b kj
k 1

c 11 c 12

c 21 c 22

c
 n 1 c n 2
a 11 a 12
c 1n 


c 2n 
a 21 a 22




a
c nn 

 n 1 a n 2
b 11 b 12
a 1n 


a 2n 
b 21 b 22




b
a nn 

 n 1 b n 2
b 1n 

b 2n 

b nn 

 Brute force. (n3) arithmetic operations.

 Fundamental question. Can we improve upon brute
force?
Divide and Conquer Technique
Friday, 20 May 2016
55
Matrix Multiplication: Warmup
 Divide-and-conquer.



Divide: partition A and B into ½n-by-½n blocks.
Conquer: multiply 8 ½n-by-½n recursively.
Combine: add appropriate products using 4
matrix additions.
C 11 C 12  A 11

  
C 21 C 22  A 21
A 12  B 11
  
A 22  B 21
B 12 

B 22 
C 11

C 12

C 21
C 22


A 11  B 11   A 12  B 21 
A 11  B 12   A 12  B 22 
A 21  B 11   A 22  B 21 
A 21  B 12   A 22  B 22 

T(n )  8T n /2  
recursive calls
Divide and Conquer Technique
(n 2 )
3
 T(n )  (n )

add, form submatrices
Friday, 20 May 2016
56
Matrix Multiplication: Key Idea
 Key idea. multiply 2-by-2 block matrices with
only 7 multiplications.
C 11

C 21
C 12  A 11
  
C 22  A 21
C 11
C 12
C 21
C 22








A 12  B 11
  
A 22  B 21
B 12 

B 22 
P5  P 4  P2  P6
P1  P2
P3  P4
P5  P1  P 3  P 7
P1
P2
P3
P4
P5
P6
P7







A 11  ( B 12  B 22 )
( A 11  A 12 )  B 22
( A 21  A 22 )  B 11
A 22  ( B 21  B 11 )
( A 11  A 22 )  ( B 11  B 22 )
( A 12  A 22 )  ( B 21  B 22 )
( A 11  A 21 )  ( B 11  B 12 )
7 multiplications.

18 = 10 + 8 additions (or subtractions).
Divide and Conquer Technique
Friday, 20 May 2016
57
Fast Matrix Multiplication
 Fast matrix multiplication. (Strassen, 1969)




Divide: partition A and B into ½n-by-½n blocks.
Compute: 14 ½n-by-½n matrices via 10 matrix
additions.
Conquer: multiply 7 ½n-by-½n matrices recursively.
Combine: 7 products into 4 terms using 8 matrix
additions.
T(n )  7T n / 2 
recursive calls
(n 2 )
 T(n )  (n log 2 7 )  O (n 2.81 )
add, subtract
 Analysis.



Assume n is a power of 2.
T(n) = # arithmetic operations.
Divide and Conquer Technique
Friday, 20 May 2016
58
Fast Matrix Multiplication in Practice
 Implementation issues.
 Sparsity.
 Caching effects.
 Numerical stability.
 Odd matrix dimensions.
 Crossover to classical algorithm around n = 128.
 Common misperception: "Strassen is only a theoretical
curiosity."


Advanced Computation Group at Apple Computer reports
8x speedup on G4 Velocity Engine when n ~ 2,500.
Range of instances where it's useful is a subject of
controversy.
Divide and Conquer Technique
Friday, 20 May 2016
59
Fast Matrix Multiplication in Theory
 Q. Multiply two 2-by-2 matrices with only 7 scalar multiplications?
(n log 2 7 )  O (n
 A. Yes! [Strassen, 1969]
2.81
)
 Q. Multiply two 2-by-2 matrices with only 6 scalar multiplications?
 A. Impossible. [Hopcroft and Kerr, 1971]

(n
log 2 6
)  O (n
2.59
)
 Q. Two 3-by-3 matrices with only 21 scalar multiplications?
 A. Also impossible.

(n log 3 21 )  O (n
2.77
)
 Q. Two 70-by-70 matrices with only 143,640 scalar multiplications?
 A. Yes! [Pan, 1980]
Divide and Conquer Technique


Friday, 20 May 2016
(n log 70 143640 )  O (n
2.80
)
60
Fast Matrix Multiplication in Theory
 Decimal wars.


December, 1979: O(n2.521813).
January, 1980: O(n2.521801).
 Best known. O(n2.376) [Coppersmith-Winograd, 1987.]
 Conjecture. O(n2+) for any  > 0.
 Caveat. Theoretical improvements to Strassen are
progressively less practical.
Divide and Conquer Technique
Friday, 20 May 2016
61
Observations
 Comparison: n= 70
 Direct multiplication: 703 = 343,000
 Strassen: 70lg 7 is approximately 150,000
 Crossover point typically around n = 20
 Hopcroft and Kerr have shown 7 multiplications are
necessary to multiply 2 by 2 matrices

But we can do better with larger matrices
 Best lower bound is W(n2) (since there are n2 entries)
 Matrix multiplication can be used in some graph
algorithms as a fundamental step, so theoretical
improvements in the efficiency of this operation can lead
to theoretical improvements in those algorithms
Divide and Conquer Technique
Friday, 20 May 2016
62
Closest Pair of Points
Closest Pair of Points
 Closest pair. Given n points in the plane, find a pair with smallest
Euclidean distance between them.
 Fundamental geometric primitive.
 Graphics, computer vision, geographic information systems,
molecular modeling, air traffic control.
 Special case of nearest neighbor, Euclidean MST, Voronoi.
fast closest pair inspired fast algorithms for these problems
 Brute force. Check all pairs of points p and q with (n2)
comparisons.
 1-D version. O(n log n) easy if points are on a line.
 Assumption. No two points have same x coordinate.
to make presentation cleaner
Divide and Conquer Technique
Friday, 20 May 2016
64
Closest Pair of Points: First Attempt
 Divide. Sub-divide region into 4 quadrants.
L
Divide and Conquer Technique
Friday, 20 May 2016
65
Closest Pair of Points: First Attempt
 Divide. Sub-divide region into 4 quadrants.
 Obstacle. Impossible to ensure n/4 points in
each piece.
L
Divide and Conquer Technique
Friday, 20 May 2016
66
Closest Pair of Points
 Algorithm.

Divide: draw vertical line L so that roughly ½n points on each side.
L
Divide and Conquer Technique
Friday, 20 May 2016
67
Closest Pair of Points
 Algorithm.


Divide: draw vertical line L so that roughly ½n points on each side.
Conquer: find closest pair in each side recursively.
L
21
12
Divide and Conquer Technique
Friday, 20 May 2016
68
Closest Pair of Points
 Algorithm.




Divide: draw vertical line L so that roughly ½n points on each side.
seems like (n2)
Conquer: find closest pair in each side recursively.
Combine: find closest pair with one point in each side.
Return best of 3 solutions.
L
8
21
12
Divide and Conquer Technique
Friday, 20 May 2016
69
Closest Pair of Points
 Find closest pair with one point in each side, assuming that distance < .
L
21
 = min(12, 21)
12
Divide and Conquer Technique
Friday, 20 May 2016
70
Closest Pair of Points
 Find closest pair with one point in each side, assuming that distance < .

Observation: only need to consider points within  of line L.
L
21
 = min(12, 21)
12
Divide and Conquer Technique
Friday, 20 May 2016
71
Closest Pair of Points
 Find closest pair with one point in each side, assuming that distance < .


Observation: only need to consider points within  of line L.
Sort points in 2-strip by their y coordinate.
L
7
6
4
12
5
21
 = min(12, 21)
3
2
1
Divide and Conquer Technique
Friday, 20 May 2016
72
Closest Pair of Points
 Find closest pair with one point in each side, assuming that distance < .



Observation: only need to consider points within  of line L.
Sort points in 2-strip by their y coordinate.
Only check distances of those within 11 positions in sorted list!
L
7
6
4
12
5
21
 = min(12, 21)
3
2
1
Divide and Conquer Technique
Friday, 20 May 2016
73
Closest Pair of Points
 Def. Let si be the point in the 2-strip, with
the ith smallest y-coordinate.
j
39
 Claim. If |i – j|  12, then the distance between
si and sj is at least .
 Pf.
 No two points lie in same ½-by-½ box.
2 rows
 Two points at least 2 rows apart
have distance  2(½). ▪
i
 Fact. Still true if we replace 12 with 7.
Divide and Conquer Technique
Friday, 20 May 2016
31
½
½
30
29
28
27
½
26
25


74
Closest Pair Algorithm
Closest-Pair(p1, …, pn) {
Compute separation line L such that half the points
are on one side and half on the other side.
1 = Closest-Pair(left half)
2 = Closest-Pair(right half)
 = min(1, 2)
O(n log n)
2T(n / 2)
Delete all points further than  from separation line L
O(n)
Sort remaining points by y-coordinate.
O(n log n)
Scan points in y-order and compare distance between
each point and next 11 neighbors. If any of these
distances is less than , update .
O(n)
return .
}
Divide and Conquer Technique
Friday, 20 May 2016
75
Closest Pair of Points: Analysis
 Running time.
T(n )  2T n /2  O (n log n )  T(n )  O (n log 2 n )
 Q. Can we achieve O(n log n)?

 A. Yes. Don't sort points in strip from scratch each time.
 Each recursive returns two lists: all points sorted by y
coordinate, and all points sorted by x coordinate.
 Sort by merging two pre-sorted lists.
T (n )  2T n /2  O (n )  T(n )  O (n log n )

Divide and Conquer Technique
Friday, 20 May 2016
76
Other Applications




Find median (Probabilistic)
Convex Hull
Reversing array
Maximum consecutive subsequence
Etc…
Divide and Conquer Technique
Friday, 20 May 2016
77