Tags: best and worst case, lecture-06, mergesort
True or False: the best case time complexity of mergesort is \(\Theta(n)\).
False.
Mergesort doesn't stop early: it always splits the array in half all the way down and merges everything back together, no matter what the input is (even if it is already sorted). Each of the \(\Theta(\log n)\) levels of recursion does \(\Theta(n)\) merging work, so the best case is \(\Theta(n \log n)\), the same as the worst case.
Tags: best and worst case, lecture-06
True or False: the best case time complexity of mergesort is \(\Theta(n)\).
False.
Mergesort doesn't stop early: it always splits the array in half all the way down and merges everything back together, no matter what the input is (even if it is already sorted). Each of the \(\Theta(\log n)\) levels of recursion does \(\Theta(n)\) merging work, so the best case is \(\Theta(n \log n)\), the same as the worst case.
Tags: lecture-06, mergesort
Suppose mergesort is called on an array of size \(n\). In total, how many calls to merge are made over all calls to mergesort?
\(\Theta(n)\)
Tags: recurrence relations, lecture-06
What is the solution of the below recurrence? State your answer using asymptotic notation as a function of \(n\).
\(\Theta(n \log n)\)
Tags: recurrence relations, lecture-06
State (but do not solve) the recurrence describing the runtime of the following function.
def foo(n):
if n <= 1:
return
foo(n // 2)
for j in range(n):
print(j)
foo(n // 2)
\(\Theta(n \log n)\)
Tags: recurrence relations, lecture-06
State (but do not solve) the recurrence describing the runtime of the following function.
def foo(n):
if n < 10:
print("Hello world.")
return
for i in range(n):
print(i)
for i in range(6):
foo(n//3)
\(T(n) = 6 T(n/3) + \Theta(n)\).
Tags: recurrence relations, lecture-06
State (but do not solve) the recurrence describing the runtime of the following function.
def boo(n):
if n <= 1:
return
for i in range(3):
boo(n/4)
for j in range(n):
print(j)
\( T(n) = \begin{cases} \Theta(1), & n = 1\\ & n > 1 \end{cases}\)
Even though the recursive calls are made in a for-loop, the basic idea is the same: there will be three recursive calls, each on arrays of size \(n/4\), and thus the recurrence will involve \(3 T(n/4)\). There is \(\Theta(n)\) work done outside the recursive calls, for a full recurrence of \(T(n) = 3 T(n/4) + \Theta(n)\).
Tags: recurrence relations, lecture-06
State (but do not solve) the recurrence relation describing this function's run time.
import random
def foo(n):
if n <= 2:
return
for i in range(n):
for j in range(i, n):
print(i)
return foo(n//2) + foo(n//2)
\(T(n) = 2 T(n/2) + \Theta(n^2)\). The nested loops are dependent: the inner loop runs \(n - i\) times, for a total of \(n + (n-1) + \ldots + 1 = \Theta(n^2)\). Then there are two recursive calls, each on a problem of size \(n/2\).
Tags: recursion, lecture-06, mergesort
Consider the code for mergesort and merge where a new print-statement has been added to merge:
def mergesort(arr):
"""Sort array in-place."""
if len(arr) > 1:
middle = math.floor(len(arr) / 2)
left = arr[:middle]
right = arr[middle:]
mergesort(left)
mergesort(right)
merge(left, right, arr)
def merge(left, right, out):
"""Merge sorted arrays, store in out."""
left.append(float('inf'))
right.append(float('inf'))
left_ix = 0
right_ix = 0
for ix in range(len(out)):
print("Here!") # <---- the newly-added line
if left[left_ix] < right[right_ix]:
out[ix] = left[left_ix]
left_ix += 1
else:
out[ix] = right[right_ix]
right_ix += 1
Suppose mergesort is called on an array of size \(n\). In total, how many times will "Here!" be printed?
\(\Theta(n \log n)\)
Tags: time complexity, lecture-06
import math
def mediansort(arr, start, stop):
"""Claims to sort the array, in-place"""
if stop - start <= 1:
return
# finds the index of the median of arr[start:stop]
median_ix = find_median(arr, start, stop)
middle_ix = math.floor((start + stop) / 2)
# move the median to the middle by swapping
arr[median_ix], arr[middle_ix] = arr[middle_ix], arr[median_ix]
# recurse on the left and right halves
mediansort(arr, start, middle_ix)
mediansort(arr, middle_ix + 1, stop)
Consider the mediansort function from above. Suppose that find_median takes \(\Theta(n)\) time. What is the time complexity of mediansort?
\(\Theta(n \log n)\)
Tags: recurrence relations, lecture-06, mergesort
Consider the modification of mergesort shown below, where one of the recursive calls has been replaced by an in-place version of selection_sort. Recall that selection_sort takes \(\Theta(n^2)\) time.
def kinda_mergesort(arr):
"""Sort array in-place."""
if len(arr) > 1:
middle = math.floor(len(arr) / 2)
left = arr[:middle]
right = arr[middle:]
mergesort(left)
selection_sort(right)
merge(left, right, arr)
What is the time complexity of kinda_mergesort?
\(\Theta(n^2)\)
Tags: best and worst case, lecture-06, mergesort
Suppose arr is a list-of-lists with \(n\) inner lists, each containing \(n\) numbers. What is the best case time complexity of the following function?
import math
def average_median(arr):
"""Compute the mean of the row medians."""
msum = 0
for i, row in enumerate(arr):
mergesort(row) # sort the row using the mergesort algorithm
msum += row[math.floor(i / 2)]
return msum / len(arr)
\(\Theta(n^2 \log n)\). Mergesort takes \(\Theta(k \log k)\) time on an array of size \(k\) in every case, including the best case. The loop sorts \(n\) rows of size \(n\), taking \(\Theta(n \log n)\) time for each, and the rest of the loop body takes constant time. So the best case is \(\Theta(n^2 \log n)\).
Tags: best and worst case, lecture-06, mergesort
Your friend has written a modified version of mergesort that they claim has an improved best-case time complexity:
def better_mergesort(arr):
if is_already_sorted(arr):
return
else:
mergesort(arr)
Here, is_already_sorted is a function that takes in an array and returns True if and only if the array is in sorted order. Assume that is_already_sorted is implemented efficiently (meaning, it has the best possible time complexity). What is the best case time complexity of better_mergesort?
\(\Theta(n)\). The best case is when arr is already sorted, so that only is_already_sorted runs. Checking whether an array is sorted requires looking at every element, and can be done in \(\Theta(n)\) time by comparing each pair of adjacent elements. So the best case time complexity is \(\Theta(n)\).
Tags: loop invariants, lecture-06, binary search
Consider the iterative implementation of binary search shown below:
import math
def iterative_binary_search(arr, target):
start = 0
stop = len(arr)
while (stop - start) > 0:
print(arr[start])
middle = math.floor((start + stop) / 2)
if arr[middle] == target:
return middle
elif arr[middle] > target:
stop = middle
else:
start = middle + 1
return None
Which of the following loop invariants is true, assuming that arr is sorted and non-empty, and target is not in the array? Select all that apply.
The first option is correct.
Tags: loop invariants, lecture-06, selection sort
Suppose you are sorting an array using selection sort. At the current iteration of selection sort, the array being sorted looks like this:
[0, 1, 2, 2, 3, 4, 2, 10, 13, 4]
Which of the following are possible options for how many times the outer for loop has run? Select all that apply.
1, 2, 3, and 4. Remember the selection sort loop invariants: after the \(\alpha\)th iteration, the first \(\alpha\) numbers are \(\leq\) the remaining numbers, and the first \(\alpha\) numbers are in sorted order.
To answer this question, we can ask, for each \(\alpha\), are the loop invariants true? If yes, then it is possible that \(\alpha\) iterations have been made; if not, then it is not possible.
It is true for \(\alpha=1\), since the first number is \(\leq\) the remaining numbers, and the first 1 numbers are in sorted order. Same for \(\alpha = 2, 3, \) and \(4\). However, it is not true at \(\alpha = 5\). One of the loop invariants is true (the first five numbers are in sorted order), but the other is not: the first 5 numbers are not all \(\leq\) the remaining numbers. In particular, the fifth number (3) is not \(\leq\) the 7th number (2). Therefore, this could not possibly be the result after 5 iterations.
Tags: lecture-06, mergesort
Below is the merge function from the mergesort algorithm, with an added print statement:
def merge(left, right, out):
"""Merge sorted arrays, store in out."""
print(len(left))
print(len(right))
left.append(float('inf'))
right.append(float('inf'))
left_ix = 0
right_ix = 0
for ix in range(len(out)):
if left[left_ix] < right[right_ix]:
out[ix] = left[left_ix]
left_ix += 1
else:
out[ix] = right[right_ix]
right_ix += 1
Suppose this version of merge is used in mergesort, and mergesort is called on the following list:
[6, 2, 7, 1, 9, 3, 8, 5].
If all of the numbers that are printed are added up, what will the sum be?
24
Tags: loop invariants, lecture-06
Consider the following code, which takes in a list of integers and moves all of the odd numbers to the front and all of the even numbers to the end, in-place.
Remember that the \(\%\) operator computes the remainder of division. For example \(5 \% 2 = 1\) and \(4 \% 2 = 0\). This can be used to check if a number is even; if \(x \% 2 = 0\), then x is even.
def separate_odd_from_even(numbers):
"""Moves the odd numbers to the front, in-place."""
n = len(numbers)
def swap(i, j):
numbers[i], numbers[j] = numbers[j], numbers[i]
odd_barrier = 0
even_barrier = n - 1
while odd_barrier < even_barrier:
if numbers[odd_barrier] % 2 == 0:
# the number is even
swap(odd_barrier, even_barrier)
even_barrier -= 1
else:
# the number is odd
odd_barrier += 1
Which of the following are valid loop invariants for the while-loop? Select all that apply.
3rd option: After each iteration, the elements in numbers[:odd_barrier] are odd.
6th option: After each iteration, the elements in numbers[even_barrier+1:] are even.
8th option: After the \(\alpha\)th iteration, even_barrier - odd_barrier = n - 1 - \(\alpha\).
How many iterations will the while-loop in separate_odd_from_even make if numbers has length \(100\)? Your answer should be a number.
\(99\)
Tags: loop invariants, lecture-06
Consider this code which partitions a list in a special way:
def is_even(i):
"""Returns True if i is even, False otherwise."""
return i % 2 == 0
def mystery_partition(numbers):
def swap(i, j):
numbers[i], numbers[j] = numbers[j], numbers[i]
barrier_ix = 0
for i in range(len(numbers)):
if is_even(numbers[i]):
swap(i, barrier_ix)
if numbers[barrier_ix] > numbers[0]:
swap(0, barrier_ix)
barrier_ix += 1
Which of the following loop invariants are true for this code? Select all that apply.
Note: any statement about an empty set or list is considered to be automatically true.
1st option: After each iteration of the for loop, all numbers in numbers[:barrier_ix] are even.
3rd option: After each iteration of the for loop, numbers[0] is greater than or equal to all numbers in numbers[:barrier_ix].
Tags: lecture-06, theoretical lower bounds
Consider the problem of determining whether there exists a pair of numbers in an array which, when added together, equal the maximum number in the array. Additionally, assume that the array is sorted.
True or False: \(\Theta(n)\) is a tight theoretical lower bound for this problem.
True.
Any algorithm must take \(\Omega(n)\) time in the worst case, since in the worst case all elements of the array must be read.
This is tight because there is an algorithm that will compute the answer in worst-case \(\Theta(n)\) time. Namely, this is an instance of the ``movie problem'' from lecture, where instead of finding two numbers which add to an arbitrary target, we're looking for a specific target: the maximum. We saw an algorithm that solved the movie problem in \(\Theta(n)\) time, where \(n\) was the number of movies. Since the maximum is computed in \(\Theta(1)\) time for a sorted array, we can use the same algorithm to solve this in \(\Theta(n)\) time as well.
Tags: lecture-06, theoretical lower bounds
In this problem, assume that \(A\) and \(B\) are sorted lists, each containing \(n\) numbers.
What is the best possible (worst case) time complexity of any algorithm for finding the smallest difference between a number in \(A\) and a number in \(B\)?
\(\Theta(n)\)
What is the best possible (worst case) time complexity of any algorithm for finding the biggest difference between a number in \(A\) and a number in \(B\)?
\(\Theta(1)\)
Tags: best and worst case, lecture-06, theoretical lower bounds
Consider the following problem: given two sorted lists, A and B, each containing \(n\) numbers, determine if the two lists share an element in common. That is, determine if there is a number \(x\) which appears in both lists.
What is a tight theoretical lower bound for this problem? Again, you may assume that the lists are both sorted. State your answer in asymptotic notation as a function of \(n\).
\(\Theta(n)\)
(2 points) Briefly (in 50 words or less) describe an algorithm for solving this problem and state its worst case time complexity. To earn full credit, you should give the optimal algorithm, but partial credit will be awarded for sub-optimal solutions. You may write pseudocode if you like, but an explanation in words is also sufficient. Make sure that you only state one algorithm!
Here's an approach that takes \(\Theta(n)\); it's similar to what we did in lecture for the movie problem.
Start with i = j = 0. If A[i] == A[j], return True, since you've found an element in common. If A[i] > A[j], increment j, otherwise increment i. Takes \(\Theta(n)\) time.
The brute force approach also earns credit: Loop over all pairs, checking if A[i] == B[j] for each pair. This takes \(\Theta(n^2)\) time.
Tags: recurrence relations, lecture-06
Solve the below recurrence, stating the solution in asymptotic notation. Show your work.
\(\Theta(n)\).
We'll unroll several times:
The pattern seems to be that on the \(k\)th unroll:
The base case is reached when \(k = \log_4 n\). Plugging this back in, we find:
If you reached this point, you got most of the credit. If you're unsure of what to do with \(2^{\log_4 n}\), there are a couple of ways foward.
The first (and easiest) way is to realize that \(2^{\log_4 n}\) is smaller than \(2^{\log_2 n} = n\), so \(2^{\log_4 n} = O(n)\). Similarly, we've seen the summation of \(1 + 1/2 + 1/4 + \ldots\) several times -- this is a geometric sum, and converges to 2 when there are infinitely many terms. There are a finite number of terms here, so the sum is \(< 2\). Altogether, we have:
The second approach is to remember log properties to simplify \(2^{\log_4 n}\) to \(\sqrt{n}\). This can be seen by using the ``change of base'' formula:
In this case:
And therefore \(2^{\log_4 n} = 2^{(\log_2 n) / 2} = \left(2^{\log_2 n}\right)^{1/2} = n^{1/2} = \sqrt n\). This shows that the first term in the recurrence is \(\Theta(\sqrt n)\). The second term is still \(\Theta(n)\), so the solution is \(\Theta(n)\).
Tags: recurrence relations, lecture-06
Solve the recurrence below, stating the solution in asymptotic notation. Show your work.
\(\Theta(n)\)
Tags: recurrence relations, lecture-06
What is the time complexity of this function? Here \(n\) is the length of the input array.
import math
def foo(arr):
if len(arr) == 0:
return 0
if len(arr) == 1:
return arr[0]
if len(arr) == 2:
return arr[0] + arr[1]
one_third = math.floor(len(arr) / 3)
left = arr[:one_third]
middle = arr[one_third: one_third * 2]
right = arr[one_third * 2:]
for i in range(len(arr)):
if i < one_third:
arr[i] = 2 * arr[i] - sum(left)/len(left)
if i >= one_third and i < one_third * 2:
arr[i] = 2 * arr[i] - sum(middle)/len(middle)
if i >= one_third * 2:
arr[i] = 2 * arr[i] - sum(right)/len(right)
total = sum(left) + sum(middle) + sum(right)
return foo(left) + foo(middle) + foo(right) + total
\(\Theta(n^2)\). This does \(\Theta(n^2)\) work outside of the recursive calls. The recurrence relation is \(T(n) = 3T(n/3) + \Theta(n^2)\), whose solution is \(\Theta(n^2)\).
Solving this recurrence involves a little bit more than usual, because you have to recognize a geometric sum. When unrolling, you should find that on step \(k\) you get a term of the form \(n^2 + n^2/3 + n^2/3^2 + n^2/3^3 + ... + n^2/3^{k-1}\). This is a geometric sum, which we saw in an earlier homework. If you factor out the \(n^2\), you get something of the form \(1 + 1/3 + 1/3^2 + ...\). There is an exact formula for this, but for our purposes all that matters is that the sum converges to some number (a constant), and so the recurrence is \(\Theta(n^2)\).