Quick Sort
Recursive
package com.programming.ds.sorting;
import com.programming.tool.ProgrammingTools;
/**
* There are many different versions of quickSort that pick pivot in different
* ways.
*
* Always pick first element as pivot.
*
* Always pick last element as pivot (implemented below)
*
* Pick a random element as pivot.
*
* Pick median as pivot.
*
* The key process in quickSort is partition(). Target of partitions is, given
* an array and an element x of array as pivot, put x at its correct position in
* sorted array and put all smaller elements (smaller than x) before x, and put
* all greater elements (greater than x) after x. All this should be done in
* linear time.
*
* 1) QuickSort is a divide and conquer algorithm. Large list is divided into
* two and sorted separately (conquered), sorted list is merge later.
*
* 2) On "in-place" implementation of quick sort, list is sorted using same
* array, no additional array is required. Numbers are re-arranged pivot, also
* known as partitioning.
*
* 3) Partitioning happen around pivot, which is usually middle element of
* array.
*
* 4) Average case time complexity of Quicksort is O(n log n) and worst case
* time complexity is O(n ^ 2), which makes it one of the fasted sorting
* algorithm. Interesting thing is it's worst case performance is equal to
* Bubble Sort
*
* 5) Quicksort can be implemented with an in-place partitioning algorithm, so
* the entire sort can be done with only O(log n) additional space used by the
* stack during the recursion.
*
* 6) Quicksort is also a good example of algorithm which makes best use of CPU
* caches, because of it's divide and conquer nature.
*
* 7) In Java, Arrays.sort() method uses quick sort algorithm to sort array of
* primitives. It's different than our algorithm, and uses two pivots. Good
* thing is that it perform much better than most of the quicksort algorithm
* available on internet for different data sets, where traditional quick sort
* perform poorly. One more reason, not to reinvent the wheel but to use the
* library method, when it comes to write production code.
*
* 8) Quick sort is not stable sorting algorithm
*
* 9) What is the best sorting algorithm to use for the elements in array are
* more than 1 million in general?
*
* Most practical implementations of Quick Sort use randomized version. The
* randomized version has expected time complexity of O(nLogn). The worst case
* is possible in randomized version also, but worst case doesn’t occur for a
* particular pattern (like sorted array) and randomized Quick Sort works well
* in practice. Quick Sort is also a cache friendly sorting algorithm as it has
* good locality of reference when used for arrays. Quick Sort is also tail
* recursive, therefore tail call optimizations is done.
*
* @author rmis16
*
*/
public class QuickSortRecursive extends ProgrammingTools {
/**
* Recursive quicksort logic
*
* @param array input array
* @param startIdx start index of the array
* @param endIdx end index of the array
*
*/
public void quickSort(int[] input, int left, int right) {
if (left < right) {
// idx is partitioning index. input[idx] is now at right place
int idx = partition_with_mid_element(input, left, right);
// Recursively sort element before and after partition
quickSort(input, left, idx - 1);
quickSort(input, idx + 1, right);
}
}
/**
* This function takes last element as pivot, places the pivot element at its
* correct position in sorted array, and places all smaller (smaller than pivot)
* to left of pivot and all greater elements to right of pivot
*
* @param array array to partitioned *
* @param left lower bound of the array *
* @param right upper bound of the array *
* @return the partition index
*/
public int partition(int[] input, int left, int right) {
int pivot = input[right];
int i = left - 1;
for(int j = left; j < right; j++) {
if(input[j] <= pivot) {
i++;
swap(input, i, j);
}
}
swap(input, i + 1, right);
return i + 1; //pivot is at right position
}
/**
* This approach is partitioning with mid element
*
* @param array array to partitioned *
* @param left lower bound of the array *
* @param right upper bound of the array *
* @return the partition index
*/
public int partition_with_mid_element(int[] input, int left, int right) {
int index = (left + right) / 2;
int pivot = input[index];
int i = left - 1;
for(int j = left; j <= right; j++) {
if(j == index) {
continue;//skip index element
}
if(input[j] <= pivot) {
i++;
if(i == index && (i + 1) != right) {
i++;//skip index element
}
if(i != j) {
swap(input, i, j);
}
}
}
if(i < index) {
//i did not moved.
swap(input, i + 1, index);
return i + 1;
}
swap(input, i, index);
return i;
}
void print(int[] input) {
System.out.print("\n\nInput array before sort :");
printArray(input);
quickSort(input, 0, input.length - 1);
System.out.print("\nInput array after quick sort :");
printArray(input);
System.out.println("\n");
}
public static void main(String a[]) {
QuickSortRecursive obj = new QuickSortRecursive();
int[] input1 = { 24, 2, 45, 20, 56, 75, 2, 56, 99, 53, 12 };
obj.print(input1);
int[] input2 = { 10, 5, 2, 20, 1, 3 };
obj.print(input2);
int[] input3 = { 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3 };
obj.print(input3);
int[] input4 = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
obj.print(input4);
int[] input5 = { 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 };
obj.print(input5);
}
}
Quick Sort Iterative:
public void quickSort(int[] input, int left, int right) {
// Create an auxiliary stack
Stack<Integer> stack = new Stack<>();
// push initial values of l and h to stack
stack.push(left);
stack.push(right);
// Keep popping from stack while is not empty
while (!stack.isEmpty()) {
// pop left and right
right = stack.pop();
left = stack.pop();
// Set pivot element at its correct position
// in sorted array
int pivot = partition_with_mid_element(input, left, right);
// If there are elements on left side of pivot,
// then push left side to stack
if (pivot - 1 > left) {
stack.push(left);
stack.push(pivot - 1);
}
// If there are elements on right side of pivot,
// then push right side to stack
if (pivot + 1 < right) {
stack.push(pivot + 1);
stack.push(right);
}
}
}
package com.programming.ds.sorting;
import com.programming.tool.ProgrammingTools;
/**
* There are many different versions of quickSort that pick pivot in different
* ways.
*
* Always pick first element as pivot.
*
* Always pick last element as pivot (implemented below)
*
* Pick a random element as pivot.
*
* Pick median as pivot.
*
* The key process in quickSort is partition(). Target of partitions is, given
* an array and an element x of array as pivot, put x at its correct position in
* sorted array and put all smaller elements (smaller than x) before x, and put
* all greater elements (greater than x) after x. All this should be done in
* linear time.
*
* 1) QuickSort is a divide and conquer algorithm. Large list is divided into
* two and sorted separately (conquered), sorted list is merge later.
*
* 2) On "in-place" implementation of quick sort, list is sorted using same
* array, no additional array is required. Numbers are re-arranged pivot, also
* known as partitioning.
*
* 3) Partitioning happen around pivot, which is usually middle element of
* array.
*
* 4) Average case time complexity of Quicksort is O(n log n) and worst case
* time complexity is O(n ^ 2), which makes it one of the fasted sorting
* algorithm. Interesting thing is it's worst case performance is equal to
* Bubble Sort
*
* 5) Quicksort can be implemented with an in-place partitioning algorithm, so
* the entire sort can be done with only O(log n) additional space used by the
* stack during the recursion.
*
* 6) Quicksort is also a good example of algorithm which makes best use of CPU
* caches, because of it's divide and conquer nature.
*
* 7) In Java, Arrays.sort() method uses quick sort algorithm to sort array of
* primitives. It's different than our algorithm, and uses two pivots. Good
* thing is that it perform much better than most of the quicksort algorithm
* available on internet for different data sets, where traditional quick sort
* perform poorly. One more reason, not to reinvent the wheel but to use the
* library method, when it comes to write production code.
*
* 8) Quick sort is not stable sorting algorithm
*
* 9) What is the best sorting algorithm to use for the elements in array are
* more than 1 million in general?
*
* Most practical implementations of Quick Sort use randomized version. The
* randomized version has expected time complexity of O(nLogn). The worst case
* is possible in randomized version also, but worst case doesn’t occur for a
* particular pattern (like sorted array) and randomized Quick Sort works well
* in practice. Quick Sort is also a cache friendly sorting algorithm as it has
* good locality of reference when used for arrays. Quick Sort is also tail
* recursive, therefore tail call optimizations is done.
*
* @author rmis16
*
*/
public class QuickSortRecursive extends ProgrammingTools {
/**
* Recursive quicksort logic
*
* @param array input array
* @param startIdx start index of the array
* @param endIdx end index of the array
*
*/
public void quickSort(int[] input, int left, int right) {
if (left < right) {
// idx is partitioning index. input[idx] is now at right place
int idx = partition_with_mid_element(input, left, right);
// Recursively sort element before and after partition
quickSort(input, left, idx - 1);
quickSort(input, idx + 1, right);
}
}
/**
* This function takes last element as pivot, places the pivot element at its
* correct position in sorted array, and places all smaller (smaller than pivot)
* to left of pivot and all greater elements to right of pivot
*
* @param array array to partitioned *
* @param left lower bound of the array *
* @param right upper bound of the array *
* @return the partition index
*/
public int partition(int[] input, int left, int right) {
int pivot = input[right];
int i = left - 1;
for(int j = left; j < right; j++) {
if(input[j] <= pivot) {
i++;
swap(input, i, j);
}
}
swap(input, i + 1, right);
return i + 1; //pivot is at right position
}
/**
* This approach is partitioning with mid element
*
* @param array array to partitioned *
* @param left lower bound of the array *
* @param right upper bound of the array *
* @return the partition index
*/
public int partition_with_mid_element(int[] input, int left, int right) {
int index = (left + right) / 2;
int pivot = input[index];
int i = left - 1;
for(int j = left; j <= right; j++) {
if(j == index) {
continue;//skip index element
}
if(input[j] <= pivot) {
i++;
if(i == index && (i + 1) != right) {
i++;//skip index element
}
if(i != j) {
swap(input, i, j);
}
}
}
if(i < index) {
//i did not moved.
swap(input, i + 1, index);
return i + 1;
}
swap(input, i, index);
return i;
}
void print(int[] input) {
System.out.print("\n\nInput array before sort :");
printArray(input);
quickSort(input, 0, input.length - 1);
System.out.print("\nInput array after quick sort :");
printArray(input);
System.out.println("\n");
}
public static void main(String a[]) {
QuickSortRecursive obj = new QuickSortRecursive();
int[] input1 = { 24, 2, 45, 20, 56, 75, 2, 56, 99, 53, 12 };
obj.print(input1);
int[] input2 = { 10, 5, 2, 20, 1, 3 };
obj.print(input2);
int[] input3 = { 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3 };
obj.print(input3);
int[] input4 = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
obj.print(input4);
int[] input5 = { 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 };
obj.print(input5);
}
}
Quick Sort Iterative:
public void quickSort(int[] input, int left, int right) {
// Create an auxiliary stack
Stack<Integer> stack = new Stack<>();
// push initial values of l and h to stack
stack.push(left);
stack.push(right);
// Keep popping from stack while is not empty
while (!stack.isEmpty()) {
// pop left and right
right = stack.pop();
left = stack.pop();
// Set pivot element at its correct position
// in sorted array
int pivot = partition_with_mid_element(input, left, right);
// If there are elements on left side of pivot,
// then push left side to stack
if (pivot - 1 > left) {
stack.push(left);
stack.push(pivot - 1);
}
// If there are elements on right side of pivot,
// then push right side to stack
if (pivot + 1 < right) {
stack.push(pivot + 1);
stack.push(right);
}
}
}
Analysis of QuickSort
Time taken by QuickSort in general can be written as following.
Time taken by QuickSort in general can be written as following.
T(n) = T(k) + T(n-k-1) +(n)
The first two terms are for two recursive calls, the last term is for the partition process. k is the number of elements which are smaller than pivot.
The time taken by QuickSort depends upon the input array and partition strategy. Following are three cases.
The time taken by QuickSort depends upon the input array and partition strategy. Following are three cases.
Worst Case: The worst case occurs when the partition process always picks greatest or smallest element as pivot. If we consider above partition strategy where last element is always picked as pivot, the worst case would occur when the array is already sorted in increasing or decreasing order. Following is recurrence for worst case.
T(n) = T(0) + T(n-1) +
The solution of above recurrence is (n2).
(n2).
Best Case: The best case occurs when the partition process always picks the middle element as pivot. Following is recurrence for best case.
T(n) = 2T(n/2) +
The solution of above recurrence is (nLogn). It can be solved using case 2 of Master Theorem.
(nLogn). It can be solved using case 2 of Master Theorem.
Average Case:
To do average case analysis, we need to consider all possible permutation of array and calculate time taken by every permutation which doesn’t look easy.
We can get an idea of average case by considering the case when partition puts O(n/9) elements in one set and O(9n/10) elements in other set. Following is recurrence for this case.
To do average case analysis, we need to consider all possible permutation of array and calculate time taken by every permutation which doesn’t look easy.
We can get an idea of average case by considering the case when partition puts O(n/9) elements in one set and O(9n/10) elements in other set. Following is recurrence for this case.
T(n) = T(n/9) + T(9n/10) +
Solution of above recurrence is also O(nLogn)
Although the worst case time complexity of QuickSort is O(n2) which is more than many other sorting algorithms like Merge Sort and Heap Sort, QuickSort is faster in practice, because its inner loop can be efficiently implemented on most architectures, and in most real-world data. QuickSort can be implemented in different ways by changing the choice of pivot, so that the worst case rarely occurs for a given type of data. However, merge sort is generally considered better when data is huge and stored in external storage.
Is QuickSort stable?
The default implementation is not stable. However any sorting algorithm can be made stable by considering indexes as comparison parameter.
The default implementation is not stable. However any sorting algorithm can be made stable by considering indexes as comparison parameter.
What is 3-Way QuickSort?
In simple QuickSort algorithm, we select an element as pivot, partition the array around pivot and recur for subarrays on left and right of pivot.
Consider an array which has many redundant elements. For example, {1, 4, 2, 4, 2, 4, 1, 2, 4, 1, 2, 2, 2, 2, 4, 1, 4, 4, 4}. If 4 is picked as pivot in Simple QuickSort, we fix only one 4 and recursively process remaining occurrences. In 3 Way QuickSort, an array arr[l..r] is divided in 3 parts:
a) arr[l..i] elements less than pivot.
b) arr[i+1..j-1] elements equal to pivot.
c) arr[j..r] elements greater than pivot.
See this for implementation.
In simple QuickSort algorithm, we select an element as pivot, partition the array around pivot and recur for subarrays on left and right of pivot.
Consider an array which has many redundant elements. For example, {1, 4, 2, 4, 2, 4, 1, 2, 4, 1, 2, 2, 2, 2, 4, 1, 4, 4, 4}. If 4 is picked as pivot in Simple QuickSort, we fix only one 4 and recursively process remaining occurrences. In 3 Way QuickSort, an array arr[l..r] is divided in 3 parts:
a) arr[l..i] elements less than pivot.
b) arr[i+1..j-1] elements equal to pivot.
c) arr[j..r] elements greater than pivot.
See this for implementation.
How to implement QuickSort for Linked Lists?
QuickSort on Singly Linked List
QuickSort on Doubly Linked List
QuickSort on Singly Linked List
QuickSort on Doubly Linked List
Can we implement QuickSort Iteratively?
Yes, please refer Iterative Quick Sort.
Yes, please refer Iterative Quick Sort.
Why Quick Sort is preferred over MergeSort for sorting Arrays
Quick Sort in its general form is an in-place sort (i.e. it doesn’t require any extra storage) whereas merge sort requires O(N) extra storage, N denoting the array size which may be quite expensive. Allocating and de-allocating the extra space used for merge sort increases the running time of the algorithm. Comparing average complexity we find that both type of sorts have O(NlogN) average complexity but the constants differ. For arrays, merge sort loses due to the use of extra O(N) storage space.
Quick Sort in its general form is an in-place sort (i.e. it doesn’t require any extra storage) whereas merge sort requires O(N) extra storage, N denoting the array size which may be quite expensive. Allocating and de-allocating the extra space used for merge sort increases the running time of the algorithm. Comparing average complexity we find that both type of sorts have O(NlogN) average complexity but the constants differ. For arrays, merge sort loses due to the use of extra O(N) storage space.
Most practical implementations of Quick Sort use randomized version. The randomized version has expected time complexity of O(nLogn). The worst case is possible in randomized version also, but worst case doesn’t occur for a particular pattern (like sorted array) and randomized Quick Sort works well in practice.
Quick Sort is also a cache friendly sorting algorithm as it has good locality of reference when used for arrays.
Quick Sort is also tail recursive, therefore tail call optimizations is done.
Why MergeSort is preferred over QuickSort for Linked Lists?
In case of linked lists the case is different mainly due to difference in memory allocation of arrays and linked lists. Unlike arrays, linked list nodes may not be adjacent in memory. Unlike array, in linked list, we can insert items in the middle in O(1) extra space and O(1) time. Therefore merge operation of merge sort can be implemented without extra space for linked lists.
In case of linked lists the case is different mainly due to difference in memory allocation of arrays and linked lists. Unlike arrays, linked list nodes may not be adjacent in memory. Unlike array, in linked list, we can insert items in the middle in O(1) extra space and O(1) time. Therefore merge operation of merge sort can be implemented without extra space for linked lists.
In arrays, we can do random access as elements are continuous in memory. Let us say we have an integer (4-byte) array A and let the address of A[0] be x then to access A[i], we can directly access the memory at (x + i*4). Unlike arrays, we can not do random access in linked list. Quick Sort requires a lot of this kind of access. In linked list to access i’th index, we have to travel each and every node from the head to i’th node as we don’t have continuous block of memory. Therefore, the overhead increases for quick sort. Merge sort accesses data sequentially and the need of random access is low.
How to optimize QuickSort so that it takes O(Log n) extra space in worst case?
Please see QuickSort Tail Call Optimization (Reducing worst case space to Log n )
Please see QuickSort Tail Call Optimization (Reducing worst case space to Log n )
Comments
Post a Comment