10.2 MeldableHeap: A Randomized Meldable Heap

In this section, we describe the MeldableHeap, a priority Queue implementation in which the underlying structure is also a heap-ordered binary tree. However, unlike a BinaryHeap in which the underlying binary tree is completely defined by the number of elements, there are no restrictions on the shape of the binary tree that underlies a MeldableHeap; anything goes.

The $\ensuremath{\mathrm{add}(\ensuremath{\mathit{x}})}$ and $\ensuremath{\mathrm{remove}()}$ operations in a MeldableHeap are implemented in terms of the $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ operation. This operation takes two heap nodes $\ensuremath{\ensuremath{\mathit{h_1}}}$ and $\ensuremath{\ensuremath{\mathit{h_2}}}$ and merges them, returning a heap node that is the root of a heap that contains all elements in the subtree rooted at $\ensuremath{\ensuremath{\mathit{h_1}}}$ and all elements in the subtree rooted at $\ensuremath{\ensuremath{\mathit{h_2}}}$.

The nice thing about a $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ operation is that it can be defined recursively. See Figure 10.4. If either $\ensuremath{\ensuremath{\mathit{h_1}}}$ or $\ensuremath{\ensuremath{\mathit{h_2}}}$ is $\ensuremath{\ensuremath{\mathit{nil}}}$, then we are merging with an empty set, so we return $\ensuremath{\ensuremath{\mathit{h_2}}}$ or $\ensuremath{\ensuremath{\mathit{h_1}}}$, respectively. Otherwise, assume $\ensuremath{\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{x}}}} \le \ensuremath{\ensuremath{\ensuremath{\mathit{h_2}}.\ensuremath{\mathit{x}}}}$ since, if $\ensuremath{\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{x}}}} > \ensuremath{\ensuremath{\ensuremath{\mathit{h_2}}.\ensuremath{\mathit{x}}}}$, then we can reverse the roles of $\ensuremath{\ensuremath{\mathit{h_1}}}$ and $\ensuremath{\ensuremath{\mathit{h_2}}}$. Then we know that the root of the merged heap will contain $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{x}}}$, and we can recursively merge $\ensuremath{\ensuremath{\mathit{h_2}}}$ with $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{left}}}$ or $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{right}}}$, as we wish. This is where randomization comes in, and we toss a coin to decide whether to merge $\ensuremath{\ensuremath{\mathit{h_2}}}$ with $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{left}}}$ or $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{right}}}$:

Figure 10.4: Merging $\ensuremath{\ensuremath{\mathit{h_1}}}$ and $\ensuremath{\ensuremath{\mathit{h_2}}}$ is done by merging $\ensuremath{\ensuremath{\mathit{h_2}}}$ with one of $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{left}}}$ or $\ensuremath{\ensuremath{\mathit{h_1}}.\ensuremath{\mathit{right}}}$.

In the next section, we show that $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ runs in $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time, where $\ensuremath{\ensuremath{\mathit{n}}}$ is the total number of elements in $\ensuremath{\ensuremath{\mathit{h_1}}}$ and $\ensuremath{\ensuremath{\mathit{h_2}}}$.

With access to a $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ operation, the $\ensuremath{\mathrm{add}(\ensuremath{\mathit{x}})}$ operation is easy. We create a new node $\ensuremath{\ensuremath{\mathit{u}}}$ containing $\ensuremath{\ensuremath{\mathit{x}}}$ and then merge $\ensuremath{\ensuremath{\mathit{u}}}$ with the root of our heap:

This takes $O(\log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1)) = O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time.

The $\ensuremath{\mathrm{remove}()}$ operation is similarly easy. The node we want to remove is the root, so we just merge its two children and make the result the root:

Again, this takes $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time.

Additionally, a MeldableHeap can implement many other operations in $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time, including:

• $\ensuremath{\mathrm{remove}(\ensuremath{\mathit{u}})}$: remove the node $\ensuremath{\ensuremath{\mathit{u}}}$ (and its key $\ensuremath{\ensuremath{\mathit{u}}.\ensuremath{\mathit{x}}}$) from the heap.
• $\ensuremath{\mathrm{absorb}(\ensuremath{\mathit{h}})}$: add all the elements of the MeldableHeap $\ensuremath{\ensuremath{\mathit{h}}}$ to this heap, emptying $\ensuremath{\ensuremath{\mathit{h}}}$ in the process.

Each of these operations can be implemented using a constant number of $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ operations that each take $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time.

10.2.1 Analysis of $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$

The analysis of $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ is based on the analysis of a random walk in a binary tree. A random walk in a binary tree starts at the root of the tree. At each step in the random walk, a coin is tossed and, depending on the result of this coin toss, the walk proceeds to the left or to the right child of the current node. The walk ends when it falls off the tree (the current node becomes $\ensuremath{\ensuremath{\mathit{nil}}}$).

The following lemma is somewhat remarkable because it does not depend at all on the shape of the binary tree:

Lemma 10..1   The expected length of a random walk in a binary tree with $\ensuremath{\ensuremath{\mathit{n}}}$ nodes is at most $\log \ensuremath{(\ensuremath{\mathit{n}}+1)}$.

Proof. The proof is by induction on $\ensuremath{\ensuremath{\mathit{n}}}$. In the base case, $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}=0$ and the walk has length $0=\log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1)$. Suppose now that the result is true for all non-negative integers $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}'< \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}$.

Let $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1$ denote the size of the root's left subtree, so that $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2=\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}-\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1-1$ is the size of the root's right subtree. Starting at the root, the walk takes one step and then continues in a subtree of size $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1$ or $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2$. By our inductive hypothesis, the expected length of the walk is then

$$\mathrm{E}[W] = 1 + \frac{1}{2}\log (\ensuremath{\ensuremath{\ens... ...{1}{2}\log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2+1) \enspace ,$$

since each of $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1$ and $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2$ are less than $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}$. Since $\log$ is a concave function, $\mathrm{E}[W]$ is maximized when $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1=\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2=(\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}-1)/2$. Therefore, the expected number of steps taken by the random walk is

$$\mathrm{E}[W] = 1 + \frac{1}{2}\log (\ensuremath{\ensuremath{\ensuremath{\mathi... ..._1+1) + \frac{1}{2}\log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2+1)$$

$$\le 1 + \log ((\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}-1)/2+1)$$

$$= 1 + \log ((\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1)/2)$$

$$= \log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1) \enspace . \qedhere$$

$\qedsymbol$

We make a quick digression to note that, for readers who know a little about information theory, the proof of Lemma 10.1 can be stated in terms of entropy.

Proof. [Information Theoretic Proof of Lemma 10.1] Let $d_i$ denote the depth of the $i$th external node and recall that a binary tree with $\ensuremath{\ensuremath{\mathit{n}}}$ nodes has $\ensuremath{\ensuremath{\mathit{n}}+1}$ external nodes. The probability of the random walk reaching the $i$th external node is exactly $p_i=1/2^{d_i}$, so the expected length of the random walk is given by

$$H=\sum_{i=0}^{\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}} ... ...\sum_{i=0}^{\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}}p_i\log({1/p_i})$$

The right hand side of this equation is easily recognizable as the entropy of a probability distribution over $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1$ elements. A basic fact about the entropy of a distribution over $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1$ elements is that it does not exceed $\log(\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}+1)$, which proves the lemma. $\qedsymbol$

With this result on random walks, we can now easily prove that the running time of the $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ operation is $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$.

Lemma 10..2   If $\ensuremath{\ensuremath{\mathit{h_1}}}$ and $\ensuremath{\ensuremath{\mathit{h_2}}}$ are the roots of two heaps containing $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1$ and $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2$ nodes, respectively, then the expected running time of $\ensuremath{\mathrm{merge}(\ensuremath{\mathit{h_1}},\ensuremath{\mathit{h_2}})}$ is at most $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$, where $\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}=\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1+\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_2$.

Proof. Each step of the merge algorithm takes one step of a random walk, either in the heap rooted at $\ensuremath{\ensuremath{\mathit{h_1}}}$ or the heap rooted at $\ensuremath{\ensuremath{\mathit{h_2}}}$. The algorithm terminates when either of these two random walks fall out of its corresponding tree (when $\ensuremath{\ensuremath{\ensuremath{\mathit{h_1}}}}=\ensuremath{\ensuremath{\ensuremath{\mathit{nil}}}}$ or $\ensuremath{\ensuremath{\ensuremath{\mathit{h_2}}}}=\ensuremath{\ensuremath{\ensuremath{\mathit{nil}}}}$). Therefore, the expected number of steps performed by the merge algorithm is at most

$$\log (\ensuremath{\ensuremath{\ensuremath{\mathit{n}}}}_1+1) + \l... ...e 2\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}} \enspace . \qedhere$$

$\qedsymbol$

10.2.2 Summary

The following theorem summarizes the performance of a MeldableHeap:

Theorem 10..2   A MeldableHeap implements the (priority) Queue interface. A MeldableHeap supports the operations $\ensuremath{\mathrm{add}(\ensuremath{\mathit{x}})}$ and $\ensuremath{\mathrm{remove}()}$ in $O(\log \ensuremath{\ensuremath{\ensuremath{\mathit{n}}}})$ expected time per operation.