Greedy Algorithms

This page briefly introduces greedy algorithms.

Introduction

A greedy algorithm (English: greedy algorithm) is a process that uses a computer to simulate a “greedy” person making decisions. This person is very greedy: at every step, each action always selects the optimal operation according to some criterion. They are also short-sighted, always considering only the immediate situation and not the possible effects later on.

As you can imagine, greedy methods do not always produce the optimal solution, so whenever a greedy method is used, its correctness generally needs to be proved.

Explanation

Scope of Application

Greedy algorithms are particularly effective for problems with optimal substructure. Optimal substructure means that a problem can be decomposed into subproblems, and the optimal solutions to the subproblems can be extended recursively to the optimal solution of the final problem.1

Proof

There are two methods for proving greedy algorithms: proof by contradiction and induction. In general, a problem uses only one of these methods for its proof.

  1. Proof by contradiction: If exchanging any two elements/two adjacent elements in the exchange scheme does not make the answer better, then it can be inferred that the current solution is already optimal.
  2. Induction: First calculate the optimal solution F1F_1 for the base case (for example, n=1n = 1), and then prove that, for every nn, Fn+1F_{n+1} can be derived from FnF_{n}.

Key Points

Common Problem Types

Below the difficulty of the advanced group, there are two most common types of greedy methods.

  • “We sort XXX in a certain order, and then select in some order (for example, from small to large).”
  • “Each time, we take the largest/smallest item in XXX and update XXX.” (Sometimes “the largest/smallest item in XXX” can be optimized, for example by maintaining it with a priority queue.)

The difference between the two is that one is offline: process first, then select; the other is online: process and select at the same time.

Sorting Solution

A common situation for the sorting method is an array containing several (generally one or two) weights. The optimal value is obtained by sorting and then traversing the array to simulate the calculation.

Regret Solution

The idea is to accept every current option regardless of whether it is optimal, and then compare it. If the choice is no longer optimal after being selected, regret it and discard the option; otherwise, formally accept it. Repeat this process.

Differences

Difference from Dynamic Programming

The difference between greedy algorithms and dynamic programming is that greedy algorithms make a choice for the solution of each subproblem and cannot roll back. Dynamic programming saves previous computation results and makes the current choice based on previous results, so it has rollback functionality.

Detailed Examples

Example of the Adjacent-Exchange Method

NOIP 2012 The King's Game

On the national holiday of country H, the king invited n ministers to play a game with prizes. First, he asked each minister to write an integer on each of their left and right hands, and the king himself also wrote an integer on each of his left and right hands. He then arranged the n ministers in a line, with the king standing at the very front of the line. After they were arranged, all ministers would receive some gold coins as a reward from the king. The reward received by each minister is the product of the numbers on the left hands of all people in front of that minister, divided by the number on that minister's right hand, with the result rounded down.

The king does not want any particular minister to receive an especially large reward, so he asks you to rearrange the order of the line so that the reward received by the minister with the largest reward is as small as possible. Note that the king's position is always at the front of the line.

Solution

Let the numbers on the left and right hands of the ii-th minister after sorting be ai,bia_i, b_i, respectively. Consider deriving a greedy strategy using the adjacent-exchange method.

Let ss denote the product of all aia_i values of the people in front of the ii-th minister. Then the reward received by the ii-th minister is sbi\dfrac{s} {b_i}, and the reward received by the i+1i + 1-th minister is s⋅aibi+1\dfrac{s \cdot a_i} {b_{i+1}}.

If the ii-th and i+1i + 1-th ministers are exchanged, the reward received by the current ii-th minister is sbi+1\dfrac{s} {b_{i+1}}, and the reward received by the current i+1i + 1-th minister is s⋅ai+1bi\dfrac{s \cdot a_{i+1}} {b_i}.

The arrangement before the exchange is better if and only if

max⁡(sbi,s⋅aibi+1)<max⁡(sbi+1,s⋅ai+1bi)\max \left(\dfrac{s} {b_i}, \dfrac{s \cdot a_i} {b_{i+1}}\right) < \max \left(\dfrac{s} {b_{i+1}}, \dfrac{s \cdot a_{i+1}} {b_i}\right)

Extracting the common ss and reducing gives

max⁡(1bi,aibi+1)<max⁡(1bi+1,ai+1bi)\max \left(\dfrac{1} {b_i}, \dfrac{a_i} {b_{i+1}}\right) < \max \left(\dfrac{1} {b_{i+1}}, \dfrac{a_{i+1}} {b_i}\right)

Converting the fractions to integer expressions gives

max⁡(bi+1,ai⋅bi)<max⁡(bi,ai+1⋅bi+1)\max (b_{i+1}, a_i\cdot b_i) < \max (b_i, a_{i+1}\cdot b_{i+1})

In the implementation, we save the two input numbers in a struct and overload the operator:

struct uv {
  int a, b;

  bool operator<(const uv &x) const {
    return max(x.b, a * b) < max(b, x.a * x.b);
  }
};

Example of the Regret Method

“USACO09OPEN” Work Scheduling

John's workday starts at time 00 and has 10910^9 units of time. At any unit of time, he can choose any one of the N jobs numbered 11 through NN, where N(1≤N≤105)N(1 \leq N \leq 10^5). The deadline of job ii is Di(1≤Di≤109)D_i(1 \leq D_i \leq 10^9), and its profit after completion is Pi(1≤Pi≤109)P_i( 1\leq P_i\leq 10^9 ). Given the job profits and deadlines, find the maximum profit John can obtain.

Solution
  1. First assume that every job is done, sort all jobs by deadline, and add them to the queue;
  2. When deciding whether to do job i, if its deadline satisfies the condition, compare it with the element having the smallest reward in the queue. If the reward of job i is higher (regret), then ans += a[i].p - q.top().
    Use a priority queue (min-heap) to maintain the smallest element at the front of the queue.
  3. When a[i].d<=q.size(), this can be understood as follows: from time 0 to time a[i].d, only a[i].d tasks can be done. If q.size()>=a[i].d, completing q.size() tasks takes at least a[i].d units of time, so when the profit of job i is relatively large, the smallest task should be swapped out of the priority queue.
Reference Code

C++

#include <algorithm>
#include <cmath>
#include <cstring>
#include <iostream>
#include <queue>
using namespace std;

struct f {
  long long d;
  long long p;
} a[100005];

bool cmp(f A, f B) { return A.d < B.d; }

// Maintain the minimum value with a min-heap
priority_queue<long long, vector<long long>, greater<long long>> q;

int main() {
  long long n, i;
  cin >> n;
  for (i = 1; i <= n; i++) {
    cin >> a[i].d >> a[i].p;
  }
  sort(a + 1, a + n + 1, cmp);
  long long ans = 0;
  for (i = 1; i <= n; i++) {
    if (a[i].d <= (int)q.size()) {  // Past the deadline
      if (q.top() < a[i].p) {       // Regret
        ans += a[i].p - q.top();
        q.pop();
        q.push(a[i].p);
      }
    } else {  // Add directly to the queue
      ans += a[i].p;
      q.push(a[i].p);
    }
  }
  cout << ans << endl;
  return 0;
}

Python

from collections import defaultdict
from heapq import heappush, heappop

a = defaultdict(list)
for _ in range(int(input())):
    d, p = map(int, input().split())
    a[d].append(p)  # Store the profit for the corresponding time

ans = 0  # Record the total profit
q = []  # Maintain the minimum value with a min-heap
l = sorted(a.keys(), reverse=True)
for i, j in zip(l, l[1:] + [0]):
    for k in a.pop(i):
        heappush(q, ~k)
    for _ in range(i - j):
        if q:  # Take the job with the highest profit from the heap
            ans += ~heappop(q)
        else:  # Exit the loop when the heap is empty
            break
print(ans)
Complexity Analysis
  • Space complexity: when nn tasks are input, nn elements are used in the aa array, and in the worst case the priority queue stores nn elements, so the space complexity is O(n)O(n).
  • Time complexity: the time complexity of std::sort is O(nlog⁡n)O(n\log n), and the time complexity of maintaining the priority queue is O(nlog⁡n)O(n\log n). Therefore, the overall time complexity is O(nlog⁡n)O(n\log n).

Exercises

References and notes

Footnotes

  1. Greedy algorithm - Wikipedia, the free encyclopedia ↩

This content comes from OI Wiki: Greedy Algorithms. See the original for authors and revision history. AIOJ has adapted the formatting, in-site links, and resource paths.

Content is shared under CC BY-SA 4.0 and SATA license; additional terms are governed by the original notice. Thanks to the OI Wiki authors and contributors; per the SATA terms, please visit the OI Wiki repository and star it to show your support.