Parallelization runs independent tasks at the same time, instead of one after another, and then combines the results. It has two distinct uses.
Divide and conquer#
Split a large task into independent subtasks and run them all at once.
Example: you need a summary of each section of a 50-page report. Instead of sending the sections one at a time, you send all 50 in parallel and combine the summaries at the end. What would take 50 calls in a row now takes roughly the time of one.
This works only when the subtasks are truly independent. If section 12's summary needs section 11's, they cannot run side by side.
Voting#
Run the same task several times and take the majority answer.
Example: classify a document as safe or unsafe. Run the classification 5 times and keep the answer that 4 or 5 runs agree on.
Voting trades cost for confidence. If a single run is 85% accurate, the majority of 5 runs is much more reliable, because several runs rarely make the same mistake at once. You pay for 5 calls instead of 1.
Comparing the two#
| Divide and conquer | Voting | |
|---|---|---|
| Each call does | A different part of the job | The same job |
| You combine by | Merging the parts | Taking the majority |
| What you gain | Speed | Confidence |
| What you pay | About the same total tokens | Several times the cost |
Short line#
Parallelization runs independent tasks at the same time and combines the results. Divide and conquer splits a big job into parts for speed. Voting runs the same task several times and takes the majority, trading cost for confidence.
Key takeaways#
- Run independent tasks at the same time, then combine the results.
- Divide and conquer: split the job into independent parts, so 50 calls take about the time of one.
- The parts must not depend on each other.
- Voting: run the same task several times and keep the majority answer.
- Voting trades cost for confidence.