The Pareto Principle in Data Engineering | by Hugo Lu | Sep, 2024
We’ve all heard of the Pareto Principle. It’s also known as the 80/20 rule. It’s the idea that you can get 80% of the work done with 20% of the effort; the final 20% of the job takes up the other 80% of the effort.
It derives from the work of an economist that’s so famous you even study them at school. Their name is Vilfredo Pareto. The first thing they teach you at school is a different concept that’s also eponymous with Pareto: Pareto Efficiency.
An outcome is said to be Pareto Efficient or Pareto optimal if you cannot improve it without damaging the welfare of any of the parties involved. For example, while it might be a “net improvement” for society to have stricter laws on the environment, such policies do not constitute a pareto improvement. This is because companies that pollute the environment are made worse off.
Formally, a state is Pareto-optimal if there is no alternative state where at least one participant’s well-being is higher, and nobody else’s well-being is lower. If there is a state change that satisfies this condition, the new state is called a “Pareto…