In trying to understand humans well enough to create general models of behavior I begin with a trait common to ALL humans: in any dilemma, humans make the choice which least disturbs their habitual patterns of activity. Note that I didn't say the easiest, fastest or smartest choice. With this in mind, let's revisit yesterday's post where I quoted from Clay Shirky's Many-to-Many post, "Inequality," after a brief excursion to secure an understanding of the term "power law distribution":
In systems where many people are free to choose between many options, a small subset of the whole will get a disproportionate amount of traffic (or attention, or income), even if no members of the system actively work towards such an outcome. This has nothing to do with moral weakness, selling out, or any other psychological explanation. The very act of choosing, spread widely enough and freely enough, creates a power law distribution.
A Predictable Imbalance
Power law distributions, the shape that has spawned a number of catch-phrases like the 80/20 Rule and the Winner-Take-All Society, are finally being understood clearly enough to be useful. For much of the last century, investigators have been finding power law distributions in human systems. The economist Vilfredo Pareto observed that wealth follows a "predictable imbalance", with 20% of the population holding 80% of the wealth. The linguist George Zipf observed that word frequency falls in a power law pattern, with a small number of high frequency words (I, of, the), a moderate number of common words (book, cat cup), and a huge number of low frequency words (peripatetic, hypognathous). Jacob Nielsen observed power law distributions in web site page views, and so on.
More precisely (without excess precision)
The basic shape is simple - in any system sorted by rank, the value for the Nth position will be 1/N. For whatever is being ranked -- income, links, traffic -- the value of second place will be half that of first place, and tenth place will be one-tenth of first place. (There are other, more complex formulae that make the slope more or less extreme, but they all relate to this curve.) We've seen this shape in many systems. What've we've been lacking, until recently, is a theory to go with these observed patterns.
Now, thanks to a series of breakthroughs in network theory by researchers like Albert-Laszlo Barabasi, Duncan Watts, and Bernardo Huberman among others, breakthroughs being described in books like Linked, Six Degrees, and The Laws of the Web, we know that power law distributions tend to arise in social systems where many people express their preferences among many options. We also know that as the number of options rise, the curve becomes more extreme. This is a counter-intuitive finding - most of us would expect a rising number of choices to flatten the curve, but in fact, increasing the size of the system increases the gap between the #1 spot and the median spot.
A second counter-intuitive aspect of power laws is that most elements in a power law system are below average, because the curve is so heavily weighted towards the top performers. In Figure #1, the average number of inbound links (cumulative links divided by the number of blogs) is 31. The first blog below 31 links is 142nd on the list, meaning two-thirds of the listed blogs have a below average number of inbound links. We are so used to the evenness of the bell curve, where the median position has the average value, that the idea of two-thirds of a population being below average sounds strange. (The actual median, 217th of 433, has only 15 inbound links.)
This pattern would explain a lot of specific economic and social facts (like that average income can rise at the same time the number of broke people increases). A lot of worthy discussion could proceed from this point, but I really did mean this diversion to be brief.
The main thought here comes from combining my opening assumption about human nature with the four methods he mentioned of responding to a power law distribution. Keeping in mind who does the official responding to socioeconomic events, how do you think the economic equivalent of each technique would be appraised by Conservatives™?
You can tax the system, moving more of X (whatever is being ranked) from where it is abundant to where it is scarce. This is how progressive taxation works, transferring money from those with most to those with least. The result is still a power law distribution (the economist Vilfredo Pareto called the distribution “a predictable imbalance” and found it in all market economies), its just got a shallower slope, and an average that’s further from the #1 position and closer to the median.
No need to guess here. Only the various Messiahs get more respect than the idea of cutting taxes.
You can limit the head of the curve. Sometimes this happens naturally — even Joi Ito falls short of the number of connections required for a raw power law distribution on LinkedIn, while over at Friendster, they have placed an artificial cap at 200 links.
This translates to setting limits on oneself; NOT your typical Conservative™'s habitual behavior.
You can also try to make the system more dynamic, by making it possible for new nodes to get attention. This is what Sifry is up to with his Interesting Newcomers list. These are not weblogs with a high number of inbound links, but rather high growth.
Facilitating the competition. Not the habitual behavior of Capitalists™, of which Conservatives™ are a subset. That leaves:
You can truncate the tail, periodically dumping those links that are below median or some other threshold.
This is the Conservative™ plan…the way unemployment figures are calculated is a near archetypical example (no, the idea of not counting those who are no longer looking for work isn't a Conservative™ plot, though last summer's decision to change the method of adjusting the figures…a change that coincidentally made the Bushista's Boys look better…probably qualifies as a plot). It requires no change on their part, and by insisting that amoral market forces set the truncation threshold they can avoid the moral repercussions of allowing amoral market forces to set the threshold in the first place.
It's pure self interest at work. But it's not an irrational decision. It's not stupid or evil. That's a good thing because you can't actually talk to stupid, irrational or evil.
Posted by P6 at January 21, 2004 05:48 AM
Trackback URL: http://www.niggerati.net/mt/mt-tb.cgi/71
Comments
"It's pure self interest at work. But it's not an irrational decision."
That is because self-interest is purely rational, even when that self-interest is disguised by interest in the well-being of others.
As for the rest of the article, I'll have to read it over again. There seemed to be some interesting points in there, especially the fact that diversity and freedom lead to inequality. This is not a novel idea though, since this was known before Plato, and first put into action during the French Revolution through the words of Babeuf and Marechal. Since you, and most on the left, seem to favor equality at any costs, it makes sense why, despite claims to the contrary, the left has always been hostile toward freedom and diversity.
Posted by Brian at January 21, 2004 11:20 AMYep, hostile to freedom and diversity, that's me! When you get back in touch with planet Earth, Brian, be sure to let us know.
Posted by Al-Muhajabah at January 21, 2004 12:38 PMI don't know who you are, nor do I know anything about your views, Al-.
But I stand by my contention that those who seek equality at any cost will do so at the expense of liberty and diversity. Just to clarify, diversity does not consist of variations of concentrations of certain skin-coloring pigments. Communism sought to destroy diversity by imposing a uniformity of thought, a single ideology on everyone, reinforced through repition of mindless slogans. French revolutionaries saw diversity of talents, intelligence and wisdom a threat to the dream of absolute equality.
Posted by Brian at January 21, 2004 01:04 PMTell me what you mean by freedom and diversity. I'm sure you're right, that I'm hostile to your understanding of the terms.
Posted by P6 at January 21, 2004 01:14 PMTo me, freedom is negative. It is what you can possess without depriving another person of theirs. Freedom is my ability to act so long as my actions harm no one, or impede no one else's ability to act.
Posted by Brian at January 21, 2004 01:19 PMWhat about diversity? You forgot that. Let's see if YOU are an enemy of diversity or not.
Posted by P6 at January 21, 2004 02:11 PMthose who seek equality at any cost will do so at the expense of liberty and diversity
People expect equal opportunity, and should should have it. That does not mean everyone has to be able to afford a Hummer, it just means people should all have an equal opportunity to. That does not take anyones liberty away. It expands it.
And a quick note on freedom. IMHO, true freedom is being left alone.
Posted by Prince C. at January 21, 2004 06:06 PMBrian will agree with your definition of freedom, Prince. He's a Libertarian, and Libertairians annoy me because I feel the almost get it, but don't want it.
Posted by P6 at January 21, 2004 06:17 PMBrian, it's the fact that you don't know me that means you shouldn't make generalizations about me because I'm a liberal. Get it?
Posted by Al-Muhajabah at January 21, 2004 08:47 PMUnfortunately, Prometheus' extremely interesting citation got hijacked by a lame digression. I move that we start posting comments about Clay Shirky's essay.
First, the notion that freedom is somehow at odds with equality is illogical. Let me begin by rebutting the converse: inequality is a consequence of freedom, including economic freedom (here, freedom by owners to exchange their endowments freely).
[unfortunately I was just told to leave the building. Anyone interested in what I was going to say should drop me a line. Sorry to do that!]
Posted by James R MacLean at January 21, 2004 10:24 PMSo as I was saying--
If you have little freedom, including little economic freedom, it's extremely probable that the extra state power (or warlord power, or company town power, or union busting thug power) is used to expropriate rents from the unfree. This was true under all of the various fascist states, under Leninist states, under kleptocratic autocracies, etc. Conversely , freedom could stimulate investment in public & merit goods; a vibrant democracy means--in econ lingo--efficient, or deepened, markets for public goods.
Indeed, it's reasonable to suspect that freedom and equality are mutually reinforcing (because strong democratic institutions prevent the tragedy of the commons, and they also afford greater protection for dissent and nonconformer rights.
Posted by James R MacLean at January 22, 2004 09:17 PM