Do not make the mistake of thinking that because my conclusion is the same as another person's that my reasoning is the same
A Good Cause or Two
nbuf_button.gif bootbush.jpg
Click for more info

The Best of P6
The Racism Series The Reparations Series Installing a negro in your head Identity Blogging Where We Stand The LimbaughDiscussion That has Nothing To Do With Limbaugh
Updated when I write something really cool

Search
Local Links
The Attack on Civil Rights Corporate Influence on Government The Development of Race Basic Laws of Human Stupidity Blogger Archives
EMAIL ME AT
email.gif
Blogroll Me!
Blog-related mail may be published

The Public Library
The Black Experience in America The Souls of Black Folks My Bondage and My Freedom The Martin Luther King Jr. Collection Walker's AppealThe Shaping of Black America, Ch. 3
Updated as frequently as possible

Archives
April 18, 2004 - April 24, 2004 April 11, 2004 - April 17, 2004 April 04, 2004 - April 10, 2004 March 28, 2004 - April 03, 2004 March 21, 2004 - March 27, 2004 March 14, 2004 - March 20, 2004 March 07, 2004 - March 13, 2004 February 29, 2004 - March 06, 2004 February 22, 2004 - February 28, 2004 February 15, 2004 - February 21, 2004 February 08, 2004 - February 14, 2004 February 01, 2004 - February 07, 2004 January 25, 2004 - January 31, 2004 January 18, 2004 - January 24, 2004 January 11, 2004 - January 17, 2004 January 11, 2004 - January 17, 2004January 04, 2004 - January 10, 2004December 28, 2003 - January 03, 2004December 21, 2003 - December 27, 2003December 14, 2003 - December 20, 2003December 07, 2003 - December 13, 2003November 30, 2003 - December 06, 2003November 23, 2003 - November 29, 2003November 16, 2003 - November 22, 2003November 09, 2003 - November 15, 2003November 02, 2003 - November 08, 2003October 26, 2003 - November 01, 2003October 19, 2003 - October 25, 2003October 12, 2003 - October 18, 2003October 05, 2003 - October 11, 2003September 28, 2003 - October 04, 2003September 21, 2003 - September 27, 2003September 14, 2003 - September 20, 2003September 07, 2003 - September 13, 2003August 31, 2003 - September 06, 2003August 24, 2003 - August 30, 2003August 17, 2003 - August 23, 2003August 10, 2003 - August 16, 2003August 03, 2003 - August 09, 2003 July 27, 2003 - August 02, 2003 July 20, 2003 - July 26, 2003 July 13, 2003 - July 19, 2003 July 06, 2003 - July 12, 2003 June 29, 2003 - July 05, 2003 June 22, 2003 - June 28, 2003 June 15, 2003 - June 21, 2003 June 08, 2003 - June 14, 2003 June 01, 2003 - June 07, 2003 May 25, 2003 - May 31, 2003 May 18, 2003 - May 24, 2003 May 11, 2003 - May 17, 2003 May 04, 2003 - May 10, 2003 April 27, 2003 - May 03, 2003 April 20, 2003 - April 26, 2003 April 13, 2003 - April 19, 2003 April 06, 2003 - April 12, 2003
« Almost missed this | Main | That, of course, is the goal »

January 20, 2004
Taking statements out of context 

Ever since Clay Shirky wrote "Power Laws, Weblogs, and Inequality he's been dogged by follow-up articles on repercussions, reinterpretations and discussions about what the discussions about it mean.

Mr. Shirky just posted something at Many-to-Many titled Inequality that is about networks but since society is a kind of network I don't feel I'm reaching when I suggest you apply the analysis to politics, society and all that.

So here’s what I do believe: inequality is inevitable, and that being for or against it makes no more sense than being for or against the weather.

Now there are many ways to treat inequality as inevitable — you can adopt such a posture because you are or have become cynical, worldly wise, passive, or an adherent of realpolitik — but I have a very particular way in which I believe inequality is inevitable. I believe that wanting large networks without inequality is like wanting mortar without sand. Inequality is not some removable side-effect of networks; inequality is what holds networks together, inequality is core to how networks work.

Networks are deep patterns, but we have often treated them as shallow. Over the last hundred years, we have observed networks at work in a variety of places — food chains, the spread of gossip, the electrical grid, the connection of neurons — but we have often regarded those networks as second-order entities, whose behavior is mainly the product of their constituent parts.

This is wrong. Networks are 0th order patterns, deeper than their constituent parts. The way packets moves through the network is similar to the way gossip moves through an office, even though a router is nothing like an office worker. Once you network things — any things — the subsequent patterns are more affected by the characteristics of networks than the characteristics of the things the network is made of.

One of the characteristics of networks is a kind of structural inequality that holds things together. Call this a Zipf or Pareto distribution or a power law or any of the other names it’s been given. If a system is large, heterogeneous, and robustly but sparsely connected, it will exhibit power law distributions in its arrangement, and that pattern will be reflected in whatever binds the network together, both statically and dynamically — link density, popularity, messages sent or received, and so on.

If this hypothesis is correct, there are two workable responses, and one obviously unworkable one. The first workable response is to exit the system by violating one or more of the core conditions.

  • You can get out of a system with power law distributions by giving up on scale. As Simon St. Laurent points out in Escaping the Googlearchy, one way to avoid the inequality of large systems is not to have large systems. This seems to me to be an incredibly fruitful area of inquiry — the web made us all size queens for a decade, but now that hardware, system tools, and most importantly programming talent and time are abundant enough, not everything has to scale. I believe we are seeing and will continue to see work on small systems that avoid power law distributions because, as Ross noted long ago in Ecosystem of Networks, a group of 12 is different from a group of 150.
  • The second way to get out of such systems is to violate heterogeneity. The US Army has systems that don’t exhibit power law distributions of links in the command structure, because they control all the units. They can achieve hierarchical coordination by avoiding heterogeneity.
  • The third way to get out is to drop the idea that the network needs to be robust. If you do not require a network with the six degrees of separation principle, and can tolerate lots of cases where the nodes simply can’t reach each other, you have much greater freedom in topological models. This is what Gnutella was like in its earliest incarnation, before the advent of super-peers.
However, simply escaping systems that happen to have power law distributions is a really lousy solution for all sorts of environments we like to be a part of. This trio of characteristics — large, heterogeneous, and robust but sparse — is the one that allows us to have, say, billions of people separated by 6 degrees, or an internet where any two machines can reach one another in fewer (usually quite a bit fewer) than 30 hops.

This brings up the other workable response to this sort of inequality — accept it, but change its terms. Once you accept that there will be a power law distribution, instead of fighting it, you can concentrate on modifying it. There are several possible strategies here as well.

  • You can tax the system, moving more of X (whatever is being ranked) from where it is abundant to where it is scarce. This is how progressive taxation works, transferring money from those with most to those with least. The result is still a power law distribution (the economist Vilfredo Pareto called the distribution “a predictable imbalance” and found it in all market economies), its just got a shallower slope, and an average that’s further from the #1 position and closer to the median.
  • You can limit the head of the curve. Sometimes this happens naturally — even Joi Ito falls short of the number of connections required for a raw power law distribution on LinkedIn, while over at Friendster, they have placed an artificial cap at 200 links.
  • You can truncate the tail, periodically dumping those links that are below median or some other threshold. Brucker-Cohen’s BumpList is an example of auto-truncation, though at a scale much too small to demonstrate power laws. I don’t know of any systems that use auto-truncation at large scale.
  • You can also try to make the system more dynamic, by making it possible for new nodes to get attention. This is what Sifry is up to with his Interesting Newcomers list. These are not weblogs with a high number of inbound links, but rather high growth.
All of these responses leave the power law intact, just altered.

The unworkable response is to assume you can destroy the power law distribution without also destroying (or at least altering beyond recognition) the factors in which caused it to arise in the first place — size, diversity, connectedness.

Last summer, a study came out describing the characteristics of the conservative mindset, which included, among other things, a tolerance for inequality. By this definition, I believe we will all be conservatives soon. Evidence that inequality is a core aspect of our large systems, is in many ways the signature of those systems in fact, will make utopian declarations of being ‘against’ inequality an impossibility for anyone who regards reality as a constraint on their world view.



Posted by P6 at January 20, 2004 12:51 PM
Trackback URL: http://www.niggerati.net/mt/mt-tb.cgi/65
Comments
Post a comment









Remember personal info?