Black Holes in the Blogosphere

This is an attempt at explaining why finding weblog communities based on linking is so difficult. Fortunately, even if the problem is not solvable it is still a nice idea to share this knowledge with others (sigh). That is not to say that I have given up, and neither have Lilia and Stephanie who inspired this anti-sleeping pill.

First, a couple of disclaimers. What follows is based on posts and links in 2004. In addition, the weblog spidering is not perfect so some potential community candidates that could not be spidered had to be omitted.

Below is a screenshot from the BlogTrace community finder that illustrates the behaviour of the current algorithm.

On the left is the community and the right shows who links to whom. Unselfishly I started with my own blog in the knowledge that Lilia is really the core. The algorithm is based on the idea to invite new members once they are socially "acceptable" to the existing members. The top-to-bottom order on the left is therefore relevant and reflects order of invitation. Socially acceptable is operationalised mathematically by considering whether candidate members are linked to by existing members and also whether they link back (reciprocity). The specific operationalisation is based on the idea of resistance: the more a weblog is linked to within the community, the easier it will be found by others.

The first seven members of the community I know personally, I link to most of them and most of them link to me. This is actually an excellent result. However, Lilia is really the community core and several blogs she links to frequently are not made part of the community.

One of the reasons the algorithm starts drifting is that it has sucked dry the most likely candidates (links from the top-7 in the figure) and then finds new energy by following the links from two recent entries and core blogs: corante/many and zephoria/thoughts. Unfortunately, this starts an entirely new community and, socially, it should have acknowledged that potential community candidates existing members link to are to be preferred. The core blogs therefore act as black holes: once the algorithm hits on one of them it is very difficult to escape.

Another seed for which the same algorithm was tried is Peter Caputa's weblog.

Here it picks up corante and zephoria/thoughts a little earlier, and Peter has commented that in his case this is probably more correct (and most of the other community members also are familiar to him).

The next step is to find the magic potion that makes the algorithm behave a little better socially. After all, resistance is not the basis weblog communities are made of :-).

Comments

[Roland]

Hi Anjo,
Have you noticed NeuroGrid?
(http://www.neurogrid.net/php/whitepaper.php)
Maybe a social potion for your algorithm.

Greetings
Roland

BTW; and Pharming? .... ;-)