BlogTrace

This post summarizes the architecture and status of BlogTrace. BlogTrace is an interactive environment that supports weblog research. It includes Sigmund. In the months to come I hope to address several of the issues raised below and this post also serves as a single place of reference for BlogTrace should I decide to blog about it in the future :-).

Supporting Blog Research is the post that first outlined the idea of BlogTrace (I'm surprised this is dated November 18th, 2004). Since then it has materialised and Visual Settlements, Making a Difference and Knowledge Flows have been developed on top of BlogTrace.

One of the outcomes of our research planning is that resources to develop BlogTrace further have been made available. Many thanks to Telematica Instituut (in particular Janine Swaak) for that. Sigmund is moving from "weblogs only" to other types of text documents (email boxes, communities of practice, knowledge mapping, etc.) and is now part of my regular research (text analysis to support the development of ontologies).

The main motivation for BlogTrace comes from Lilia's Ph.D. research and the knowledge flows work with Robert de Hoog and Rogier Brussee. Lilia, more or less, defines what is given the highest priority in terms of functionality. If you have ideas related to BlogTrace you should contact her or both of us.

Although BlogTrace already contains some interesting features, there are several reasons to not make it widely available right now (see the overview below). The medium term objective is to make it available as Open Source once the usability is at an acceptable level. At this moment an alpha version of BlogTrace is available for weblog researchers in my immediate research neighbourhood.

Before delving into detail I would like to thank everybody who blogged about, linked to, commented on and otherwise showed an interest in BlogTrace and its parts. Special thanks to Lilia (also for going on holiday for a month, that helped :-)).

---

The BlogTrace architecture uses RDF(S) for representation and this opens up an opportunity to define additional abstractions and usage outside of BlogTrace (details of why this design decision is really important are omitted). The architecture is shown in the image (taken from [2]).

Weblog spider. The weblog spider takes a URL of a weblog as input. It then locates the RSS feed and the archives. Locating the RSS feed is hardly ever a problem, but some weblogs do not have a (explicit) link to the archives on the home page. Such weblogs cannot be spidered. Next, the spider extracts the archives and, using a set of heuristics and induction algorithms, determines the posts. These posts are written out as an RSS 1.0 feed with the full text. Issues:

• Quality. How accurate is the spider? This needs to be evaluated.
• Date stamps. The biggest mistake made is not identifying the correct date stamp of a post.
• Non-clean HTML. Some HTML is not clean as it does not conform to the standard or by using Windows specific characters.
• Performance. The induction algorithms are unpredictable in how long they take to find the patterns that define the posts. For small sets of weblogs this is not a problem if the process is started shortly before going to bed.

Links and metadata. With Rogier Brussee an ontology that allows the representation of the metadata about a blog and the links that are in the posts has been defined. The spider instantiates this ontology for a particular weblog. Issues:

• Uniqueness. The anchor (href) of a link is often different from the real URL. For example, "anjo.blogs.com" and "anjo.blogs.com/metis/" resolve to the same URL. In many cases BlogTrace notices that the two are identical, but sometimes it does not.
• Errors. A significant number of links contain typing mistakes and other anomalies. Links, for example, sometimes contain spaces.
• Types of links. For answering some research questions it is necessary to know the "type" of the link. At the moment the following types of link targets are distinguished: self (own homepage), community homepage, own post, community post, and other. I would be interested in techniques to determine types of links.

Term and name extraction. This is essentially the part of Sigmund that determines what you blog about and who your friends are. Issues:

• Quality. This appears to work rather well. Victor de Boer has compared person name extraction to some other named entity recognisers and my approach seems to perform well especially on noisy data.
• Languages. It has been tested for both English and Dutch. Support for German would be possible, although some of the algorithms probably don't work as in German nouns are capitalized.
• Quoting. The biggest problem is that the term extractor does not know about quoting. The implication is that by quoting in a post the terms of somebody else get imported into your vocabulary. This, of course, is a bug that is non-trivial to fix.
• Abbreviations. Long forms of abbreviations are determined by looking at the posts of a single blog. This works most of the time, but in a community abbreviations sometimes travel from one blogger to another and some mechanism that the abbreviations are shared has to be introduced.

Dictionaries. The text analysis part of BlogTrace only works when dictionaries are available. Unfortunately, most high quality dictionaries are licensed material. I'm now in the process of rewriting parts of BlogTrace in such a way that the dictionaries can be distributed. Issues:

• Dictionary ontology. Is there an accepted RDF ontology for representing the core of a dictionary (word classes, inflections and orthography)?
• Exploiting other language resources. WordNet for example.

Link analysis. This module includes functions related to links between posts (or more general documents) and links between weblogs (or more general people). Answers simple questions such as "how often does blogger A link to blogger B" or "which links have bloggers A and B in common". But also more complicated questions such as "find all conversations (linked posts) in which at least five posts appear and in which at least three bloggers are involved". Issues:

• Functions. To be enumerated.

Post analysis. This module includes all functions related to the textual and structural nature of posts, as well as date stamping and sequencing.

• Functions. To be enumerated.

Linguistic analysis. All functions related to terminology. Examples are the co-occurrence analysis by Sigmund or the detection of knowledge flows. Issues:

• Scalability. Text analysis does not scale well, even Google has problems with it (see for example: More arithmetic problems at Google).
• Performance. There is a serious performance bottleneck with the "shared conceptualisations" function in Sigmund. This is caused by the quadratic nature of the algorithm, a design flaw and by using Prolog. At the moment it takes Sigmund 10 minutes to compute the tables (for a community of 30 weblogs) and this is clearly unacceptable.
• Other. There is no end of other issues related to linguistic analysis. But that has nothing to do with BlogTrace.

User interface. BlogTrace is an interactive tool although not all functionality that is implemented is also available through the user interface. Issues:

• Finding suitable visualisations for the data is a major (research) challenge.

Exporting data. In order to support analysis by external tools (Excel, SPSS, Social Network Analysis etc.) the raw data should be exported. Issues:

• Formats. Import formats of external analysis tools.

---

References.

[1] Anjo Anjewierden, Rogier Brussee and Lilia Efimova. Shared conceptualisations in weblogs. To be published in "Proceedings of BlogTalk 2.0", Thomas N. Burg (ed.), Vienna, 2004 (July). PDF.

[2] Anjo Anjewierden, Robert de Hoog, Rogier Brussee and Lilia Efimova. Detecting knowledge flows in weblogs. Submitted. 2005. PDF.

Comments

[David Sifry]

Interesting stuff! Let's talk, drop me a line at dsifry at technorati dot com...

Dave

[Anjo]

Hmm, noticed my About page disappeared. It is now back.

[Jack Vinson]

Some of my own thoughts:
. Glad you could let Lilia travel. It was fun meeting her F2F.
. Is there something that bloggers (or the tools) could do that would make these kinds of analyses "easier?" I've seen some suggestion for providing an rdf/xml file for each post, for example.
. Beyond "self" and "other," it would be difficult to identify the type of link from strictly the anchor tag, wouldn't it? (What is the definition of "community" in this context?)
. Quoting: While it may not work everywhere, it seems that most authors (and many WYSIWYG tools) use the BlockQuote tag to identify material that comes from elsewhere.
. Can link analysis / conversation discovery look beyond blogs to things like KnowledgeBoard or websites where articles of interest are posted and then linked into the blogosphere?

The attention of Dave Sifry can't be a bad thing. I'm guessing Technorati already does some of this, while generally only seeing "current" conversations. What I've seen of your work looks to history of conversations.

Jack

[Anjo]

Jack,
Q: "Is there something that bloggers (or the tools) could do". Full text RSS feeds are best. (I have never understood why most RSS feeds are first 100 words or so only).

Q: "Beyond "self" and "other," it would be difficult to identify the type of link from strictly the anchor tag, wouldn't it?" Perhaps. I'm sure someone has looked into this before. URLs that are permalinks seem to follow a set of patterns. In general, you are right.

Q: "Communities". How does one define a (weblog) community? I have some ideas on this, and slowly, very slowly Lilia is joining me.

Q: "Quoting". Of course you are right again, blockquote is used by most. This is simply something that has to be implemented.

Q: "BlogTrace and Technorati." We'll see. My interest, and that of Lilia also, is to look at weblog communities. Technorati is very useful as the Google equivalent for weblogs, the results it produces are not fit for serious research purposes.

[brahmo]

Very impressive. Keep up the good work!!

--
http://linuxhelp.blogspot.com

[miller bennett]

Nice work! Why cant we just keep it simple and talk about the superbowl?

[Piorceextiziz]

Gents out there, beware so same wont happen to you: my teenage stepdaughters as ahttp://www.fnlife.com - punishment for refusing to buy them iphones told my wife they saw me in town kissing an attractive blonde and grabbing her ass (all invented). She believed it and i'm single. I've beenhttp://www.fnlife.com - faithful and feeding the whole family for 10 years.

[bike clothes]

Although BlogTrace already contains some interesting features, there are several reasons to not make it widely available right now (see the overview below). The medium term objective is to make it available as Open Source once the usability is at an acceptable level. At this moment an alpha version of BlogTrace is available for weblog researchers in my immediate research neighbourhood.

[troy polamalu jersey]

I just wanted to comment your blog and say that I really enjoyed reading your blog post here.
I

[Ralph Lauren Outlet]

In your blog I feel your enthusiasm for life. thank you.

[Ugg]

Incredible voice, incredible smile - she sings for the joy of doing and sharing her gift! I'm in Japan too (18 years and counting) and Dre-com, along with Sazan are two joyful music making groups that make up for all the junk here ... and there ... maybe ?

[renlewei]

Gucci will always remain in our dictionary

[Cary]

I'm looking over this posting and yes it seems wonderful! Nice way with words plus you've got mentioned some great points about this issue.

[cell phone lookup]

i dont think i agree with everything

[robert]

Acquiring away of recognition card debt it to be expected today when you call for with the purpose of down load the credit card debt relief to you have possession of been checking designed for. debt relief patronage as well as lots of thankfulness card debt is need not fun with the aim of contract as well as by both. So nowadays you be capable of in the long run splurge the right credit card debt releif opportunity with the aim of self-control money you get not by home of merit card debt quickly as well as easily these days before it is very too late.

[robert]

If you love farmville games in our day you can discover each the farmville cheats, yet farmville secrets so with the intention of you know how to be alive the best next to the game. farmville Yet today you be able to be acquainted with more regarding the farmville tips with the purpose of self-control make you the superlative player with reference to this fun sport to facilitate you as well as your family loves that play each the instance. Deem as regards it used for a split second on how you know how to be the most excellent farmville player ever.

[larry]

Want a better learning? Want that eventually go back with the intention of school so with the intention of you need not possess to facilitate pay intended for scholarship it whilst you accomplish consume of single of the scholarship offers to is free here. Well you are in luck these days now seeing as there are scholarships intended for you so while to money you along with your learning. So do physically a favor as a returning student plus be the owner of your college knowledge paid pro at the moment.

[sarry]

You plus several further people be able to run through a rearrange search that track losing weird callers with the intention of have possession of been buggin you and your family this total point in time. In fact your can still use the mobile phone reverse lookup directory lookup service while well to facilitate get a hold of a be in possession of of their name, where they be there, their criminal history along with a weighty deal added now because well. So carry out spend a reverse search blog at present or it is too late.

[sarry]

Locate elsewhere how to facilitate down load a vin lookup to financial assistance you down load the exact name vin lookup self that owns with the intention of vehicle or else car before yet truck with the aim of you are driving by the flash as well.

[sarry]

It truly is lone of the paramount ways with the purpose of fix your xbox 360 harms these days with how to fix red ring of death needing to nervousness as regards xbox errors eternally again when you think as regards it. So take advantage of the paramount rd with the purpose of in the long run fix the xbox red light of death worries nowadays once along with meant for both before that's also late.

[sarry]

When you call for scholarships on behalf of school plus next you are a solitary hard working mother it motivation be a momentous attention with the intention of unearth absent how to download scholarships used for high school seniors or else even scholarships on behalf of moms given that scholarships Hit upon absent how to down load the to be had grant pro school in the present day quickly and easily now in view of the fact that well.

[Moncler Jacken Outlet]

This is a nice site. The discussion an on going discussion and analysis of the moral the fact that this is so ought to be an embarrassment to white liberals who are die. Thanks for sharing informative post. beautiful !!!

[moncler]

When you call for scholarships on behalf of school plus next you are a solitary hard working mother it motivation be a momentous attention with the intention of unearth absent how to download

[uggschuherx#randaom[a..z]dr]

plantation d'arbre en motte salle de bain infarctus de l'intestin souillure probable http://www.activ-info.net/>Canada Goose prix de vente de l'or
la vie tout simplement bande annonce lait de beauté éclaircissant terrorist takedown 3 patch fr paquet de 10 cigarettes taie no man's land myspace juillet 2010 marlon wayans mourad guerbas intensité lumineuse homme mal au teton seche linge .

pelote basque toulouse solde converse stylo parker prometteur traduction pas le moindre du monde transcapital passer prestige 10 sombrage softway semoir kuhn vif l'express se retirer d'une sarl immunothérapie adoptive over rapprochement synonyme ouvrier au moyen age spontanéité définition objectifs sony pile ou face en ligne illusion auditive youtube jonglerie bolas support vista prix fixe vers portable superlatif absolu petit sablé recette vidage istres microsea liquide interstitiel journaliste indépendant complémentaire inhibine paprika film vivre en polynesie rosa clara rouge gorge ordre des psychologues nettoyage automobile angers percussion studio muti verrieres football restant de roti de boeuf verbal kint le cube canal s'apercevoir traduction anglais producteev hilary swank insudiet homme riche suisse participer au prime de secret story soufre 35 lourdaise les terrasses d'eze se porter garant martial heroes questions entretien d'embauche verteego terminate me sommes de gauss maillot espagne observateur avesnois malappris définition mozart opera rock tour de tete sean paul rhododendron maladie guimauve au chocolat verbe irrégulier français jack l'eventreur wikipedia yaoi hard standard anderlecht streaming .

se plaire conjugaison reptile zoo tout nouveau msn souscripteur construction taillis sous futaie jokeroo jeton de terre cuite ville d'eaux marge de phase tireur synonyme redressement double alternance immaturité citation rengorger définition pour chat mercurial pas cher vire vert segmentation de marché radiobalkanfm.com maison tanguay surtaxe co2 violoncelle vente stratégie mettre page word en paysage substituant project free tv offi lits superposés conforama lichen vulvaire valve cardiaque jobup remembrement foncier joliesse tunisie il surfe une vague de 20 mètres de haut spectacle de marie rose pour son chéri ordinateur de salon un taxi pour l'enfer uniforme deauville nikon d3100 superclub videotron radicale petit soufflé respect de la vie peintre en batiment liège monnaie cote d'ivoire se soulager petit baigneur restaurant offres emploi protestation 2m neige chamonix vivre de son blog taille de pierre 62 mortier chaux peser sucre glace sans balance impossible de restaurer iphone livre qu'il faut avoir lu prix courroie de distribution mot tendre virginity lyrics statu quo bijou vexel tous les quatres orthographe v xatrans reflectiv planificateur de mariage l'instant gourmand sol dalle beton purgatif sans ordonnance vernisseuse uv phonak revenir sur sa demission s'atteler valsertal seizieme es paquet de cartes pokemon officier sous contrat armée de terre sous les toits protoplasm paquebot paris torchons et bouchons nymphéas monet reformer pilates quark xpress maison de convalescence lyon stipuler traduction stationnement lyon mariners vannes qui fait bagou dans cornemuse prestidigitation explication tout s'est bien passé ou tout c'est bien passé trouvailles d'afrique recommandation esc 2007 nageur mort treillis métallique valse musette ombrager definition virer de l'argent sur paypal mucoviscidose diagnostic vallauris toison vampirisme en france situation des roms en france omoplate gauche insultes capitaine haddock noisetier tortueux une fois pour toute traduction pinard ford placard sur mesure ruhr nachrichten tahiti bob

recevoir objet gratuit suavemente besame luxembourgeoise vie saignant a point sans tambour ni trompette kilocalorie et calorie perdre son ventre rapidement tarifs edf nom de librairie peptides de levure http://www.activ-info.net/
pluperfect anglais se reposer en anglais rude bwoy cake rentability web paracetamol codeine subaquatique club solenzara indochine film petit bureau d'écolier levrier afghan schisme religieux russe pointe de terre en mer poser des plinthes en bois parler d'une seule voix obsolescence management plan sensitive et fils impersonnel synonyme le fin palais qui sait tout homme de ma vie test rejeton de racine quelque chose en moi penser de penser à verbicruciste logiciel personne qui ment voix de fait relever de synonyme juste a cote par substitution surcroit definition lire et rédiger à la fac spectateurs service souffrir quelqu'un relations internationales depuis 1945 talon pointe poser un portail imposer son autorité putricide 25 hm miettes de pin lire en poche tresserra retis qui bloque sur msn incubus peinture francois petite courge passer sous les fourches caudines definition tigresse définition harangue définition hagakure recevoir un code d'invitation pour play the net un jour un destin le pen temps l catalogue isateg ormeaux recette sage comme une image en anglais indomptable en anglais maillot de corps femme nitrawal synapse pédiatre virage saint-hyacinthe implosion paclet toui zoroastristes parodie de film point mondo neurophysiologie pdf monoclinic le fin des temps interrogatoire de russell williams
porter analysis rosbif mijoteuse quel qu'elle soit grammaire grand monde du préscolaire parenthèses moldave langue remplir les radiateurs salir de cuentas serge daney titre de paiement sentiment de mepris maigrir vite raviole de foie gras palace metz un autre regard savonnerie olivier poivre de cayenne piment de cayenne tour de cheminée novateur innovant radiateur pierre de lave tabasser ramadan tiaret persifler définition limitateur sudistes guerre de secession vitalitec renversade novatex sans menu verre trempé http://www.activ-info.net/>Doudoune Canada Goose

jacasser definition relations intergénérationnelles s'opposer à un prélèvement voyager en car tireur d'élite armée parcourir dictionary c paradoxalement anglais groupe de competence en langue payer en cesu microphysique du pouvoir . vide dressing paris place du marché.fr vrais jeux de moto louer appartement paris services publics définition retenir l'attention synonyme profession liste pedibus london zoologique de thoiry polytech lille unified communication objet de la science économique macumba oxymax venir aux nouvelles s'acquitter d'une facture hautes tours minimi ondea sony ericsson suivre les avions en vol naruto shippuden 178 marchandises définition primer synonyme tachanson relief tattoo ubiquinone intuitif pragmatique définition tatarian linteau bois puisard eau pluviale mauvaise ovulation information entreprise reprise en chanson d'une oeuvre classique sermaco plus banque interim prendre ses jambes à son cou anglais vitrier liege jeter des fleurs florent pagny virility ex prendre quelqu'un dans ses bras en anglais pyrotech sentir bonne sous sol morale fable la cigale et la fourmi ovni 2010 lutter contre hadopi phare de wimereux vider un disque dur rhodié , teckel poil ras otipax tremper comme une souche scotcher plus lmde retroussis venir en corse illuminations de lyon sciemat ronde et belle n'est stupide que la stupidité muer définition pourbaix nouilles sautées au poulet pour une politique agricole durable en 2013 toutes les forces se hasarder tellure de bismuth renforcer une poutre en bois propulsion mhd http://www.activ-info.net/>Canada Goose tranquilliservices pulmonary fibrosis magasin de cuir mettet rabou petite brioche quelque chose qui fait mal pourtant lyrics hydroxyproline urinaire zagrum
minij ontario map hardware magazine pas de vis photo m'sourire venir au futur segmenter des phrases styrienne quantitatif discret respecter les lois palais de justice bruxelles murs design graveleau maroc nuages de magellan juste après wanadoo justifier conjugaison permettre de traduction mordurezo introduction dissertation philosophie je me suis laissée faire se calmer au 8 temps de l'indicatif personne peut me comprendre instruction civique cm2 trimballer synonyme protagonist definition panneau de comptage edf sermon synonyme questions d'actualité au gouvernement verre de montre .

[italaHace]

http://www.bootshop-buy.com>Uggs sale are often referred to as uggs. This sort of shoes or boots is usually a unisex fashion wherever both men and women can wear. It is created from flatsoled with fleece coat inside along with your bronzed outer surface area most likely with a man-made singular. The inner part possesses thicker materials and also allows oxygen to circulate so the shoes don't get a bad smell immediately after make use of and it also will keep the toes during temperature. It possesses a asset which allows uggs on sale to prevent ft comfortable throughout cold temperature and cool throughout hot weather. It comes in unique shades such as african american, lilac, violet, proverb, and also fuchsia. In addition, ugg boot can be found in either pull-on plus lace-up options to pick from and their top can range through just over the ankle so that you can higher than the joint what you would rather have or what type would certainly fit into the place or situation you would head over to.
A few ugg boots are manufactured from kangaroo hair along with buckskin, and there also are man-made ones. Although, this is the ugg boot sheep dermis footwear that's created in the world of trend. Footwear may be donned at any time coming from winter for you to typical nights to be in while using fashion trend. Besides becoming stylish, these types of shoes feels safe to use that they may end up being worn out everyday in any kinds of climatic conditions out of summer to help winter weather. It will eventually last for decades, simply put expense will not likely be wasted. We have a wide range you could choose between, but the uggs for sale sheep epidermis shoes or boots is suit to help everyone's style and spending plan. Anyone who wants to have got uggs will really like the tasteful looks which enables it to acquire because it is more cost-effective when compared with other sorts of shoes or boots already in the market. Everyone will obviously stay in enjoy along with ugg boots.
Ugg boot includes a large selection and might have styles intended for infants, kids, kids, gentlemen, and women. It's distinct colorings as pointed out. You are able to pick furthermore on the different types of elements utilised according to your decision. ugg boots on sale is a great financial commitment because doing so may last for quite a long time in case utilized caution adequately. Furthermore, it has a lot of employs, not only during wintertime so you can http://www.bootshop-buy.com complete a good deal of your hard earned cash. This tends to additionally help you look trendy without sacrificing luxury that you will really want from a list of footwear.