My (intended) research on reputation

The email below tries to explain (also to myself) what the hell I am doing.. I sent it to a company called Fog Creek software, creators of the StackExchange platform, which is used widely by online communities to manage questions and answers. I will soon add a graphical representation of what I mean, but until then, this is it.

…

Dear StackOverflow,

I am currently doing my PhD research on reputation in online communities to support online learning, self-organization, and reputation building. I sincerely like your reputation system, and find it a good benchmark to work from. It includes many aspects of the Yahoo design patterns, and it’s quite user-friendly and customizable.
I am involved in several projects where I could test new strands of your reputation system, and this could be interesting for you. I will shortly explain my approach, and if you’re interested, we could connect and maybe do business/research together.
In short, it’s about calculating someone’s expertise on a subject by looking at his/her contributions. Rather than looking at the contributions itself, I would like to look at the value of those contributions. At StackExchange you look at “Votes up/down”. Now what I am interested in, is not the “Votes”, but the determination of what these votes mean in terms of expertise or competencies. Consider the following difference:

• I am a member of the online community MathOverflow, and earned 178 points.
• I am a member of the online community MathOverflow, and earned 
  • 71 points for euler-characteristics (Top 5%)
  • 33 points for fourier-analysis (Top 10%)
  • 21 points for polynomials (Top 25%)
  • and some more points assigned to various tags

Now, the second reputation is much more specific, which can be useful when someone needs a person for a specific task, or for providing recommendations based on reputation. But it also contains some important challenges: 

• Who makes up the tags?
• How are tags interrelated (if you’re an expert in algebra, are you also an expert in communitative theory?)
• What is the value of your reputation outside this community? How to convey trust to outsiders? What about the community reputation?

Some other issues I have in mind that are interesting to research:

• Voting (rating) is one way to indicate quality of a contribution, but there are more (see the example of citations below). These include viewing (popularity), embedding, tagging, favoriting, endorsements, #return visits, etc.
• There are not only different ways to indicate quality, they probably mean something different as well. 
  • Varying degrees of importance. In real life, different authorities have different reputations for different subjects. So how does this influence the rating (or quality indication)? What does it mean when a professor in mathematics endorses an answer, or when a student does this. Reflecting on StackExchange, it could be interesting to relate badges and the importance of a vote.
  • Different meanings: take the voting system at Slashdot > Insightful, Informative, etc. This could also be linked back to badges, and people build up different reputations on the same community.
• Tagging (keywords) is one way to indicate the subject and context of the contribution, but there are more ways to infer information about a contribution’s content and context. This really depends on the location (group/community) of the contribution, the contributor, and the people who visit/make use of the contribution.

Hence, my research is about how people interact with contributions and what this means for the contributor’s reputation. Each interaction contains a value statement and possibly some contextual factors. These should be combined to come to a valid statement of someone’s value in a certain context. Reputation points without context is like a car without fuel, quite useless.
My goal is to create a model for setting up a reputation system that improves current ‘points’ & ‘badges’ based reputation systems in online learning communities. These approaches are useful, but I am convinced that adding an extra dimension to it is quite useful, especially in large corporations and open knowledge ecosystems, such as online communities. It can also be useful in educational institutes that support self-regulated learning and peer-support, and would like to endorse people who are active and helpful online.
This model will contain steps for 

• determining types of contributions (i.e. Questions & Answers, Articles, Comments, Aggregations).. 
  • StackExchange; I also like the concept of wiki/community contribution
• determining badges/authority types/roles (i.e. knowledgeable, popular, instigator, writer, active, aggregator)
  • the question here is: ‘How can you be valuable in this community? What are the roles?’
• defining and managing the taxonomy (i.e. tags, groups, categories)

• determining quality indicators (i.e. ‘view’, ‘vote’ or ‘endorsement’), and sub-qualification (‘vote:insightful’ or ‘endorse:on-topic’)

And steps for determining the relation between the different elements of the model

• relating different levels of taxonomy (tags and categories)
  • inference rules that describe how keywords are assigned to a contribution, and indirectly, to a person’s reputation
• relating ‘contribution type’ and ‘badge/authority type/role’ (i.e. Question ~ Instigator, Answer ~ Knowledgeable)
• the meaning and relative importance of a quality indicator
  • meaning: ‘# views’ ~ ‘Popularity’, ‘Endorsement’ ~ Knowledgeable, ‘# votes’ ~ Knowledgeable, ‘Best Answer’ ~ Knowledgeable
  • relative importance: 1 endorsement = 10 votes
• the relation between authority type, quality indicator, and contribution type. 
  • For example; only a ‘pro’ (authority type) can ‘endorse’ (quality indicator) ‘an answer’ (contribution type)

Etcetera: there are some other factors to play with, and that deal with the ‘value’ question.
I do not intend to make a system so complex that it will not be adopted, but try to find a way to improve current reputation systems by providing a model that includes both value statements & context parameters with each interaction, as has been explained in the rather simple example of a researcher publishing a paper.
Simple example of writing a paper and reputation in a scientific community:

1. I publish paper P1.
2. Person A cites my paper in his article A1.
3. The citation is a value statement.
4. The article A1 contains keywords k<1..n>
5. Paper P1 increases its value for keywords k<1..n>, 
   • in other words; the ‘value’ of paper P1 is represented in the keywords k<1..n> of article A1 ==> if more articles cite paper P1, their respective keywords are added as well and aggregated in the user’s reputation profile.

This is a very simple example for only one type of interaction: citing a resource. Clearly, there are much more interactions that mean something. It’s quite interesting to look at these interactions and see what they mean and how they can be assessed and represented automatically (and aggregated into a reputation profile). It is also interesting to look at how the reputation of Person A influences the relative importance of his value statement. 
Enough for now.. will focus on making a graphical representation of this quite philosophical concept soon, to make it all a bit clearer.
I hope the right person at your organization has found the time to read this email, and am looking forward to a reply.
Best regards,
Thieme Hennis
Thieme HennisFaculty of Systems Engineering, Policy Analysis and ManagementDelft University of Technology+31 15 278 73 71 (work)+31 6 51855 22 0 (mobile)IM/Skype username: thiemehennis