Link: http://shop.oreilly.com/product/9780596159801.do
Authors: Randall Farmer, Bryce Glass
Summary
- Reputation systems compensate for an individual's scarcer resource - attention, by substituting a community's greatest asset - collective energy.
- Reputation is highly contextual, and needs to be designed for specific use cases
- Physics analogy: atoms (reputation statements) are made up of particles (sources, claims, and targets), which are bound with forces (messages and processes) to make (reputation models) which can be mixed in solutions (reputation systems) to constitute the universe.
- Reputation containers can hold multiple aspects in one review
- Reputation points should not be spendable
- Otherwise you lose track of people's worth. Your system won't be able to tell the difference between truly valuable contributors and those who are just good hoarders and never spend the points.
- Better to link systems but allow them to remain independent and orthogonal
- Quantity and quality of contributions can be combined into robust Karma models
- Karma models must be reversible to prevent abuse and gaming
- Folksonomy is tags applied to a photo
- Content with TOO high attention velocity should be flagged for review of explicit content probability
- Keep reputation metrics more or less obscured to prevent gaming. Expect peeking
- Integrate decay and delay to obscure algorithm tracking. Integrate randomness where needed, e.g. in top image display
- Predictably Irrational, Dan Ariely Behavioural Economics
- Incentives can be social norms or market norms, which DO NOT MIX
- Social norms make us feel warm and fuzzy, are done because we were asked (altruistic, egocentric motivations)
- Marketing norms are cold and mediated by wages, prices and cash (commercial, egocentric motivations)
- Market-like reputation systems can create successful with incentives for egocentric motivations, blurring the two categories in new social virtual environments - online reputation based incentive systems - where market and social norms can coexist.
- Free is disproportionately better than cheap: Predictably Irrational, Ariely
- egocentric incentives and Karma do provide very powerful motivations, but they are almost antithetical to altruistic ones. The egocentric incentive many systems have been over designed, leaving two communities consisting almost exclusively of experts.
- Mastery incentives include achievements (e.g. goal accomplishments), ranks or levels with caps (e.g. Ensign, corporal, general), and performance scores such as percentage accuracy.
- Altruistic motivations:
- Tit for tat or pay it forward: Done because someone else did it for me first
- Friendship incentives: done because I care about others who will consume this
- Know-it-all / crusader / opinionated incentives: done because I know something everyone else needs to know
- Commercial motivations:
- Direct revenue incentive
- Branding incentive
- Egocentric motivation
- Fulfillment incentive: desire to complete a task
- Recognition incentive
- The quest for mastery
- The competitive spectrum of your community helps determine appropriate reputation system:
- {Elevator pitch: A tool to empower communities in the creation of local reputation, governance and economic systems, so they can raise money, make decisions and take action on what matters to them most}
- Good inputs
- Reward firsts, but not repetition
- Rate the thing, not the person
- Points-as-currency
- Push motivations for using application from altruism towards commercial
- Recommend against reputation points being monetizable
- Read into the effects of RMT (real money trading)
- New users give you all sorts of information for inferring reputation: IP, cookies, related reputation in other systems
- Negative karma just leads to new account creation: Karma bankruptcy
- Avoid negative karma, unless as a separate score, or moderator/internal only.
- E.g. User with 75 pts who recently accumulated lots of negative scores, or newer user with 69 pts who has never done the system wrong - how can you tell the diff?
- Karma should be a complex Reputation and should be shown rarely
- Built of multiple indirect inputs to paint a broad picture within specific context
- Rating users directly should be avoided, evaluating each other is awkward
- 'revealed preference' is what people do, more reliable (vs what they say)
- Whatever you do measure will be taken WAY too seriously
- Reputation is an extremely useful LENS that you can hold up to the content of your application. Reveals quality, obscures noise, powered by the opinions of others on your platform.
- Broken windows and online behavior:
- Disorder and crime often linked at the community level
- One unrepaired window signals no one cares, do breaking more windows costs nothing
- Content moderation and the allowance of negative content on a forum permits more negative content.
- Don't presume user behavior/preferences in terms of ranked information, let them choose the quality they'd like to see (Slashdot)
- Keeping hidden reputation on users (e.g. abuse reporter reputation) can help to give me information on case of disputes
- How many past reports were legitimate? What volume of reports is filed?
- Beware feedback loops
- Getting a job with a bad credit score is hard
- Improving your credit score without a job is hard
- Program in a way to get out: conditional credit score pass to safely exit loop, e.g.
- Hawthorne effect: spikes in behavior in response to being studied
- Ensure you leave time for stabilization after initial reputation system installation to avoid
- We tend to over extrapolate human behaviors from data. For every data point we measure, there are at least two more we are missing. Do qualitative and quantitative research. Ask them why they do what they do.
- Yahoo! Answers looking to reduce moderation overhead on forums.
- Shift content removal burden onto users, with exception handling done by customer care
- When customer reported abusive content, it was "hidden", and an email was sent to author with an explanation and offer to appeal.
- Trolls using fake emails can't appeal.
- {Ryan only putting a star rating on a place of he feels the current rating is inaccurate.}