News, research and discussion on virtual goods, currencies and economies globally.

Virtual field experiments in WoW

Photo by Flickr user drurydramaEconomic experiments in virtual worlds (or, more generally, virtual economies) have been proposed a number of times and also conducted to some extent. A recent addition to the conducting category is “Reciprocity and status in a virtual field experiment” by Andreas Nicklisch and Tobias Salz. In their study, the authors conducted a field study in World of Warcraft, seeking to investigate the role of reciprocity, and the effect of status on reciprocity, in employment relations. Reciprocity here means the observed tendency to react kindly to kind actions, even though such reactions are not enforced. Unlike previous economic experiments conducted in virtual worlds, the authors did not create an artificial setup for the experiment. Below, I summarize and briefly discuss the study and its findings.

In practice, the experiment went as follows: The experimenter (principal) asked the participants (agents) to perform a real-effort task in exchange of a reward. The task in question was fishing in a certain lake for 30 minutes, and the reward was either 4 or 12 gold pieces. The experimenter used two kinds of characters when proposing the task: either a low-level or a high-level character. So two variables were controlled: the reward of the task and the social status (and expected wealth) of the principal. The agents were unaware of participating in an experiment, paid in advance, and asked to deliver the proceedings of the fishing via mail. The agent, then, could choose to accept the reward and not perform the task.

In this case, according to the authors, the two main upsides of using a virtual world as the experimentation field are: first, the possibility to analyze the principal’s status (as measured by the level) in a meaningful social situation and second, the possibility to examine reciprocation rates controlling the effect of skills of the agent. Other features of the virtual world can also be considered beneficial. The main thing that comes to mind is that, true to the field experiment mindset, it is possible to recruit agents to perform tasks in manner that is natural to the environment they perform actions in. I’m not sure if asking a fellow player to perform a task for a reward is a typical thing to do in WoW, but I’m sure it is not the type of action that would never take place.

Control treatments, i.e. the varying of principal’s level and the wages, were carried out on different servers in order to reach participants not aware of other treatment conditions. The use of separate servers may in some cases lead to problematic results, as discussed in a recent Terra Nova thread. This is probably not a problem in this context since, as I see it, the authors are not trying to reach a separate instance of the world. Instead, they are trying to make sure the participants do not participate in multiple treatments. It is unlikely that the experimenter’s actions take place on such a large scale that information on them would spread to other servers. Hypothetically, though, it is possible that the participants are familiar to other treatment variations, one way or another.

What of the results then? Mainly, the authors observed that the behavior of agents is affected by reciprocity in virtual worlds as well. Wage offers are reciprocated with effort, even though such behavior is not enforced. Increasing the wage increases the probability that effort is provided, as well as the amount of effort provided, even if the level of the agent’s skills is controlled. The status of the principal has an effect: high-status principals are less likely to be provided with effort, but the level of effort is not affected by the status. It would seem that the agents evaluate the exchange offer with respect to the status, and thus the expected wealth, of the principal.

The issue of generalization is often brought up when virtual world experimentation is discussed. Reciprocity is a well-known effect, and the study confirms that it plays a role when exchange is considered in a virtual world. So, it confirms behavior that was already known to exist in other circumstances: in this sense, generalization is not necessary. Regarding the effect the principal’s status has on reciprocity, the study seems to confirm some theories on the issue. Whether they generalize on a larger scale is not discussed in the paper.

I think the most important finding is the additional confirmation that the users of a virtual world behave in a manner that is similar to behavior elsewhere. Also, the authors show how sensible field experiments can be run in a virtual world. We’ll see if this type of experimentation becomes increasingly popular.

Contribution, methodological vs. substantive

Nice paper, clear research problem and easy to follow. But:

I think the most important finding is the additional confirmation that the users of a virtual world behave in a manner that is similar to behavior elsewhere.

I wonder how much new information such findings really contribute. Was there any reason to expect results to the contrary? Even if you did get results suggesting otherwise, which do you think would be a more plausible explanation — that there is something wrong with the experiment design (e.g. some variable in the setting that you failed to account for) or that people somehow think differently in this particular context compared to all other contexts?

I was similarly not terribly impressed with Arden’s results — there is similarity, no?

I think we are both looking forward to seeing the first paper where experimentation in a virtual economy is used to produce results that make a significant contribution to general economic or sociological research.

Value of analogues

I am ambivalent in my agreement with Vili. On the one hand, simple, low-level experiments that demonstrate 1:1 analogies between behavior in game and behavior IRL are necessary to establish for the broader economic and/or sociological academy that generally accepted rules of behavior are maintained.

On the other hand, it may be time for VEconomists to really take the plunge and see whether there aren’t some more complex models – which assume notions like the laws of demand and supply – and see whether they ‘work’. Studies and models of auctions, industrial organization, and monetary transmission mechanisms are just a few that would be worthwhile to examine in the laboratory of an EVE or a WOW. All that seems to be missing is the appropriate set of data (which, btw, could probably be acquired w/o explicit cooperation from the Blizzards of the world).