Although it won't do too much, Empirica works straight out of the box. The first step to customizing your Empirica app is to launch a test game so you can see your edits in action.
First, follow the Quick Start guide to launch your app
server. Watch the terminal output to see your automatically generated admin
login and password.
Then, visit your admin panel http://localhost:3000/admin.
Log in using the password generated in step 1.
Click "configuration" in the upper right corner to access app settings. This
is where you'll define all your experimental conditions.
Click factors
in the menu bar at the top. This is where you set the most
basic parameters for your experiment. The only default parameter is
playerCount. Add a new factor by clicking the +
button. Set the name to
One
and the value to 1
, then click Create Factor Value
.
Click treatments
. This is where you combine factors to create an
experimental condition. Create a new treatment using the factor you just
created and name whatever you want.
Click Lobby Configurations
and create a new lobby configuration using the
default settings.
Click monitoring
in the upper right corner to toggle your admin panel from
configuration to monitoring. This is where you'll interact with all the
Empirica features that have to do with actually running experiments.
Click New Batch
in the batches panel, select the treatment you just
created, and click Create Batch
.
Press the play button (triangle) to start the batch.
Navigate to http://localhost:3000/.
Follow through the consent, identification (this would be, e.g., an MTurk
ID), instruction pages, and attention check (you have to answer correctly).
You're running an experiment! This is the default app, just a slider with 10
rounds of input. We're going to edit that.
The root file for displaying your subject interface is located at /client/game/Round.jsx
.
By default, this is divided into two main components, Task
and SocialExposure
. The Task
itself is composed of TaskStimulus
which contains the stimulus (e.g., a survey question — or in this example, an estimation task) and TaskResponse
which contains the input for users to response to the stimulus.
Edit /client/game/TaskStimulus.jsx
to the following. Although Empirica is built on top of a sophisticated Meteor framework, all we have to interact with directly is standard Javascript with a ReactJS framework.
Notice that your props passed into the React component include round
, stage
, and player
. These are your interface with Empirica, and allow you to both read and set data for the state of the experiment.
All we're doing here is adding an image and question to set up the display. Later, we'll make the image path and question text configurable by reading in values from the round
prop.
/client/game/TaskStimulus.jsx
:
import React from "react";export default class TaskStimulus extends React.Component {render() {const { round, stage, player } = this.props;const imagePath = "/experiment/images/candies.jpg";const questionText = "How many candies are in the jar?";return (<div className="task-stimulus"><div className="task-image"><img src={imagePath} height={"300px"} /></div><div className="task-question"><b>Please answer the following question:</b><br />How many candies are in the jar?</div></div>);}}
All we're going to do in this section is change the input type from a slider to a numeric input box. Because we're asking people to estimate the number of candies in a jar, we'll set the minimum value to zero. We recommend adding additional form verification methods beyond the default to ensure a smooth user experience. Note that if you do choose to use a slider input, we strongly recommend using the Empirica slider, which is made with experimenters in mind — no default values that might create anchoring effects.
The complete final code for this file is available on GitHub. Just update your renderSlider
according to the following code and be sure to update the call in the main render
method.
Notice that this method uses the player
data to control the form input. This player data is created with player.round.set()
in the handleChange
method which is called every time the input is updated, and will save this data to the database on the server, automagically.
renderInput() {const { player } = this.props;const value = player.round.get("value");return (<inputtype={"number"}min={1}onChange={this.handleChange}value={value}required/>);}
Because we changed the input type, we also need to update the handleChange
method because input events return a different type of data than slider events:
handleChange = event => {const value = Number(event.currentTarget.value);const { player } = this.props;player.round.set("value", value);};
The next step is to allow the experimenter to modify the stimulus without having to update the experiment code directly. We'll accomplish this by adding a JS object to a constants.js
file, that will contain all the stimulus information, and then use that in the experiment via the Empirica.gameInit
method.
This all happens server-side, so you need to create this file in your /server/
directory instead of your /client/
directory. Create a file called /server/constants.js
, and add the following code to it.
export const taskData = {candies: {path: "/experiment/images/candies.jpg",questionText:"The jar in this image contains nothing but standard M&M's. How many M&M's are in the jar?",correctAnswer: 797},survey: {questionText:"A 2014 survey asked Americans whether science and technology make our lives better (easier, healthier, more comfortable). What percentage of respondents agreed that science and technology are making our lives better?",correctAnswer: 80.5}};
We're not actually going to use the correctAnwser
value in this demo, but if you include all relevant data in your experiment, then it life is a little easier when you download your data for analysis.
Next, we need to use this data in the Empirica.gameInit
callback in /server/main.js
.
First, we import the constants.js
data with import { taskData } from "./constants";
added to the head of main.js
.
We're also going to update this method to randomly assign each player a set of "neighbors" to follow. We'll use this information when we update SocialExposure.jsx
.
Note also that these examples make use of underscore.js a library of convenient javascript tools.
/server/main.js
import Empirica from "meteor/empirica:core";import "./callbacks.js";import "./bots.js";import { taskData } from "./constants";// gameInit is where the structure of a game is defined.// Just before every game starts, once all the players needed are ready, this// function is called with the treatment and the list of players.// You must then add rounds and stages to the game, depending on the treatment// and the players. You can also get/set initial values on your game, players,// rounds and stages (with get/set methods), that will be able to use later in// the game.Empirica.gameInit(game => {// Establish node listconst nodes = [];for (let i = 0; i <= game.players.length; i++) {nodes.push(i);}game.players.forEach((player, i) => {player.set("avatar", `/avatars/jdenticon/${player._id}`);player.set("score", 0);// Give each player a nodeIdplayer.set("nodeId", i);// Assign each node as a neighbor with probability 0.5const networkNeighbors = _.filter(nodes, () => {return _.random(1) === 1;});player.set("neighbors", networkNeighbors);});_.each(taskData, (task, taskName) => {const round = game.addRound({data: {taskName: taskName,questionText: task.questionText,imagePath: task.pathath,correctAnswer: task.correctAnswer}});round.addStage({name: "response",displayName: "Response",durationInSeconds: 120});});});
Finally, we're ready to incorporate our new configurable task data into Task.jsx
. We'll do this in TaskStimulus.jsx
It's easy! All we do is change the value of the constants to pull dynamically with round.get()
instead of setting them manually:
const imagePath = round.get("imagePath");const questionText = round.get("questionText");
We also add logic so that we only display an image of a path is given:
<div className="task-image">{imagePath == undefined ? "" : <img src={imagePath} height={"300px"} />}</div>
Social information is included in the default Empirica template via SocialExposure.jsx
. However, nothing will show if we don't have any other players!
The first thing to do is create a multi-player game by adding a new factor value to playerCount (following the same procedure as above) with more than one player, then creating a new treatment and a new batch. You can then launch the Empirica app multiple times in the same web browser by clicking "New Player" in the header tab. The app will start once the required number of players has entered the lobby.
In this example we're going to modify the default behavior so that (a) the social information only shows after subjects have entered their initial respnose, and (b) social information shows only for a subject's network neighbors.
Within each round
are multiple stages
. We can add more stages by returning to the Empirica.gameInit
method and adding the following:
round.addStage({name: "social",displayName: "Social Information",durationInSeconds: 120});
We can access the stage information (including the name) within the app, so we're going to modify Round.jsx
to display the SocialExposure
component only when stage.name === "social"
:
{stage.name === "social" ? (<SocialExposure stage={stage} player={player} game={game} />) : ("");}
Finally, we'll modify SocialExposure.jsx
to only show information for players listed in player.get("neighbors")
by replacing the declaration for const otherPlayers
with
const otherPlayers = _.filter(game.players,p => p._id != player._id && player.get("neighbors").includes(p.get("nodeId")));
We're also going to remove the slider element, since we are not using that in this example, and just print the number:
renderSocialInteraction(otherPlayer) {const value = otherPlayer.round.get("value");return (<div className="alter" key={otherPlayer._id}><img src={otherPlayer.get("avatar")} className="profile-avatar" />Guess: {value}</div>);}
This simple feature shows just how powerful Empirica can be. Just by virtue of storing user information with player.set()
and displaying that information in the SocialExposure
component, the subject interface is automatically updated. This is due to the way ReactJS works with Meteor: any time a piece of information passed as one of the props is updated, then the display re-renders to show the new information.
One of the key features of Empirica is the ability to think like a scientist and control your app with experimental conditions. To show how this works we'll add a factor, allowing us to modify the stage length without changing the code.
First, we add the new factor through the admin panel by navigating to the factors interface and clicking "New Factor". We will create a stageLength
factor of type Integer
.
Then, add a new value (such as 60
, for 60 seconds), make a new treatment, and create a new batch. You can archive your earlier treatment(s) so they don't clutter up the interface. You'll always be able to unarchive these if necessary.
We use this treatment information in the Empirica.gameInit
callback by simply changing the definition in /server/main.js
to be durationInSeconds: game.treatment.stageLength
for both stages.
Empirica provides a set of methods that will run at the start and end of each round and stage. These can be found in /server/callbacks.js
.
By default, the score is equal to the total sum of responses, but this is not very informative. We'll modify this to show percentage of error subtracted from 1:
// onRoundEnd is triggered after each round.// It receives the same options as onGameEnd, and the round that just ended.Empirica.onRoundEnd((game, round) => {game.players.forEach(player => {let value = player.round.get("value") || 0;const prevScore = player.get("score") || 0;let newScore = 1 - value / round.get("correctAnswer");if (newScore < 0) newScore = 0;player.set("score", prevScore + newScore);});});
In this tutorial, we have created a simple that prompts subjects to complete numeric estimation tasks and allows them to revise their answers while observing the responses of other subjects.
Right now, this app ONLY works for social conditions ; if you don't have any neighbors, things will look a bit weird because the stage name is called "Social Information" and there are no instructions.
Here are some examples of how we can use Empirica features to expand the functionality of this experiment:
Add logic to use different stage names for social and non-social conditions
Add configurable task instructions to the constants.js that say things like
"Please answer the question, you will have a chance to revise your answer" and
"Please revise your answer"
Add a factor "questionSet" that takes a comma-separated list of question
identifiers, and uses only those questions. This allows different treatments
to use different conditions, based on the same constants.js file.
If you want to use more complex network structures, you could store those in a
json format and import them in main.js
There are also several very basic features from the Intro and Outro we haven't touched yet that you'll need to update:
The Consent form
The Instructions pages
The attention check
The Exit Survey and Thank You page