wsl --install

Now you may be prompted to create a username and password. Make sure to write down your login credentials for the future!

Note: Copying and pasting may be disabled in the default settings of your WLS terminal. To remedy this, right-click the title bar and select 'Properties --> Options --> Use Ctrl+Shift+C/V as Copy/Paste.'

Install Empirica in your WSL terminal with the following command. You may be prompted to use the WSL password you just created.

curl https://install.empirica.dev | sh

🎉 Congrats, you've now installed Empirica! 🎉

Step 1. Install WSL 2 and Ubuntu 20.04

1. Follow either the written instructions or video tutorial below to install WSL2.

2. During this process, select "Ubuntu 20.04" or higher as your Linux version.

3. See our overview below for a summary with screenshots.

Our Summary of these Instructions:

1. Enable the Windows Subsystem for Linux

2. Ensure you meet the requirements for WSL2

3. Enable the Virtual Machine feature in Windows (see screenshot below)

4. Download the WSL2 update package

5. Set WSL2 as your default Linux system

6. Open Microsoft store, search for Ubuntu 20.04, and install it

7. Launch the terminal and follow the prompt to create a new username/password (see screenshot below).

Screenshot: Enabling Virtual Machine feature in Windows:

Screenshot: Running Ubuntu 20.04 in Windows for the first time:

Step 2. Install NVM

Note: Copy these commands and then paste into WSL2 by right-clicking on the terminal.

Enter the following commands into your Ubuntu terminal:

curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash  # Download and install
source ~/.profile  # Reload system environment
nvm install lts/erbium

Step 3. Editing Files in WSL2

In order to access your WSL2 file directory, run the following command in WSL2:

explorer.exe .

This will allow you to browse WSL2 as if it were another computer on a local network.

Using code editors

Empirica is an open-source JavaScript framework for running multiplayer interactive experiments and games in the browser. Empirica makes it easy to develop and iterate on sophisticated designs in a statistically sound manner. It offers a unique combination of power, flexibility, and speed. Empirica is a powerful framework that facilitates the management of your games and experiments and is useful even for single-player research.

Empirica uses industry-standard open-source technologies (Go, GraphQL, Typescript, Javascript, React) Empirica provides a helpful structure to how your Games, Players, Rounds, and Stages interact. Furthermore, Empirica gives you an Admin Panel that makes organizing and running your Games for data collection easy and intuitive.

Empirica is the one method that provides enough flexibility to build any experiment while still being accessible and helpful to the researcher.

Quick start

More details can be found on Quick start. If you just want to get on with it:

curl -fsS https://install.empirica.dev | sh
empirica create myexperiment
cd myexperiment
empirica

I want to use Empirica. Where do I start?

What skills do I need to use Empirica?

Empirica helps you build online apps for online research. As with many online tools, basic notions of HTML, CSS, and JavaScript will be helpful. Knowing a bit about React.js can also be beneficial, although you do not need to be an expert. There are many tutorials online for all the technologies mentioned here.

Why should I use Empirica instead of just Node.js and React.js directly?

While Node.js and React.js are powerful tools, and you could build sophisticated experiments with them, Empirica makes building and managing your experiments much easier.

Notably, Empirica provides an Admin Panel that allows you to manage the conditions of your experiments (Treatments and Factors) and which Games are running, collecting data, or waiting for players.

Empirica does the heavy lifting in managing games, player connections, game life cycles, and more.

What's new with Empirica v2?

Empirica v2 is a major departure from v1. It goes well beyond a few API changes since we have completely rebuilt Empirica from the ground up with new technologies. Here are a few key elements of the change:

• New Features! We have introduced a bunch of new capabilities. We are still working on the documentation, but here are a couple of examples of what's now possible:

  • Custom assignments: we have opened up the entire assignment flow. You can hook in anywhere and do all kinds of fun new assignment logic.

  • (Re-)Assignment to ongoing games: players can join ongoing games. They can also be reassigned between games.

• Developer UX Loading is fast; changes refresh instantly; we use standard npm libraries; the empirica command helps through the development process, etc. We have built a strong foundation and have only started the developer experience improvements.

• New Technologies We are still using open-source technologies, but we no longer rely on Meteor.js. We have built a custom framework with a GraphQL server in Go. The backend is still on Node.js, though we no longer use Fibers, a deprecated Node.js technology used in Meteor. And we are up to date with React v18+ on the frontend.

• Open Architecture We have purposefully centered our design on a Graphql API. We are not locked into one language or framework. We currently support a Node.js backend and a React front. But someone could develop a python backend to integrate with their existing libraries and a Flutter frontend to build a mobile app.

1. Run the installation script:

That's it, you're done! Proceed to create your experiment:

A note about Windows Setup

Empirica is built on industry-standard open-source web technologies which run best in Unix-like operating systems such as Linux and macOS. ****

The installation script requires curl to be installed, and some Linux distributions ship without it. You can install curl by first running the following lines in the terminal:

Ubuntu / Debian / Debian based distributions

sudo apt update && sudo apt upgrade
sudo apt install curl
curl --version

Red-hat / Fedora / CentOS / Other Red-hat based distributions

yum install curl

After installing curl, proceed with the setup directions as normal in "Setup".

Running the app locally

Run the app on your local machine with one command:

empirica

This can take a few minutes.

Loading the factors and treatments

Treatments and factors can be entered via the admin panel.

Testing the app

To run a game, create a new batch with the games of treatments you want to use and click start on the batch.

• Click on the Reset current session button in the bottom right menu to reset this player, and create a new game for this player to join.

• Click on the New Player button in the bottom right menu to open a new tab with a different player (you will see tab with an id).

The app will automatically ("hot") reload when save changes to the code.

Simply run the following command, where my-experiment is your experiment name (no spaces or uppercase letters):

empirica create my-experiment

Once the installation is done, open your project folder:

cd my-experiment

You can now read more about the running your experiment, the structure of an Empirica experiment, and you can try our tutorial:

When players submit (end) a stage

Instead of having players wait until the end of the stage, you can have them submit their answer. This will set the Stage as submitted for them. If all the other players have also submitted the Stage, then the players move on to the next Stage or Round.

To set the stage to submitted you need to run this method from the player prop:

player.stage.set("submit", true);

This will change this property, that you can use to see if the stage has been submitted (it is a Boolean, true or false):

player.stage.get("submit");

For example, you could create a button, that has a handle method for onClick that will do the necessary with the player's answer and call player.stage.set("submit", true).

What players see when they have submitted a stage

Once a player submits for the stage, you might want to show something different on the screen. For example, instead of showing the stage's question/task (and avoiding players submitting multiple responses), you could show a message thanking the player for submitting the stage and telling them that they will have to wait for all players to submit the stage before they can move on to the next one.

You can do this by setting a conditional that renders different elements depending on what player.stage.get("submit") returns.

For example:

{
player.stage.get("submit")
    ? <div> Thank you for your answer. The next stage will start when all the other 
    players have submitted their answer. </div>
    : <div><Question player={player} /></div>
}

When new features or bug fixes are added to Empirica, you should try to upgrade to the latest version.

Check the current version of your experiment

At the root of your experiment, run empirica version:

$ empirica version
Version: v1.0.0-rc.24
SHA:     bc4184e
Build:   169
Branch:  main
Time:    2022-12-28T08:21:58Z

Client:  1.0.0-rc.24
Server:  1.0.0-rc.24

Updating Empirica in your Experiment

You can update the packages in your experiment by running the following command at the root of your experiment:

$ empirica upgrade

Within your current experiment, this will upgrade both the empirica command and the npm packages used in client/ and server/. This will not upgrade the empirica command globally. To upgrade the empirica command globally, see Updating the Empirica command

Change to a specific version in your experiment

If you need to upgrade (or downgrade) to a specific version of Empirica, you can pass the exact version like so:

$ empirica upgrade --version "v1.2.3"

You can also choose to only upgrade the command line or the npm packages individually with either:

$ empirica upgrade --commandOnly
$ empirica upgrade --packagesOnly

Updating the Empirica command

To upgrade the empirica command globally (outside any existing experiment), run the following:

$ empirica upgrade --global

Game

A Game is a single run of the experiment. It is also used to describe the experiment that was developed. One or more Players participate in a Game. A Game is divided into three main parts: Intro steps, Game, and Exit Steps.

The Intro Steps are when Players arrive in the game. Before arriving at the Lobby, they can read instructions, answer quizzes, etc.. The Intro Steps are asynchronous (each player completes them at their own pace). Once all the Players have reached the Lobby stage, the Game initializes. The Game is composed of rounds divided into Stages. Players must complete every stage of a round before moving on to the next Round. The Game is synchronous (every player has to finish a stage before they can all move on to the next stage). Once every Round is complete, the Players will reach the Exit Steps where they can read a debrief, answer questionnaires, etc. The Exit Steps are asynchronous.

Player

Players are users participating in a Game. A game must have a least 1 Player. Each Player entry in the database will contain information such as their last login, their unique ID, the ID of the Game they are participating in, which steps they have accomplished, and any other data you additionally assign to Players (e.g., an avatar to represent the Player).

Batch

A Batch is a set of 1 or more Games that run in parallel. Batches allow the creation of different assignment methods for player distribution: simple and complete. Batches are run one at a time in order of creation time. If you start multiple batches at once in the admin, only the first will accept players. Once full, the next batch will automatically start receiving players.

Round

A Game is made up of 1 or more ordered Rounds. A round contains one or more stages. Rounds are usually composed of the same stages that repeat throughout the game.

For example, in the guess the correlation experiment, we ask the player to guess the correlation between a graph and a numerical value on each round. In the first stage of each round, it's the guess. The second stage is the reveal of the result. And in the third stage, we show what other players have guessed. All in the same round. And in the next round, we repeat those stages.

All rounds are not required to contain the same stages or even the same number of stages.

Stage

A Round may contain one or more stages. A stage has a duration.

The duration is a required field that sets the duration of the stage in seconds, with a minimum of 5 seconds and no maximum. You can also allow Players to submit a stage, which marks it as done for the given player. When all players have submitted, the stage is considered done. If the stage timer runs out, the stage is also considered done, whichever happens first.

Usually, a name field is used to programmatic reference the stage. It can later be used in the UI code to differentiate stages.

If you do not want to have a timer and wait until players submit, you can set the timer to an unreasonable value (e.g. 30000000 ~= a year), and hide the timer in the UI.

The Rounds and Stages are synchronous (every player has to finish a stage before they can all move on to the next stage). The timer allows you to avoid one Player not responding and stopping the other Players from continuing because a new stage starts when the timer ends.

Factors

Factors are variables that will affect how a game will play. One Factor Type is mandatory for any Empirica game, the playerCount number.

Factors are assembled into Treatments, which are themselves assigned to Games.

To create Factor Values, you must first create Factor Types. A Factor Type describes the Factor. A _Factor Type) has a name, a description, a type, and a required marker.

• The name must be a code-friendly name, such a playerCount, which does not

  contain spaces or odd characters and is written in
• The description is a human-readable description used in the admin UI to help

  other admins understand what this factor does.

• Finally, the type is a computing type (integer, string, ...) to limit what

  _Factor Values can be created for this _Factor Type.

• Factor Types can also be marked as required, meaning they are required in

  all treatments.

The Factors can then be accessed as a property of the Game. This can be used to represent different conditions. For example, say you want some Games to show information from the other Players' answers to each other and some Games not to. You can have a "socialInformation" factor set to "yes" and "no", and only show the information from other Players if game.socialInformation == "yes" .

Treatment

A Treatment is a named set of Factors. Each Factor can appear at most once in a treatment, and required Factors must be present. The set of Factors in a treatment must be unique. No two treatments with the same set of factors can be created.

Game Lobby

A Game Lobby is where the players wait for other players to arrive until the game starts.

Lobby Config

Lobby configs configure the Game Lobby for common scenarios. It contains a timer duration for the lobby and handles the experiment's behavior in case of timeouts. Such as whether it start anyway or cancel the game once the timer runs out.

Intro Steps

The Intro Steps are pages shown to the user after registering and before the game or game lobby. The game designer can configure these steps to contain whatever is needed: instructions, a quiz, forms, etc.

The steps can be altered depending on the treatment assigned to the current player or the treatments.

The Intro Steps are asynchronous (each player completes them at their own pace).

Exit Steps

The Exit Steps are pages shown to the user after the game. They are configured by the experiment designer and may contain: results, reward instructions, quiz, thank-you note...

The steps shown may be altered depending on any parameter of the player (results, etc.). If you want to use parameters that are not originally on the player (results that might be on the game for example), you should copy the parameters that you will need in the exit steps onto the player object in the onGameEnded callback.

The Exit Steps are asynchronous (each player completes them at their own pace).

Creating batches

In the Admin panel you can create a batch of games. Select the games of the different treatments you want and create the batch.

To start the games in the batch click the ► Start button.

Now players can join the games in that batch.

If all the games from that batch are full, they will be allocated to games in other open batches that have the same treatments. If there are no such games available, the players will receive a gameFull and told that there are no games available.

What do the assignment methods 'simple' and 'complete' do?

These are different methods for randomly allocating players to different Games.

When you create a Batch, you can create multiple games within it, and these games can have different treatments. With experiments, one usually wants to randomly allocate players across the different games (Treatments).

Simple will randomly allocate players to either of the games. Imagine rolling a die for each player to determine which game they are sent to. This might lead to situations where more than the maximum number of players of a game are allocated to a game. In that case, the first players to reach the lobby will be those who get the play the game and the others will be unable to join and their status will be set to gameFull.

Complete will randomly allocate players to either of the games except those who already have the maximum number of players. This avoids the potential gameFull issue of the simple allocation method.

What's the difference between N batches of 1 game each or 1 batch of N games?

The assignment of players to games is done in batches sequentially and within each batch players will be randomly assigned to one game. Therefore, when you have 1 batch with multiple games, players will be assigned randomly to each of the X games according to the assignment method. This means it's possible that none of the games fill up even though enough players join for at least one game to proceed past the lobby as the players are distributed across various games. This would not happen if each batch has only one game: the first game will fill up with the first players who move past the lobby and the remaining players will transition to the game in the second batch.

If you want to ensure the maximum possible number of players get assigned to a game, a good strategy would be to start batches each with 1 game per each treatment condition. For example, if you have two treatment conditions of 8 players each, your batches should contain 1 game of each treatment. This way you can be sure if 16 players join, all 16 will be randomized between only 2 games and you don't lose any of your players in games that never fill up. This approach however has a drawback as it does not randomize between players with different arrival time or completion time of instructions.

Overriding default assignment

The default assignment algorithm privileges randomness for better experimental results, but this results in what we call "overbooking", in other terms, assigning more players than a game is deisgned for. This can lead to some games getting "underbooked", and to not being able to overflow players from an overbooked game into another game if the treatments do not match up (we do not overflow overbooked players between games with different treatment since the intro steps the players go through might be different between treatments).

We have added a couple options to modify the behavior of the assignment that reduces the randomness and increases assignment percentage:

• The preferUnderassignedGames option will try to assign players to games that are underassigned, before assigning to games that are already full, resuming the assignment process as usual if no underassigned games are available, in the current batch (this option does not try to prefer games that are underassigned across batches).

• The neverOverbookGames option will never assign players to games that are already full. This will push players into the next batches, if no games are available in the current batch. If no games are available in the next batches, the player will be sent to exit. This option is a bit more strict than preferUnderassignedGames and it can result in longer waiting times for players, and potentially game that never start if a player never finishes the intro steps.

Given the strict nature of the neverOverbookGames option, it is generally recommended to use preferUnderassignedGames option if you do not want the normal behavior of the assignment algorithm. If you use a single batch, preferUnderassignedGames should fill optimally.

Note that neverOverbookGames takes precedence over preferUnderassignedGames, meaning that if both options are set to true, preferUnderassignedGames will be ignored.

To apply these options, in server/src/index.js, you can add the following options to the Classic function:

ctx.register(
  Classic({
    preferUnderassignedGames: true,
  })
);

ctx.register(
  Classic({
    neverOverbookGames: true,
  })
);

Recording the data

There are multiple places where you can record data in an Empirica experiment.

To the player

One way of recording the data of players' responses is to set them to the player prop itself. You can do so with this command:

player.set("name_of_property", value)

You can retrieve what you have set as a specific property/answer for the player with:

player.get("name_of_property")

To the game, round or stage

If you want to save general data (not specific to one player), you can save it to the game, round or stage with:

game.set("name_of_property", value)
round.set("name_of_property", value)
stage.set("name_of_property", value)

You can retrieve what you have set as a specific property/answer with:

game.get("name_of_property")
round.get("name_of_property")
stage.get("name_of_property")

To the player.game, player.round or player.stage

One way of recording the data of players' responses is to set them to the player.game, player.round or player.stage prop to identify a particular data/response of a particular player to a particular game, round or stage. You can do so with this command:

player.game.set("name_of_property", value)
player.round.set("name_of_property", value)
player.stage.set("name_of_property", value)

You can retrieve what you have set as a specific property/answer with:

player.game.get("name_of_property")
player.round.get("name_of_property")
player.stage.get("name_of_property")

Exporting the data

Empirica can export your data to CSV format by running from the root of your experiment:

$ empirica export

This will create a zip file containing multiple csv files. Each file represents an object type (batches, games, players, rounds...). Each line is one of those objects. And each column is one of the keys in the object. For example, if you do player.set("myvar", 42), in the export, you will find a line in the players.csv file where the myvar column will contain 42.

Empirica experiments are event driven. The image below summarizes the lifecycle from the beginning of a game through exit steps.

Beginning and intro steps

In the admin panel, you would create a Batch of games for players to join. Players enter the game where they go through a consent form (optional), they provide an identifier (which can be set via URL queries), and they do the intro steps (the instructions and other components you have set in the intro steps. All of these are set by you in the ./client . During these steps, players are asynchronous; namely, they complete these steps at their own pace until they have finished them, independent from the pace of other players.

gameStart

Once players finish the intro steps they are sent to a lobby where they wait until every player has joined. When every player has joined, the Game object is created based on what is set in ./server/src/callbacks.js in the Empirica.onGameStart(). This is where certain parameters are set, and where the Rounds and their Stages are created.

Rounds and Stages

Then players go through each Stage of each Round. Once a Stage is finished, they move on to the next. If this was the last Stage of a Round, they move on to the next Round. If this was the last Round, they move on to the Exit Steps.

During these steps, players are synchronous; namely, every player has to finish each stage before they can all move on to the next.

In the ./server/src/callbacks.js, a series of callbacks are triggered during this process:

See our guide on customising when player submit a stage:

Game end and exit steps

After which, the players goes through the exit steps. During these steps, players are asynchronous; namely, they complete these steps at their own pace until they have finished them, independent from the pace of other players.

List of callbacks

The list of callbacks goes as follows in order:

• onGameStart Required

• onRoundStart Repeated for each Round

• onStageStart Repeated for each Stage

• onStageEnd Repeated for each Stage

• onRoundEnd Repeated for each Round

• onGameEnd

V2 is a rewrite and, therefore, introduces a lot of changes. But we have worked hard to keep the core experience similar to v1. This guide explains the differences with v1, what you need to know to start a new experiment, and how you might migrate your existing experiments.

Differences with v1

The significant difference is the backing technology: Empirica is no longer built on top of Meteor. It is a bespoke framework focused on experiment development. Day to day, this only changes things a little.

The empirica command

The command we use is now called empirica instead of meteor, but you still have an empirica create command to create a new project. And you start your development server by running the command without argument: empirica.

Code organization

We still have server/ and client/ directories. They are separate npm projects now. We used to have one package.json at the root of the project. Now both the client and server have their package.json, meaning they have their own npm dependencies. This is cleaner and easier in the long run. Also, in both directories, you will now see a subdirectory called src/, where all your code will reside. The base directory for the server and client is now only used for the configuration of each project.

Server-side

On the server, we have the same callbacks except for one: onGameInit is not called onGameStart (we already had an onGameStart that was not very used, and we have now merged them).

The arguments for each callback have changed a bit. We only pass the principal object for the callback. For example, onGameStart passes a Game object, and onStageEnd passes a Stage object. You can walk through the objects if you need to reach related objects. For example, if you need the Game object in a Stage callback (say onStageStart), you can do: stage.round.game. It follows the logical structure of the objects: batches > games > rounds > stages. Players for a game are on the Game object.

Client-side

The main objects (game, round, stage...) are no longer passed down between Components. Instead, you use the custom React Hooks provided by the empirica package. The main hooks are useGame, usePlayer (current player), usePlayers (all players in the current Game), useRound, and useStage. The initial template demonstrates how to use these.

Configuration for intro and exit steps, as well as Consent, NewPlayerForm, Loading, and NoBatches components, are now all done in the App.jsx. See the Special Empirica Components page for details.

Data

The database is no longer MongoDB. It is a single JSON file that resides in the .empirica/local directory, called tajriba.json (Tajriba is the name of our API Server). To back up or clear the database is simply a matter of copying or deleting that file.

We do not recommend reading that file directly, as the format for that file might change over time. Also, if you upgrade empirica, any previous tajriba.json file will be invalid with the newer version. Using the export command, you can still export the data from that file, even on a different version. However, we recommend exporting data before upgrading, then clearing the Tajriba file after the upgrade.

The empirica export is the preferred way to access the data. See Exporting the data for details about the export command.

Deployment

Deployment is no longer done in Meteor Galaxy. Deployment is much easier done manually than it used to be, but it is still very much manual at the moment. See the Deploying Your Experiment page for an example deployment. We are working on further simplifying deployment.

API changes

Game Init

You should make game initialization calls (addRound, addStage...) you used to make in onGameInit in onGameStart. Note that you can now add Rounds and Stages in any callback. For example, you can add a new Stage to a Round in onStageEnd. This allows for a dynamic number of Rounds and Stages. You could start by creating only 1 Round and 1 Stage in onGameStart, and add Stages and Rounds as needed during the Game.

Stage Submit

Stage submission is now done with player.stage.set("submit", true) (it used to be player.stage.submit()). And you can check whether the player has submitted the stage (the on client and server) with player.stage.get("submit") (it used to be player.stage.submitted). You can now cancel a player's submit status by setting "submit" to false player.stage.set("submit", false).

Game Treatment

Game treatment is no longer in game.treatment. It is now found in game.get("treatment").

Identifiers

Identifiers (IDs) are now .id (it used to be ._id). For example, you can do game.id (instead of game._id).

Migrating an existing experiment

The process of migrating an existing experiment is not automatic. If you do not need the new features of v2 and are done with developing your experiment, we recommend keeping it on v1. If you have just started or are about to start, moving to v2 is a good idea.

If you wish to convert your existing experiment, we recommend you create a new v2 experiment from scratch (see Creating your experiment).

Then, move over your callbacks, making sure to change the reference to your objects (see theServer-side section above).

On the client side, you will need to move over your configuration manually (intro/exit steps, component overrides... – see the Client-side section above). For Components, we recommend starting by migrating everything over as is and simply using the React Hooks on the very top Components and letting your existing Components pass down the main object (game, round...) as they used to in v1. Later you can migrate your components one by one to the new Hooks model for added clarity/simplicity (as well as improved performance). However, it is not a mandatory change!

Missing features compared to v1

There are a few known missing features compared to v1:

• Bot There is currently no bot system in v2. We have everything we need to implement a great bot system, we it's not there yet. We will release this as soon as possible. Let us know your use case in a ticket.

• Improved Admin There is, of course, an admin UI already in v2, but it could be more feature rich. We are working on improvements there. Again, if you have particular features you want to see, create a ticket with your experience so to help us prioritize what to work on first.

• Automated Deployment Deployment currently requires setting up a server manually. We want to make this experience much more accessible. We're working on it.

This page describes how to deploy Empirica using any server or VM provider. Empirica is a single command and uses an embedded database, so there is only one executable to install.

Bundling your experiment

Bundling your experiment is the task of building all the code you've created for your experiment and packaging it, along with the configuration, into a compressed "bundle" file.

To do so, run at the root of your experiment:

empirica bundle

This should create a file like [name-of-your-experiment].tar.zst. This is your bundled experiment. This is what you will deploy to your server. You should not add this to your code repository.

Setting up the server

You will need the empirica binary:

curl https://install.empirica.dev | sh -s

Running the experiment

Just so you know, you will need to send your experiment bundle to the server. For example, using scp:

scp myexperiment.tar.zst someuser@example.com:~

And finally, on the server, run the experiment with:

empirica serve myexperiment.tar.zst

Your experiment will be available at the address of that machine on port 3000.

Optional: setup a reverse proxy

You will need a domain name to use HTTPS (encrypted connections). We will not go into much detail here, but you can find many registrars online, such as Google Domains, Cloudflare Registrar, Namecheap, or Hover.

Then you will need to configure your domain name to point to your server. This is also beyond the scope of this guide. Still, you should be able to look up the documentation of your hosting provider (e.g., Digital Ocean) and your Registrar (e.g., Google Domains) to figure out how to point your domain name to your server using either an A Record or CNAME Record.

Once your domain name is set up (we will use www.example.com in our example), you can run:

caddy reverse-proxy --from www.example.com --to localhost:3000

This will set up the HTTPS certificate and redirect all incoming traffic to empirica.

Saving the database

The data in empirica will be stored in .empirica/local/tajriba.json. Copy that file to a safe place after running your experiment. You can take as many snapshots of your data as you wish by copying that file every so often.

You can also export the data from the server by running the export command. See the export section on the Managing the Data page.

Structure

The Admin Panel can be accessed at /admin. It is made of two parts:

• The monitoring section

• The configuration section

The Monitoring section

Here you can create and run batches of games and monitor player statuses.

The Configuring section

Here you can create factors and treatments before creating batches.

Clearing Players: How players can play again, even if they were led to the exit steps.

Players can only play once. If there is a problem (e.g., the lobby timed out or the game was full), or if you want players to play again, you can clear them, which allows them to play again.

In the monitoring section, in the players tab, you can see the status of every Player. A button called Clear, to the right of each player line, allows you to clear them one by one.

Once retired, players can refresh their page/return to the link of your game, and they will be randomly allocated to a game with the same treatment as before. They are allocated to a game with the same treatment, so they don't discover a game with different conditions or instructions.

Treatments and factors settings

The treatments and factors are saved in a .empirica/treatments.yaml. It is often shared by simply committing it to your code versioning system (e.g., Git).

You can edit treatments and factors manually in that file (make sure to reload the admin page after edits).

Alternatively, you can edit these directly in the admin UI, which will update the .empirica/treatment.yaml file.

No Games

The "No experiments available" page can be overriden in the App.jsx file with the noGames prop on <EmpiricaContext> like so (file simplified for clarity):

import { MyNoGames } from "./MyNoGames.jsx";
// ...
export default function App() {
  // ...
  return (
    <EmpiricaParticipant>
      <EmpiricaContext noGames={MyNoGames}>
        <Game />
      </EmpiricaContext>
    </EmpiricaParticipant>
  )
}

The "no games" component does not receive any special props.

Consent

The consent page can be overriden in the App.jsx file with the consent prop on <EmpiricaContext> like so (file simplified for clarity):

import { MyConsent } from "./MyConsent.jsx";
// ...
export default function App() {
  // ...
  return (
    <EmpiricaParticipant>
      <EmpiricaContext consent={MyConsent}>
        <Game />
      </EmpiricaContext>
    </EmpiricaParticipant>
  )
}

The Consent component will receive one prop:

• onConsent: this is a function that should be called if the player consents. If they do not consent, you can either modify something on the page (e.g. hide the consent form) or redirect them away from the experiment.

Here's a simplistic Consent component example:

export function MyConsent({ onConsent }) {
  return (
    <div>
      <div>Do you consent?</div>
      <div>
        <button type="button" onClick={onConsent}>
          Yes!
        </button>
      </div>
    </div>
  );
}

If you don't want to fundamentaly change the Consent, and only want to change the text of the default consent, you can use the Consent component provided with the Empirica package:

import { Consent } from "@empirica/core/player/react";

export function MyConsent({ onConsent }) {
  return (
    <Consent
      title="Do you consent?"
      text="It's all good, mate."
      buttonText="OK"
      onConsent={onConsent}
    />
  );
}

PlayerCreate (where players set their id)

Players have to set their id at a PlayerCreate page just after the consent page. This could be a name, an MTurk/Prolific id, a student id, an email, etc.

You might want to change its design, format, or instructions. For example, if you want participants to provide a student id but no other personal information, you would want to make your own clear instructions.

The default PlayerCreate.jsx page looks like this:

import React, { useState } from "react";

export function MyPlayerForm({ onPlayerID, connecting }) {
  // For the text input field.
  const [playerID, setPlayerID] = useState("");

  // Handling the player submitting their ID.
  const handleSubmit = (evt) => {
    evt.preventDefault();
    if (!playerID || playerID.trim() === "") {
      return;
    }

    onPlayerID(playerID);
  };

  return (
    <div>
      <div>Enter your Player Identifier</div>

      <form action="#" method="POST" onSubmit={handleSubmit}>
        <fieldset disabled={connecting}>
          <label htmlFor="playerID">Identifier</label>
          <input
            id="playerID"
            name="playerID"
            type="text"
            autoComplete="off"
            required
            autoFocus
            value={playerID}
            onChange={(e) => setPlayerID(e.target.value)}
          />

          <button type="submit">Enter</button>
        </fieldset>
      </form>
    </div>
  );
}

There are 2 props passed to the PlayerCreate component:

• onPlayerID: the callback you must call with the player identifier you collected from the Player.

• connecting: this will be true while the player create being identified (after you have submitted with onPlayerID), until the player is "logged in". This field allows you to disable or hide the form while sending the login data to the server.

Once you've made your version of the PlayerCreate component, you can import it into the App.jsx file, and set the playerCreate prop on <EmpiricaContext> like so (file simplified for clarity):

import { MyPlayerForm } from "./MyPlayerForm.jsx";
// ...
export default function App() {
  // ...
  return (
    <EmpiricaParticipant>
      <EmpiricaContext playerCreate={MyPlayerForm}>
        <Game />
      </EmpiricaContext>
    </EmpiricaParticipant>
  )
}

Lobby

The lobby page can be overriden in the App.jsx file with the lobby prop on <EmpiricaContext> like so (file simplified for clarity):

import { MyLobby } from "./MyLobby.jsx";
// ...
export default function App() {
  // ...
  return (
    <EmpiricaParticipant>
      <EmpiricaContext lobby={MyLobby}>
        <Game />
      </EmpiricaContext>
    </EmpiricaParticipant>
  )
}

At the moment, the Lobby does not have any special fields, but you do have access the current Player (const player = usePlayer()) and the current treatment (player.get("treatment")).

Timer

There is a Timer.jsx component in the default game template that represents the stage timer and shows players how much time is left at that stage.

The time remaining on the current stage (in milliseconds) can be accessed with the useStageTimer() React Hook.

Note that the timer might not exist if there is no current stage (before the Game starts, or in between Stages)

Here is the code for a basic Timer component:

import { useStageTimer } from "@empirica/core/player/classic/react";
import React from "react";

export function MyTimer() {
  const timer = useStageTimer();

  return (
    <div>
      {timer?.remaining ? Math.round(timer?.remaining / 1000) : 0} seconds
    </div>
  );
}

Finished, Loading and Connecting

The Finished, Loading and Connecting pages can be overriden in the App.jsx file with the finished, loading, and connecting props on <EmpiricaContext> like so (file simplified for clarity):

import { Myloading } from "./Myloading.jsx";
import { Finished } from "./Finished.jsx";
// ...
export default function App() {
  // ...
  return (
    <EmpiricaParticipant>
      <EmpiricaContext
        loading={Loading}
        connecting={Loading}
        finished={Finished}
      >
        <Game />
      </EmpiricaContext>
    </EmpiricaParticipant>
  )
}

These components do not receive any special props.

Function of these components:

• loading: is shown any time the experiment is loading, including between stages.

• connecting: us shown while the experiment is connecting to the server. This will happen on the initial page start, or if the server disconnects at any point. By default, the connecting page is the same as the loading page.

• finished: is the last page shown to the player after they have finished the Game and all the exit steps.

Empirica Chat

Installing

First, install Empirica Chat with:

npm install --save @empirica/chat

Import and Usage

In the components you want to use the chat, import this:

import { Chat } from "@empirica/chat";

Then you can create the chat component with:

<Chat player={player} scope={game} />

chat expects 2 required props:

• player: the current player

• scope: object that the chat will be attached to, can be game, round, or stage objects.

Scoping

The scope is important because it determines where the chat data (the messages) will be stored. You can access them again afterwards based on the scope. For example, if the scope is set to the Round:

const chat = round.get("chat") ?? [];

Multiple chat instances within the same scope

You can pass an optional customKey string prop to differentiate different chats within the same scope. This changes which get/set key on the given scope the chat will be recorded.

<Chat player={player} scope={game} customKey="casual_chat" />

Display names

Chat also displays a name for each participant, which you need to set in the experiment independently of the playerId: player.set('name', "myPseudonym")

Other functionalities

Server

Empirica.onGameStart(callback)

The onGameStart callback is called just before a game starts, when all players are ready, and it must create rounds and stages for the game.

One (and one only) onGameStart callback is required for Empirica to work.

It also offers the addRound() method, which allows to add a round to the game. The returned Round object will implement the addStage(stageArgs) method, which allows to add a Stage to the Round. The stageArgs object to be passed to the stage creation method must contain:

• duration: the stage duration, in seconds

Note that the Game has not yet been created when the callback is called, and you do not have access to the other properties of the Game which will be created subsequently.

Example

Empirica.gameInit(game => {
  game.players.forEach((player, i) => {
    player.set("score", 0);
  });

  const round1 = game.addRound();
  round1.addStage({
    duration: 120
    name: "Response",
  });

  if (game.treatment.playerCount > 1) {
    round1.addStage({
      duration: 120
      name: "Result",
    });
  }

  const round2 = game.addRound();
  round1.addStage({
    duration: 300
    name: "Response",
    someotherfield: "mydata"
  });

  if (game.treatment.playerCount > 1) {
    round1.addStage({
      duration: 120
      name: "Result",
    });
  }
});

Game Callbacks

Game hooks are optional methods attached to various events throughout the game life cycle to update data on the server-side.

Contrary to client side data updates, sever-side updates are synchronous, there is no risk of conflicting updates, and important calculations can be taken at precise points along the game.

``

Empirica.onRoundStart(callback)

Example

Empirica.onRoundStart(({ round }) => {
  round.set("scoreToReach", round.currentGame.get("maxScore"));
});

Empirica.onStageStart(callback)

Example

Empirica.onStageStart(({ stage }) => {
  stage.set("randomColor", myRandomColorGenerator());
});

Empirica.onStageEnded(callback)

Example

Empirica.onStageEnded(({ stage }) => {
  const expectedScore = stage.round.get("expectedScore");
  const group = stage.get("score") > expectedScore ? "great" : "not_great";
  stage.set("scoreGroup", group);
});

Empirica.onRoundEnded(callback)

Example

Empirica.onRoundEnded(({ round }) => {
  let maxScore = 0;
  round.currentGame.players.forEach((player) => {
    const playerScore = player.round.get("score") || 0;
    if (playerScore > maxScore) {
      maxScore = playerScore;
    }
  });
  round.set("maxScore", maxScore);
});

Empirica.onGameEnded(callback)

onGameEnded is triggered when the game ends. It receives the game that just ended.

Example

Empirica.onGameEnded(({ game }) => {
  let maxScore = 0;
  game.rounds.forEach((round) => {
    const roundMaxScore = round.get("maxScore") || 0;
    if (roundMaxScore > maxScore) {
      maxScore = roundMaxScore;
    }
  });
  game.set("maxScore", maxScore);
});

Empirica.on(model, callback)

on(model, callback) listens to the creation of new mode objects. Model objects are the "game", "round", "stage", "player" and "batch". The callback receives a ctx object, and the model object listened to. In the example below, we're listening to new games, so we receive the game as argument.

Empirica.on("game", (ctx, { game }) => {
  if (game.get("initCalc")) return;
  
  game.set("initCalc", initCalcs());
});

Empirica.on(model, attributeName, callback)

on(model, attributeName, callback) listens on changes to the attribute of attributeName on model. Model objects are the "game", "round", "stage", "player" and "batch". The callback receives a ctx object, the model object, and the attribute value. In the example below, we're listening on changes to choice on the player model.

Empirica.on("player", "choice", (ctx, { player, choice }) => {
  if (choice === "yes") {
    // ...
  }
});

Empirica.flush()

Empirica.flush(): Promise<void> adds the ability to manually flush changes outside of callback functions. This is useful when you want to make changes asynchronously without blocking the callback. For example, if you want to call an external API and update an attribute with the results, but your don't want to block the callback while waiting for the API to respond.

Empirica.flush() will return a promise that resolves when the changes have been flushed. You can use await Empirica.flush() to wait for the changes to be flushed. But you do not have to wait for the flush. It is only useful if you have multiple flushes in a single function, where you want to commit the changes in steps.

The simplest example:

Empirica.on("stage", "myTrigger", (ctx, { stage, myTrigger }) => {
  callMyAPI.then((value) => {
    stage.set("value", value);
    Empirica.flush();
  });
});

An example with different use cases:

Empirica.on("stage", "myTrigger", (ctx, { stage, myTrigger }) => {
  // Copying the stage to the global namespace so we can use it outside of the
  // callback function. See below this callback function.
  myStage = stage;

  // flush
  setTimeout(() => {
    stage.set("b", (stage.get("b") || 0) + 1);
    Empirica.flush();
  }, 2000);
});

// Note: copying a stage to the global namaspace like this is not recommended,
// this is just for illustration purposes. If you have multiple games running
// at the same time, stages will overwrite each other.
let myStage;
setInterval(() => {
  if (myStage) {
    myStage.set("c", (myStage.get("c") || 0) + 1);
    Empirica.flush();
  }
}, 1000);

Note: if you perform blocking operations between changes, the changes before the blocking operations might be flushed before flushing manually. The change log is global, so while you're blocking, other changes might happen in a callback elsewhere, and that could trigger a flush of all latent changes. If you want to make a set of changes atomic, you should not do any blocking operations between them (i.e. use await or Promises) or collect your changes in a local datastructure and apply them at once.

If you are starting an interval (setInterval), you should make sure to clear it properly. Here's an example with an interval per game:

const timers = {};

// We use the game event to start the interval. this ensures that the interval
// is started on a server restart.
Empirica.on("game", (ctx, { game }) => {
  if (!game.isRunning) return;

  timers[game.id] ||= setInterval(() => {
    // do something
    Empirica.flush();
  }, 1000);
});

Empirica.onGameEnd(({ game }) => {
  clearInterval(timers[game.id]);
});

Or have a single interval that manages all games:

const activeGames = new Set();

// We use the game event to start the interval. this ensures that the interval
// is started on a server restart.
Empirica.on("game", (ctx, { game }) => {
  if (game.isRunning) {
    activeGames.add(game);
  }
});

setInterval(() => {
  for (const game of activeGames) {
    // do something
  }
  Empirica.flush();
}, 1000);

Empirica.onGameEnd(({ game }) => {
  activeGames.delete(game.id);
});

Server Objects

Game object

Round object

Stage object

Player object

Client

This will go through all the steps needed to deploy an Empirica experiment in a generally secure manner, how to update your experiment, and how to retrieve your data. We are using Digital Ocean, which is relatively easy to use, but you can use any hosting provider and the steps will be similar if you use Ubuntu 22+.

This tutorial does assume a few things:

• You have minimal Linux, command line, and ssh experience. If you don't check out some quick intro tutorials on Youtube.

• You have a domain name. A subdomain on empirica.app can be provided, see Setup your domain below for details.

Bundle your experiment

At the root of your experiment, run:

This will create a file called myexperiment.tar.zst where myexperiment is the name of your experiment or empirca. This file will be useful later in the process.

Create your dropplet on Digital Ocean

2. Click Create in the top left, then Droplets

3. Choose a region close to you ; datacenter doesn't matter much, leave the default

4. For the image selection, choose Ubuntu then 23.10 x64

5. For size, choose Basic, then Premium AMD, then 4GB/2CPU or 8GB/4CPU depending on your needs. Generally, you can get by with 2 CPUs, espcially during development, and you can always upgrade your droplet later.

6. Enable automated backup..., daily, doesn't hurt

8. Enable Add improved metrics...

9. Quantity: 1

10. Hostname: you choose (e.g. myexperiment), this is unrelated to your domain name, it's mostly just the name in your Digital Ocean admin UI.

11. Click Create Droplet

12. Wait for the Droplet to start, and note down the IP address

Setup your domain

Your own domain

If you have a access to a domain name, we'll assume you know how to setup an A Record (name is the domain or subdomain name you want to use, the value is the IP address of your Droplet, and for TTL either automatic/default, or 5 min if you expect to change the address soon).

You might want to double check your domain is configured correctly (it can take a few minutes to propagate). On macos and linux you can use dig:

empirica.app subdomain

If you do not have a domain name, you can contact Nicolas P. on Slack with the subdomain on empirica.app you'd like to use (e.g. mycoolexperiment.empirica.app). Please provider 2-3 options that are not too generic (e.g. experiment is too generic) so we don't conflict, and the IP address of your Droplet.

A couple of notes:

• please do not do anything illegal on your domain. We are not responsible for anything sketchy you do on your subdomain

• if you're taking down your experiment, please let us know so we can cleanup the DNS entry. We reserve the right to clear the subdomain if we haven't heard from your in a year or two

Setup your server

SSH into your server using the IP address from your Droplet. For example:

Install Empirica

To install empirica on your server, run:

Install caddy

Caddy is the reverse proxy that will enable HTTPS on your experiment. To install, run the following commands:

Setup Empirica

We want Empirica to reload automatically in case of a crash, a server restart, or an update. For that we will create a few folders and configuration files.

First, create a folder for the deployment at the root of your user (where you ssh-ed in). We will use a directory called empirica:

We will create 2 files to start and restart empirica using systemctl. First, run:

In this file, paste the following:

Save the file (with nano, ctrl+o then Enter) and exit (ctrl+x).

We will then create a restarter file:

And paste the following:

We will then create a watcher file:

And paste the following:

Save and exit nano. This file will restart empirica when the empirica bundle changes.

Run the following to register these 2 files:

First deployment

Next, we'll upload your Empirica experiment bundle (the .tar.zst file we created in the first step) from your machine to your server. You can do that with scp (replace myexerpiment with the name of your bundle and the IP address with your Droplet IP address):

We can now start empirica like so:

Check that it's running as expected:

It should return a bunch of html with an HTTP code 200 (HTTP/1.1 200 OK).

Setup Caddy

Delete the existing configuration for Caddy:

And add a new one:

With the following, where the domain name if the domain name you setup:

And restart caddy:

It might take a minute for the HTTPS certificates to generate. Go to your domain in your browser, and if all goes well, your experiment is deployed, with HTTPS.

Restart empirica on changes

We will enable our restarter units:

Run experiment

Access the admin

Go to /admin on your domain in the browser. You will be asked for a username and password. You can find the default username (admin by default) and password (randomly generated with your new project) in your project in .empirica/empirica.toml on your local machine. You can change or add more users from that file and redeploy.

Export the data

On your server, the data will be in ~/empirica, and you can run the following from root:

To copy the data back to your machine, you can run, from your machine (* will pull all the exports, you can specify the exact zip file your want instead) :

Update your experiment

To update your experiment, bundle and scp again from your local machine:

You will need to reload the browser to see the changes. It does not reload the page automatically as it does in local development mode.

Administration tasks

Stop, restart, or start empirica manually

View logs

Setting up the dependencies

Before you complete the tutorial, you will need to install Empirica as described in the link below:

Setting up the editing environment

This tutorial was developed using Visual Studio Code. You are welcome to use any editor or IDE you are comfortable with, but if you don't have a favorite, or you want to follow along precisely with the tutorial, you may want to give it a try. You can find instructions on the page linked below:

This tutorial can be used independently or as a supplement to the Empirica workshop and tutorial (to be) filmed at the International Conference for Computational Social Science in July 2023.

How social scientists can use this tutorial

This tutorial provides detailed instructions that will help you get started building your first experiment in Empirica. The tutorial doesn't assume any prior knowledge of Javascript or React, although some experience with programming will be helpful.

Many experiments can be built up from this basic foundation with small changes to the way information is presented or the choices participants are presented with. The goal of this tutorial is to help you become more comfortable with Empirica and understand how to translate your experiments from ideas to tangible code.

For resources that can help you become a more capable developer, see:

How software developers can use this tutorial

If you are a software developer and comfortable using React, JavaScript, CSS, etc., this tutorial will help you quickly understand the basic structure of an Empirica experiment. This tutorial may act as a helpful example to refer to when implementing other experiments that your team wants to explore. To get more familiar with the Empirica conceptual model, see:

About the experiment

The Prisoner's Dilemma is a classic concept in game theory that explores the tension between cooperation and self-interest. It is often used as an experimental tool in social psychology to study decision-making, trust, and cooperation.

In the standard version of the Prisoner's Dilemma, two individuals are arrested for a crime and are held in separate cells. They are both given the opportunity to cooperate with each other or betray each other by confessing to the crime. The outcomes are as follows:

1. If both prisoners remain silent (cooperate), they each receive a moderate sentence for a lesser charge.

2. If one prisoner confesses (betrays) and the other remains silent, the betrayer receives a minimal sentence, while the other prisoner receives a severe sentence.

3. If both prisoners confess (betray), they both receive a moderately severe sentence.

The dilemma arises because each prisoner must decide whether to trust the other person and cooperate for a potentially better outcome for both or act in their self-interest by betraying the other, which might lead to a better personal outcome if the other remains silent.

Special thanks

This tutorial was developed by James Houghton, Mohammad Alsobay, and Michael Li, with support from Millie Gu and others.

To create your experiment, open your terminal and navigate to the directory where you want to create your project. (e.g. cd ~/Desktop). You can find a refresher on terminal commands used to navigate directories here:

Set up a project using the template experiment

To set up a template experiment, enter the following command in your terminal window:

empirica create prisoners-dilemma

This will create a folder named 'prisoners-dilemma' for the project and populate it with a template experiment. (e.g. ~/Desktop/prisoners-dilemma)

You should see something similar to the following:

    ✓  Setup directory
    ✓  Copy project files
    ✓  Install client packages
    ✓  Install server packages
    ✓  Upgrade empirica packages
    ✓  Generate default settings
                                                                                                                          
    Empirica (local build) experiment created.                                                                                   
    Get started:                                                   
                                                                  
        > cd prisoners-dilemma                                        
        > empirica   

To work within the newly created folder, change into the newly created directory:

cd prisoners-dilemma

Test-drive the template experiment

To start the experiment, run the command:

empirica

You should see something similar to the following in the terminal window:

    Empirica (local build) server running:                                       
                                                                                
        Player  http://localhost:3000                                             
        Admin   http://localhost:3000/admin  

To view the experiment, type http://localhost:3000/admin/ into your browser address bar. (You can use any browser you like, the images in this tutorial show the Chrome browser.) You will see the Empirica admin interface, which handles setting up treatments and launching batches of games:

Note: if you see a login page like the following, you should use the username "admin".

You can print the auto-generated password by running the terminal command cat .empirica/empirica.toml within your prisoners-dilemma folder.

Start a "batch"

The admin console opens to the "Batches" tab. To create a new batch, click the "New Batch" button to open the batch creation drawer. We will create one game with the "Two Players" treatment. Edit your settings to look like this:

Click Create to return to the main batch screen. You'll see that a batch has been added to the batches list. Click "Start" to begin the batch.

If you find it helpful to see the process animated:

Play the game

To play the game, open a new tab to url http://localhost:3000/. You should see something like the following:

For now, enter your name as the identifier. When your experiment is deployed, this will be where participants enter their MTurk Worker ID, Prolific Identifier, etc.

You will need to create a second player to join the game. While you are developing the game on your local computer, you will see a toolbar in the lower left corner:

Click the second button to add a new participant - a view for this second participant will open in a new tab. Enter a different identifier, and walk through the experiment by playing for both participants. Try opening each of the players' tabs side-by-side so you can see how each player's view is updated in real time!

As a brief rundown, the stages of the experiment are as follows:

1. Player Consent

2. Player Identification

3. Intro Steps (Here an "Instructions" page)

4. A lobby, where you wait for enough players to complete the introduction for the game to launch

5. Synchronous Game Stages (here a guessing game, and minesweeper, just for fun)

6. Exit Steps (surveys, payment instructions, debrief, etc)

Stop the development server

Once you have had a chance to play through the game, return to your terminal window and shut down the empirica server by pressing the control key and the 'c' key simultaneously (<ctrl>+c). If it asks you for confirmation, you can push the same key combination again.

In this section, we break down the initial process of creating an experiment into subparts, for clarity.

To get a glimpse of the actual code that runs Empirica, open Visual Studio Code to the prisoners-dilemma folder.

You can either do this manually, by going to File > Open Folder and selecting the prisoners-dilemma folder, or by typing in your terminal code . within the prisoners-dilemma directory.

The template experiment includes three subdirectories:

1. .empirica stores data and config files that we need to run the experiment.

2. client is where the code that runs on your experiment participant's browser lives.

3. server is where all the code that runs on the server lives.

The client and server directories are each structured as a node package, and contain some boilerplate code that we can ignore:

• node_modules/ is a folder containing dependencies, and is managed for us (we never need to touch it)

• dist/ is a folder for the compiled version of our code, which gets built automatically

• package.json and package-lock.json contain references to other packages that our experiment depends on

• client/index.html, client/vite.config.js, client/uno.config.ts, client/jsconfig.json server/jsconfig.js are all configuration files we can leave as they are

In this tutorial, we will make changes in the following places:

• client/src/ is a folder containing the views the participant sees, built up as React components.

• server/src/callbacks.js is a file for server-side code that is called when specific actions happen in the game.

• .empirica/local/tajriba.json is where our experiment data is stored. It is often helpful to delete this file when you restart the server, to start with a clean slate.

Have a quick browse through the files in client/src/examples to see how the jelly bean and minesweeper examples are implemented.

When you create an experiment with the empirica create command, Empirica populates a template experiment with the jelly bean and minesweeper code that you saw in the previous step. We won't use this example code in our experiment, so let's clean it out. There are a few steps:

• delete the entire client/src/examples folder

• In client/src/Stage.jsx:

  • remove the imports for JellyBeans and MineSweeper (2 lines)

  • replace the switch statement and its three cases (8 lines, including brackets) with the code `

    Copy

    return (<p>Not yet implemented...</p>)

The file should look like this when you are done:

import {
  usePlayer,
  usePlayers,
  useRound,
} from "@empirica/core/player/classic/react";
import { Loading } from "@empirica/core/player/react";
import React from "react";

export function Stage() {
  const player = usePlayer();
  const players = usePlayers();
  const round = useRound();

  if (player.stage.get("submit")) {
    if (players.length === 1) {
      return <Loading />;
    }

    return (
      <div className="text-center text-gray-400 pointer-events-none">
        Please wait for other player(s).
      </div>
    );
  }

  return (<p>Not yet implemented...</p>)
}

• In the server/src/callbacks.js file:

  • empty the contents of the onGameStart and onStageEnded functions, keeping the function definitions

  • delete everything after the empty function definition for Empirica.onGameEnded({}).

When you are done, the file should look like this:

import { ClassicListenersCollector } from "@empirica/core/admin/classic";
export const Empirica = new ClassicListenersCollector();

Empirica.onGameStart(({ game }) => {});

Empirica.onRoundStart(({ round }) => {});

Empirica.onStageStart(({ stage }) => {});

Empirica.onStageEnded(({ stage }) => {});

Empirica.onRoundEnded(({ round }) => {});

Empirica.onGameEnded(({ game }) => {});

Note: In JavaScript, a callback function is a function that is passed as an argument to another function and is invoked or called at a specific point within that function's execution. The callback function is typically used to ensure that certain code is executed only after the completion of a specific task or operation. In our case, these callbacks are invoked when parts of the experiment start and end.

Restart empirica

Now we can restart Empirica to have it reflect our changes. The command below will remove our existing datafile and start the server.

rm .empirica/local/tajriba.json; empirica

The first step in coding our experiment will be to replace the filler text in the Introduction page with actual instructions for the participant. This text is stored the file client/src/intro-exit/Introduction.jsx. The component will initially look like this:

import React from "react";
import { Button } from "../components/Button";

export function Introduction({ next }) {
  return (
    <div className="mt-3 sm:mt-5 p-20">
      <h3 className="text-lg leading-6 font-medium text-gray-900">
        Instruction One
      </h3>
      <div className="mt-2 mb-6">
        <p className="text-sm text-gray-500">
          Lorem ipsum, dolor sit amet consectetur adipisicing elit. Eius aliquam
          laudantium explicabo pariatur iste dolorem animi vitae error totam. At
          sapiente aliquam accusamus facere veritatis.
        </p>
      </div>
      <Button handleClick={next} autoFocus>
        <p>Next</p>
      </Button>
    </div>
  );
}

The function contained in this file is called a React Component. This is just a function that returns a piece of a page for your browser to display, and it can contain other React components within it. By assembling a number of different React components in a hierarchy, we can build up the whole page, while being able to separate reusable pieces into their own files.

For example, the Button element is another React component that we import into this file. The button's behavior and styling are defined in a separate file (src/components/Button.jsx) so that we can reuse it in a number of pages in our experiment.

All components need to return something that React will translate into plain HTML to display on the page. In this case, it is returning mostly HTML, with the Button component and a little inline styling thrown in (more on this later).

For now, lets replace just the display text (and it's paragraph tags <p>...</p>, lines 11-15 above) with instructions related to the prisoners dilemma.

Feel free to experiment with your own text and styling. The code below is what we will use in this tutorial:

      <p>
        In this game, you will be partnered with another experiment participant.
        <br />
        In <strong>each round of the game</strong>, you and your partner will:
      </p>
      <p>
        (a) choose whether to work together or to look out for your own
        interests
      </p>
      <p>
        (b) see each others choices and the results of your choices
      </p>

Within the browser, you can see these changes reflected immediately by simply saving the changed file. (In some cases, you may need to refresh the page.) This is called 'hot reloading', which is incredibly useful for debugging and UI editing.

Your edited file should look like this:

import React from "react";
import { Button } from "../components/Button";

export function Introduction({ next }) {
  return (
    <div className="mt-3 sm:mt-5 p-20">
      <h3 className="text-lg leading-6 font-medium text-gray-900">
        Instruction One
      </h3>
      <div className="mt-2 mb-6">
        <p>
          In this game, you will be partnered with another experiment
          participant.
          <br />
          In <strong>each round of the game</strong>, you and your partner will:
        </p>
        <p>
          (a) choose whether to work together or to look out for your own
          interests
        </p>
        <p>(b) see each others choices and the results of your choices</p>
      </div>
      <Button handleClick={next} autoFocus>
        <p>Next</p>
      </Button>
    </div>
  );
}

Now we will create placeholders for the two stages that the participants will use to play the game. First, in the 'Choice' stage, participants will decide whether to cooperate, or defect. Second, in the 'Result' stage, they will see the choice that their partner made, and their payoff.

Create placeholder components

Create a folder in the client/src. We will call it stages, as it will hold the code for the various stages the participant will progress through.

Within this folder, we will create 2 files. First, create Choice.jsx with the following placeholder code:

import React from "react";

export function Choice() {
  return <div>"Choice" component not yet implemented</div>;
}

Second, create Result.jsx with the following boilerplate code:

import React from "react";

export function Result() {
  return <div>"Result" component not yet implemented</div>;
}

Set up stage sequencing

Now we need to tell Empirica that within each round of the game, participants will advance through two sequential stages: one called "Choice" and one called "Result".

We do this on the server, so that Empirica can coordinate advancing all players through the stages simultaneously. The place for this is in the onGameStart callback, found in server/src/callbacks.js. This function is called when enough players have advanced to the lobby for the game to start, and it is where we can most easily modify the sequence of the game.

First we need to add a 'round' to the game, using the game.addRound function. This function takes a dictionary with various keys, for now we'll just use the name key.

Empirica.onGameStart(({ game }) => {
  const round = game.addRound({
    name: `Round`,
   });
});

After we have created the round, we need to attach a stage to it for each of the choice and result stages:

Empirica.onGameStart(({ game }) => {
  const round = game.addRound({
    name: `Round`,
   });
   round.addStage({name: "choice", duration:10000});
   round.addStage({name: "result", duration:10000});
});
   

For now, we'll set the maximum duration of each stage to 10,000 seconds, after which the game will advance even if one or more players has not submitted an answer. This is to make it easier to develop the experiment. When we are getting ready to deploy our experiment, we will set this to a more reasonable number.

The server/src/callbacks.js file should now look like this:

import { ClassicListenersCollector } from "@empirica/core/admin/classic";
export const Empirica = new ClassicListenersCollector();

Empirica.onGameStart(({ game }) => {
  const round = game.addRound({
    name: `Round`,
  });
  round.addStage({ name: "choice", duration: 10000 });
  round.addStage({ name: "result", duration: 10000 });
});

Empirica.onRoundStart(({ round }) => {});

Empirica.onStageStart(({ stage }) => {});

Empirica.onStageEnded(({ stage }) => {});

Empirica.onRoundEnded(({ round }) => {});

Empirica.onGameEnded(({ game }) => {});

Display components during the appropriate stages

Now that we have placeholders for the components that will be displayed during the 'choice' and 'result' stages, and Empirica knows to coordinate these two stages, we need to tell the participant display to show the correct component during the correct time window.

For this, we will return to client/src/Stage.jsx, which by convention handles switching displays between different stages of the game. (This is just convention, you can use any structure of React components that works for your experiment.)

First we need to import this hook by adding it to the hooks already in our import list:

import {
  usePlayer,
  usePlayers,
  useRound,
  useStage,
} from "@empirica/core/player/classic/react";

We then need to call the hook to gain access to the stage object. We put calls to all of the hooks at the top of the component, so the following line can go under the calls to the useRound hook:

const stage = useStage();

Now we can use the value of the stage object in a switch statement to display a different component for each value of the stage's name. This code will replace the code return <p>Not yet implemented...</p>; that we used as a placeholder earlier.

switch (stage.get("name")) {
    case "choice":
      return <></>;
    case "result":
      return <></>;
    default:
      return <Loading />;
  }

For now, we're using a pair of empty tags in place of the objects.

Lastly, we need to import the placeholder components we defined above. Import statements such as these all belong together at the top of the file:

import { Choice } from "./stages/Choice";
import { Result } from "./stages/Result";

Finally, we can use these components inside the switch statement instead of the empty tags <></>.

  switch (stage.get("name")) {
    case "choice":
      return <Choice />;
    case "result":
      return <Result />;
    default:
      return <Loading />;
  }

With all changes made, the Stage.jsx file should look like this:

import {
  usePlayer,
  usePlayers,
  useRound,
  useStage,
} from "@empirica/core/player/classic/react";
import { Loading } from "@empirica/core/player/react";
import React from "react";
import { Choice } from "./stages/Choice";
import { Result } from "./stages/Result";

export function Stage() {
  const player = usePlayer();
  const players = usePlayers();
  const round = useRound();
  const stage = useStage();

  if (player.stage.get("submit")) {
    if (players.length === 1) {
      return <Loading />;
    }

    return (
      <div className="text-center text-gray-400 pointer-events-none">
        Please wait for other player(s).
      </div>
    );
  }

  switch (stage.get("name")) {
    case "choice":
      return <Choice />;
    case "result":
      return <Result />;
    default:
      return <Loading />;
  }
}

Note: Remember to restart your server using the command rm .empirica/local/tajriba.json; empirica in your terminal whenever you make an update to a server-side file.

Restart your server, and walk through your experiment. It should look like this:

So far, we've implemented a live, single-round Prisoner's Dilemma between two players. Now, we'll add customizations that allow us to explore the effects of different interventions on participants' behavior. After all - to run an experiment, you need a comparison.

First, we'll explore the effect of varying the number of rounds of the game participants play together. Then we'll explore the effect of allowing participants to communicate via text chat.

We'll parameterize both of these interventions as different "factors" in the treatment definition, and we'll talk about how Empirica helps us randomize players between conditions to explore a causal effect.

The last major piece needed to complete this section of the demo is to compute the result that a player receives based on their and their partner's choices.

Identify the correct time to compute the score

The score should be computed once, at the end of the "choice" stage. To coordinate this computation, we do it on the server side, in the onStageEnded callback function in server/src/callbacks.js. This function gets triggered at the end of each stage, and receives the recently closed stage object as an argument.

Empirica.onStageEnded(({ stage }) => {
  if (stage.get("name") !== "choice") return;
  console.log("End of choice stage");
});

Loop over all the players

We don't have access to Empirica's 'hooks' on the server side, but we can get access to the players from the stage argument. stage.currentGame will give us access to the game object, and stage.currentGame.players will return the list of players:

const players = stage.currentGame.players;

Once we have the list of players, we can loop over them and compute the list of scores:

for (const player of players) {
  console.log("computing score for player ", player.id)

}

For each player, we need to identify the partner, using the same strategy as before. We'll need to find the partner for each player as we compute the player's score, so this will be inside our for loop:

  const partner = players.filter((p) => p.id !== player.id)[0];

Now we need to get the choices made by both the player and their partner. This code looks mostly familiar

  const playerChoice = player.round.get("decision");
  const partnerChoice = partner.round.get("decision");

Compute the payoff for each player

Once we know the choice that the player and their partner each made, we can compute the payoff for the player. In this case, we define a variable called score that we will assign a value (in months) based on both the player's choice and their partner's choice:

  let score;
  if (playerChoice === "testify" && partnerChoice === "testify") {
    score = 6;
  } else if (playerChoice === "testify" && partnerChoice === "silent") {
    score = 1;
  } else if (playerChoice === "silent" && partnerChoice === "testify") {
    score = 12;
  } else {
    score = 2;
  }

Now we need to save the score to the player.round object, so that is available for our Results component to display. This will be the last line inside our for loop:

  player.round.set("score", score);

Your complete code should now look like this:

import { ClassicListenersCollector } from "@empirica/core/admin/classic";
export const Empirica = new ClassicListenersCollector();

Empirica.onGameStart(({ game }) => {
  const round = game.addRound({
    name: `Round`,
  });
  round.addStage({ name: "choice", duration: 10000 });
  round.addStage({ name: "result", duration: 10000 });
});

Empirica.onRoundStart(({ round }) => {});

Empirica.onStageStart(({ stage }) => {});

Empirica.onStageEnded(({ stage }) => {
  if (stage.get("name") !== "choice") return;
  console.log("End of choice stage");

  const players = stage.currentGame.players;
  
  for (const player of players) {
    console.log("computing score for player ", player.id);
    const partner = players.filter((p) => p.id !== player.id)[0];
    const playerChoice = player.round.get("decision");
    const partnerChoice = partner.round.get("decision");

    let score;
    if (playerChoice === "testify" && partnerChoice === "testify") {
      score = 6;
    } else if (playerChoice === "testify" && partnerChoice === "silent") {
      score = 1;
    } else if (playerChoice === "silent" && partnerChoice === "testify") {
      score = 12;
    } else {
      score = 2;
    }
    player.round.set("score", score);
  }
});

Empirica.onRoundEnded(({ round }) => {});

Empirica.onGameEnded(({ game }) => {});

Restart and run through the experiment again

At this point, we have completed the coding for the basic prisoner's dilemma game. To restart your server and test the game, close the existing server by pressing <ctrl>+c in the terminal window, and then entering the following command:

rm .empirica/local/tajriba.json; empirica

When participants play multiple rounds of the prisoner's dilemma, knowing that they will have repeated opportunities to coordinate, we may expect them to behave more prosocially, in the hope that their partner will cooperate as well. We can test this by varying the number of rounds in our game.

The first thing we'll do is modify the "Treatments" file, found in .empirica/treatments.yaml. When we create a new empirica experiment from the template, the treatments file looks like this:

factors:
  - desc: playerCount determines the number of Players are in a Game.
    name: playerCount
    values:
      - value: 1
      - value: 2
      - value: 3
      - value: 5
      - value: 8
      - value: 13
treatments:
  - desc: "Single-player Game"
    factors:
      playerCount: 1
    name: Solo
  - desc: "Two-player Game"
    factors:
      playerCount: 2
    name: Two Players

The file is broken down into two sections: factors, and treatments. The factors section defines all of the ways that an experiment could vary - in the template experiment, there is only one dimension listed, the playerCount. The treatments section defines how different factors combine to create a particular treatment condition that a player will experience. This will become clearer as we go along.

Add a factor for the number of rounds

The first change we want to make is to simplify the factors and treatments that currently exist. Our current prisoner's dilemma experiment only makes sense with two players, so we can simplify the treatment file by cutting out the other options. (We still need to include playerCount to be in the file, however, or empirica won't know what our default should be.) We can also adjust the name and description of our baseline two-player game:

factors:
  - desc: playerCount determines the number of Players are in a Game.
    name: playerCount
    values:
      - value: 2
      
treatments:
  - desc: Standard one-shot prisoner's dilemma
    factors:
      playerCount: 2
    name: baseline

We can now add a factor for the number of rounds we want players to play. Indentation matters in yaml syntax, so this addition should be included directly beneath the playerCount factor, and parallel it exactly:

  - desc: How many times participants will make a choice
    name: numRounds
    values:
      - value: 1
      - value: 3
      - value: 7

Create treatments that vary the numRounds factor

Now we need to add this factor to the existing baseline treatment, by adding the new factor specification to the treatment:

treatments:
  - desc: Standard one-shot prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 1
    name: baseline
    

Now we can add an additional treatment underneath the baseline treatment for a short or long iterated game:

  - desc: A short iterated prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 3
    name: short_iterated
  - desc: A long iterated prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 7
    name: long_iterated

At this point, your treatment file should look like the following:

factors:
  - desc: playerCount determines the number of Players are in a Game.
    name: playerCount
    values:
      - value: 2
  - desc: How many times participants will make a choice
    name: numRounds
    values:
      - value: 1
      - value: 3
      - value: 7
      
treatments:
  - desc: Standard one-shot prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 1
    name: baseline
  - desc: A short iterated prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 3
    name: short_iterated
  - desc: A long iterated prisoner's dilemma
    factors:
      playerCount: 2
      numRounds: 7
    name: long_iterated

Access the new treatments variable inside the code

Our changes to the treatments file tell Empirica that we'd like to vary the number of rounds, but Empirica still doesn't know what to do with this information yet. We need to update our code to use this information in how it creates games.

As we're using the new variable to set the number of rounds, we'll want to get access to it in the onGameStart callback where we set up the round and stages of the game, in the file server/src/callbacks.js.

We'll add a line at the very top of the onGameStart callback that will give us access to the treatment object for this particular game:

  const treatment = game.get("treatment");

  const { numRounds } = treatment;

Create multiple rounds

Now that we have the numRounds variable, we can use it to set up multiple rounds of the game. We'll take our existing round and stage creation code and wrap it in a for loop. At the same time, we'll make sure each round gets a unique name:

  for (let i = 0; i < numRounds; i++) {
    const round = game.addRound({
      name: `Round ${i}`,
    });
    round.addStage({ name: "choice", duration: 10000 });
    round.addStage({ name: "result", duration: 10000 });
  }

Now, if you play through the game, when you press "continue" after the Result stage, you'll be taken to another round of the Choice stage.

Compute a cumulative score

As this is a multi-round game, we can compute a cumulative score that changes after each round. We'll set this cumulative score on the player object, as we want its value to persist across rounds in the game.

In the onStageEnded callback, after we set the player's score for the round, we can first get the current cumulative score as below:

  const currentScore = player.get("score") || 0;

Notice that here we're using the "or" operator || as before to provide a default value for currentScore. This is because in the first round of the game, there will be no existing value for score, and so the player.get(...) command will return undefined. Using the "or" operator means we can recognize this as the start-of-game state, and handle it appropriately.

We can now update the player's score to be the previous currentScore, plus the score for the round:

  player.set("score", score + currentScore);

Your complete code for server/src/callbacks.js should now look like this:

import { ClassicListenersCollector } from "@empirica/core/admin/classic";
export const Empirica = new ClassicListenersCollector();

Empirica.onGameStart(({ game }) => {
  const treatment = game.get("treatment");
  const { numRounds } = treatment;
  for (let i = 0; i < numRounds; i++) {
    const round = game.addRound({
      name: `Round ${i}`,
    });
    round.addStage({ name: "choice", duration: 10000 });
    round.addStage({ name: "result", duration: 10000 });
  }
});

Empirica.onRoundStart(({ round }) => {});

Empirica.onStageStart(({ stage }) => {});

Empirica.onStageEnded(({ stage }) => {
  if (stage.get("name") !== "choice") return;
  console.log("End of choice stage");

  const players = stage.currentGame.players;
  
  for (const player of players) {
    console.log("computing score for player ", player.id);
    const partner = players.filter((p) => p.id !== player.id)[0];
    const playerChoice = player.round.get("decision");
    const partnerChoice = partner.round.get("decision");

    let score;
    if (playerChoice === "testify" && partnerChoice === "testify") {
      score = 6;
    } else if (playerChoice === "testify" && partnerChoice === "silent") {
      score = 1;
    } else if (playerChoice === "silent" && partnerChoice === "testify") {
      score = 12;
    } else {
      score = 2;
    }
    player.round.set("score", score);
    const currentScore = player.get("score") || 0;
    player.set("score", score + currentScore);
  }
});

Empirica.onRoundEnded(({ round }) => {});

Empirica.onGameEnded(({ game }) => {});

Remember to restart the server to incorporate server-side code changes by pressing <ctrl>+c within the terminal window, and then using the command below to clear the database and restart the server:

rm .empirica/local/tajriba.json; empirica

When you go to the Empirca admin interface to create a batch, you have the option to select between the baseline and the short and long iterated prisoner's dilemma games. You can add multiple games to a batch, and as participants arrive they will be randomly assigned to a treatment within the batch. This helps you know that there are no systematic differences between the groups assigned to each condition. For more information, see:

Notice now that when you play through the game, the score listed in the upper corner of the timer bar updates with your cumulative score. This is because the default empirica experiment uses player.get("score") as its data source for this display. You can update how these components are visualized in client/src/Profile.jsx.

At this point, we'll get into a little bit more user interface design, by filling out the "Choice.jsx" placeholder we created earlier.

Describe the Prisoner's Dilemma in Text

First, we'll enter some text to describe the Prisoner's Dilemma to the participant in more detail. In our client/src/stages/Choice.jsx file, we can replace the "Choice component not yet implemented" text we had earlier with the following description, keeping the outer <div> ... </div> tags.

Note: we need to use parentheses to enclose the return statement when it spans more than one line.

  return (
    <div>
      <h2>🙀 You and your partner in crime have been arrested! 🙀</h2>
      <ul className="list-disc list-inside">
        <li>
          If you both 🤐🤐 keep silent, you will both get
          <strong>2 months</strong> in jail.
        </li>
        <li>
          If you 📣 testify against your partner and he 🤐 keeps silent,
          <strong>you will only spend 1 month in jail</strong>.
        </li>
        <li>
          If you 🤐 keep silent, but your partner 📣 testifies, you get 
          <strong>12 months</strong> in jail.
        </li>
        <li>
          If you both 📣📣 testify, you both get <strong>6 months</strong> in
          jail.
        </li>
      </ul>
      <br />
      <p>What do you do?</p>
    </div>
  );

Add buttons for each option

We also want to add a button for each choice. First we need to import the button component at the top of the file:

import { Button } from "../components/Button";

Now we can create the button components themselves. These will sit below the text, but within the outermost <div> tag. We will enclose them in additional <div></div> tags so that we can apply some styling to them independently of the rest of the page

      <div className="flex justify-center">
        <Button>🤐 Keep silent</Button>
        <Button>📣 Testify</Button>
      </div>

In addition to styling the <div> we can also apply style to the buttons themselves. In this case, we'll add 5 pixels of margin around each button, to separate them from one another and the remaining content, as below:

      <div className="flex justify-center">
        <Button className="m-5">🤐 Keep silent</Button>
        <Button className="m-5">📣 Testify</Button>
      </div>

Handle the button clicks

If you click the buttons at this point, nothing will happen, because we haven't "handled" the button click event. To do so, we need to create a new function that will be called when a button is clicked. This function will live inside the Choice component, prior to the return statement, and should take a single parameter that indicates which button was clicked. For now, all it will do is log the value of the parameter that was passed into it:

  function onClick (choice) {
    console.log("You chose:", choice)
  };

Now we need to make sure this function gets called when the button is clicked. If you look at the file in which the Button component is defined, you'll notice that it has a handleClick argument that accepts a function to be called when the button is clicked. We can't pass our onClick function directly into this argument, however, because we need to specify the value of the choice that will be passed to onClick.

() => onClick("silent")

defines a function we can pass along to our button component that will then call our onClick function with the choice "silent". Our code for the button components now should look like this:

<Button className="m-5" handleClick={() => onClick("silent")}>🤐 Keep silent</Button>
<Button className="m-5" handleClick={() => onClick("testify")}>📣 Testify</Button>

Now if you return to the browser, and click the buttons, you should see the name of the button output to the browser console.

Save the response

Now we need to save the value of the player's choice. We may want to have multiple rounds in the future, so we'll save it to the player.round object, which is unique to the player and to the round.

First we need to import the usePlayer hook, which will give us access to the player object. This belongs with the other import statements at the top of the file:

import { usePlayer } from "@empirica/core/player/classic/react";

Now we need to access the player object using the hook we just imported. Hooks need to be called first in a React component, so this statement should be the first line in the Choice function.

const player = usePlayer();

We want to save the value to the player.round object. Empirica provides a .set("key", value) function for this object (along with the player itself, the round, the stage, the game, etc). We can replace the console.log(...) statement in our onClick function with the following set statement:

player.round.set("decision", choice);

Submit the stage

The last thing we need to do is tell Empirica when the player clicks one of the buttons that they are ready to move onto the next stage. We can do this immediately after we set the value to the player.round object, inside the onClick callback function. We just need to set a submit value on the player.stage object to true:

player.stage.set("submit", true);

Now the Choice component is complete. Your full code should look like the following:

import React from "react";
import { Button } from "../components/Button";
import { usePlayer } from "@empirica/core/player/classic/react";

export function Choice() {
  const player = usePlayer();

  function onClick(choice) {
    player.round.set("decision", choice);
    player.stage.set("submit", true);
  }

  return (
    <div>
      <h2>🙀 You and your partner in crime have been arrested! 🙀</h2>
      <ul className="list-disc list-inside">
        <li>
          If you both 🤐🤐 keep silent, you will both get
          <strong>2 months</strong> in jail.
        </li>
        <li>
          If you 📣 testify against your partner and he 🤐 keeps silent,
          <strong>you will only spend 1 month in jail</strong>.
        </li>
        <li>
          If you 🤐 keep silent, but your partner 📣 testifies, you get
          <strong>12 months</strong> in jail.
        </li>
        <li>
          If you both 📣📣 testify, you both get <strong>6 months</strong> in
          jail.
        </li>
      </ul>
      <br />
      <p>What do you do?</p>

      <div className="flex justify-center">
        <Button className="m-5" handleClick={() => onClick("silent")}>
          🤐 Keep silent
        </Button>
        <Button className="m-5" handleClick={() => onClick("testify")}>
          📣 Testify
        </Button>
      </div>
    </div>
  );
}

You should now see the experiment behave as below:

Online tutorials

Reference material

Building outResult.jsx will be much like the Choice.jsxfile, but will give us an opportunity to use the .get() method on empirica objects, and explore pulling data from all of the players in the game.

Develop the text display

As in the Result component, we'll start by replacing the return statement with what we hope to actually show. For now we'll use XX, YY, and ZZ as placeholders for the values we will insert later:

return (
    <div>
      <p>You chose: XX</p>
      <p>Your partner chose: YY</p>
      <br />
      <p>You get ZZ months in jail!</p>
    </div>
  );

Our task now will be to collect the values for the player's choice, their opponent's choice, and the resulting score. Remember, these values were set based on the player's actions in the previous "Choice" stage.

Report the player's choice

In the "Choice" stage, we saved the player's choice to the player.round object. Here we can use the associated get method to include it in our display. First we need to import the usePlayer hook at the top of our client/src/stages/Result.jsx file:

import { usePlayer } from "@empirica/core/player/classic/react";

As before, we capture the player object at the top of the component function, before the return statement:

const player = usePlayer();

Finally, we can update the display code to insert a value from the player.round object:

<p>You chose: {player.round.get("decision")}</p>

Note that player.round.get(...) needs to be enclosed in curly braces, to distinguish that we are inserting a piece of Javascript into the display syntax.

If you refresh the page, you should now see the player's choice displayed.

Report the partner's choice

We will now add a new hook to the file, usePlayers, which returns a list of all player objects in the game. Include it in the import statement above:

import { usePlayer, usePlayers } from "@empirica/core/player/classic/react";

And as with usePlayer, set the result of the usePlayers hook to a constant at the top of the Result component function:

const players = usePlayers();

We need to identify which player in this list is the "partner" to the current player. We do this by filtering the players list for all players with different id values to the current player. As this is a two-person game, that list will contain exactly one element, and so we can set the first element of the filtered list to be the player's partner. This code needs to live before the return statement, but after we have defined the players object.

const partner = players.filter((p) => p.id !== player.id)[0];

Now that we have the partner object, we can display the value they set in the previous stage by replacing "YY" in our display with the appropriate get statement:

<p>Your partner chose: {partner.round.get("decision")}</p>

At this point, you should be able to see both your own and your partner's responses in the browser window.

Display the result

Now we need to display the computed result. We will actually compute this result in the next section (Part 4.6) so for now we'll use code to display the value once it has been completed. In the next section we'll save the player's score to the player.round object, so here we'll use the following code to get that result:

<p>You get {player.round.get("score")} months in jail!</p>

Unfortunately, right now there is no value set for "score" on the current player.round object, and so the get function returns undefined. This means that the above code will just display "You get months in jail!".

Instead, we'll use the "or" operator || to define a default value ("TBD") that will get inserted in the event that player.round.get(...) returns undefined:

<p>You get {player.round.get("score") || "TBD"} months in jail!</p>

Add a "continue" button

The last thing we want to do in this component is add a button to allow the player to continue on after they have read the result. Remember to import it at the top of the file:

import { Button } from "../components/Button";

As we only have one button in this page, we can add it simply at the end of the return statement, within the outermost <div>.

  <Button>
    Continue
  </Button>

We can build up our click handler inline, as it only needs to set the stage submission:

<Button handleClick={() => player.stage.set("submit", true)}>

Your final code should look like this:

import React from "react";
import { usePlayer, usePlayers } from "@empirica/core/player/classic/react";
import { Button } from "../components/Button";

export function Result() {
  const player = usePlayer();
  const players = usePlayers();
  const partner = players.filter((p) => p.id !== player.id)[0];

  return (
    <div>
      <p>You chose: {player.round.get("decision")}</p>
      <p>Your partner chose: {partner.round.get("decision")}</p>
      <br />
      <p>You get {player.round.get("score") || "TBD"} months in jail!</p>

      <Button handleClick={() => player.stage.set("submit", true)}>
        Continue
      </Button>
    </div>
  );
}

If you step all the way through the experiment at this point, you will be directed to the exit survey, which asks some default demographic questions and your thoughts about the game. At this point the "synchronous" portion of the game is over, and participants have as much time as they need to complete the experiment.

We won't modify the exit survey in this tutorial, but if you would like to do so, it can be found under client/src/intro-exit/ExitSurvey.jsx.

In the standard prisoner's dilemma experiment, participants are not able to communicate with one another. We've left the chat enabled from the template experiment up until this point, but now we should set up a flag to enable or disable it, so we can measure the effect of the chat on the outcomes of the game.

Create a factor and treatments

Let's return to the file .empirica/treatments.yaml. As we did when we changed the number of rounds, we'll create a factor for the chat:

We update our existing conditions, and create new ones with the chat in different states. This time, we'll set the default behavior of the chat to be disabled:

As we add additional manipulations to the game, the number of different treatments begins to grow rapidly. Managing this type of complexity is exactly what Empirica was built to accomplish. Imagine creating a separate setup for each of these different conditions, rather than being able to simply parameterize the factors!

Enable or disable chat in the player display

In order to make use of the chat feature, we'll make changes to client/src/Game.jsx. Notice in this file that the <Chat...> component is embedded in syntax that looks like this:

All we need to do at this point is replace the left-hand side condition with one that looks for chatEnabled. We have two changes to make - first to replace playerCount in the treatment object destructuring line with chatEnabled:

Then we need to use chatEnabled in the left-hand side of the short-circuit operation. As chatEnabled already returns a true or false value, we can just use it directly as the left-hand side expression:

Your full code for Game.jsx should now look like this:

Now when you restart empirica and play through a game, you'll be able to turn the chat on or off by selecting the appropriate treatment condition.

🎉 Congratulations 🎉, you've finished building your first experiment with Empirica! Now you need to deploy it to a server, so that you can send participants to it.

There are a number of ways to do this, including setting up your own servers on Amazon Web Services, DigitalOcean, or a similar platform. Check out the deployment documentation here:

I have a question about how to do something in Empirica, where can I ask it?

I have a bug/technical issue with Empirica, where should I mention it?

What is a React.js component?

A React.js component is the main building block of the frontend of your Empirica App. There are many tutorials online to help with your understanding of React.js.

In Empirica, we assign certain components for the Intro Steps, the Round, the Exit Steps, and a few other elements of the Game.

A component is composed of:

• states that affect what is rendered for the user, but that can also be changed by the user interacting with the rendered elements of the app.

• props provided from other components that can be used to determine what is rendered for the user.

• other components that it imports and builds into what it renders for the user.

• a render function that determines what is shown to the user with a mix of HTML (with <> tags) and JavaScript (with {}).

Each component is generally made into one .jsx file. Components can be imported into each other to build more complex components. Usually, the type of components used in Empirica are class-based.

A major perk of React.js components is that whenever one of their states change (e.g., because of the action of user), it will refresh every component affected and update what is rendered depending on the new states. This makes for web apps that live update what they look like and do. This is particularly useful for Empirica because you want to update what you show to players depending on states such as which responses they have given, which stage of the Game they are at, what other players are doing, etc.

Hence, there are some elements of a component you might want to render differently depending on certain props and states. You can use syntax such as { condition ? true : false }or { condition && true} where condition is a condition that is tested, true is what is rendered if this condition tests true, and false is what is rendered if this condition tests false.

A component might look like this:

How can I redirect a player if I detect they are using a certain browser or a mobile device?

To install react-device-detect use:

react-device-detect has different Booleans that you can import and use in one of your components to detect whether the player is using a certain browser.

• isMobile for whether they are using a mobile device

• isChrome for whether they are using Chrome

• isFirefox for whether they are using Firefox

• isSafari for whether they are using Safari

• ...

Import them into the component with:

For example, if you want to render a different consent form if the player is using a mobile device or is not using Chrome:

How can I show a different Exit Step to players depending on whether they have finished the game or if the game was cancelled/had a problem?

In the client/src/App.jsx you set which components form the Exit Steps with the exitSteps prop on EmpiricaContext. You can use the player.get("exitStatus") to separate out whether they have finished the game or if they were sent to the exit steps because the game was cancelled/had an issue and send them to different Exit Steps.

For example:

Can players navigate back and forth between the Exit Steps?

For now, players cannot navigate back and forth between the Exit Steps.

Players can only move from one Exit Step to the next if the component has an element (e.g., a button) that will call the next prop. For example:

If you wanted multiple pages within the Exit Steps that players can navigate through, you could create a component within one Exit Step that has different components to form "pages" and with a state that knows which page it is at and navigating to and from them.

Can a manually sent a player to an exit step?

You can have a piece of code to manually send a player to an exit stage if they do something (e.g., you want to give them a quit button) by using:

where the string is the name of the exit stage you want to send them to.

When are players playing synchronously and asynchronously? Can I modify this?

Players are asynchronous during the Intro Steps (i.e., they can finish each part at their own pace). At the lobby, players have to wait for each other to finish the Intro Steps. Then players are synchronous during the Game (i.e., they must all finish a Stage before they can move on to the next Stage or Round). Finally, once all the players have finished the Game, they are asynchronous again during the Exit Steps.

It is not recommended to try and modify this aspect of an Empirica app. You should consider what you want players to be doing at their own pace and what you want them to be doing synchronously. Remember that you can use your Treatments to modify how certain aspects of the game are presented throughout the app, and you can set different times for different Stages.

Can players leave and re-join the app? Can players refresh the page?

Yes, the browser cache of the player records they Player id and the database records the status and position of the Player, so it knows whether the Player is in the Intro Steps, the Lobby, a specific Round and Stage, or the Exit Steps. If a player refreshes the page or leaves and comes back, they will be sent back to their position.

However, remember that in Stages the timer is still ticking and during the Intro Steps other players might be waiting in the lobby; hence, it is not a good idea for players to leave the game.

It can be useful for players to know that they can safely refresh the page in case they have a graphical bug or if the page takes too long to render.

How can I assign players to different versions of my game based on a condition set outside of the game?

Imagine you have players that have different favourite colors, and you want to create games with one player for each favourite color. This is currently hard to make with Empirica because the players are allocated to a game and then they are asked questions, not the other way round.

Another solution is to create the groups outside of Empirica and invite them one group at a time to a game.

Can I have an idea of what my app would look like in production/deployment on my local machine?

When you launch your meteor app locally, it will look different to what it looks like when players see the deployed version. This is because there are tools that you only to have access to as the designer when the app is running locally on your machine (e.g., New Player, Reset current session, and Open Admin buttons).

If you want to see what your app will look like once deployed, but still run it locally, you can run the app with:

Commands

To find in which directory you currently are, use:

To list all the files and folders in your directory, use:

To navigate between directories use cd followed by the path or name of the directory you want to access. If you start typing a location that is present in your current directory, you can then use the TAB key to autocomplete the name of a directory.

To go up one directory, use:

To go back to the root (/home/<user>), use:

To delete a file, use:

To delete a directory, use:

Now that you have run multiple commands, and that you will be launching Empirica apps, there are a few useful tricks you should know:

• Use ctrl+l to clear your command line.

• Use ctrl+c to cancel (whilst writing it) or interrupt (when it is running) a command.

Tips for Windows WSL users

Viewing, accessing, and modifying your files

If you correctly installed WSL 2, you can view, access, and modify the files and folders in your WSL directly from your explorer. To open up the WSL in your explorer use:

Nevertheless, you will also need to get accustomed to using the command line to navigate in, and use, the the files and folders in your WSL directory. Here are some useful commands.

VS Code

Visual Studio Code (VS Code) is a cross-platform editor from Microsoft. It has a plethora of features, built-in tools, and plugins to help your write and debug your Empirica code. And it is particularly easy to install, launch, and use.