I recently had a talk about the state of open-source programmable systems for generating experiments for behavioral research. I decided to write up a series of blog posts capturing the most important points of my presentation, involving introductions on a few browser-based systems for creating experiments, a discussion on the challenges and benefits of these web-based solutions, and how the development of programmable behavioral experiments could be improved.

So, what kind of experiments are we talking about here? Experiments for behavioural research, such as tasks for assessing cognitive functions, consist of simple tests that can be used to evaluate different aspects of a respondent’s psychological functions. Examples of such function are processing speed, simple or complex reaction times, working memory capacity, task switching and so on. These tasks are often computerized, and as they have gained popularity and are relatively easy to implement, a miscellaneous set of solutions has emerged as academics and enthusiasts develop their own solutions according to their own needs, skills, preferences and resources. These solutions range from readily available, extensive and customizable test batteries in a “custom” language (e.g. PEBL) to more loosely defined software that may provide a GUI as well as the option to program some of the code (e.g. Psychopy). Several more or less entirely graphical, commercial systems exist, as well as a large set of initially promising but nearly dead open-source projects that have gotten left behind for a reason or another (unfortunately including the rather promising PXLab project after the developer passed away).

When it comes to web experiments, or experiments that the participant could run through a browser without having to download anything or going to a lab to participate, a rather short list of options exists. I found only a few non-commercial systems for creating web experiments (WebExp, PXLab and jsPsych), each with their own setbacks and benefits. Systems for web experiments come in a smaller supply, they have gone through less research and development than programmatic offline experiment systems, and they involve a set of problems that can be relatively successfully mitigated by lab experimentation. But even so, there are still scenarios where one might want to take the risk and choose highly optimized web experimentation ahead of laboratory experiments, especially since with developments in technology, many of these shortcomings become less and less significant.

After this short introduction, I invite you to read the next blog post, where I will discuss more in detail the setbacks and benefits of web-based systems.