Q & A with October 2015 DLABSS Gift Card Winner

The latest winner of our monthly $50 Amazon gift card lottery agreed to answer a few questions about his experience working with DLABSS on multiple experiments. We have posted his responses below:

1. How did you first hear about Harvard DLABSS, and why did you decide to participate for the first time?

I believe I heard about DLABSS in a craigslist posting. I decided to participate because I thought that the subject matter seemed interesting and I could voice my opinion on the matter. It seemed a good way to give back to the data on my community. 

2. Why have you continued to participate in experiments on DLABSS?

I continued participating in the studies because there was a big breadth of topics that I could take part in, which made it very interesting. A big part of the reason why I took some time to do surveys was the chance to be part of social science research, plus most surveys were not too cumbersome or long. 

3. What is one thing you have learned from one of the experiments you participated in?

I learned a whole lot more about US presidents than I ever thought I would! Some of the surveys asked about political ideologies that I was not too knowledgeable about, so afterwards I would look them up online.

Also, I learned that survey development can be a complex thing!

4. In your opinion, what is the best feature of DLABSS and/or its website?

I think the best, and most unique, feature would be being able to take part of this master database that offers participants a chance to be part of different surveys and areas of science. Moreover, being able to receive notices about surveys is great!

5. What about DLABSS do you think could be improved for experiment participants?

I was sometimes confused when the emails mentioned I had completed X% of surveys, and maybe language could be a bit clearer when talking about those. Furthermore, I tended to get duplicate emails a lot of the times.

6. If you were talking to someone considering participating in DLABSS for the first time, how would you describe your overall experience?

I'd say it seems to be an easy way to take part of surveys and research within the comfort of your own personal space; usually [experiments] are very interesting and timely which helps for those of us who are busy. An added bonus is the chance to be able to win a gift certificate.  

Time or Space? Experimental Research on Self Control and Commute Tradeoffs

DLABSS Note: This is a guest post by Julia Lee, currently a postdoc fellow at the University of Michigan's Ross School of Business. Previously she was a doctoral student at the Harvard Kennedy School of Government as well as a lab fellow at Harvard University's Edmond J. Safra Center for Ethics. We'll be periodically checking in with researchers running survey experiments with us to gauge their experience with DLABSS.

I have been interested in the seemingly-irrelevant factors that may influence our motivation and cognition at work.  For example, some of my research looks into how weather can influence worker productivity despite the fact that it is often neglected as an important factor when people think and make plans about how much work they can get done. Recently my colleagues and I began exploring a similar puzzle using a survey experiment on DLABSS.   

My collaborators (Jon JachimowiczBradley StaatsFrancesca Gino, and Jochen Menges) and I were interested in how people strategize about their daily commutes.  People spend not-so-insignificant amounts of time commuting to and from work in a given day, and we already know that long commutes in heavy traffic can have negative effects on work-related attitudes and behavior, such as lower job satisfaction and exhaustion.  Nevertheless, when people are asked to consider two different housing options, their choices rarely seem to factor in the negative impact of commuting. According to Dutch psychologist Ap Dijksterhuis’s thought experiment, if offered a 3-bedroom apartment with 10-minute commute time and a 5-bedroom house with a 45-minute commute, many will choose the 5-bedroom house because they underestimate the pains of a lengthy commute.

We set out to test this commuting paradox with participants recruited from DLABSS, with a twist — we wanted to see if this relationship is likely to change based on the individual levels of self-control. Our preliminary results suggested that self-control is an important factor that can potentially influence how different people make commute-related decisions. Individuals high in trait self-control were more likely to choose an apartment that was closer to work, while those who rated low in trait self-control were more likely to choose a house that was farther away from work.  Similarly, those high in trait self-control were more likely to choose a job that was geographically close but paid less, while those low in trait self-control were more likely to choose a job that paid more, but required a longer commute.  

We are currently running a few follow-up studies to examine why people who have different levels of trait self-control make different decisions when facing tradeoffs between commuting and space, salary or other considerations.

Q & A with September 2015 Gift Card Winner

The latest winner of our monthly $50 Amazon gift card raffle was kind enough to answer a few questions about her experience working with DLABSS. We have posted her responses below:

How did you first hear about Harvard DLABSS, and why did you decide to participate for the first time?

Honestly, I just got an email one day asking me to take a survey. So I signed up and did it, and I liked participating so I kept doing them whenever I got an email.

Why did you continue to participate in experiments on DLABSS?

I know they're helpful to the group who puts them together. They spend a lot of time putting them together and they need the results, so it's easy for me to keep doing them.

What is one thing you learned from one of the experiments you participated in?

Honestly, I've learned I don't know a lot about politics, which is a problem and I need to brush up!

In your opinion, what is the best feature of DLABSS and/or its website?

The surveys are really easy to fill out, and I always get an email telling me when there is a new experiment, so I don't have to go searching for them.

If you were talking to someone considering participating in DLABSS for the first time, how would you describe your overall experience?

It was pretty fun for me. I enjoy filling out surveys in general, and knowing the results of my survey actually helped a group of people so it made it a more pleasant experience.

Q&A with Another DLABSS Gift Card Winner

DLABSS was lucky enough to have another one of our Amazon gift card winners complete a brief interview so their experiences can be shared with the DLABSS community! We have posted our interview questions and Diana’s answers below. 

Please tell us a little about yourself such as where you are from, your occupation, and anything else relevant about yourself that you would like to include.

I am a native New Yorker, a former English professor and current freelance tutor/editor/translator. 

How did you first hear about Harvard DLABSS, and why did you decide to participate for the first time? 

Perhaps I found you on Craigslist. I enjoy answering questionnaires.

Why did you continue to participate in experiments on DLABSS? 

I enjoy answering questionnaires.

What is one thing you learned from one of the experiments you participated in? 

I can earn money from answering questionnaires. 

In your opinion, what is the best feature of DLABSS and/or its website? 

[It’s] affiliation with Harvard

If you were talking to someone considering participating in DLABSS for the first time, how would you describe your overall experience?  

Interesting, not too long per questionnaire, and possibly lucrative.

Q & A with our Second Gift Card Winner

Last month another DLABSS participant won an Amazon gift card from our monthly raffle! As our second winner, we wanted to follow up with a few questions for Jason to learn what DLABSS' participants like him are thinking. Our questions and Jason’s answers are below.  

How did you first hear about Harvard DLABSS, and why did you decide to participate for the first time?

“I first heard about Harvard DLABSS [through] Craigslist. I personally enjoy surveys that contribute to real life issues, as opposed to consumer products or something similar to that.”

Why did you continue to participate in experiments on DLABSS?

“What compels me to continue participating in DLABSS experiments is the variety of issues. Every time they send me an e-mail inviting me to take a survey its always a surprise as to what the subject matter will be. This gives me an opportunity to answer in a fresh state of mind and also learn something new internally and externally.”

What is one thing you learned from one of the experiments in which you participated?

“One thing I have learned is how our environment can influence our subconscious mind and contribute to preconceived notions and judgements.”

In your opinion, what is the best feature of DLABSS and/or its website?

“The automated e-mail system they use is very convenient. On average I receive one e-mail every two weeks, the surveys are for the most part very brief and user friendly.” 

If you were talking to someone considering participating in DLABSS for the first time, how would you describe your overall experience?

“It’s a fun easy way to voice your opinions, and help Harvard students get their assignments completed. There's also a chance you could win a $50 gift card to Amazon, which I have won ; )”

MTurk Results have been Replicated by DLABSS!

Professor Enos has successfully replicated the results of a study performed in Mechanical Turk using DLABSS. This means that DLABSS could be a suitable replacement for MTurk as a way of gathering data, and is great news for researchers currently using DLABSS or researchers considering using DLABSS in the future. The report written by Professor Enos explaining his findings is below:

    DLABSS has successfully replicated several studies in Mechanical Turk. By replicating studies—meaning the results obtained in DLABSS and Mechanical Turk were substantively the same—we are demonstrating that volunteers of DLABSS can potentially substitute for Mechanical Turk. Mechanical Turk (MTurk) is currently the primary source of online subjects among social science researchers, so this signals significant potential for DLABSS as a tool for researchers. In this post, I describe the details of one of those replications.

    In this study, a call for volunteers was posted on DLABSS and a very similar advertisement was posted on MTurk. I paid subjects $1 to complete the study on MTurk. The study took the average subject about 9 minutes to complete. I collected similar demographics on MTurk and DLABSS, so we can compare the differences between a volunteer and paid sample. These basic demographics are displayed in the table below. The DLABSS sample looks largely similar to the MTurk sample, however it is more liberal and better educated. Of course, one of the primary advantages of DLABSS is that subjects can be easily targeted, so that if a researcher wants, for example, fewer college graduates, this can be easily obtained.

    In this particular study, I asked people to judge the appearance of faces. I was interested in whether subjects thought that these faces looked more like the face of an African American person or more like the face of a Caucasian person. I showed them groups of faces for five seconds. One particular face was highlighted and I asked subjects to judge the appearance on a 7-point scale from ”Completely African American” (1) to ”Completely Caucasian” (7).

    The test of interest is that subjects were shown three conditions and in each condition asked to judge the same face. In two conditions, the faces on the screen were segregated by race (white faces separated from Black faces). The highlighted face, which the subjects were asked to judge, could be grouped next to white faces or Black faces (see image below). In the other condition, the faces were integrated by race (white faces and black faces together).

    My hypothesis is that subjects will use segregation as a heuristic in judging the faces, so that when the face is segregated and grouped with Black faces that subjects will say the face is more African American, when it is grouped with white faces, subjects will say it is more Caucasian, and when it is integrated, subjects will be more likely to say it evenly split in appearance.

    The figure below shows that this was the result in both Mechanical Turk (the red bars) and DLABSS (the gray bars). The bars represent differences in judgments of the faces between the integrated and segregated conditions. Negative numbers mean more African American, and positive numbers are more Caucasian. The Black segregated faces were judged to be more African American and the white segregated faces were judged to be more Caucasian than the baseline integrated condition. A T-test for a difference of means between conditions yields p < .05 in Mechanical Turk and p < .01 in DLABSS.

    Of course, the average differences between conditions are not the same: -.11 in MTurk and -.24 in DLABSS—but exact replication is rare in social science. The important takeaway though is that a researcher using either MTurk or DLABSS would have come to same conclusion from this data—indicating that DLABSS is a worthwhile replacement for MTurk.


Q & A with the First Gift Card Raffle Winner

Last month was the first month a DLABSS participant won an Amazon gift card from our monthly raffle! Because he was our first winner, we sent Matthew a few questions to answer so everyone could learn a little bit more about the individuals that contribute to the success of DLABSS. Our questions and Matthew’s answers are below.  

How did you first hear about Harvard DLABSS, and why did you decide to participate for the first time?

“I saw an ad for volunteers to take surveys. My degree is in psych and I remember what it was like looking for [participants] to take our surveys and experiments.”

Why did you continue to participate in experiments on DLABSS?

“It's actually fun to take these surveys.”

What is one thing you learned from one of the experiments in which you participated?

“One never really knows [what] the other has in mind. With these surveys you may think you know what they are about but it can really surprise you to see where they are taking these questions.”

In your opinion, what is the best feature of DLABSS and/or its website?

“You get to participate in helping generate data that someone thinks is significant.” 

If you were talking to someone considering participating in DLABSS for the first time, how would you describe your overall experience?

“If you want to share an opinion on things you may never have even thought about it, go for it!”

A Visualization of DLABSS' Growth Since its Start

Since its start in August, Harvard Digital Lab for the Social Sciences has experienced substantial growth in the number of active participants, researchers, and experiments. The time plot above provides a visual representation of the progress DLABSS has made since it began. The black line shows the total number of individual participants that have participated in an experiment hosted on DLABSS has surpassed 1000. The different colored lines each represent a different experiment hosted by DLABSS, and the number of participants that have participated over time.

As you can see, the number of experiments has continued to increase, which offers participants a diverse pool of interesting studies that focus on many different topics. Also the longer DLABSS has been active, the faster researchers are provided with responses for their studies. The experiments posted more recently have exhibited a much faster increase in participants compared to earlier experiments. This is great news for researchers who are looking to get quick feedback on their experiments. We would like to thank the researchers and participants that have helped make DLABSS so successful this fall.

DLABSS is implementing a gift card raffle this month!

We at DLABSS understand that all of our participants voluntarily donate their time to help our cause and further Harvard University research. Thanks to our valued participants, DLABSS has gotten off to a successful start this semester. To show our appreciation we will begin a new system this month that raffles off a $50 Amazon gift card once every month. Each time you complete an experiment your email will be entered into the raffle. This means the more experiments that you complete, the greater the chance you have of winning. The winner will receive a digital gift card by email. There is no limit on the number of experiments you can participate in, except you cannot participate in the same one twice. With your continued participation we can make the next semester even more successful!

DLABSS has made News and was Published in the Harvard Gazette

Since our last update, DLABSS has added three new experiments by Professor Ryan EnosProfessor Julia Minson, and Professor Gwyneth McClendon. We are happy to see that our collection of experiments continues to grow as DLABSS progresses. Expect more experiments to be posted soon! Also Harvard DLABSS has recently appeared in the Harvard Gazette! Check out the story here if you want to know a little bit more about us or are interested in the main goal of DLABSS, how DLABSS came to be, and where we expect it to go in the future. We thank everyone who has helped to get DLABSS where it is today and expect to progress even more in the coming months!

An Update as DLABSS Continues to Grow

Greeting from DLABSS! Research is underway as several researchers have posted experiments. The newest study, MID Crowd by Dr. Vito D'Orazio, is now available for participants to take and contribute to Harvard's research. Researchers are showing continued interest in having their experiments posted on DLABSS. We are currently expanding our pool of studies to accommodate more areas of interest within the social sciences, and we are excited about the valuable research that DLABSS can offer this academic year. The participant pool is also continuing to grow as more and more people find out about the opportunity to be involved in Harvard research. Stay tuned to see what exciting experiments we are hosting next!

Official Launch of the Harvard Digital Lab for the Social Sciences!

We are happy to announce the official launch of the Harvard Digital Lab for the Social Sciences (DLABSS). During the month of August, we plan to build a sizable pool of study participants and begin facilitating research projects. Participants can now sign up on our Master Survey, and see all experiments currently posted! As the DLABSS grows this summer and during the next academic year, we will blog approximately once a month with updates. We will also post results of studies affiliated with DLABSS as these become available.