Tutorial on Big Data and Online Crowd-Sourcing for Vision Research

Creating Big DataSets

Vision Sciences Society Satellite Event, Friday, May 18, 2018.
Organizer: Wilma A. Bainbridge, NIMH
Speakers: Tim Brady (UCSD), Gijsbert Stoet (Leeds Beckett University), Wilma Bainbridge (NIMH), Dwight Kravitz (George Washington University)

Online experiments and Big Data are becoming big topics in the field of vision science, but can be hard to access for people not familiar with web development and coding. This tutorial will teach attendees the basics of creating online crowd-sourced experiments, and how to think about collecting and analyzing Big Data related to vision research. Four experts in the field will discuss how they use and collect Big Data, and give hands-on practice to tutorial attendees. We will discuss Amazon Mechanical Turk, its strengths and weaknesses, and how to leverage it in creative ways to collect powerful, large-scale data. We will then discuss Psytoolkit, an online experimental platform for coding timed behavioral and psychophysical tasks, that can integrate with Amazon Mechanical Turk. We will then discuss how to create Big Datasets using various ways of “scraping” large-scale data from the internet. Finally, we will discuss other sources of useful crowd-sourced data, such as performance on mobile games, and methods for scaling down and analyzing these large data sets.

Here, the speakers have made material related to the course available.

Tim Brady - Introduction to Amazon Mechanical Turk
8:30am - 9:20am
The slides for his talk can be found here. You can view his previous tutorials and example scripts on Amazon Mechanical Turk here.

Gijsbert Stoet - Introduction to Psytoolkit
9:20am - 10:10am
You can watch his tutorial on Psytoolkit here, and learn specifically about how you can integrate Psytoolkit with Amazon Mechanical Turk here. Learn more about Psytoolkit and access their website here.

Wilma Bainbridge - Creating Big Datasets and Data-scraping
You can view the slides, demonstrations, example code, and tutorial materials from her talk .

Dwight Kravitz - Crowd-sourcing Data: Practical Lessons and Potential Applications
The slides for his talk can be found here.

Disclaimer: I collect minor usage statistics (geographic location and time) for downloads of my tutorial materials.