Skip to main content

Subject to change as the term progresses.

Date Topic Readings
Aug 27 Intro Lecture
  • Relationship between crowdsourcing, human computation, collective intelligence, data mining, social computing
  • Examples of crowdsourcing and human computation
Sep 01 Labor Day (no classes)
Sep 03 The Mechanical Turk crowdsourcing platform - part 1
  • Working on Mechanical Turk, demographics of Mechanical Turk workers, estimating the size of the Mechanical Turk Marketplace
  • Terminology and mechanics: Turkers, Requesters, HITs, micropayments
Sep 08 The Mechanical Turk crowdsourcing platform - part 2
  • Amazon Mechanical Turk from the Requester's perspective: Designing HITs, qualifications, pricing HITs, approving/rejecting
Sep 10 Taxonomy of crowdsourcing and human computation
  • Categorization system: motivation, quality control, aggregation, human skill, process flow
Sep 15 Machine Learning part 1
  • Examples of machine learning applications
  • Bayes Rule
  • Reasoning with Probabilities
Sep 17 Machine Learning - part 2
  • Labeled training data - what it is, where to get it, and how to put it in a format that a classifier will be able to use
  • Walkthrough of building a classifier in Python
  • We do the homework in front of your eyes
  • Experiment design - training / test splits, overfitting, baselines, evaluation
Sep 22 Programming concepts for human computation
  • People as function calls, decomposing complex tasks into simpler subtasks
  • Memoizataion of expensive function calls, "Crash and Re-Run"
  • Quicksort for kittens
Sep 24 Iterative and Parallel Processing
  • Experiments with TurKit
  • Iterative versus parallel processing
Sep 29 Firearm Violence: A Public Health Issue by Douglas Wiebe, Associate Professor of Epidemiology, University of Pennsylvania Perelman School of Medicine
  • Firearm injury as a public health issue
  • Importance of data in epidemiology
Oct 01 Crowdsourcing and human computer interaction (HCI) design by Ellie Pavlick (Instructor will be traveling)
  • Next generation interfaces
  • Soylent word processor ("it's made of people")
  • The Wisdom of Crowds: Chapter 6 ("Society Does Exist: Taxes, Tipping, Television and Trust")
  • Soylent: A Word Processor with a Crowd Inside by Michael Bernstein, Greg Little, Rob Miller, Björn Hartmann, Mark Ackerman, David Karger, David Crowell, and Katrina Panovich
Oct 06 Crowdsourcing and HCI part 2
  • E-mail Valet
  • Adrenaline
Oct 08 In-class presentations: Pitch your term project ideas
Oct 13 In-class presentations: Pitch your term project ideas
Oct 15 No class today. Instructor and TA will be traveling for a DARPA meeting.
Oct 20 Quality Control part 1
  • Agreement-based methods
  • Embedded gold standard
Oct 22 Quality Control part 2
  • Economic incentives
  • Reputation systems
Oct 27 Quality Control part 3
  • More on reputation systems
  • The EM algorithm
Oct 29 A/B Testing
  • Active versus passive crowdsourcing
  • Optimizing web sites using A/B testing
Nov 03 Prediction Markets by Ellie Pavlick
Nov 05 Distilling Collective Intelligence from Twitter part 1
  • Topic Detection and Tracking
  • Approximate algorithms for scaling to large data sets
Nov 10 Distilling Collective Intelligence from Twitter part 2
  • Public health and Twitter
Nov 12 Detecting Emotional Contagion in Massive Social Networks
  • Human subjects research
  • Privacy
  • Are you part of a social experiment?
Nov 17 Crowdsourcing Translation
  • The language demographics of Mechanical Turk
  • Using crowdsourcing to collect data to train SMT systems
Nov 19 Crowdsourcing for Automatic Speech Recognition
  • A high level overview of automatic speech recognition
  • Using crowdsourcing to train ASR systems
Nov 24 What big data can tell us about sex and relationships
Nov 26 No class (Friday class schedule today)
Dec 01 TBD
Dec 03 Personality, Gender and Age in the Language of Social Media by Andy Schwartz
Dec 08 Course wrap-up
Dec 15 Final Presentations from 3pm-5pm in Berger Auditorium - room 101 in Skirkanich Hall