Skip to main content

Subject to change as the term progresses.

Date Topic Readings
Aug 27 Intro Lecture
  • Relationship between crowdsourcing, human computation, collective intelligence, data mining, social computing
  • Examples of crowdsourcing and human computation
Sep 03 The Mechanical Turk crowdsourcing platform - part 1
  • Working on Mechanical Turk, demographics of Mechanical Turk workers, estimating the size of the Mechanical Turk Marketplace
  • Terminology and mechanics: Turkers, Requesters, HITs, micropayments
  • The Wisdom of Crowds: Chapter 2 ("The Difference Difference Makes")
  • The Demographics of MTurk by Panos Ipeirotis
  • Being A Turker by David Martin, Benjamin Hanrahan, Jacki O’Neill and Neha Gupta
Sep 08 The Mechanical Turk crowdsourcing platform - part 2
  • Amazon Mechanical Turk from the Requester's perspective: Designing HITs, qualifications, pricing HITs, approving/rejecting
Sep 10 Company Profile: In-Trade by Ellie Pavlick
Sep 10 Taxonomy of crowdsourcing and human computation
  • Categorization system: motivation, quality control, aggregation, human skill, process flow
Sep 15 Programming concepts for human computation
  • People as function calls, decomposing complex tasks into simpler subtasks
  • Memoizataion of expensive function calls, "Crash and Re-Run"
  • Quicksort for kittens
Sep 22 Crowdsourcing and human computer interaction (HCI) design
  • Next generation interfaces
  • Soylent word processor ("it's made of people")
  • The Wisdom of Crowds: Chapter 6 ("Society Does Exist: Taxes, Tipping, Television and Trust")
  • Soylent: A Word Processor with a Crowd Inside by Michael Bernstein, Greg Little, Rob Miller, Björn Hartmann, Mark Ackerman, David Karger, David Crowell, and Katrina Panovich
Sep 24 Crowdsourcing and HCI part 2
  • E-mail Valet
  • Adrenaline
Sep 22 Crowdsourcing and human computer interaction (HCI) design
  • Next generation interfaces
  • Soylent word processor ("it's made of people")
  • The Wisdom of Crowds: Chapter 6 ("Society Does Exist: Taxes, Tipping, Television and Trust")
  • Soylent: A Word Processor with a Crowd Inside by Michael Bernstein, Greg Little, Rob Miller, Björn Hartmann, Mark Ackerman, David Karger, David Crowell, and Katrina Panovich
Sep 29 Quality Control part 1
  • Agreement-based methods
  • Embedded gold standard
Oct 01 Quality Control part 2
  • Reputation systems
  • Economic incentives
Oct 06 Quality Control part 3
  • The EM algorithm
Oct 08 In-class presentations: Pitch your term project ideas
Oct 13 In-class presentations: Pitch your term project ideas
Oct 14 Machine Learning part 1
  • Examples of machine learning applications
  • Naive Bayes
  • Text classification
  • The Wisdom of Crowds: Chapter 9 ("Committees, Juries and Teams: The Columbia Disaster and how Small Groups can be Made to Work")
  • Intro to Machine Learning in Python by Hilary Mason
Oct 20 Machine Learning part 2
  • A high level perspective on how machine learning works and why it can fail
Oct 22 Machine Learning part 3
  • Integrating machine learning and crowdsourcing
Oct 27 Distilling Collective Intelligence from Twitter part 1
  • Topic Detection and Tracking
  • Approximate algorithms for scaling to large data sets
Oct 29 Distilling Collective Intelligence from Twitter part 2 by Michael Paul
  • Public health and Twitter
Nov 05 AskSheet: Efficient Human Computation for Decision Making with Spreadsheets by Alex Quinn
Nov 10 A/B Testing
  • Active versus passive crowdsourcing
  • Optimizing web sites using A/B testing
Nov 12 Prediction Markets by Ellie Pavlick
Nov 17 Crowdsourcing Translation
  • The language demographics of Mechanical Turk
  • Using crowdsourcing to collect data to train SMT systems
Nov 19 Crowdsourcing for Automatic Speech Recognition by Scott Novotney
  • A high level overview of automatic speech recognition
  • Using crowdsourcing to train ASR systems
Nov 19 What big data can tell us about sex and relationships
Dec 01 Review of machine learning homework by Ellie Pavlick
Dec 03 Personality, Gender and Age in the Language of Social Media by Andy Schwartz
Dec 08 Course wrap-up