Skip to main content

Crowdsourcing and human computation are emerging fields that sit squarely at the intersection of economics and computer science. They examine how people can be used to solve complex tasks that are currently beyond the capabilities of artificial intelligence algorithms. Online marketplaces like Mechanical Turk provide an infrastructure that allows micropayments to be given to people in return for completing human intelligence tasks. This opens up previously unthinkable possibilities like people being used as function calls in software. We will investigate how crowdsourcing can be used for computer science applications like machine learning, next-generation interfaces, and data mining. Beyond these computer science aspects, we will also delve into topics like the sharing economy, prediction markets, how businesses can capitalize on collective intelligence, and the fundamental principles that underlie democracy and other group decision-making processes.

Homework 7 "Quality control" has been released. It is due before 11:59PM on Thursday, April 15, 2021.
Final Project Part 2 "Quality Control and Aggregation" has been released. The assignment has multiple deliverables. The first part is due before 11:59PM on Tuesday, April 13, 2021. The second part is due before 11:59PM on Tuesday, April 20, 2021.
Final Project part 3 "Working Prototype" has been released. It is due before 11:59PM on Tuesday, April 27, 2021.
Final Project part 4 "Preliminary Analysis" has been released. It is due before 11:59PM on Thursday, April 29, 2021.
Presentation and Final Report has been released. It is due before 12:00PM on Tuesday, May 11, 2021.
Course number
NETS 213 - students from all majors are welcome!
Instructor
Chris Callison-Burch
Teaching Assistants
Course Staff
Website
http://crowdsourcing-class.org/
Discussion Forum
Piazza
Time and place
Spring 2021, Tuesdays and Thursdays 3-4:30pm Eastern live via Zoom
The first day of class will be Thursday January 21, 2021.
The final day of class will be Thursday April 29, 2021.
Office Hours
To join the office hour queue, please go to OHQ.io.
Monday 10:30am-noon Eastern - Anna
Monday 6pm-8pm Eastern – Ashley
Tuesday 2pm-3pm Eastern - Bryan
Wednesday 9:30am-11:30am Eastern - Zach
Wednesday 1pm-3pm Eastern - Jason
Wednesday 3pm-4:30pm Eastern - CCB
Wednesday 7pm-9pm Eastern - Joe
Wednesday 9pm-10pm Eastern - Sarah
Thursday 8am-9:30am Eastern - Anka
Thursday 9:30am-10:30am Eastern - Zach
Thursday noon-2pm Eastern - Eddie
Thursday 6pm-9pm Eastern - David
Friday 10:30am-noon Eastern - Anna
Friday 2pm-3pm Eastern - Bryan
Friday 9pm-10pm Eastern - Sarah
Saturday 10:30am-12:30pm Eastern - Megha
Sunday 1pm-3pm Eastern - Ajay
Prerequisites
CIS 120 or prior programming experience.
Course Readings
Each lecture has an accompanying set of academic papers
Grading
This is a project-based course. Instead of exams, you will do a series of hands-on assignments and a final project.
  • Weekly assignments (50%)
  • Final project (45%)
  • Participation (5%)
Late day policy
Each student has ten free “late days”. Homeworks can be submitted at most two days late. If you are out of late days, then you will not be able to submit your homework. One “day” is defined as anytime between 1 second and 24 hours after the homework deadline.
Regrade Policy
If you have a regrade request, please submit regrade requests to Gradescope not private posts on Piazza. If you submit the regrade request to Gradescope, it makes it easier and faster for us to re-grade your homework. If you need to attach files, you can provide public shared links to files on your Google Drive. Regrading will typically be available for a 2 week period following the release of an assignment’s grade, so please don’t wait until the end of the semester to submit a regrade request.
Course materials
Students should expect to spend around $50-$100 of their own money on MTurk or other crowdsourcing platforms. If this will cause you undue financial hardship, please let the instructor know.