Skip to main content
Warning: this assignment is out of date. It may still need to be updated for this year's class. Check with your instructor before you start working on this assignment.
This assignment is before 11:59PM due on Monday, April 18, 2016.

Final Project : Part 3

The final project is due on Thursday May 5th, 2016. By that time, your system will need to be fully operational, and you’ll have to have collected enough data to do an interesting analysis. We recommend looking over the questions on the final questionnaire, so that you get a sense of what is expected of the final project. You should pay attention to what sorts of analyses you can do for your final project.

For your final project, you can collect data by employing a real crowd, or by having your classmates simulate the crowd. For this part of the project, you will finish a working version of your system, so that your classmates can help you as a simulated crowd, with the goals of debugging your system before you turn the real crowd loose on it, and/or creating some data that you can use to analyze for your final project.

Your classmates will make contributions before the last day of classes, and will get credit toward their participation grade for their contributions.

What you should do to prepare

  • Get a working prototype ready
  • Be sure that it is capable of collecting data from the crowd.
  • Write a step-by-step README that walks your classmates through how to use your system, including all details about how to access it and what you want them to do. If you are using a crowdsourcing platform to collect your crowd contributions, then you should set up your project to be running on the Mechanical Turk Worker Sandbox. If you are using CrowdFlower, you should launch your HIT to the “Internal Crowd.” You can do this by going to “Contributors” and then to the “Channels” tab, and turning off “On-Demand Workforce.” Then you can follow the link at the bottom of the dashboard to test your own HIT.
  • Put together a short video tutorial for a new member of the crowd (like one of your classmates) shows you using the system, and explains to them how you would like them to make contributions to your project
  • Think about what sort of analysis you want to do.
    • Do you want to compare your system against an existing non-crowdsourcing method?
    • Do you want to analyze how accurate the crowd is at making predictions?
    • Do you want to test how well your method could scale in terms of time/money?
  • Make sure that your capturing the appropriate information so that you can do your analysis.

What to submit

Please fill out this questionnaire. It asks for the following:

  • Details about your team members
  • Name of your project
  • A one sentence description of your project.
  • A logo for your project (PNG format, stored on github)
  • What type of project is it?
    • Human computation algorithm
    • Social science experiment with the crowd
    • A tool for crowdsourcing
    • A business idea that uses crowdsourcing
    • Other
  • Provide a link to your training video for new members of your crowd (Vimeo link, and password details)
  • A written step-by-step guide for how your classmates should make contributions to your project. This should include where to access it, and who to contact if it isn’t working or if they have any questions.
  • A link to your github repo for the code that you use to run your system
  • A README describing how the code runs

Grading Rubric

This assignment is worth 7 points of your overall grade in the course. The rubric for the assignment is given below.

  • 5 points - have a working prototype with sufficient documentation so that the course staff can verify that it’s working
    • 2 points - allowing us to make a submission in the way that a member of the crowd would
    • 2 points - successfully implemented features/milestones set in README
    • 1 point - documentation for code relevant to QC, aggregation, and all supporting code
  • 1 point - for your training video for new members of your crowd
  • 1 point - for clearly written step-by-step instructions on how they can make contributions