Final Project : Part 1

Here are a few final project ideas. You are welcome to adapt one of these ideas into your final project, or to come up with your own idea. My expectation is that your final project will represent a substantial amount of work, and that it will be something that you’re proud of and that you would like to show off to potential employers or to graduate schools.

Text Adventure Games (suggested by CCB and Daphne Ippolito)

Long before video games had the amazing graphics they have now, there existed text adventure games like Zork or The Hitchhiker’s Guide to the Galaxy. Text adventure games were also known as Interactive Fiction. There was a recent paper from Facebook Research that used MTurk to create data to help Learning to Speak and Act in a Fantasy Text Adventure Game. The Facebook project is probably a better example of a Multi-User Dungeon (MUD) than interactive finction. Your project could create data in a similar fashion to the Facebook paper, with some augmentations suggested by CCB and his PhD student Daphne.

Roll Call

Design a crowdsourced app that can collect the contact information for all legislators in a state and poll their offices to see where each of them stand on a piece of upcoming legislation. For instance, you could use the app to poll all members of the PA house and the PA senate about whether or not they would support the National Popular Vote Interstate Compact.

Crowdsourced Matchmaking

Come up with a human computation algorithm that helps people find a better match in online dating. Some people have tried to use machine learning or crowdsourcing to optimize their dating experience on OKCupid. Can you come up with a better way of matching people up via crowdsourcing? Maybe you can have the crowd act as Cyrano de Bergerac, feeding users better lines than they could think of themselves. Maybe you could have people in a social network nominate people who they think would be good matches.

Translate Children’s Books

The International Children’s Digital Library is a collection of children’s books from around the the world. Volunteer translators have translated a subset of their books into different languages. We could try to translate many more of the books using crowdsourcing. There could be different tasks for monolingual speakers and bilingual speakers. Monolinguals could transcribe the text of the books (which is usually embedded in images). Bilinguals could translate it. Monolinguals could edit the translations.

Crowdsourcing and Creativity

Create a human computation algorithm to convert prose into poetry. Your algorithm should model two aspects of poetry rhyme and meter. NLP researchers have been working on text-to-text generation algorithms that can rewrite sentences in many different ways. This software can generate a huge number of alternatives, some of which may fit the constraints of a poem. However, the software is currently poor at determining which of the generated sentences are grammatical versus ungrammatical, and which correctly retain the its original meaning. Your job will be to incorporate humans into the process to make those decisions. Meet with your professor to learn more about the NLP software (it takes quite a lot of effort to learn), and then design a set of MTurk HITs to filter generated sentences down to ones that are poetic, grammatical and mean the same thing as the original prose.

Fairness in rejection

Design an adjudication system for work rejected by Mechanical Turk Requesters. The system should allow Workers to appeal rejections, and should have a mechanism for deciding whether the rejection was fair (in which case it would stand), or unfair (in which case it should be overturned, and the Worker should be paid). Possible ideas: design a second pass HIT that has other Turkers review the work, and decide whether it is acceptable or not. As part of this project you should specify what constraints are on the original HIT design to allow easy second pass reviewing and highlighting / explanation of why an assignment was rejected. You should also quantify the expected increase in costs to Requesters, based on variables like: rejection rate, original reward amount, reviewing cost estimate.

Design a system for confederated qualifications

Design an implement a method for Mechanical Turk Requesters to share qualification tests and the results of who passed the tests. Write a short paper describing the value of such a system, and comparing it to MTurk’s master’s qualification. Design a few qualifications of your own that you think would be broadly useful, possibly by reviewing the tasks currently posted on MTurk and generalizing the skill sets that are needed.

Cognitive science experiments

Choose some aspect of the cognitive science that can be tested through experiments on human subjects. One of my favorite examples of this is Lera Boroditsky’s work testing various aspects of the Sapir–Whorf hypothesis that language influences they way that we think. Read Lera’s article on how she used MTurk to test whether metaphors change the way people reason. Choose your own topic (or reimplement several classic experiments). Write a paper discussing your results, discussing whether MTurk provides a representative sample of subjects, and describing how to go about applying for Institutional Review Board approval for cognitive science experiments on MTurk.

Behavioral Economics

Write a suite of HITs on Mechanical Turk to test behavioral economic theories by implementing a set of games like the “ultimatum game”. In this game two people are paired up. (They can communicate with each other, but otherwise they’re anonymous to each other.) They’re given $10 to divide between them, according to this rule: One person (the proposer) decides, on his own, what the split should be (fifty-fifty, seventy-thirty, or whatever). He then makes a take-it-or-leave-it offer to the other person (the responder). The responder can either accept the offer, in which case both players pocket their respective shares of the cash, or reject it, in which case both players walk away empty-handed. See more details in “The Wisdom of Crowds”. Note that this requires pairing two people simultaneously or simulating their interactions.

Track the spread of the flu using social media

Did you know that you can catch the flu from social media? Well, you can’t. But you can use it as a tool to track the spread of certain diseases. You could try re-creating one of the publications by this cool researcher. What sorts of health problems do you think social media can give us information about?

Wikitopics

Wikipedia provides hour-by-hour page view statistics for every one of its pages. Write a human computation algorithm that uses these statistics as input to detect trending topics in the news. Use humans to (1) review the trending pages to say whether they describe something newsworthy, (2) cluster them into pages about the same event, and (3) write short summaries of the event that triggered them to become popular. Design good mechanisms for quality control for clustering, and for describing something as newsworthy. Read this paper about a baseline computational algorithm.

MTurk Prediction Market

Prediction markets use collective intelligence to try to predict the outcome of future events. Prediction markets answer questions that have definite, verifiable answers on a particular date (like “Will the government shutdown still be in effect on October 31, 2013?”). They let people buy and sell shares in the outcomes, and track the value of each outcome’s shares over time. You should implement a prediction market that sets that value of the shares. You should hire workers on Mechanical Turk to make the predictions. The major design challenge will be to formulate the system so that it incentivizes Turkers to make well-considered predictions instead of random predictions. For instance, you may consider designing a HIT that pays nothing initially, but that gives people up to $10 if all of their predictions are accurate.

Political Descriptors by Demographic

The words we use to describe politicians and public figures in general depends a lot on their background. Pick one characteristic to keep track of (age, gender, party, country or state of origin, relationship status, time in office, anything), figure out which words correlate most strongly with politicians who possess that characteristic, and use the crowd to assign an intensity and sentiment to some of these words – maybe even design a HIT that swaps out the names and pronouns of one politician for another and ask the Turker to assess the clarity and cohesion of the article to see how background affects descriptions in the media.