Instructions for posting a simple paraphrase judgement HIT to CrowdFlower using the web interface.

1. From your account's main page, click on "Your Jobs" on the left, and then on the "Create Job" button.

2. Select "Start from scratch" and then "Upload File".

3. Upload the paraphrase_input.csv file from this directory. This file contains one unit per row- i.e. one phrase, sentence, and set of 10 paraphrases to be judged. 

4. On the right, select "Build Job." You can build your job interactively using the graphical editor, or you can click on "Switch to CML editor". This instructions will continue using the CML editor.

5. From the CML editor, you can copy and paste the contents of instructions.html into the "Instructions" box. The instructions are given in standard html.

6. You can copy and paste the contents of hit_template.html into the CML box. The CML box requires that you use special CrowdFlower Markup Language (CML) tags to designate your form elements. Full documentation of CML is provided here: http://success.crowdflower.com/hc/en-us/articles/202817989-CML-and-Instructions-CML-CrowdFlower-Markup-Language-Overview

Within the CML editor, you can access variables from your spreadsheet using the {{VARIABLE_NAME}} syntax. E.g. in hit_template.html, we use {{sentence}} at the top of the HIT. When you launch your HIT, Crowdflower will create a new instance of your CML questin for each row in the uploaded csv. In each instance, {{sentence}} will be replaced by the value of the column labeled "sentence" in the given row.

NOTE: CML offers some options to control logic in your HIT, e.g. "show question B only if the answer to question A was TRUE". You also have an option to include code in the custom CSS and Javascript editors. However, you do not have access to the variables from your spreadsheet from within the JavaScript editor. CrowdFlower makes it easy to get up and running quickly and ensures high quality for HITs that are simple and very uniform across instances. However, if you want to build very complex HITs with more JS bells and whistles, I recommend using MTurk.

7. Once you are happy with your HIT design, click on "Job Settings" on the lefthand panel. On the main page you can set 1) the number of unit per HIT (recall that the CML you wrote in step 6 corresponds to one row/unit), 2) the level of redundancy, and 3) the pay. 

NOTE: If you click on "Show Advanced Settings" at the bottom, and then select the "Contributors" -> "Channels" tab, you can restrict your HIT to only the "Internal Contributors." This is analogous to launching on the Sandbox on MTurk.

8. Click on "Launch" and then "Launch Job."



