Amazon Mechanical Turk
Encyclopedia
The Amazon Mechanical Turk (MTurk) is a crowdsourcing
Internet
marketplace
that enables computer programmers (known as Requesters) to co-ordinate the use of human intelligence to perform tasks that computers are unable to do yet. It is one of the suites of Amazon Web Services
. The Requesters are able to post tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a store-front, writing product descriptions, or identifying performers on music CDs. Workers (called Providers in Mechanical Turk's Terms of Service) can then browse among existing tasks and complete them for a monetary payment set by the Requester. To place HITs, the requesting programs use an open Application Programming Interface
, or the more limited MTurk Requester site. Requestors are restricted to US-based entities.
Requesters can ask that Workers fulfill Qualifications before engaging a task, and they can set up a test in order to verify the Qualification. They can also accept or reject the result sent by the Worker, which reflects on the Worker's reputation. Currently, workers can have an address anywhere in the world. Payments for completing tasks can be redeemed on Amazon.com via gift certificate or be later transferred to a Worker's U.S. bank account. Requesters, which are typically corporations, pay 10 percent over the price of successfully completed HITs to Amazon.
," a chess-playing automaton
of the 18th century, which was made by Wolfgang von Kempelen
. It toured Europe, beating the likes of Napoleon Bonaparte and Benjamin Franklin
. It was later revealed that this "machine" was not an automaton at all, but was in fact a chess master hidden in a special compartment controlling its operations. Likewise, the Mechanical Turk web service allows humans to help the machines of today perform tasks they aren't suited for.
, to find duplicates among its web pages describing products.
MTurk was launched publicly on November 2, 2005 and, as of July 2011, is still in beta. Following its launch, the Mechanical Turk user base grew quickly, in part the result of the Slashdot effect
. At that time, there were a huge number of "Human Intelligence Tasks" (HITs) in the system. In early- to mid-November 2005, there were tens of thousands of HITs, all of them uploaded to the system by Amazon itself for some of its internal tasks that required human intelligence. Most of these were related to music CD items. Web traffic grew to a massive amount near the beginning of December.
However, the number of HITs in the system soon decreased, and by December 20, there were less than 100 groups of HITs on the average page load. In January, new types of HITs were set up, such as top three lists ranking for the (now defunct) Amazon Unspun site, and third-party HITs began to appear as well. By April 2006, there were only the occasional batch of 25 HIT groups being offered, and the service had slowed to a crawl.
As of January 2007, there were new HITs being offered of podcast
transcribing, rating and image tagging. The transcription HITs, mostly offered by CastingWords, are still posted regularly as of July 2011. Other common HIT types ask Turkers to write or rewrite sentences, paragraphs, or whole articles. These have rewards ranging from one cent to about $10. HITs that reward people for linking to or commenting on a blog
, or friending a person on Facebook
are also often encountered, as are surveys.
In March 2007, there were reportedly more than 100,000 workers in over 100 countries. In 2011, techlist published an interactive map pinpointing the locations of 50,000 of their MTurk workers around the world.
According to a survey conducted through one MTurk HIT, Turkers are primarily located in the United States
with demographics generally similar to the overall Internet population in the US.
, but his body was found before any technical progress was made. That summer, computer scientist Jim Gray disappeared on his yacht and Amazon's Werner Vogels
, a personal friend, made arrangements for DigitalGlobe
, which provides satellite data for Google Maps
and Google Earth
, to put recent photography of the Farallon Islands
on the Mechanical Turk. A front-page story on Digg
attracted 12,000 searchers who worked with imaging professionals on the same data. The search was unsuccessful. In September 2007, a similar arrangement was repeated in the search for aviator Steve Fossett. Satellite data was divided into 85 squared meter sections, and Mechanical Turk users were asked to flag images with "foreign objects" that might be a crash site or other evidence that should be examined more closely. The search was also unsuccessful; the crash site was eventually found by hikers about a year later.
designed to simplify the process of completing HITs. According to the Amazon Web Services Blog, however, Amazon appears to disapprove of the ones that automate the process 100% and take out the human element. Accounts using so-called automated bots have been banned.
service. However, the Mechanical Turk is a more general marketplace that can potentially help distribute any kind of work tasks all over the World. The Collaborative Human Interpreter
(CHI) by Philipp Lenssen also suggested using distributed human intelligence to help computer programs perform tasks that computers cannot do well. MTurk could be used as the execution engine for the CHI.
". At the same time, workers set their own hours and are not under any obligation to accept any work they do not wish to do. Because workers are paid as contractors rather than employees, requesters do not have to file forms for, nor pay, payroll taxes, and they avoid laws regarding minimum wage
, overtime
, and workers compensation. Workers, though, must report their income as self-employment income. In addition, some requesters have taken advantage of workers by having them do the tasks, then rejecting their submissions in order to avoid paying. However, at least some workers on Mechanical Turk are people who do the work for fun. Quality of the entries can vary from usable to useless.
Amazon.com does not monitor the service and refers all complaints to the poster of the HIT.
Crowdsourcing
Crowdsourcing is the act of sourcing tasks traditionally performed by specific individuals to a group of people or community through an open call....
Internet
Internet
The Internet is a global system of interconnected computer networks that use the standard Internet protocol suite to serve billions of users worldwide...
marketplace
Marketplace
A marketplace is the space, actual, virtual or metaphorical, in which a market operates. The term is also used in a trademark law context to denote the actual consumer environment, ie. the 'real world' in which products and services are provided and consumed.-Marketplaces and street markets:A...
that enables computer programmers (known as Requesters) to co-ordinate the use of human intelligence to perform tasks that computers are unable to do yet. It is one of the suites of Amazon Web Services
Amazon Web Services
Amazon Web Services is a collection of remote computing services that together make up a cloud computing platform, offered over the Internet by Amazon.com...
. The Requesters are able to post tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a store-front, writing product descriptions, or identifying performers on music CDs. Workers (called Providers in Mechanical Turk's Terms of Service) can then browse among existing tasks and complete them for a monetary payment set by the Requester. To place HITs, the requesting programs use an open Application Programming Interface
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
, or the more limited MTurk Requester site. Requestors are restricted to US-based entities.
Requesters can ask that Workers fulfill Qualifications before engaging a task, and they can set up a test in order to verify the Qualification. They can also accept or reject the result sent by the Worker, which reflects on the Worker's reputation. Currently, workers can have an address anywhere in the world. Payments for completing tasks can be redeemed on Amazon.com via gift certificate or be later transferred to a Worker's U.S. bank account. Requesters, which are typically corporations, pay 10 percent over the price of successfully completed HITs to Amazon.
Name
The name Mechanical Turk comes from "The TurkThe Turk
The Turk, also known as the Mechanical Turk or Automaton Chess Player , was a fake chess-playing machine constructed in the late 18th century. From 1770 until its destruction by fire in 1854, it was exhibited by various owners as an automaton, though it was exposed in the early 1820s as an...
," a chess-playing automaton
Automaton
An automaton is a self-operating machine. The word is sometimes used to describe a robot, more specifically an autonomous robot. An alternative spelling, now obsolete, is automation.-Etymology:...
of the 18th century, which was made by Wolfgang von Kempelen
Wolfgang von Kempelen
Johann Wolfgang Ritter von Kempelen de Pázmánd was a Hungarian author and inventor with Irish ancestors.-Life:...
. It toured Europe, beating the likes of Napoleon Bonaparte and Benjamin Franklin
Benjamin Franklin
Dr. Benjamin Franklin was one of the Founding Fathers of the United States. A noted polymath, Franklin was a leading author, printer, political theorist, politician, postmaster, scientist, musician, inventor, satirist, civic activist, statesman, and diplomat...
. It was later revealed that this "machine" was not an automaton at all, but was in fact a chess master hidden in a special compartment controlling its operations. Likewise, the Mechanical Turk web service allows humans to help the machines of today perform tasks they aren't suited for.
History, HIT types, and user demographics
The service was initially invented for Amazon's in-house use by Peter CohenPeter Cohen
Peter Cohen may refer to:*Peter A. Cohen, former Chairman and CEO of Shearson Lehman Brothers and current Chairman and CEO of Cowen Group*Peter-Adrian Cohen, American writer*Peter Cohen , Swedish film director, writer, editor and producer...
, to find duplicates among its web pages describing products.
MTurk was launched publicly on November 2, 2005 and, as of July 2011, is still in beta. Following its launch, the Mechanical Turk user base grew quickly, in part the result of the Slashdot effect
Slashdot effect
The Slashdot effect, also known as slashdotting, occurs when a popular website links to a smaller site, causing a massive increase in traffic. This overloads the smaller site, causing it to slow down or even temporarily close. The name stems from the huge influx of web traffic that results from...
. At that time, there were a huge number of "Human Intelligence Tasks" (HITs) in the system. In early- to mid-November 2005, there were tens of thousands of HITs, all of them uploaded to the system by Amazon itself for some of its internal tasks that required human intelligence. Most of these were related to music CD items. Web traffic grew to a massive amount near the beginning of December.
However, the number of HITs in the system soon decreased, and by December 20, there were less than 100 groups of HITs on the average page load. In January, new types of HITs were set up, such as top three lists ranking for the (now defunct) Amazon Unspun site, and third-party HITs began to appear as well. By April 2006, there were only the occasional batch of 25 HIT groups being offered, and the service had slowed to a crawl.
As of January 2007, there were new HITs being offered of podcast
Podcast
A podcast is a series of digital media files that are released episodically and often downloaded through web syndication...
transcribing, rating and image tagging. The transcription HITs, mostly offered by CastingWords, are still posted regularly as of July 2011. Other common HIT types ask Turkers to write or rewrite sentences, paragraphs, or whole articles. These have rewards ranging from one cent to about $10. HITs that reward people for linking to or commenting on a blog
Blog
A blog is a type of website or part of a website supposed to be updated with new content from time to time. Blogs are usually maintained by an individual with regular entries of commentary, descriptions of events, or other material such as graphics or video. Entries are commonly displayed in...
, or friending a person on Facebook
Facebook
Facebook is a social networking service and website launched in February 2004, operated and privately owned by Facebook, Inc. , Facebook has more than 800 million active users. Users must register before using the site, after which they may create a personal profile, add other users as...
are also often encountered, as are surveys.
In March 2007, there were reportedly more than 100,000 workers in over 100 countries. In 2011, techlist published an interactive map pinpointing the locations of 50,000 of their MTurk workers around the world.
According to a survey conducted through one MTurk HIT, Turkers are primarily located in the United States
United States
The United States of America is a federal constitutional republic comprising fifty states and a federal district...
with demographics generally similar to the overall Internet population in the US.
Missing persons searches
In 2007, the service began to be used to search for prominent missing individuals. It was first suggested during the search for James KimJames Kim
James Kim was an American television personality and technology analyst for the former TechTV international cable television network, reviewing products for shows including The Screen Savers, Call for Help, and Fresh Gear...
, but his body was found before any technical progress was made. That summer, computer scientist Jim Gray disappeared on his yacht and Amazon's Werner Vogels
Werner Vogels
Dr. Werner Vogels is the Chief Technology Officer and Vice President of Amazon.com in Seattle, Washington. In charge of driving technology innovation within the company, Vogels has broad internal and external responsibilities. He is the only executive apart from Amazon’s CEO Jeff Bezos to speak...
, a personal friend, made arrangements for DigitalGlobe
DigitalGlobe
DigitalGlobe, of Longmont, Colorado, USA, is a commercial vendor of space imagery and geospatial content, and operator of civilian remote sensing spacecraft...
, which provides satellite data for Google Maps
Google Maps
Google Maps is a web mapping service application and technology provided by Google, free , that powers many map-based services, including the Google Maps website, Google Ride Finder, Google Transit, and maps embedded on third-party websites via the Google Maps API...
and Google Earth
Google Earth
Google Earth is a virtual globe, map and geographical information program that was originally called EarthViewer 3D, and was created by Keyhole, Inc, a Central Intelligence Agency funded company acquired by Google in 2004 . It maps the Earth by the superimposition of images obtained from satellite...
, to put recent photography of the Farallon Islands
Farallon Islands
The Farallon Islands, or Farallones , are a group of islands and sea stacks in the Gulf of the Farallones, off the coast of San Francisco, California, USA. They lie outside the Golden Gate and south of Point Reyes, and are visible from the mainland on clear days...
on the Mechanical Turk. A front-page story on Digg
Digg
Digg is a social news website. Prior to Digg v4, its cornerstone function consisted of letting people vote stories up or down, called digging and burying, respectively. Digg's popularity prompted the creation of copycat social networking sites with story submission and voting systems...
attracted 12,000 searchers who worked with imaging professionals on the same data. The search was unsuccessful. In September 2007, a similar arrangement was repeated in the search for aviator Steve Fossett. Satellite data was divided into 85 squared meter sections, and Mechanical Turk users were asked to flag images with "foreign objects" that might be a crash site or other evidence that should be examined more closely. The search was also unsuccessful; the crash site was eventually found by hikers about a year later.
Third-party programming
Programmers have developed various browser extensions and scriptsScripting language
A scripting language, script language, or extension language is a programming language that allows control of one or more applications. "Scripts" are distinct from the core code of the application, as they are usually written in a different language and are often created or at least modified by the...
designed to simplify the process of completing HITs. According to the Amazon Web Services Blog, however, Amazon appears to disapprove of the ones that automate the process 100% and take out the human element. Accounts using so-called automated bots have been banned.
API
Amazon makes available an API to give users another access point to the MTurk system. The MTurk API lets a programmer submit HITs to MTurk, retrieve completed work, and approve/deny that work. Web sites and Web services can use the API to integrate MTurk work into other web applications, providing users with alternatives to the interface Amazon has built for these functions.Related systems
MTurk is comparable in some respects to the now discontinued Google AnswersGoogle Answers
Google Answers was an online knowledge market offered by Google that allowed users to post bounties for well researched answers to their queries. Asker-accepted answers cost $2 to $200. Google retained 25% of the researcher's reward and a 50 cent fee per question. In addition to the researcher's...
service. However, the Mechanical Turk is a more general marketplace that can potentially help distribute any kind of work tasks all over the World. The Collaborative Human Interpreter
Collaborative human interpreter
The Collaborative Human Interpreter is a proposed software interface for human-based computation specially designed for collectingand making use of human intelligence in a computer program.One typical usage is implementing...
(CHI) by Philipp Lenssen also suggested using distributed human intelligence to help computer programs perform tasks that computers cannot do well. MTurk could be used as the execution engine for the CHI.
Criticism
Because HITs are typically simple, repetitive tasks and users are paid often only a few cents to complete them, some have criticized Mechanical Turk as a "virtual sweatshopSweatshop
Sweatshop is a negatively connoted term for any working environment considered to be unacceptably difficult or dangerous. Sweatshop workers often work long hours for very low pay, regardless of laws mandating overtime pay or a minimum wage. Child labour laws may be violated. Sweatshops may have...
". At the same time, workers set their own hours and are not under any obligation to accept any work they do not wish to do. Because workers are paid as contractors rather than employees, requesters do not have to file forms for, nor pay, payroll taxes, and they avoid laws regarding minimum wage
Minimum wage
A minimum wage is the lowest hourly, daily or monthly remuneration that employers may legally pay to workers. Equivalently, it is the lowest wage at which workers may sell their labour. Although minimum wage laws are in effect in a great many jurisdictions, there are differences of opinion about...
, overtime
Overtime
Overtime is the amount of time someone works beyond normal working hours. Normal hours may be determined in several ways:*by custom ,*by practices of a given trade or profession,*by legislation,...
, and workers compensation. Workers, though, must report their income as self-employment income. In addition, some requesters have taken advantage of workers by having them do the tasks, then rejecting their submissions in order to avoid paying. However, at least some workers on Mechanical Turk are people who do the work for fun. Quality of the entries can vary from usable to useless.
Amazon.com does not monitor the service and refers all complaints to the poster of the HIT.