Google Image Labeler

Google Image Labeler is a feature, in form of a game, of Google Image Search that allows the user to label random images to help improve the quality of Google's image search results.

History

On August 31, 2006 Google launched this service, as a beta.

Luis von Ahn developed the ESP Game [ [http://video.google.com/videoplay?docid=-8246463980976635143 Human Computation - Google Video] ] , a game in which two people are simultaneously given an image, with no way to communicate, other than knowing the matching label for each picture or the pass signal. The ESP Game has been licensed by Google in the form of the Google Image Labeler.

Players have noticed various subtle changes in the game over time. In the earliest months, through about November, 2006, you could see your partner's guesses during play by mousing over the image. When the "congenita abuse" started (see below) you could see if your partner was using those terms, while the game was underway. The game was changed so that only at the end of the game could you click "see partner's guesses" and learn what he or she had typed. "Congenita abuse" was finally stopped by changes in the structure of the game in Feb. 2007 (see below). During the first few months of 2007 regular players grew to recognize a group of images that signified a "robot" partner, always with the same labels in the same order. This appears to have changed as of about March 13, 2007. Suddenly most of the images seen are brand new, and the older images come with extensive off-limits lists.

As of May 4, 2007 there have been fundamental and substantial changes made to the game. Instead of 90 seconds, players now have 2 minutes. Instead of 100 points per image, the score is varied to reward higher specificity. "Man" might get 50 points whereas "Bill Gates" might get 140 points. On August 7, 2007 another change was made. Instead of simply showing the point values of each match as the match occurs, the value of each match is shown next to the matching word at the end of the game. This makes it much easier to see the exact value of specific versus general labeling. A further change was observed on October 15, 2007. The new version was put into place and then seems to have been withdrawn. In the new version you see only the image you are labelling, whereas in the old version the images are collected in the lower part of the screen as you play. Other changes are subtle, for example the score is in green letters in the new version and red in the old. The most significant change is that the clock freezes during the image change, and that time used to be essentially subtracted from the two minutes of play. The changes appear to have gone into full effect on October 18, 2007.

Rules

The user will be randomly paired with a partner who is online and using the feature. Users can be registered players who accumulate a total score over all games played, or guests who just play for one game at a time. Note that players come from around the world, some practicing their English, and both American and British English will be encountered (soccer vs. football). When an uneven number of players are online, a player will play against a prerecorded set of words.

The current rules follow. For changes in the rules, see the history section. Over a 2 minute period, the user and his/her partner will be shown the same set of images and asked to provide as many labels as possible to describe each image you see. When the user's label matches the partner's label, both will earn points and move on to the next image until time runs out. It is possible to pass on an image but both users must agree to do this. The score is variable from 50 to 150 depending on the specificity of the answer. The 150 score is rare but 140 points will be awarded for a name or specific word, even if the word is spelled out in the image. Terms with low specificity like "trees" or "man" earn only 50 points. There has never been any screening for correctness, so that if both players type "Jupiter" for an image of Saturn, they would presumably both get 140 points.

Labels that have been agreed on by previous users may show on an "off limits" list and cannot be used in that round. Some players think that the game staggers appearance of the images, and that sometimes it takes the first words typed by one player to form an "off limits" list for the other player. In other words, the off limits words may be unilateral, asymmetrical. This would explain the rather frequent circumstance when it seems a partner can't think of words like "car," "bird," or "girl." Very rarely, at the end of the match it becomes obvious that one image was different for the two players. Perhaps this is simply an error, or perhaps it is a test to see how quickly people will pass when their descriptions do not match, but it may also be a mechanism implemented to view cheaters, if the words for the different images are similar. At times, one user's computer will fail to load a new image, or continue to display the previous image shown. Times likes these also call for a mutual "pass" on the part of both players.

End of game

After the 120 second time expires, the game is over. The user can see the user name of the partner for the first time, their score for the game (with which both are credited) and their individual cumulative score to date. These are compared to the daily high scores for teams and the all-time individual high scores. Google is betting on users' competitiveness to rack up high scores to swell the number of images ranked.

The game's end screen also shows all the images presented during the game along with the list of labels guessed by the other player. The images themselves then link to the websites where those images were found and can be explored for more information and to satisfy curiosity.

Benefits to Google

The game is not designed simply for fun. Though the feature is enjoyable for the users, it is also a clever way for Google to ensure that its keywords are matched to correct images. Each matched word will help Google to build an accurate database used when using the Google Image Search.

Without human tagging of images, Google Images search has in the past relied on the context of the image. For example, a photo that is captioned "Portrait of Bill Gates" might have "Bill Gates" associated as a possible search term. The Google Image Labeler relies on humans that tag the meaning or content of the image, rather than its context looking on at where the image was used. By storing more information about the image, Google stores more possible avenues of discovering the image in response to a user's search.

Issues and Problems

Additional Rules

* Some users complain that the rules are difficult to decipher. Nowhere is it stated, for example, that a player should press Return after typing a label.
* Beginners often make the mistake of typing several terms into the first box, not realizing that those words are then all considered, together, as a phrase.
* The "pass" option is also not explained; although it means that the player does not want to rate a word, some players have thought that this button is to be pressed after making a guess. Either of these mistakes can easily result in a zero score.

While these rules are not explained on the playing screen, some are in fact explained on the Help Page. [ [http://images.google.com/imagelabeler/help.html Google Help Page] ]

Other issues include:

* Some images may fail to load, or will load very slowly, using time off the clock
* Experienced users type the letter "x" to avoid simply passing (and scoring zero) when this happens. Note that "x" (nor any other word) will not work for two successive images so other terms ("blank" or "none") could also be utilized, since sometimes many images do not load.
* Only six labels appear on the screen during a round, if more are added some scroll off the top. Observation shows that all of these labels count, and with a fast partner you can see nine or ten words for one image. Also, after hitting "Pass" you will no longer see the count of your partner's labels. If you continue to type, you can get a match on words your partner had already typed when you passed. If your partner types a matching word after you pass, it will not count.
* Scores are typically low when several of the images presented have a number of "off limits" tags. In some cases, the "off limits" tags are quite extensive or exhaustive, making it difficult for both partners to create a novel tag that matches. This tends to use time and reduce the total score.

Abuse

Less than a month after the launch, the game began to be abused [Nas Raja in [http://www.mattcutts.com/blog/google-image-labeler-game/#comment-87075 Matt Cutts: Gadgets, Google, and SEO] ] . It appeared that Google was getting spammed with words from the following list: "abrasives, accretion, bequeathing, carcinoma, congenita, diphosphonate, entrepreneurialism, forbearance", "googley", and "nager beast". Because players can see the responses of their partner at the end of each round, they learned that other players were using these words. Some then incorporated these words into their answers for entertainment and increased scores. As of Feb. 7, 2007 Google changed the game to cut down on the abuse. The words on the list above were filtered out. Also certain images that had become triggers for the random words came with an immediate "Your Partner Wants to Pass." In the game revision of March 13, 2007, the "trigger images" were removed for a while but they were back in play as of March 27. []

References

External links

* [http://images.google.com/imagelabeler/ Google Image Labeler Beta]
* [http://www.rci.rutgers.edu/~deis/labeler.html Discussion and Tips from Players, comments on May 4 changes]