CrowdFlower – IDEA https://www.idea.org/blog Fresh ideas to advance scientific and cultural literacy. Wed, 26 Feb 2025 17:45:04 +0000 en-US hourly 1 https://wordpress.org/?v=4.9.26 Keeping adult pics out of community generated content https://www.idea.org/blog/2012/10/11/community-generated-content/ https://www.idea.org/blog/2012/10/11/community-generated-content/#respond Thu, 11 Oct 2012 17:09:08 +0000 http://www.idea.org/blog/?p=3865 Getting the community involved in contributing content to an online project is a great form of public participation, and also a way to build  large repositories of content. However, the underbelly of community-generated content is bad taste, inappropriate content, and outright abuse.

This seedy side is particularly evident in social networks sites, where users upload photos from their cell phones. To keep it clean,  social media sites hire legions of inexpensive laborers via crowdsourcing sites like Amazon’s Mechanical Turk and CrowdFlower to screen uploaded content.

To get a flavor for the kinds of content people upload, check out the rules which have been developed by the Skout social network. Photo reviewers are paid 3-4 cents to scan one hundred photos at a time, and marking inappropriate photos which violate any of the following. From topless shots to erections to gang symbols…

(The greyed images were erections.)

  • No nudity – including photos of frontal, back and side nudity.
  • No bare skin one inch above the pubic area.
  • No shirtless body shots indoors, even if with face – only allowed in natural settings (e.g., a swimming pool or beach)
  • No cleavage shots without a face.
  • No pubic hair can be visible.
  • No underwear can be visible including the underwear waistband showing above pants.
  • No faceless body/torso shots without clothes or in underwear, including no crotch/butt only photos, nor abs only photos
  • Appropriate public swimwear is allowed if the following is observed: No pubic hair, no women’s nipples, no outline of genitals and no portion of the butt can be present. And, Only in natural settings (e.g., a swimming pool or beach)
  • Pants and shorts must be worn normally, buttoned, and not pulled or hanging down.
  • No images of hands or fingers placed in pants or pulling underwear outward.
  • No copyrighted pictures or illustrations.
  • No images that display semen (or any fluid made to look like semen or ejaculation) on anything in photo.
  • No images of sexual acts, either real, illustrated or simulated.
  • No photo that is sexually explicit or overly suggestive.
  • No photos that have been altered to disguise sexual acts including a black box or blurred filters to hide sexual images such as touching of genitals by hand.
  • Nudity (particularly the genitals) covered up by a towel, hat or other means is not allowed.
  • No grabbing/holding or touching genitals or genital area.
  • No photos with sheer, or otherwise see-through or wet material below the waist, or covering women’s nipples/breasts.
  • No erections or outline of genitals through clothing will be allowed.
  • No photos that contain sex props and toys, including the use of fruits/vegetables.
  • No images of illegal drug use and paraphernalia.
  • No images of firearms or weapons.
  • No photos of any obscene gestures and/or lewd behavior. (e.g., the middle finger or a banana used to simulate male genitalia)
  • No photos of violent acts to yourself, someone else, or animals; including blood in photo.
  • No image used to advertise services, goods, events, websites, or apps.
  • No profanity or curse words.

The guidelines are this excruciatingly detailed  because all these kinds of content are often uploaded to the social network. The Skout network’s screening finds 5-10% of their content to be in violation.

Typical educational projects have a more cultured vibe, and a self-selected community of participants, and should encounter much less depravity. But any outreach projects planning large amounts of community-generated photos should have moderation mechanisms in place.

]]>
https://www.idea.org/blog/2012/10/11/community-generated-content/feed/ 0
Do more by outsourcing some outreach tasks to freelancers https://www.idea.org/blog/2012/05/30/do-more-by-outsourcing-some-outreach-tasks-to-freelancers/ https://www.idea.org/blog/2012/05/30/do-more-by-outsourcing-some-outreach-tasks-to-freelancers/#comments Wed, 30 May 2012 16:49:45 +0000 http://www.idea.org/blog/?p=3742 Expand your outreach capacity with multiple media, multiple languages, new sites and apps, and other features by hiring freelancers. Here’s an overview of marketplaces we’ve successfully used at IDEA.

Translations

In our global world, there’s no excuse for staying limited to English speakers, especially when there’s greater need for education in non-English places. Even if you have multilingual staff, most translations will be better and more cost effective if you outsource. Professional translators are efficient, and skilled at adapting idioms and phrases. They tend to be detail-oriented, soft spoken individuals.

ProZ – Text translations of any size, e.g., articles, reports, books. ProZ also has interpreters. Quality translation into a Western language costs $0.07-$0.15 per word, languages from developing economies cost a bit less. Target the upper range, e.g., €0.10 EUR per word if you were going to French. If you know a native speaker, have them review a few of the most promising applicants’ samples. The workflow is that you send the translator your text, and they email you back the translation. If you have large volumes, a translator can translate 30k words a month, going up to 50k if you have bursts of work. Over 300 thousand translators represented.

ICanLocalize – Software translations, such as the short text labels which are part of any web site or app. Little texts like “Back,” “Next,” “Go to article.” We recently used them to translate our WikiNodes app into 18 languages. The cost is $0.10-0.15 per word, including their 20% commission and a 50% fee for peer review. Your programmers first consolidate all the text blurbs from your app into single files (“localization”). Conveniently, you can upload many common formats to their site, and then download them back to put in your app a few days later. Their translators are savvy to the nuances of software localization, and will know that “back” is navigation, not the region between your neck and your buttocks.

Voiceovers

Voice123 – If you are creating a radio advertisement, podcast, game, or need narration for a video, and you want a specific kind of professional voice, there’s an infinite world of voiceover artists. Voice actors have a range of styles, from youthful to movie trailer. Voice123 has had 220 thousand voice talents sign up, giving 3.1 million audition samples, since being founded in 2003. There are a few voiceover marketplaces, Voices.com is another option. With both sites, voice actors — with a diverse range of prices and skills — will give you short samples to listen to. These professionals have studios (microphones and audio software) in their office, and can often send back short recordings within a matter of hours. A ~1 minute recording (e.g., voice track for a video) could cost $50-100. But the price is not linear, so 2 minutes is not 2x the cost of 1 minute.

Simple, repetitive tasks

Some tasks need a person to be involved (i.e., can’t be done by a computer), but don’t need a highly skilled professional. For example, reviewing thousands of blog posts or comments for spam or abuse, categorizing posts or photos, retyping entries from print or PDF into a spreadsheet, or looking up thousands of web sites or Twitter handles for your members. Workers can be paid around $3-10/hour, typically paid 5-10 cents per task. You never know who the workers are, it could a a retiree in Montana, or a 19 -year-old in Delhi.

CrowdFlower makes these tasks easy to manage. They deal with the details of hiring remote workers, and charge a commission for serving as an intermediary.

If you want to go straight to the source, check out Amazon Mechanical Turk (which CrowdFlower and others use as their backend). With Mechanical Turk, you define small tasks, called a “HIT,” (e.g., looking at a photo from your site, and assigning a category), upload the task as a spreadsheet file (each spreadsheet row is a different photo URL). Depending on the price you set, you may have none, or dozens of workers working on different rows from your spreadsheet at once.

Transcription

Converting audio to written text is tedious. Several services will manage the details of finding transcriptionists, and managing them. (Behind the scenes, these services often use Amazon Mechanical Turk).

CastingWords is one of many transcription services. You email audio or provide URLs, and 1-10 days later, you receive a text file. Price depends on speed. $2.50/minute for 1 day turn around, $1/minute for slow turnaround.

Programming

There are two main sites for hiring web programmers for defined tasks like creating web sites, writing simple blog posts or making web graphics. These can also be a good way to find an ongoing part time web master, if you first hire then for a small, defined task.

ScriptLance is the best site for freelance programmers, and has good management features. No need to pay extra for a ‘featured’ project.

vWorker  is a bit more cumbersome, has fewer users, but tends to have slightly more technical workers. There’s no harm in posting jobs to both sites. (Previously called “RentACoder”)

Local help

If you need local help, or need to work in person with the worker — take a look at your local Craigslist and Idealist.

Everything else

From accounting to data entry, writing to design, transcription to programming, the Coke and Pepsi of outsourcing are eLance and Guru. We tend to have a slight preference for eLance, but they are similar. Both sites are focused on web sites, ecommerce, programming and databases, but also include creative tasks, administrative support, and accounting and legal (e.g., drafting a terms of use for your web site).

Below, this map from eLance shows where their talent is based. Darker blue is more common:

eLance – Claims over 68 thousand jobs posted in the last month, over 1.4 million registered contractors, and nearly $550 million in projects to date, with $43 million in 2011.

Guru –  Guru touts that over 1 million users have completed over 3 million tasks, working on over 150 million dollars in projects (average of $50 per project, though project budgets can often go into the $thousands). Today they have 4716 open jobs.

Freelancer.com – A slightly smaller community, but could be worth posting here also. Over 3.5 million freelancers registered.

Specific skills

oDesk – Specializes in buying time from workers on an hourly basis. Their workers tend to be more highly skilled, so expect higher hourly rates — in the $20-100/hr range. But if you have a specific technical task, this can be a place to fine workers who are not on the other sites. Some other sites (e.g., Freelancer) are expanding into hourly work also.

Design

Designers and other creatives can be challenging to track down because the best designers tend not to use the above sites. Here are some options:

Sortfolio – Created by 37Signals (who created BaseCamp project management software), provides a convenient way to browse portfolios and filter by budget ranges and location. Has a lot of good designers on board.

Coroflot and Behance – Browse portfolios for various kinds of creative work, and then write personalized emails to a few designers praising their work and inviting them to work on your project. Both sites also have full time job boards. Coroflot has 1.4 million images of creative work, from the portfolios of over 150,000 creative professionals and students. Behance has 10.6 million images, 1.6 million projects, and recently raised $6.5 million in funding from investors including Jeff Bezos.

Other marketplaces?

Do you have experience with other marketplaces? Share them in the comments…

]]>
https://www.idea.org/blog/2012/05/30/do-more-by-outsourcing-some-outreach-tasks-to-freelancers/feed/ 7