For some insight, we look at a crowdsourced history site: Historypin is an appealing database of historical photos, with dates, locations, captions, and other metadata. It’s called History “pin” because the photos are pinned on a map. (See recent article about Changes over time, in photos and maps.) Some locations have photos from multiple dates, showing how a place has changed over time, or cross-referenced with Google Maps StreetView. Currently, Historypin has 308k items, from 51k users, and 1.4k institutions. This is a graph of pins over the last three years:

Aside from a change in their growth rate in early 2012, growth is linear. Since new users are always being added, the linear rate of content growth means users are losing interest. The following are activity rates:

These graphs show some trends:

• New users: In early 2013, Historypin was pulling in around 23 new users a day, and that rate nearly doubled by late summer. But there was a precipitous fall in July 2013, and Historypin currently averages 17 new users per day.
• New institutions: This rate is more consistent, hovering around two new institutions per day. This rate will eventually limit as they saturate the market.
• Daily pins: Between personal and institutional users, and ignoring a one-time spike in May 2013, the rate of new contributions hovers around 200-300 items per day.

How much do users participate? Users can join for many reasons; e.g., to contribute, to be able to make favorites (bookmarks), of curiosity, or because they are spammers. As with many sites, most users are dormant, with just a few users doing most of the activity. The following are the number of pins posted per user:

Four out of five users never post a photo, and 9% only upload one image. The remaining 12% are enumerated in the above-right graph (starting at the top clockwise).

Not another drop in the bucket

It can be thankless to contribute to a crowdsourcing site, so successful projects provide a broader context and community. Historypin illustrates some good practices.

One strategy is to have themes. Rather than generic task (“do stuff”), a theme narrows the scope (“do stuff about the San Francisco Bay”). For example, here are three current Historypin themes:

Themes are also used by the DIY History site from University of Iowa Libraries, where volunteers have been recruited to transcribe 37,507 handwritten pages of stories of Civil War soldiers and their families, of Iowa women making lives for themselves and their communities, and other themes like cookbooks, women’s lives, and the machinations of railroad barons. In DIY history, they have a large archive collection, organized into these themes.

Here’s one of their letters, which needs transcription:

Narrow themes create a manageable and achievable workload, offsetting the drudgery of transcription, and providing satisfaction to volunteers.

Other good features

Projects must have robust, easy to use technology so that volunteers can easily get started and participate.

Also, draw on gamefication principles: Keep it fun and satisfying, include some challenge and sense of accomplishments. Old Weather has a clear process. It’s clear how to get involved, and easy to follow individual and project progress.

Another motivator for contributors is to have tangible outcomes or context. For example, Old Weather has tangible outcomes, providing Arctic and worldwide weather observations which are fed into climate models of past environmental conditions; and tracking past ship movements so historians can tell the stories of the people on board.

Appeal and ease of use matters. A similar idea, “Papers of the War Department” from the Center for History and New Media at George Mason University is progressing much slower, with 1.6k registered users (since June 2010), and approximately 200 people volunteering on the site per three month period. Volunteers have transcribed approx 4 thousand (out of 43k total) documents. At this rate, the Mason project will be done in 30 years. Check out their site, and you’ll see how it falls flat.

So what’s happening with Historypin?

Historypin is not growing exponentially. It’s not viral. Rather, Historypin’s rates of new users, new content, and new content per user have been falling in 2013. Here are a few theories:

• Failure to tap a nerve – Not many people care to help stick photo pins on a map, despite their themed projects. Some ideas don’t stick.
• Hasn’t reached a critical density – The earth has 149 million square kilometers of land. London is 1.5k square kilometers. A density of 1 photo per square kilometer would be require ~500x more pins.
• Monolingual – Historypin wants to have a global community, but the site is exclusively English. It’s not hard to translate a user interface. (See our 2012 article about outsourcing translations.)
• Closed system – There’s no way to export your content, link it to another system, nor is there an API. Similarly, they did not pursue a way for institutions to use their site on the backend, e.g., for a historical society in a small town to create a local site on their platform.
• Questionable owner & future – The absence of clear funding sources undermines confidence in the long term prospects. Historypin was created by We Are What We Do, a “not-for-profit behaviour change company.” Their other projects range from branding a series of plastic shopping bags for a British retailer, to embroidered napkins with Tweets. The sole mention of a funding source for Historypin is a vague comment about support from Google.
• Bugs – Most of the technology on the Historypin site is slick and polished, but the core action of browsing photos on a map is buggy and awkward unless you zoom in close. Their mobile app is criticized by many users for being buggy and inadequate.

It’s a shame, because they are doing lots of things right, with a visually appealing site, an active and authentic blog, smooth technology, a mobile app, and some social media (Twitter, Facebook) to give contributors a sense of what’s new. In August 2013, Historypin acquired a smaller, similar project, LookBackMaps, but that has not boosted activity.

Invaluable labor

These considerations are important because crowdsourcing with volunteers has strong potential to deliver two kinds of content:

(a) Content which requires human judgement. For example, transcription projects focus on documents which are impossible to digitize automatically with OCR, but cost prohibitive to be transcribed by paid staff. Volunteers to the rescue! Once digitized, the documents become more vital, searchable in full text, read online, cross-referenced and mined by researchers. Fascinating insights can be extracted from large amounts of historical text.

(b) Content in which volunteers have unique knowledge. For example, the beauty of Historypin is that so much of the world’s photographic history is owned by individuals. When people share these old photos, scanning their family albums and the like, many other, interesting questions can be asked and explored.

What do you think?

What other crowdsourced projects in science and culture should folks know about? And what do you think are key ingredients for success?

Data source: Statistics from the home page and site of HistoryPin and other projects, over time.

Here’s an overview of several facets of crowdsourcing.

Motivations

Competition vs. collaboration are two common frameworks for projects. A competition can draw dozens or or thousands of participants who seek a prize. Unlike grant solicitations, these competitions are based on objective results, not on resumes, prior work or personal history. A collaboration typically involves a participant working on a small piece of a larger project.  

The Walker Art Center ran a crowdsourced video festival & awards competition (First International Cat Video Festival) attracting 10,000 entries and over 10,000 attendees.

Fundraising

Crowd funding is when educational projects are funded by individual, online contributors or investors.  Most crowdfunding is done via web sites which list projects, and provide a means for donors to commit. Typically, project funding is all-or-nothing.

One of the leading sites is Kickstarter, which since their launch in spring 2009, has funded over $417 million, funding over 36,000 creative projects. The “Let’s Build a Goddamn Tesla Museum” crowdfunding campaign on IndieGogo, another major crowdfunding site, enable a nonprofit group to buy Tesla’s old lab, which was threatened with development. The group raised over $1,370,511, reaching their original $800k goal in under a week.

Crowd funding tends to work best for a hip projects, on average, a third of projects are funded.

Crowdfunding is distinct from traditional, online fundraising, in that it is focused on projects, not general operations. In the traditional realm, in Q2 2012, charities reported $204-million in total online gifts (10.9% growth over Q2 2011) and $180.9-million for Q3 2012 (8.9% over Q3 2011), with an average gift of $77, according to data provided to The Chronicle of Philanthropy. More people are giving online, albeit in smaller amounts, than in past years.

See a prior blog post on crowdfunding virtual exhibits.

Workers

Cloud Labor is hiring a distributed virtual labor pool, available on-demand, to fulfull a range of tasks from simple to complex. With enticing projects, this can mean a ton of volunteers …

• The New York Public Library is developing a citizen cartography tool that lets the public take information archived on digitized historical maps and use the data to tag a searchable interface built with Open Street Map. The goal: a larger, more detailed database that will help future researchers.
• The National Library of Finland created the digitalkoot project to help digitize millions of pages of archival material. Visitors to the site transcribe old books one word at a time while playing a video game. Think CAPTCHA meets Angry Birds.
• The United States Holocaust Memorial Museum (in partnership with the private company Ancestry.com) has recruited “citizen historians” to research historical documents from WWII. The Children of the Lodz Ghetto project is designed to teach historical skills while “restor[ing] names and stories to those whose identities were nearly silenced by a force that nearly succeeded in making them disappear completely from history.”
• Many natural history museums coordinate “citizen science” projects that enlist public help to tackle large research challenges, like collecting and identifying ants, transcribing data from the labels on century-old cicadas or spotting celestial phenomena.
• Many other citizen science projects have elements of crowdsourcing. See SciStarter for more citizen science projects.

In addition, administrative work can sometimes be done. Fansourcing involves recruiting fans to do administrative tasks which are more interesting to enthusiastic fans (brand advocates) than low-level staff. It can connect volunteer fans with potential visitors via live chat, or moderating online discussions and answering customer service questions. The volunteers offer their genuine enthusiasm, not necessarily a deep professional expertise.

Aside from volunteer engagement, the majority of cloud labor is paid. Simple tasks are often paid at hourly rates below $5/hour. See a prior blog post on outsourcing some outreach tasks to freelancers. It tends to drive towards the lowest common denominator, so it’s best for tasks that are suitable for non-professionals. Quality control is often maintained by double and triple-checking work through redundancy. For example, if the task was to write tags describing a painting, the same painting could be tagged by 5 workers, with software to detect spammy responses, and look for tags common to multiple workers.

Politics

Civic Engagement is collective actions that address issues of public concern. This works on both local and national levels. The White House could collect ideas on how to change the manufacturing industry from those who work in it. It asks people which technologies they think are the most important to the industry, as well as what sort of future regulation they believe would be beneficial. Soliciting responses via the internet, in public, eliminates barriers to participation.

Knowledge of the crowd

Collective Knowledge  is development of knowledge assets or information resources from a distributed pool of contributors. This type of mass collaboration is best showcased by the Wikipedia and other Wikimedia projects. Here are some smaller projects which collected votes from their community:

• The Smithsonian American Art Museum invited the public to vote via the Web on which examples of video games to include in its “Art of Video Games” exhibit.
• In 2010, the Walker had 50/50: Audience and Experts Curate the Paper Collection, in which visitors cast over a quarter million votes about which of 183 paper artworks should go into an upcoming exhibition.
• In 2008, the Brooklyn Museum invited visitors to help choose the best photos for an exhibition.

Talent of the crowd

Collective Creativity  taps into creative talent pools to design and develop original art, media or content. This can mean new creative works by professionals, or non-professionals.

• Several museums, including the Smithsonian, The Children’s Museum of Indianapolis and the British Museum, have established positions for “Wikipedians in Residence.” The Wikipedians push museum data and images into the Wikipedia universe, as well as soliciting and managing content from the wiki-editing crowd. (See my blog post on reaching the public using Wikipedia.)
• RunCoCo is advice on how to run a community collection online (see PDF).
• New Zealand was looking to revitalize their tourism campaign, and hosted a contest for young filmmakers. Their reward was the opportunity to screen their work in front of famous filmmaker Peter Jackson, plus a trip to New Zealand to shoot and produce a 3-minute film.
• In 2007, World Without Oil, was a crowdsourced public media narrative which invited players to participate in a collaborative simulation of a global oil shortage by playing an online mystery game, and later generating their own stories about the crisis and strategizing ways to manage it.

Open Sourcing is a philosophy and approach that promotes free redistribution and access to an end product’s design and implementation details. It’s the opposite of keeping secrets or paid licensing. Key benefits are broader use and publicity. Some popular projects are also able to foster a community where people outside the organization also contribute. Typically, revenue comes from selling related services, or grants. For example, Omeka is a web-publishing platform for library, museum, archives, and scholarly collections and exhibitions.

Communities

Community Building  is developing communities through active engagement of individuals who share common passions, beliefs or interests.

Preserapedia is an open encyclopedia for heritage conservation with over 1 thousand articles.

In business

The term was popularized by journalist Jeff Howe in a 2006 Wired article “The Rise of Crowdsourcing,” about outsourcing labor to the “crowd,” but the concept rapidly broadened beyond labor. In business, crowdsourcing now means obtaining services, ideas, content, or money from a large group of people. “Crowdsourcing has become a very successful business model for many startups such as YouTube, Wikipedia, Reddit, Threadless and Kickstarter – to name only a few. But so far, its usage by big companies has been sporadic and experimental,” notes François Pétavy, CEO of eYeka, a crowdsourcing platform. Pétavye says that use in the business world is growing, and that crowdsourcing now solves a variety of real world problems, have a demonstrable return on investment.

It is related to other evolving concepts. For example, Open Innovation is using of sources outside of the entity of group to generate, develop and implement ideas.

Tools are applications and platforms that support collaboration, communication, and sharing among distributed groups of people.

Source: The categories above, and the lede illustration, are adapted from Crowdsourcing.org. Several projects from AAM TrendsWatch 2012 (PDF).

The print dictionary lasted for two and a half centuries, tracing back to 1755, when Samuel Johnson published A Dictionary of the English Language, the first comprehensive, reliable English-language dictionary. Johnson’s dictionary was used by the few wealthy, literate individuals and institutions who owned books, and was the standard for over 150 years. The consumer market for dictionaries expanded with the Industrial Revolution of the mid-19th century, when paper and books became generally affordable. Despite the importance of dictionaries to literacy and communication, there have only been a handful of major dictionaries.

Although a few dictionaries become commonly accepted as standards, there is no grand authority in charge of the English language.

Dictionaries are creative works which describe the ever-changing way a society uses language. Thus, if you start reading dictionaries, you find a cacophony of subtly-different definitions. For example, a “crosswalk” could be:

• a path marked off on a street to indicate where pedestrians should cross (American Heritage)
• a crossing lane marked off for pedestrians (Collins)
• a specially marked place for people to walk across a street (Longman)
• a specially paved or marked path for pedestrians crossinga street or road (Merriam-Webster)
• a marked part of a road where pedestrians have right of way to cross (Oxford)
• a place where pedestrians can cross a street (Wiktionary)

Why the variety of definitions? For one thing, copyright law necessitates that publishers find new ways to say the same thing. But also, language evolves over time, and publishers seek to serve consumers by hiring dictionary editors to invent new approaches to their craft of dictionary writing.

Unfortunately, even for the publishers like Bullon, who get digital, the economics of dictionary publishing is unsustainable. Dictionaries have a simple, obsolete, business model: A huge up-front investment, recouped over years (or decades) of book sales.

Depending on the publisher’s budget and marketing priorities, they hire lexicographers to create definitions from scratch, or paraphrase existing dictionaries. The Webster’s Third New International Dictionary took a decade to create. Finally published in September 1961, the team of lexicographers led by  Philip Babcock Gove spent 757 editor-years and $3.5 million ($26 million in today’s dollars) to build their unabridged dictionary with 450,000 entries. To create a new, medium-sized dictionary with 35,000 words would cost hundreds of thousands of dollars (for paraphrasing) to tens of millions of dollars (for new definitions based on examining real-world usages).

Sadly for dictionaries, in this digital age, it’s impossible to earn much money from subscriptions and advertising, and unrealistic to earn millions from dictionary-type apps. There’s a stagnation of the older dictionaries — and new approaches brewing which use crowdsourcing and artificial intelligence.

The following is an overview of most major dictionaries currently available, and their online/mobile offerings.

Free dictionaries:

• Wikimedia Foundation — Wiktionary — Wiktionary is a free web site, and a data source for many dictionary web sites and apps. — Started in 2002, Wiktionary has over 3.1 million entries in English. Similar to Wikipedia, it’s a collaborative project that anyone can edit, aiming to describe all words of all languages using definitions and descriptions in English. Many of the entries are either based on the other three free sources below, or paraphrased from existing dictionaries. Many of the entries were automatically generated, adapted from commercial dictionaries. It can be used freely, as long as it is credited.
• Princeton University — WordNet — WordNet is a data source for many thesaurus-type web sites and apps. — WordNet is a different kind of dictionary. Instead of a dictionary of words (where each word has one or more senses), it is a dictionary of “word senses.” It contains 117 thousand sets of synonyms, each for a specific meaning (e.g. “a sudden brief burst of bright light”). It can be used as a dictionary by listing all the meanings in which a word appears. WordNet also contains semantic relations. It can be used freely, as long as it is credited.
• Public domain — A Dictionary of the English Language — Scanned pages available from Internet Archive, with error-prone OCR. — Published in April 1755, and written by Samuel Johnson, ‘A Dictionary of the English Language,’ also known as ‘Johnson’s Dictionary,’ was influential. Prior to the completion of the Oxford English Dictionary, 173 years later, Johnson’s was the pre-eminent English dictionary. Funded in 1746 by a consortium of London’s most successful publishers, including Robert Dodsley and Thomas Longman.
• Public domain — Century Dictionary and Cyclopedia — Available in full-text form at Wordnik, or Global Language Resources. — The largest, out-of-copyright dictionary. One of the largest encyclopedic dictionaries of the English language. Published from by The Century Company of New York, in multivolume versions, with 7,046 pages. Last updated 1914.

Provides an API:

Application Programming Interfaces (APIs) allow web sites and apps to access dictionary information in realtime. This is a great way to get information used in a variety of ways, inviting other developers to create innovative web sites and apps that use the dictionary data.

1) Scan the UPCs from your books. The UPC code is  the ISBN-13 code. There are many apps for this. I used the Red Laser app, available for iPhone, Android, or Windows phone. Make a “list” from within the app, then from within the list, start scanning with “multiscan” on. When you are done scanning a few dozen, or hundreds of books, email yourself the list. The CSV file will contain ISBN-13 numbers and the book titles.  (If your books don’t have barcodes, or you are not selling books, skip steps 1 and 2 and enter them manually in step 4.)

2) Make a spread sheet file, e.g., with Microsoft Excel. These are the column headings, and the values for each:

• sku – this can be anything unique, I duplicated the ISBN here
• product-id – the ISBN from Red Laser
• product-id-type – this explains what the “id” is. Put a “2” in this column, that means “ISBN”
• price – put “1000” (will explain later way)
• item-condition – 2 (means “very good”)
• quantity – put “1”
• add-delete – skip this (leave blank)
• will-ship-internationally – y
• expedited-shipping – y
• standard-plus – y
• item-note – Add any note you want buyers to see
• fulfillment-center-id – leave blank

This is all easier than it looks. See documentation about the row headings. You can “fill down” in Excel to repeat the same values. Export the file as a tab delimited file.

3) Make an account with Amazon to start “selling professionally”. This costs $39.99/month, one month free. Go to sign up. Once you have an account, you will log in to the “Seller Central.” They have a free plan, but you can’t do bulk uploads.

4) Upload your spreadsheet containing your listings to Amazon. Go to “Inventory” > “Upload products and inventory”. Under “Upload inventory files”, choose “Inventory loader file”, then click the “Upload Now” button. Wait for a few minutes, then look in the “Inventory” > “Manage Inventory” > “All My Inventory.” You should see all your books. They will all be priced at $1000.

5) Price your books to the market price. Select all your books, and choose “Actions” > “Match Low Price”, and then “Save”. Repeat as needed for additional pages of books. This will set your price to what customers will expect to pay. You can sort by price, and then delete any books which are too cheap to be worthwhile. Most serious books about arts and science should be valuable. Then select all your books, and choose “Actions” > “Change to Fulfilled by Amazon”. Now the system is waiting for you to mail them. You can always set any price you want.

6) Send your books to Amazon. From “Inventory” > “Manage FBA Inventory”, choose your books, and mark them for shipping. Print PDF product and shipment labels provided by Amazon. Pack your books into your own boxes. You will put one label inside each box, and tape one to the top.  Use Amazon’s discounted shipping or select your own carrier. (A box will cost $5-10 to ship with UPS).

7) Amazon stores your books. Amazon catalogs and stores your products in their ready-to-ship inventory. Amazon receives and scans your inventory. They record Item dimensions for storage. You monitor inventory using their integrated online tracking system.

8) Customers order your books. How soon this happens will depend on the popularity of your books. Customers search for and purchase your products directly on Amazon.com, or on other e-commerce channels such as your own site. Your listings on Amazon.com are eligible for Amazon Prime and Super Saver Shipping, and rank by price excluding shipping. You can expand to list and fulfill your products globally.

9) Amazon picks and packs your products. Fulfillment by Amazon picks your products from inventory and packages them. Amazon locates your products using their advanced web-to-warehouse, high-speed picking and sorting system. They manage your order volume – whether you get a few orders a day or a thousand. Customers can combine your orders with other products fulfilled by Amazon.

10) Amazon ships your products and provides support. Amazon ships products to customers from their network of fulfillment centers. Amazon ships customer orders using the method they choose. All outbound shipping costs are included in your service fees. Amazon provides tracking information for customers. For orders on Amazon.com, customers can contact us for customer service.

Then you get paid from Amazon. They can pay personal or business accounts.

This same process works for items other than books, e.g., T-Shirts. The basic process is the same, but steps 1 & 2 differ. Amazon has extensive help pages. If you can post an ad to CraigsList or eBay, you can figure this out too. Remember to cancel your monthly subscription once you sell everything. For anything that doesn’t sell, you can have Amazon dispose of it or mail it to you for a nominal fee.

Get volunteers involved! The work goes quick with a few extra hands, and once you mail off the boxes to Amazon, there’s nothing else to do.

Tom Segal is an analyst for a venture fund focused on education technology.

To counter the argument that questionable ethics play a role, Tom Segal points out that “schools deal with for-profit institutions at every level.” He argues that expanding educational technology to the private sector also creates competition which in turn would allow schools to choose the best technology. Nonprofit institutions limit the amount of money that can be put into innovative development. Technological advances will not occur as quickly and with as much innovation unless the private sector is able to become part of the process.

Segal is focused on for-profit vendors selling technology to the schools — not on the wholesale commercialization of schools.

Jon Bower responded, “While we are starting to engineer education processes around student learning, much of the instruction process is still an art. Creating good processes and embedding them in technology is difficult, as companies from Brøderbund Software to Kahn Academy have learned.”

Go to article “Incentivizing Innovation in Education; or A Role For For Profits in Education” >

Their new system, ReadCube Access, has an iTunes sales approach: a library can rent an article for less than $6 or can buy the same article for $11 (or less, depending on the source). Hrvatin and McGrath hope that ReadCube Access can not only lower expenses for universities, but also allow more journals to reach students around the country.

Other features allow for importing PDFs, finding new papers via Google Scholar or PubMed, downloads via institutions’ paywalled logins, recommendations, annotations, and citations.

They sold their idea to industry giant Nature Publishing Group and to the University of Utah’s library system. (See their blog post.)

Translations

In our global world, there’s no excuse for staying limited to English speakers, especially when there’s greater need for education in non-English places. Even if you have multilingual staff, most translations will be better and more cost effective if you outsource. Professional translators are efficient, and skilled at adapting idioms and phrases. They tend to be detail-oriented, soft spoken individuals.

ProZ – Text translations of any size, e.g., articles, reports, books. ProZ also has interpreters. Quality translation into a Western language costs $0.07-$0.15 per word, languages from developing economies cost a bit less. Target the upper range, e.g., €0.10 EUR per word if you were going to French. If you know a native speaker, have them review a few of the most promising applicants’ samples. The workflow is that you send the translator your text, and they email you back the translation. If you have large volumes, a translator can translate 30k words a month, going up to 50k if you have bursts of work. Over 300 thousand translators represented.

ICanLocalize – Software translations, such as the short text labels which are part of any web site or app. Little texts like “Back,” “Next,” “Go to article.” We recently used them to translate our WikiNodes app into 18 languages. The cost is $0.10-0.15 per word, including their 20% commission and a 50% fee for peer review. Your programmers first consolidate all the text blurbs from your app into single files (“localization”). Conveniently, you can upload many common formats to their site, and then download them back to put in your app a few days later. Their translators are savvy to the nuances of software localization, and will know that “back” is navigation, not the region between your neck and your buttocks.

Voiceovers

Voice123 – If you are creating a radio advertisement, podcast, game, or need narration for a video, and you want a specific kind of professional voice, there’s an infinite world of voiceover artists. Voice actors have a range of styles, from youthful to movie trailer. Voice123 has had 220 thousand voice talents sign up, giving 3.1 million audition samples, since being founded in 2003. There are a few voiceover marketplaces, Voices.com is another option. With both sites, voice actors — with a diverse range of prices and skills — will give you short samples to listen to. These professionals have studios (microphones and audio software) in their office, and can often send back short recordings within a matter of hours. A ~1 minute recording (e.g., voice track for a video) could cost $50-100. But the price is not linear, so 2 minutes is not 2x the cost of 1 minute.

Simple, repetitive tasks

Some tasks need a person to be involved (i.e., can’t be done by a computer), but don’t need a highly skilled professional. For example, reviewing thousands of blog posts or comments for spam or abuse, categorizing posts or photos, retyping entries from print or PDF into a spreadsheet, or looking up thousands of web sites or Twitter handles for your members. Workers can be paid around $3-10/hour, typically paid 5-10 cents per task. You never know who the workers are, it could a a retiree in Montana, or a 19 -year-old in Delhi.

CrowdFlower makes these tasks easy to manage. They deal with the details of hiring remote workers, and charge a commission for serving as an intermediary.

If you want to go straight to the source, check out Amazon Mechanical Turk (which CrowdFlower and others use as their backend). With Mechanical Turk, you define small tasks, called a “HIT,” (e.g., looking at a photo from your site, and assigning a category), upload the task as a spreadsheet file (each spreadsheet row is a different photo URL). Depending on the price you set, you may have none, or dozens of workers working on different rows from your spreadsheet at once.

Transcription

Converting audio to written text is tedious. Several services will manage the details of finding transcriptionists, and managing them. (Behind the scenes, these services often use Amazon Mechanical Turk).

CastingWords is one of many transcription services. You email audio or provide URLs, and 1-10 days later, you receive a text file. Price depends on speed. $2.50/minute for 1 day turn around, $1/minute for slow turnaround.

Programming

There are two main sites for hiring web programmers for defined tasks like creating web sites, writing simple blog posts or making web graphics. These can also be a good way to find an ongoing part time web master, if you first hire then for a small, defined task.

ScriptLance is the best site for freelance programmers, and has good management features. No need to pay extra for a ‘featured’ project.

vWorker  is a bit more cumbersome, has fewer users, but tends to have slightly more technical workers. There’s no harm in posting jobs to both sites. (Previously called “RentACoder”)

Local help

If you need local help, or need to work in person with the worker — take a look at your local Craigslist and Idealist.

Everything else

From accounting to data entry, writing to design, transcription to programming, the Coke and Pepsi of outsourcing are eLance and Guru. We tend to have a slight preference for eLance, but they are similar. Both sites are focused on web sites, ecommerce, programming and databases, but also include creative tasks, administrative support, and accounting and legal (e.g., drafting a terms of use for your web site).

Below, this map from eLance shows where their talent is based. Darker blue is more common:

eLance – Claims over 68 thousand jobs posted in the last month, over 1.4 million registered contractors, and nearly $550 million in projects to date, with $43 million in 2011.

Guru –  Guru touts that over 1 million users have completed over 3 million tasks, working on over 150 million dollars in projects (average of $50 per project, though project budgets can often go into the $thousands). Today they have 4716 open jobs.

Freelancer.com – A slightly smaller community, but could be worth posting here also. Over 3.5 million freelancers registered.

Specific skills

oDesk – Specializes in buying time from workers on an hourly basis. Their workers tend to be more highly skilled, so expect higher hourly rates — in the $20-100/hr range. But if you have a specific technical task, this can be a place to fine workers who are not on the other sites. Some other sites (e.g., Freelancer) are expanding into hourly work also.

Design

Designers and other creatives can be challenging to track down because the best designers tend not to use the above sites. Here are some options:

Sortfolio – Created by 37Signals (who created BaseCamp project management software), provides a convenient way to browse portfolios and filter by budget ranges and location. Has a lot of good designers on board.

Coroflot and Behance – Browse portfolios for various kinds of creative work, and then write personalized emails to a few designers praising their work and inviting them to work on your project. Both sites also have full time job boards. Coroflot has 1.4 million images of creative work, from the portfolios of over 150,000 creative professionals and students. Behance has 10.6 million images, 1.6 million projects, and recently raised $6.5 million in funding from investors including Jeff Bezos.

Other marketplaces?

Do you have experience with other marketplaces? Share them in the comments…

Readers are acclimated to ads, but you have to use good taste. “Many readers may blame you for misleading claims or ugliness in ads, and they may think the ads undermine your editorial integrity,” says Rennie. In a few fields, readers enjoy ads (e.g., SuperBowl ads, or fashion magazines), but in science and cultural fields, that’s rarely the case. They best you can hope for is that your ads are benign.

This article examines the ad business on a broad level, and looks closely at how to use or launch an ad network.  

Online advertising is big business

In 2011, global online advertising was ~$73 billion, comprising 16% of the $464 billion spent on ads of all kinds (print newspapers, magazines, television, radio, cinema, outdoor, internet). In four markets (Denmark, Norway, Sweden and the UK), internet advertising already accounts for more than 25% of total ad expenditure.

In a sign of the times, U.S. online advertising will exceed print for the first time in 2012. Print advertising continues to erode in the U.S., and is projected to fall from $36 billion in 2011 to $33.8 billion in 2012. Meanwhile U.S. online advertising is projected to grow 23.3% to $39.5 billion this year. Mobile ad spending is growing rapidly, $1.5 billion in the U.S. in 2011.

 Google is the top dog of online advertising, earning half of all global online advertising revenue, $36.5 billion in 2011. Another 11% of the world’s digital ad spending is earned by four other major companies, Microsoft, Yahoo!, AOL and Facebook. Collectively, these top search engines and portals have 61% of the ad market.

The other 39% of the market is most often served by ad networks. Publishers work with one of more ad middlemen — ad networks — who recruit and manage advertisers, serve ads, bill advertisers, and pay publishers. Google is also in this business, paying publishers ~70% of ad revenue. In 2011, Google paid out $7.3 billion to publishers who allow Google to place ads on their network. Other major ad networks include: ValueClick,  Tribal Fusion, AdBrite, Burst Media, Casale Media, isocket, and Blogads. (See top 50 ad networks from ComScore.)

Finding appropriate advertisers

In the realms of science and culture, these major ad networks may pay too little, or have irrelevant ads for your audience. Smaller advertiser networks “should be a good model for culture blogs, since it’s a valuable niche audience if you carefully manage the list of sites,” says Barry Hoggard, co-founder of an arts/culture ad network, Culture Pundits, which ran from 2007 to 2012. Hoggard says a custom ad network “allows the sites to have more control over the kind of ads that appear. General networks like AdSense can show some pretty lousy ads — credit cards, correspondence schools, etc.” Further, “when it’s working,” a custom ad network can pay higher revenue than a general network.

But if your primary audience is students and their educators, and the context is educational settings, ads may be inappropriate. If you run ads on these sites, since your site is positioned as an expert source, and you have a young, potentially naive audience, you have an extra responsibility to have nonintrusive ads that are not misleading.

If your audience is the general public, a large ad network like Google’s AdSense can be fine. A good quality ad network excludes misleading and scammy advertisers. We use AdSense on WebExhibits.org, and the ads tend to be nonobjectionable, drawing $0.64 CPC and $1.60 RPM in 2011. (Acronyms defined below.)

If you have a niche audience, of professionals in your field, or knowledgeable enthusiasts, you could  build your own pool of advertisers. Hoggard says a good minimum for a culture-oriented audience is 1 million page views a month. He says, “Below that, it can be hard to get the attention of a lot of media buyers.” However, “if you have a niche audience with good demographics that is attractive to advertisers… you can be a lot smaller.” Rennie agrees with that ballpark, for science-related sites, and estimates that a “few million” monthly page views is the minimum for interesting major media buyers directly.

Culture Pundits ad network

Culture Pundits was a well regarded arts & culture ad network that operated from 2007 until January 2012. Culture Pundits was founded by Barry Hoggard and Tom Schreiber. Their sites served an audience of arts professionals and enthusiasts. Their member sites were all cultural blogs or art, film, literature calendars. They excluded sites that focused on design or fashion, although those sites have higher traffic. Their advertisers tended to be art-related institutions, galleries, publishers, museums and businesses.

They ad network launched in August 2007 with ten sites, expanding to 40 websites, and 1 million page views per months, by September 2008. By May 2009, managing the ad network became unwieldy, with advertisers requesting targeting of specific web sites, and geo targeting. To make things more manageable, Culture Pundits tried reducing the pool of advertisers to 10 sponsors each month, cut back to 25 sites, and reduced visual clutter by delivering a single medium-size ad (300×250 pixels) across all of the sites in the network.

The network paid 60% of revenues to its publishers. The balance of 40% paid sales commissions and administrative costs, with a little left over to pay the two founders’ salary.

Ultimately, the network, while profitable, was not a good business. Over 5 years, ad sales fluctuated. They sold out their full ad inventory 3 times, but most months, they sold only a third. When they closed down the ad network, they were drawing in ~$2k/month, paying out ~$1200/month to member sites — each site earning less than $100/month, on average.

Hoggard says, “It’s a lot of work to educate organizations about the value of a niche network like ours, versus just doing targeted Google AdWords campaigns. There is a lot of hand-holding.” His colleague, Schreiber, did most of the negotiations. Other tasks included managing the ad serving software, managing finances, and providing technical support to publishers and advertisers. There were often delays to get creative assets for ads, and also to get paid. In all, it became too time consuming.

While they had found a good niche, with satisfied publishers and advertisers, many of whom made repeat ad buys, Hoggard says, “In the last six months [of 2011], our revenue was really down compared to any other six-month period of the last 3-4 years.” Given the other projects he is working on, Hoggard decided to spend his time on making those more successful.

More is more: History News Network

George Mason University’s History News Network (HNN) has been running for over a decade, and has had ads the whole time. The site has over 300k unique visitors a month. Rick Shenkman, the site’s founder and publisher, says that up until three years ago,  “publishers bought up most of our ad space. Today the publishers are barely advertising and we are mostly using ad networks.” Shenkman spends 6-8 hours a month managing his advertising. His approach uses a lot of ads:

The top ads are managed in-house, and link to pages at George Mason, or well respected TV, magazine and retail advertisers. They charge $400/month for  banner ads on the lower sections of their pages. When they don’t have interested advertisers, they are at the mercy of their ad network, which was promoting two misleading sites that hustle readers to get on mailing lists for commercial online and technical colleges, a scammy site selling questionable health supplements, and Johns Hopkins. At the bottom of their page, they run irrelevant text ads for discount men’s watches, ways to make money fast!, and a misleading site for Green Card applicants.

Less is more: The Deck and InfluAds

The opposite approach is to have a single, small ad on a site. Small ads usually pay 3-5 times better, and deliver a better reader experience than the the “crappy ad clutter culture that exists on the web today,” says Anibal Damião, founder and CEO of the ad network InfluAds. But total revenue can be less than packing a site with 5-10 ads.  This is what one ad looks like:

Single, small ads. Left: reformrevolution.com with an ad via InfluAds; Right: alistapart.com with an ad via The Deck.

The single ad approach was pioneered by The Deck network, which delivered 114 million ad impressions in January 2012. The ads are 120 pixels wide by 90 pixels tall and also allow for up to 80 characters of text to accompany the image. It’s an appealing model. The Deck charges $8300 USD per ad per month, and rotates 33 ads a month.

“[The Deck] inspired us and our objective is to democratize that model,” says Damião, whose network offers a number of niche topics, and runs small ad sizes (130×100, 154×70, and 250×250).  At Culture Pundits, Hoggard also used single ads, though he found that a slightly larger size (300×250) was “better-suited” for their arts & culture advertisers.

It’s an issue of quality of quantity, says Damião, “We are publisher + quality focused, while other networks are advertiser-demand + quantity focused.” For business-related sites, InfluAds pays around $4-5 CPM, and design-related sites tend to earn around $1-1.6 CPM. InfluAds has a full dashboard for publishers to monitor their accounts, and pays 10 days after the month.

Freethought & 3 Quarks

To see how this works, let’s look at how two blog networks have set up their advertising…

Freethought blogs is a blog network of 31 blogs, centered on a theme of independent thinking and skepticism. In December 2011, the site earned ~$10k. The blog network had 6.4 million pageviews in January, averaging 200k pageviews daily. The site is a spinoff of Scienceblogs begun by PZ Myers and Ed Brayton, and has some of the same contributors. They had ads from when they launched. “I spent several thousand dollars to get the network off the ground and it was the intent from the start to derive revenue from the network and to pay all of our bloggers based on their percentage of the site’s full revenue,” says Brayton.

When they first launched in August 2011, Brayton tried to manage the ads himself using Google Adsense and  other networks, putting them into a Doubleclick server. “But it was very time consuming and our revenue was pathetically low,” so he decided to turn advertising over to a private ad manager who runs the ads for independent news and politics sites, Raw Story, Alternet, Crooks and Liars, and some others — which immediately tripled their revenue. Brayton occasionally does direct ad sales, augmenting the advertisers solicited by his ad manager.

Freethought has five ads on each page: A  wider banner (728×90) on the top, and four medium size squares (300×250). In December 2011, Freethought had about 5.7 million page views, and their revenue through the Raw Story ad manager was about $6800, with a total CPM rate of ~$1.20. They draw additional revenue from a popunder ad from 3I Interactive, that earns about $800 a month, and they also run their own Google Adsense ads on our RSS feed and search feeds, which bring in anywhere from $400 to $900 a month. And he has some direct ad sales of ~$500 a month.

3 Quarks is a filter blog with diverse contributors, collecting information from around the web on science, design, literature, current affairs, and art. S. Abbas Raza, the founder and editor, says they had 430,000 impressions last month. The site launched in 2004, and they’ve had ads for over 5 years. Running ads was an easy decision. Raza says, “We didn’t wring our hands over it at all. We wanted to make some money, and so we ran ads.” He spends 2-4 hours a week managing his advertising. Their ad revenue varies widely.

The advertisements run in a strip on the right side of the screen. Raza is satisfied with his advertisers. They use Blogads, Google Adsense, and a few direct text link ads. Today, they have text ads for tickets at Las Vegas shows and Florida theme parks, and image ads for several books, a professional society, a legitimate institution promoting online courses on social research, and five nonobjectionable Google text ads.

Sponsored posts

Another form of advertising is sponsored posts, which can be appropriate for some sites. A good example of sponsored posts are these posts at business analysis blog Asymco for Carnegie Mellon University, and mobile app Scrivener. The sponsored posts on are topics the blogger would not normally cover.

The sponsorship network, The Syndicate, charges advertisers $2,750 USD for a single post on sites with combined 1,500,000 monthly page views. The Syndicate sends the blogger possible topics to approve, and draft text for the post. The blogger edits a short description of the product or service she or he’s willing to endorse, adding a personal note. Sponsored posts can be paid a flat fee, or the publisher can receive a commission for each click.

Advertisers like sponsored posts because they are an endorsement of sorts, and also catch readers attention. They also workaround the reality that many web readers use ad blockers, so some traditional ads get ignored. Brayton notes, “I don’t know how many total impressions we serve each month. Some of our visitors have ad blockers, of course, so it’s lower than the number of page views.”

Selling your soul?

More broadly, if you are thinking about adding ads, will you lose your  integrity? Not necessarily. There is vast precedent you can draw on.

“The first, most important principle is to avoid not just conflicts of interest but the perception of conflicts of interest, in so far as that’s reasonably possible,” says Rennie. “Conflicts of interest are bad for business because they make the editorial look untrustworthy and the advertisers look dishonest. Depending on the sensitivity of a topic and the audiences coming to read about it, the bar for avoiding those conflicts may get higher or lower.”

Traditional publishers have been dealing with conflict of interest for decades. The American Society of Magazine Editors outlines principles for the dance between editorial and advertising content:

• Readers are entitled to fair and accurate news and information
• The value of your site to advertisers depends on reader trust
• The difference between editorial content and marketing messages must be transparent
• Editorial integrity must not be compromised by advertiser influence

See their detailed guidelines, most of which apply to your site, or at least offer good food for thought.

“Most of the time when online publishers get into trouble with editorial-advertising conflicts, they’ve done something that goes against these longstanding rules,” says Rennie. “For example, in the notorious Pepsi-gate fiasco at Scienceblogs a few years ago, the publishers made the mistake of letting the sponsored blog have the same look and feel as the rest of the editorial network, without adequate labeling. (They made a bunch of subsequent mistakes, too, but this was the completely avoidable root error.)”

Talking the talk

The world of online advertising has its own language. A quick primer of important terms:

• Impression – Each time an ad is viewed. There can be multiple ad impressions per page.
• Publisher – This is you. The term is a holdover from the days of print marketing.
• Flat-rate – An ad runs on a site for a specific time period. Ad could be 100% of the time, or rotate with others, to appear with a specific probability.
• CPM – Cost per 1,000 impressions. (M is the Roman Numeral for one thousand.)
• CPC – Cost per click. Advertisers only pay when a reader clicks their ad, regardless of how many impressions. This is how Google’s ad system, and many others work.
• CTR – Clickthrough rate. A high CTR means ads are more appealing to readers. Number of clicks ÷ number of impressions.
• CPA – Cost per action or acquisition. Advertisers pay a commission for an action, such as becoming a new customer. e.g., if NetFlix paid a $20 commission for new customers. Similar to CPL, or cost per lead.
• RPM – Page revenue per thousand impressions. Earnings ÷ number of page views * 1000.
• eCPM – Same as RPM, but from the advertisers’ point of view. Effective cost per 1,000 impressions. CPC/A * CTR * 1,000.
• Text vs. Image or rich media – Text ads tend to be 1-2 sentences plus a link; Images are a graphic; Rich media are animated or have other interactivity.
• Mini, small – Small, subtle ads. They don’t annoy readers, and fit into many layouts.
• Skyscraper, Hi-rise – Vertically oriented ads, typically tall and skinny.
• Leaderboard, Marquee – Horizontally oriented ads, typically full screen width, and short.
• Roadblock – Exclusivity. Advertisers gets 100% of the advertisements on a site for a short time, useful for important announcements and product launches.
• Sponsored post – An alternative to ads; a sponsored post is part of the site’s content, and is clearly marked. Analogous to an advertorial, but not misleading.
• Above the fold – Newspaper display space is more valuable “above the fold” on the front page, as is the section of a website visible without having to scroll down.

Making your own

It’s possible to run your own ad network, but profit margins are tight if you focus on a niche audience. A new ad network could serve multiple publishers, or one blog network. While there are already a lot of science blog networks (see my post on blog networks last year, or this list from scienceblogging.org), there’s room for more blog networks that focus on specific fields (e.g., neuroscience, mathematics, etc.), or new networks that focus more on the general public and/or content that will be useful for educators to use in a classroom.

In culture and the humanities, there is a lot of potential for new blog networks, and many untapped advertisers. Culture Pundits solicited advertisers who were museums, galleries, shows and retailers. There may be potential interest from retailers (e.g., a retailer selling antiques for history-themed sites), online courses, and industry-specific vendors for sites that target professionals, and other advertisers.

To launch an ad network, your publishers will insert a piece of web code (HTML and JavaScript into their web pages, and that code will load the ads. You can make it as simple as one code = one ad, or run a complex system. Google’s DoubleClick for Publishers is free for small and medium-size sites, and provides extensive options for scheduling ads.  Hoggard started using a commercial ad manager, OpenX, for ad serving, but there were regular security/hacking problems. Later, they switched to DoubleClick, and it “worked very well for us.”

If you want to partner in making new ad network for a science or culture-related sites, and have over 1 million (preferably 2-3 million) views a month, InfluAds wants to expand their networks, and is a possible place to start.

Keeping it simple

Apps just need to be good enough. No need to get too fancy or reinvent the wheel. While custom apps run from $25-100k, many vendors will create an app for you for less than $25k, and some for well under $5k. This is a summary of the vendors offering apps for less than $25k at yesterday’s 3rd Museums & Mobile online conference.

The key to a tour app on a budget is creating a ‘templated’ app. You upload content (“tour stops”) to the vendor’s online content management system (similar to creating blog posts), and then the vendor packages your content into their pre-existing framework, creating an app that gets submitted to the Apple or Android app stores.

Summary of vendors: 

TourBuddy: List of tour stops from 'Savannah Walking Tour', detailed view of a stop, and their GPS view. Good for large outdoor areas.

TourBuddy – $1-2k per app, with reasonably priced updates, and options for additional features. Inspired by the IKEA model of anchoring on a low price, yet delivering good design, Yvonne Jouffrault, says Tour Buddy is creating apps for organizations who want an affordable app, and don’t need a lot of customization. Jouffrault says GPS maps are one of their best features. The GPS works well for any large, outdoor location with a clear view of the sky. The map mode allows users to pan over a map to choose a tour spot, or see their location on a Google map. TourBuddy has created approximately a dozen apps so far, and their apps run on both Apple iOS and Android. Coming soon, TourBuddy is adding a web app option, a better organization of tour stops, and more visual styles. Later in 2012, Tour Buddy may add support for tablets and video. They have two apps with many ratings: their “Savannah Walking Tour” has a free and $10 version, and 4/5 stars in the Apple App Store; “Dutch Utopia” is free and has 3.5/5 stars. See pricing.

OnCell iPhone App: On the left, list of stops, and map view. On the right, a view from their sample web app.

OnCell – $1.5k for audio tours, $3.8k for multimedia. OnCell built their business creating audio tours run from cell phones, i.e., visitors call a phone number and enter stop numbers into the phone to hear recorded tour messages. Their call-in business has hundreds of clients in the US and Canada, including many major institutions like the Grand Canyon and 80+ other National Parks, the Met, the Smithsonian, etc. The call-in market is being disrupted by the tour apps, which don’t require visitors to make a phone call and burn their cell phone minutes. They have two apps so far on the “OnCell App” platform, the “New Orleans Jazz” app was published this summer, but has no reviews yet. See pricing.

TourSphere: Underground Railroad app, list of stops, a stop, and a keypad mode. Not shown is a non-GPS map mode.

TourSphere – $500-900/month for native apps. $400/month for web apps. They have a CMS for uploading content. No upfront fees, no contracts. The price depends on which combination of phone and tablet sized iOS or Android apps you make. Their ‘National Underground Railroad Freedom Center’ has 3/5 stars. See pricing.

Tristan: Browsing stops and viewing detailed information in the 'Canadian Museum of Civilization' and the 'Infinity of Nations'

Tristan – $8-15k per app for an exhibition guide and walking tour. Tristan’s authoring platform is called Autor. Vanessa Vanzieleghem, Tristan’s Global Sales Manager, notes that Autor supports up to 7 major languages, and publishes to iPhone, Android, Blackberry, iPad and creates a web app. Their more expensive, custom apps have ratings in the range from 2.5/5 stars (“Phillips Collection,” U.S. app store), to 3.5/5 (“The Art at Cowboys Stadium” and “Smithsonian: Infinity of Nations,” U.S. app store), to 4.5/5 (Canadian Museum of Civilization, Canadian app store).

GuideOne: Alaska Native Heritage Center app. Alternating between portrait and landscape orientations, the app has coverflow-like and map methods to browse stops. The stops play as videos. At right is a scavenger hunt, visitors go find the object and answer a quiz question.

GuideOne – $15k+ per app. Their “plug and play” apps draw on a number of preset modules: audio tours, zoomable floor maps and images, scavenger hunts and quizzes, membership/donation features, and links to social networks. GuideOne has lowered their prices and improved their offerings since the Spring. According to product manager, Juan Sanabria, their Alaska Native Heritage Center app launched in May (4/5 starts), the Inupiat Heritage Center app launching soon for iOS and Android. Their app for the Chrysler Museum of Art (Norfolk, VA) was released in February, but has only garnered one review. Coming soon is a Boston Historic Sites iPad kiosk and Android and iOS app launching in launching in Spring 2012. Their apps have an offline mode, but the option to download content into the Alaskan app didn’t work for me on a high-speed network.

KanvaSys' Biosphere app: List of tours, map mode (non-GPS), a tour stop, and empty search results for "ecosystem" with no alternative suggestions.

KanvaSys – $20k+ one-time, or $8k+/year if hosted. KanvaSys is part of Ideeclic, a Canadian company which began creating web sites and educational games for the Alberta and Quebec school boards. Their platform is called “inSitu,” and they can skin it for different apps. KanvaSys’ latest app, “Biosphere” for the Environment Museum is free, but doesn’t have many reviews in the app store yet. Their “Cleveland Metroparks Zoo” app has a 3/5 rating.

Adlib – €2k EUR/ year for a 3 year contract, plus the cost for underlying Adlib Museum system, a local collection management system. The advantage is that the app content comes from the collection management system, so it does not need to be re-entered.

CMS and custom hardware from Antenna International.

Antenna International – Full pricing is not disclosed. Antenna is a big player in audio tours. They have two billing models. Their ‘Universal’ plan involves Antenna taking the risk, owning all the content, and handling all the logistics of the hardware. All visitors have access to the tour as part of their admission fee, and the museum commits to pay Antenna approximately 50 cents per head for 3-5 years. So a million visitors = $500k to Antenna, with no upfront cost to the museum. Their other main option is ‘leasing’ which costs $17-20/month per device. As with the Universal plan, Antenna handles logistics (headphones, maintenance, repainting damage, staff training on best practices, and delivers sufficient devices to meet demand). Often museums have seasonable demand, so Antenna will ship over a few hundred (rarely as many as 1000) additional devices for a few months. A baseline of 250 devices would cost around $55k per year. Prior to launching their proprietary XP Iris device, which is a touch screen and number pad, they had options to buy iPod Touches, but Antenna is phasing out commodity hardware, and pushing customers to use their devices. A downside of working with Antenna is that you usually don’t own your own tour, and especially not any celebrity voices they might recruit, so you can’t spin the content into less expensive tours, post them to YouTube, or use them for any other purposes in the future. Antenna has a new CMS, called “publisher” which allows organization staff to rearrange a tour, or upload a new piece of content. Examples of the CMS in use, offered by Ken Husband, was the marketing department adding a “stop” promoting a sale in the museum shop, or the curators removing a stop for an item which is on loan.

Other vendors at the conference are higher-end, full service firms who create custom apps. These vendors are often vague about their pricing, though  vendors try to keep costs down by reusing features between apps. Full service firms at the conference were: Art Processors, EarPrint, Espro Acoustiguide Group, GVAM, and Imagineear.

Why native apps?

With a native app, visitors don’t have to struggle to read your web site into their small screen. And while you should have a mobile-formatted version of your web site (a web app), native apps give organizations better branding with an icon directly on audience’s home screen, and provides a smoother and richer user experience. Plus, a native app can work when there’s no internet connection.

One important detail: Make sure you are the official “publisher” of your app, so that when you improve the app in the future, or change vendors, your visitors automatically get the new version of your app.

Update: 27-Oct-11. Added new information provided by Tristan.

Three suggestions

For a real-world example on a recent project, designer Ben Snyder said on his blog, “the purpose of the user test is to get feedback about the new design to understand if there are any parts of the website that are confusing to users, and to test the site for hidden bugs that might prevent a user from getting the information they need” or prevent them from completing actions on the site.

Snyder considered three constraints: (a) Two week time frame; (b) Evaluation limited to Facebook fans and current users; (c) Inexpensive.

His conclusion was to use: Feng-GUI, Inspectlet, and Usabilla. Snyder says, “With Feng-GUI we will get some great data that approximates an eye-tracking study. With Usabilla we have a tool where we can actually conduct a user test. We can write the instructions for the user and they can take the test without need for us to moderate it. It will provide us with great feedback based on the outcome of the user tests. We will use Inspectlet to record the user studies and will generate additional data and heatmaps that will show us more directly how the users behaved on the website.” Their costs came in under $100, with $25 for 10 Feng-GUI tests; $7.99 for Inspectlet (the first week is free); and $49 for 100 tests on Usabilla (or 10 for free).

More options

Here’s a broader view of testing options, adapted from a recent list by Cameron Chapman, plus several additional suggested by commentors on her blog:

The following is the verbatim article…

Do We Really Need a National Weather Service?

By Iain Murray  and David Bier — Published August 27, 2011 | FoxNews.com

As Hurricane Irene bears down on the East Coast, news stations bombard our televisions with constant updates from the National Hurricane Center.

While Americans ought to prepare for the coming storm, federal dollars need not subsidize their preparations. Although it might sound outrageous, the truth is that the National Hurricane Center and its parent agency, the National Weather Service, are relics from America’s past that have actually outlived their usefulness.

The National Weather Service (NWS) was founded in 1870. Originally, the NWS was not a public information agency. It was a national security agency and placed under the Department of War. The Service’s national security function has long since disappeared, but as agencies often do, however, it stuck around and managed to increase its budget.

Today the NWS justifies itself on public interest grounds. It issues severe weather advisories and hijacks local radio and television stations to get the message out. It presumes that citizens do not pay attention to the weather and so it must force important, perhaps lifesaving, information upon them. A few seconds’ thought reveals how silly this is. The weather might be the subject people care most about on a daily basis. There is a very successful private TV channel dedicated to it, 24 hours a day, as well as any number of phone and PC apps. Americans need not be forced to turn over part of their earnings to support weather reporting.

The NWS claims that it supports industries like aviation and shipping, but if they provide a valuable contribution to business, it stands to reason business would willingly support their services. If that is the case, the Service is just corporate welfare. If they would not, it is just a waste.

As for hurricanes, the insurance industry has a compelling interest in understanding them. In a world without a National Weather Service, the insurance industry would probably have sponsored something very like the National Hurricane Center at one or more universities. Those replacements would also not be exploited for political purposes.

As it stands today, the public is forced to pay more than $1 billion per year for the NWS.  With the federal deficit exceeding a trillion dollars, the NWS is easily overlooked, but it shouldn’t be. It may actually be dangerous.

Relying on inaccurate government reports can endanger lives. Last year the Service failed to predict major flooding in Nashville because it miscalculated the rate at which water was releasing from dams there. The NWS continued to rely on bad information, even after forecasters knew the data were inaccurate. The flooding resulted in 22 deaths.

Private weather services do exist, and unsurprisingly, they are better than the NWS. When Hurricane Katrina hit New Orleans in 2005, the National Weather Service was twelve hours behind AccuWeather in predicting that New Orleans would be affected. Unlike the NWS, AccuWeather provides precise hour-by-hour storm predictions, one of the reasons private industry supports them.

It is not just random mistakes in crises either. Forecast Watch has found that the National Weather Service predictions of snow and rain have an error rate 20 percent higher than their private alternatives. “All private forecasting companies did much better than the National Weather Service,” their report concludes. In 2008, they found that the NWS’s temperature predictions were worse than every private-sector competitor including the Weather Channel, Intellicast, and Weather Underground. Even NWS’s online ZIP code search for weather reports is in some cases totally inaccurate, giving reports for areas hundreds of miles away.

NWS claims to spread information, but when the topic of budget cuts came up earlier this year, all they spread was fear. “There is a very heightened risk for loss of life if these cuts go through,” NWS forecasters said, “The inability for warnings to be disseminated to the public, whether due to staffing inadequacies, radar maintenance problems or weather radio transmitter difficulties, would be disastrous.”

Disastrous? The $126 million in cuts would still have left the Service with a larger budget than it had a decade ago. The massive bloat in government should not get a pass just because it’s wrapped in good-of-the-community clothing. NWS services can and are better provided by the private sector. Americans will invest in weather forecasting because if there is one thing we can be certain of, people will want to protect their property and their lives.

Iain Murray is Vice President at the Competitive Enterprise Institute and author of “Stealing You Blind: How Government Fatcats Are Getting Rich Off of You.” David Bier is a Research Associate at CEI.

Update 28-August: Updated introduction, added context.

Smartphones are popular phones. According to Nielsen, 38% of Americans now own smartphones, and 55% of those who purchased a new handset in the past three months bought a smartphone (rather than a dumb one), up from 34% a year ago. Android’s growth curve flattened in 2011 while the iPhone’s got a boost. Collectively, Android and iOS are activating over 800k devices per day.

Tablet computers are dominated by Apple’s iPad which sold over 25 million units as of June 2011, since launch in April 2010. Competing manufacturers have been slow to launch compelling alternatives. The HP TouchPad (running webOS 3) is launching this week, and is lauded for being attractive and easy to use, but has started off with lukewarm reviews before launch for being bulky, and as Walt Mossberg at WSJ says, “poor battery life, a paucity of apps and other deficits.” The RIM BlackBerry Playbook (running BlackBerry Tablet OS) launched April 2011, but it was widely disliked by reviewers and the public, and sold only half a million by early June. Currently, monthly shipments of Motorola’s Xoom, Acer’s Iconia, Asustek’s Eee Pad Transformer (all running Android 3 Honeycomb) and RIM’s PlayBook average at about 100-200k units.

Tablet apps are exploding. Again this is dominated by Apple’s platform, as developers perceive it to be the best platform to invest their efforts. In 453 days since the original iPad came out on April 3, 2010, the App Store has more than 100k iPad apps available (i.e., targeting the tablet, not just a scaled up iPhone app).

Electronic books, such as Amazon’s Kindle, are gaining popularity as convenient, inexpensive devices dedicated to reading. Pew reports that 12% of U.S. households now own an eBook device. That doubled from November 2010 to May 2011. Meanwhile, in May 2011, 8% of adults report owning a tablet computer such as an iPad, Samsung Galaxy or Motorola Xoom.

Social media is related to mobile, and one new twist is Google+, Google’s latest foray into social media. It follows on Google’s prior ventures (Orkut, Buzz, and Wave), and is off to a good start with positive press and reviews. TechDirt thinks it’s main advantage is that it’s not Facebook.

Making the PDFs free is the culmination of a decade of research and sales modeling on how to finance a nimble publishing house with paid print books, with enough spare revenue to allow free release of electronic books. Here’s the evolution:

Previously, NAP priced books, including PDFs, to generate enough income to be self-supporting. According to Barbara Kline Pope, executive director for NAP, in 1994, they started offering free full text online of all reports in page-by-page format, and in 2003, they began to make all of their full books available as free PDFs in developing countries (limited by IP address). Cannibalization of sales was not a concern because a negligible number of print sales were made in the 3rd world.

Starting in 2001, they conducted careful research on buyers’ behaviors.

To optimize sales, they found they should price PDFs at approx 75% the cost of a print book, and offer a bundle deal with both the print and PDF for 120% of the printed price. (See graph at right).

But what about going free? They found that if a buyer had a book in their shopping cart, and were offered a free PDF, 42% would jettison the printed book from the shopping cart and just take the free PDF. Mapping that data over all sales channels, they estimated a 33% substitution rate of free PDF for print. This corresponded to approx a loss of $1.9 million in revenue on printed books if they gave away free PDFs.

So to give away free PDFs, the press was going to have to find $1.9 million someplace else… The largest source of money was cost savings, and NAP slashed costs. Instead of print brochures, they now do the lion’s share of marketing online. They outsourced printing in 2004, and in August 2010 they also outsourced fulfillment and warehousing. They will continue to produce printed books for readers who like paper. They dropped unprofitable products, such as their line of science books for general audiences.

In the last two years, NAP’s revenue from book sales was ~$5 million, of which ~10% was from paid PDFs. Overall, the volume of print sales has been falling. If their estimates are right, they will immediately lose all of the PDF income, and lose at least another 1/3 to buyers who will no longer pay for a print copy. So book revenue for the coming year might be around $3m. Given their mission to broadly disseminate the work of the Academies, it is now a small price to pay for much wider reach: NAP hopes free downloads of the PDFs will boost eBook downloads from approx 700k/year now to over 3 million by 2013.

In addition to PDFs, NAP also publishes to other eBook formats. They have ~200 books in Kindle, but the black & white format is not good for many of their books because of their color graphs and large tables. They have about the same number on the iPad, and are working on more. They have their entire list in Google e-books.

The Academies are a suite of private, nonprofit organizations (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council) created by congressional charters in the 19th and 20th centuries to advise the federal government on science, technology, and health policy. The NAS, NAE, and IOM are honorific organizations (members are not paid; it’s a high honor), and is supported by approx 1,100 paid staff. They conduct studies, convene meetings, and create publications. In 2010, the Academies had a total budget of $289 million, of which 84% came from U.S. agencies (e.g., the Department of Transportation) as grants and contracts, and the rest from foundations, nonprofit institutions, and state and local government. NAP’s budget is separate.

This model for providing free eBooks works for NAP, given their structure where the writing of the books are funded by the Academies’ research projects, but Ms. Pope cautions that it does not necessarily translate to other kinds of academic publishing, such as university presses.

Sources: NAS, NAP, and Pope’s 2009 paper in Marketing Science (PDF) with Kannan and Jain.

IDEA’s second mobile app, WikiNodes (see app store link) puts the encyclopedic knowledge of Wikipedia at the fingertips of iPad users. Articles are displayed as nodes that can be touched, dragged and spun around — showing the relations between articles and sections of articles. The app is currently featured in Apple’s app store.

Here’s a 30 second demo:

//www.youtube.com/watch?v=fdiXXxMnqJQ

The app is based on IDEA’s SpicyNodes system for displaying and navigating information using nodes (see SpicyNodes.org). The SpicyNodes approach has great potential for other subjects, from browsing museum collections and archives, to browsing flora & fauna, and many other kinds of linked data.

Encyclopedias contain large amounts of interlinked information — which is begging for a creative and engaging visualization. Tap any node, and it expands to give you more information. Or, switch to a full-page view to display articles as pages, then scroll up and down. Nodes link together sections of Wikipedia articles and related topics, making Wikipedia come to life. Browse in a way that mimics how you look for things in the real world.

Tablets are the new frontier for conveying art and science to the public. Industry analysts estimate that by the end of 2012, there will be 100-120 million tablets. IDEA will produce apps for whatever platforms have the lion’s share of the market.

Getting a new app noticed by the world is a huge challenge, as the app store is flooded with hundreds of new apps every day, of varying quality. Our download rates increased by more than 10x when featured by Apple as a “New and Noteworthy” app on their iTunes main screen.

Upcoming features for the app include multilingual support (many users are in non-English speaking countries), and other enhancements to browsing (e.g., bookmarks and more linked nodes). The app is currently free, but not for long.

Virtual exhibits on tablet devices (e.g., the Apple iPad) put exhibits at the fingertips of students and the public. Visitors can browse science, art or culture from classrooms, during their commutes, or from their sofas. — But where does the money come from?

As with physical museums, the problem with charging money for downloads is limiting visitation to enthusiasts. Access must free to get significant use on tablet computers in classrooms, or by people who would not otherwise pay. Aside from grant support or advertisements, are there other revenue models? Could funding come from the community?

We posit that virtual exhibit apps could be free downloads, giving a preview teaser. Then, to see the rest of the exhibit, visitors pay for access, sponsor access for others, or request free access. Here’s how it might look:

Prior thoughts

Nina Simon, an author and consultant who helps museums get the public involved in designing exhibits (see profile at Smithsonian mag or her blog), chatted last week with Andrew Taylor (Director of the Bolz Center for Arts Administration) in an online talk show about museum business models (pic at right).

Museums could take a page from public media and public radio, Simon said, where sponsors/members might make a museum available to the public for free. Simon says there needs to be a move away from memberships based on a discount transaction model (i.e., members get in free), and instead memberships should emphasize that members make the museum available for everyone else. Simon has previously written about rethinking membership and admissions (see a few of her articles, and her Feb 2011 post on switching from ‘value’ to ‘affinity’ membership programs).

Looking at an exhibit as media, there are other analogies. Experiments in “pay what you want” for intangible goods, such as music and other media, show that a subset of people will financially support an otherwise free product. (e.g., see TechDirt’s coverage of a group of indie video games which drew $1.2m, the Freakonomics screening, and how pay-what-you-want worked better for Panera when there was a charity component.)

In addition to funding after the fact, some virtual exhibits may be able to crowdsource funding beforehand to create an exhibit. The Kickstarter funding platform (for artists, designers, filmmakers, musicians, journalists, inventors, explorers, etc.), has received over $53 million in pledges for various creative projects.

There are other analogies as well, such as a shareware software. But shareware is purely a form of donation/payment, it is not specifically earmarked to make access available to others. Also, see my post about revenue models of open access journals.

Do the numbers add up?

The cost of creating a virtual exhibit could range from $5k to $100k, depending on the richness of the content and whether a lot of interactive and video content is created from scratch.

Maybe a project could collect $5k using Kickstarter, another $20k in individual payments, and $30k in sponsored payments — yielding access to 120 thousand visitors? After 10% fees to Kickstarter, and an app store’s 30% commission, that would be $39,500 in revenue, plenty for a create an interesting, informative, interactive, virtual exhibit.

What do you think?

Do you know of virtual exhibits using crowd-funding strategies? Will members of the public pay a little extra to sponsor a virtual exhibit for strangers? Should sponsored payments be pure donations, or tied to a specific number of free uses?

Are we dreaming?

Update: 2-May-2011, clarified Nina Simon’s comments.