Science game contest awarded $50k prize to ineligible candidate

The National STEM Video Game Challenge, awarded the $50k grand prize last week to a professional team that did not meet the eligibility criteria.

This story came to my attention last week, when I wrote a blog post about a cool online science game for Middle School kids which won the grand prize as part of the contest run by the Joan Ganz Cooney Center at Sesame Workshop (the parent organization of Sesame Street) and E-Line Media. That article briefly mentioned anomalies in the contest, and the Cooney Center sent me an email: “Please take this article down as soon as possible because of a legal issue that is pending.” This made me wonder, is there a scandal on Sesame Street?

Team too large?

The contest web site clearly says that “Teams may be comprised of up to four members.” Project lead Dan N. of the grand prize winning team said, they were “definitely in keeping with the principle of ‘small teams development’ that the contest wanted to embody.” However, the winning game lists 13 individuals in the credits (click red image to see credits). Dan N. says their credits screen is inaccurate, and they listed their entire staff out of a sense of solidarity. He says the actual core team was: himself (lead designer), Arthur L. (lead programmer), Trevor B. (illustrator), Amanda L. (voices), and Dan W. Moreover, special education researcher Dr. Matthew M. helped plan two vital aspects of the proposal: the research theories underlying the game, and their strategy for serving the underserved communities of kids with learning disabilities. That’s six people.

Who should be counted? The woman who supplied voices? The rest of the company’s team who may have done some minor programming or illustration? The programmers who wrote older source code which was reused? The researcher who collaborated on planning and defining the educational value proposition and impact, the research theories, and strategy for reaching underserved communities?

Wrong target age?

The contest strictly limited the game’s audiences to young children (grades pre-k through 4th). But president and founder of E-Line Media, Alan Gershenfeld, gave contradictory information in an interview, saying the methodology of the challenge was “to try to get the most talented game designers to make great games that make impact by engaging the largest number of middle school students.”

Due to ambiguity and disregard for the rules, only 8 out of 25 applicants expressively targeted the correct age range. The grand prize winner targets middle school audiences. Dan N. says their game could be rewritten for a younger audience, but it has not been.

Not mobile?

Mobility was an unofficial requirement. The contest site said “The Developer Prize challenges emerging and experienced game developers to design mobile games, including games for the mobile Web, for young children (grades pre-K through 4) that teach key STEM concepts and foster an interest in STEM subject areas.” The promotional description sent to other channels like the White House blog mirrored these guidelines. Mobile was not expressively mentioned in the official rules.

The grand prize winner is an Adobe Flash-based game designed for web browsers. It is a large-screen app that is currently unable to run on the Apple iPad, and the game design is not mobile-ready. For example, the grand prize winner uses the mouse for both pointing and clicking, which is not possible on a tablet where there is only one kind of tap. Moreover, the game is designed to require keypresses, such as the spacebar. There is no spacebar on a tablet computer:

These are important distinctions, and the game will require substantial redesign to work on a tablet computer controlled by only fingertip gestures. There are major differences in how games are designed for mouse and keyboard controlled interfaces, vs. touch-controlled interfaces. The programming process is also very different. Only 4 of 25 submitted games expressively targeted mobile.

Challenge or award?

On 20 December 2010, the White House blog promoted the contest, saying that it “challenges emerging and experienced game developers to design mobile-based video games.” They wanted developers to kick into action and make new games.

However, a week later, contest organizers expanded their scope to recruit games already in development. Clooney tweeted on 29-December, “Have a great game for kids in development? Just one week left to submit to the National STEM Video Game Challenge!”

The challenge successfully spurred many applicants to create mobile games for young kids. However, it did not spur the grand prize winners to create the game. The grand prize winners were already planning to make the game, as they had recently won a $839k contract from the U.S. Dept. of Education’s Institute of Education Sciences’ Small Business Innovation Research (SBIR) program to do so.

The honest losers

The two runners up were mobile games which targeted young children. They received no cash award. These two games are by amateurs and are less impressive than the grand prize winner.

One of the runners up was a husband and wife team, who created “A sciTunes Human Body Adventure.” He is a middle school teacher and musician, and she works in career services for a university. He has created several mobile apps already. Their game targeted elementary kids, and is designed to be a mobile app. “The winning team was from a professional design studio, so I don’t feel too bad,” said Daniel C. to the Watertown Daily Times. “The main thing was if I had won the prize, it would have helped with getting the business off the ground. It would have done quite a bit for a one-man studio.”

The other runner up was a 3-person team, which created “Doc Y and Bacon Figure Out the World” (See video.) The project lead, Bill G. is finishing up his MFA in Interactive Media at USC’s School of Cinematic Arts. The game is centered on mobile devices, using Twitter or text messaging. Doc Y is a scientist, and Bacon is his dog. The game is designed to be played over a series of days, asking kids questions they can answer by reply text message.

This ain’t DARPA

By comparison, a successful challenge is the famous DARPA Challenge for autonomous robotic cars. DARPA had three challenges to spur innovation. Inspired by glory and a $2M prize, 89 applicants entered the most recent challenge, navigating in urban settings. The contest was announced in May 2006. Applicants had 4 months indicate interest, then 8 months to submit demo papers and videos. Of those applicants, 35 passed muster and were invited to test their robot cars for real in an urban environment in November 2007. The DARPA challenge had a major impact in spurring innovation, had clear guidelines and rules which were realistic, and did careful due diligence before passing out the cash.

More about the contest

The National STEM Video Game Challenge contest was funded by AMD Foundation, Entertainment Software Association and Microsoft. Several organizations helped publicize it, including the White House.

On 16 September 2010, the White House announced an expansion of Obama’s “Educate to Innovate” campaign to improve science, technology, engineering and math (STEM) education. The White House announcement was the cornerstone of the promotion campaign for this contest. The contest itself opened for applications on 14 October 2010, when Cooney tweeted: “Gamemakers: Make FUN science/math games 4 kids + win up to $50K, STEM video game chalenge opens TODAY.”

The official rules explained the restriction on small team sizes and the audience focus:

The goal of the Competition is to challenge emerging and experienced game developers to design video games for young children (grades pre-k through 4th) that teach key Science, Technology, Engineering and Math (“STEM”) concepts. … the Team must consist of no more than four (4) individuals … and [proposals must describe each] Team member’s relevant work experience and educational background… each Applicant represents and warrants that … all ideas, creative elements and any other materials and information contained in the Submission are wholly original with the Applicant, … no person or entity has collaborated with Applicant in creating the Submission, provided any material or information for inclusion in the Submission”

At the awards ceremony, Aneesh Chopra, the U.S. Chief Technology Officer, praised the challenge, saying, “Three cheers for the National STEM Video Game Challenge for catalyzing this entertaining and educational approach to harnessing American ingenuity, all for the cause of science, technology, engineering and math education,” said Chopra. “It is efforts like these that will ensure our nation’s continued economic and technological leadership well into the 21st century.”

Summary

All this really amounts to a poorly run contest and challenge rules that don’t make much sense. It’s no fair to lump amateurs with professionals, especially if simple safeguards like team-member limits are not enforced.

The contest would have benefited from giving developers more time to design games, differentiating amateurs from professional contestants, clarifying what kind of prior work can be included, and strictly following the eligibility criteria.

Do challenges like this spur innovation in education? Maybe. But unlike robotic cars, the problem is neither a lack of technology innovation nor creative ideas. The problem is determining the learning value of mobile games for education, and finding business models which can a earn a living for science game developers. Giving a $50k prize to a company that already figured out how to get a million dollars in government contracts is not the right kind of support.

Note: The Cooney Center confirmed that they decline to comment for this article.