Mar

31

The Mechanical Turk Experiment: How I made $2.18 an hour – and how you can too!

Filed Under Crowdsourcing | Comments 

I am a reasonably intelligent person, so one day I was wondering how much I could make by signing up and working as a Turker on Amazon’s Mechanical Turk (Check it out in Part 7 of the Article series). Turns out, not much. In a little over 3 hours I made $6.56. It’s tough going too, between wading through and picking the appropriate HITs and actually executing on them. (Many of the HIT’s are limited to a certain number and the good ones run out quickly.)

If I got better at working the system I could probably kick the earnings up to $3. Assuming a 50 hour work week, I could make up to $600 a month, $7200 a year. Of course, I would have long before gone completely insane and been evicted.

So, what kind of work did I do?

    Subscribed to a YouTube channel. $0.01.
    Reviewed website layout and copy. $0.05.
    Evaluated whether 100 sites were phishing or not. $1.
    Transcribed audio, a difficult-to-hear 5 minute interview. $2.


The Original Mechanical Turk

The Original Mechanical Turk



The people using the service most are sites hoping to populate their site with some user generated content, researchers, and semi-spammers looking to build links. The work that seems to give you the best return involves the transcription of audio and scanned text that is too difficult for character recognition.

Don’t get me wrong, I love Amazon Turk and think it’s a fascinating and brilliant way to bring together small manual human jobs with those looking for some pocket change. The hourly rate is peanuts for US based folks like me, but if you are one of the billions who live on dollars a day it becomes much more relevant. Granted, you need access to an Internet connected computer and (currently) must speak English, but I am sure in the next decade there will be a non-trivial portion of this group with some sort of access to a computer and knowledge of English.

So, how much can you make in an hour?

Mar

23

Open source succeeds under a benevolent dictatorship — and so do co-creation projects

Filed Under Co-creation, Crowdsourcing, Wisdom of Crowds | Comments 

Chris Anderson (of the Long Tail) recently articulated an interesting metaphor regarding social media and driving a project/organization forward. In his post Open source is a company; social media is a country I would call particular attention to his take on successful open source projects:

Many people mistakenly think that open source projects are emergent, self-organized and democratic. The truth is just the opposite: most are run by a benevolent dictator or two. What makes successful open source projects is leadership, plain and simple. One or two people articulate a vision, start building towards it and bring others on board with specific tasks and permissions.

Remember this concept if you ever decide to run a crowdsourcing, idea generation, or co-creation system with your customers — or anyone, for that matter. Simply making the tools available will do not good. Nor will a vague sense of who is in charge. Central leadership is still necessary. Enterprises shouldn’t believe that putting a project out in the wild without definitive leadership and support will produce anything of value. Everything needs a champion to drive it forward.

Simple enough, but the real value I see created in what I write about here has sprung out of a – sometimes hypothetical – balancing and blending of external inputs or votes or intellectual property or funding or designs with a strong plan, leadership, and vision. That includes rejecting bad ideas. Saying NO to your customers when you feel strongly about it (37 Signals’ favorite past-time.). Retaining focus on what is important and getting rid of the extraneous.

Essentially: co-creation doesn’t take the work out of what you do but it can enhance it and help you more deeply understand the people you serve.

Mar

10

Guest Author Post on ReadWriteWeb: Get Satisfaction Leads Among Idea Aggregators

Filed Under Outside Innovation, Web2.0 | Comments 

I recently published my first guest author post on ReadWriteWeb covering idea/suggestion/complaint aggregators. To clarify exactly which space I talk about: in my view, the idea and suggestion management space has essentially three types of vendor offerings (some bleed across categories):

  1. Centralized aggregators: Get Satisfaction, Suggestion Box, FeVote, Featurelist

    Anyone can start a product or company page on these sites to submit ideas, suggestions, or complaints which are then voted up or down, Digg-style, and commented on. Companies pay for access to data, more powerful features, and the ability to “claim” pages and register official employee moderators. Similar to review sites like Epinions, the conversation will happen on these sites with or without you.

  2. Tool providers: SalesForce Ideas Management, Uservoice, IdeaScale, Get Satisfaction, Kindling.

    These systems provide similar functionality to the above sites but are controlled by and run by the companies themselves. They include features such as ratings or up/down votes, moderation, limiting the number of votes per user, running time-limited contests, limiting access to certain groups, and automatically searching for duplicate ideas during idea submission.

  3. Integrated innovation management suites: Imaginatik, Brainbank, SalesForce Ideas Management, BrightIdea, Spigit.

    The idea management portion of these suites generally have more robust capabilities such as weighting the contribution of particular users according to expertise and trust, creating virtual currency systems, providing enterprise class security, and customizing information captured. By integrating idea capture and prioritization into a more robust and sophisticated system, companies can then evaluate the costs of ideas, put them through formal review processes, and track performance of ideas from conception to execution.

So go check out the post: Get Satisfaction Leads Among Idea Aggregators I’m pretty happy with the end result, enjoyed the process, and hope to write another post for RWW on the tool providers soon.

Feb

23

Cognitive Surplus: What are you going to do with yours? Clay Shirky explains…

Filed Under Collaboration, Crowdsourcing, Web2.0 | Comments 

Some of you might wonder why anyone participates in many of the activities discussed here or whether it is all sustainable given that they result in little to – more commonly – no money. Well, you should ponder what Clay Shirky has to say about what he calls Cognitive Surplus, a concept I just can’t get enough of.

(Odd side-note: I spotted Clay Shirky (about half way down the page) among those who participated in a project my girlfriend ran as part of her art collective’s Windows Brooklyn project last summer.)

In a video and transcript from last April – yes, it takes me a while to process and write about things at times – Clay lays it out thus:

A British historian argued that the critical technology in the Industrial Revolution was Gin. The changes were so rapid and disruptive that the British went on a bender for a generation.

“And it wasn’t until society woke up from that collective bender that we actually started to get the institutional structures that we associate with the industrial revolution today.”

The Sitcom was the US’ palliative after World War II. We suddenly found ourselves with free time and disposable income, and we started watching a lot of TV. A lot.

“And it’s only now, as we’re waking up from that collective bender, that we’re starting to see the cognitive surplus as an asset rather than as a crisis. We’re seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody’s basement.”

My favorite part of his thinking: He was talking to a TV journalist about the recent rash of conversation surrounding Pluto’s planetary classification on Wikipedia to which she responded: “Where do people find the time?” His response:

“No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you’ve been masking for 50 years.”

He estimates we expend

“About 2,000 Wikipedia projects a year … watching television… I can tell you from personal experience it’s worse to sit in your basement and try to figure if Ginger or Mary Ann is cuter. I’m willing to raise that to a general principle. It’s better to do something than to do nothing.

Thus, the free time and brain time outside of work remains, the question is what percentage of it will be occupied by the completely unproductive versus the semi- and extremely productive? An example of a successful use of brain time, in this case for the common good, comes from InnoCentive where about 20% of their projects are non-profit. Uncompensated. Using people’s free time.

Going forward, the big thing will be experimenting and figuring out what works in collective work and production. As I have covered here, experiments abound, some successful, many not. There is a long way to go, but as Clay says, this is not something society will grow out of, but something society will grow into.

Cognitive Surplus. Love it.

Feb

14

Part 8: Collaborative Content Creation; or, Crowdsourcing your way to creativity

Filed Under Collaboration, Crowdsourcing, Wisdom of Crowds | Comments 

Crowdsourcing or collaboratively creating content of any kind becomes dicey very quickly and the odds of creating something greater than the sum of its parts are low. In fact, creating by committee generally leads to utter crap. (When was the last time you read and enjoyed a story written line by line by contributors?) But with collaborative sites and, in my view, a talented orchestrator, it might be possible to create something of quality or, at least, facilitate an interesting experience, regardless of the end product. Wikipedia has been talked to death as what is likely the greatest implementation of collaborative content creation in history but few projects have risen above the level of a chaotic mess.

Collaborative (or crowdsourced) content creation: A group effort to contribute to and edit portions of content for the expressed aim of producing a specific overarching project.

A Swarm of AngelsFor some reason, examples of collaborative creativity keep popping up in the film-making arena. It’s Our Movie began with a script and a director (Alex Jovy, an Oscar Nominated Producer) and used the community to find its actors. (While this may not technically fall under “content creation”, it does bring a community into the overall creative process of a film.) Anyone could upload an audition for one of the characters and then the best were voted up. A Swarm Of Angels was a more ambitious project in which everything from the concept to script, casting to funding was created by the crowd. (It was up and running for several years but now the website is under construction, status unclear.)

CoWrite is essentially a script writing competition in 10 page increments. Each week anyone can enter a submission (along with $10) and, each week, the best entry is awarded $3,000 by an internal panel of judges and becomes the next 10 pages of the script. The final script will then be rewritten by one of the weekly winners who will be paid $5K. Another perk of winning is a meeting with Benderspink (Production company behin American Pie, A History of Violence) who will also review the final script and decide what to do with it from there.

Assignment Zero, a project sparked by Jeff Howe who coined “crowdsourcing” and Jay Rosen, began with the goal of having “a crowd of volunteers write the definitive report on how crowds of volunteers are upending established businesses, from software to encyclopedias and beyond.” Jeff Howe considered it “a highly satisfying failure”.

In some cases, the experience of contributing itself becomes the purpose of a project, not what is produced in the end. Contributing a sentence to a growing story can be enjoyable and creatively inspiring regardless of the poor end result of the totality of the project. And, as creative types know, most of what you produce, regardless of your brilliance and success, will be poor.

Any other examples I’ve missed?

Wikipedia
A Swarm of Angels
It’s Our Movie (The movie is in production as of winter ‘08/09.)
Ze Frank The great and noble Ze Frank who has called on his community to write, draw, execute power moves, and make earth sandwiches in order to create works of art and entertainment and bring joy to the downtrodden…
Assignment Zero
Protagonize supports “addventure” which is collaborative writing in which people can branch stories off into new paths
Spike Lee Nokia Ad

Read the rest of the Crowdsourcing article series.

Jan

22

CoInnovative Roundup: 3 Odds and Ends to check out

Filed Under Crowdfunding, Crowdsourcing, Wisdom of Crowds | Comments 

Innocentive used for non-profit challenges: An interesting addendum to my previous post (Part 6 of The Series), about 20% of Innocentive’s portfolio of projects are aimed towards solving non-profit challenges. Further, they recognize the possibility of pairing this crowd-problem solving with crowdfunding: someone decides to champion a particular problem and, using the Innocentive platform, raises the money necessary to reward problem solvers. (Then, I would assume, they would have to raise another round to actually implement the solution, but that is developing as well.)

Help Obama prioritize and solve problems with the Citizens Briefing book: Of course, it’s inauguration week, so how could I resist? Leading up to the transition (and going forward, I assume) the Obama administration has set up the Citizens Briefing Book on change.gov which allows you to suggest ideas and priorities for Obama and vote on the best ones. These will be compiled and presented to the President. Regardless of whether it turns into something of substance or not, it is a great example of what I discussed in Part 3, the Digital Suggestion Box. As with any of these efforts whether it be from Dell, Starbucks, or Obama, the impact this technology will have depends on how whether it becomes a useful, two-way conduit of information. Are popular measures acted upon or, at the very least, responded to? (Case in point: the first and third most popular ideas with 16,000 votes involve relaxing or doing away with marijuana prohibitions.) And is the tool effective at surfacing quality?

Crowdsourcing classifications: via MadeForOne, Scott Klososky at TechnologyStory lists his view on classifications of crowdsourcing:

    Voluntary vs. Involuntary
    Social vs. Commercial
    Rewarded vs. Unrewarded

Jan

7

Part 7: Many hands make light work: The atomization of work

Filed Under Crowdsourcing | Comments 

Some tasks cannot be accomplished by automated, centralized means but can be broken up into smaller, manageable chunks that anyone can do. Shockingly, the internet comes in handy for these types of projects. Tasks that would be simple for a human (identifying an object or poorly scanned word in an image) can be difficult if not impossible to efficiently do on a computer.

The big kahuna in this space is Amazon’s Mechanical Turk. Break up a project into distinct HITs (human intelligence tasks), set your price for each task, and let the results roll in. Pricing ranges from as little as 1 cent to a few bucks per task depending on the complexity. The work itself ranges from the rewarding (helping search for a missing Steve Fossett and his plane using satellite images) to the unsavory (creating links and comments for spammers).

I see this as yet another example of the decreasing importance of the firm and a further lowering of the transaction costs of going outside your company for work. Have a huge project that you can break down? Without vetting and hiring a freelancer or outsourcing firm you can immediately post a project and start getting results while fine-tuning your request and pricing to get the best results. Pay can be dirt cheap (sometimes you can spend an hour doing something and only get $1 or $2), but in vast swathes of the world, that is a decent wage.

(On a side note and on the flip side of this: Some computing projects require unfathomable amounts of supercomputer time, so these projects, as well, are broken up into data crunching chunks that can be handled in a reasonable amount of time by home computers – I’ll call these CITs (computer intelligence tasks). Again the work spans a range of reputability: from helping Seti@Home search for alien signals to, unwittingly, sending emails or attacking sites for a botnet.)

In a twist on being paid to complete individual tasks, people either do it because they enjoy it, are getting an ancillary benefit from it, or, in a sort of slight of hand, are accomplishing more than they understand. GalaxyZoo was a project allowing anyone to participate in scientific research by identifying cosmological elements in telescope photos (Enjoyment). Gwap develops online games that accomplish image and music tagging while tagging sites on del.icio.us allows you to find them easily once you bookmark them (ancillary benefit). Finally, a stunningly brilliant application: reCAPTCHA. CAPTCHAs are those annoying tests given to you when you register on a site in an attempt to verify that you are a human and not a bot. But with reCAPTCHA, you are actually identifying words that computers could not identify when scanning books (Slight of hand). CAPTCHAs are a sad necessity given the current state of the world, but to put it to good use at the same time is strikingly elegant and simply awesome. (Further, by definition, you are using images that a computer couldn’t identify in the first place. Damn!)

(In a strange irony, it is possible to thwart CAPTCHAs by employing a system similar to Mechanical Turk in which you pay people a penny or two to identify CAPTCHAs.)

Any other examples out there?

For pay:
Mechanical Turk

Ancillary benefits:
Photo tagging on Flickr
Site tagging on deli.cio.us

Fun:
Gwap games
Write transcripts of Parliament sessions
GalaxyZoo

Unwitting participant:
CAPTCHA and reCAPTCHA

Let the computers do it:
Seti@home
Botnets (evil).
Hundreds of other distributed computing projects

Dec

19

Part 6: Expert sourcing for problem solving and innovation

Filed Under Crowdsourcing, Outside Innovation, Wisdom of Crowds | Comments 

My former employer Forrester and my former co-worker Chris Townsend recently wrote a report called Tapping The Wisdom of Experts. This is an extension or subset of the concept of Innovation Networks which Navi Radjou has been writing about for years at Forrester and can be seen as a subset as well, in a certain sense, of crowdsourcing. It is also a good summary of the topic covered in Part 6 of this series.

(To be sure we’re on the same page:
Innovation networks = “Firms seamlessly weave internally and externally available invention and innovation services to optimize the profitability of their products, services, and business models.”
Crowdsourcing = sourcing small and large jobs from anyone and everyone.
Expert sourcing = sourcing from specialized, professional-grade, vetted experts.
Wisdom of crowds = the wisdom of the crowd’s collective intelligence outweighs any individuals.)

expertise-innovation.jpg

Expert sourcing involves several actors that fall into the mold of innovation networks:
Inventor: the experts (enterprise R&D, academics, government labs, retired technicians, you know, experts.)
Transformers and Financiers: corporations who buy, develop, and fund the innovations
Broker: expert sourcing providers who bring together the experts and corporations.

Innovation networks and expert sourcing further erode the purpose and importance of the large corporation and a massive, closed R&D facility. With the aggregation, moderation and policing of transactions, NineSigma and Innocentive facilitate this possibility. Along with online collaboration improvements, the need to have a large building or organization under which a multitude of people need to sit are further eroded. Smaller, more focused enterprises are made possible as transaction and search costs between inventor and transformers/financiers decrease (lower transaction costs, after all, are the reason companies formed in the first place).

While it appears this market is pretty small in the grand scheme of things, companies large and small will not be able to afford to only look for internally for innovations, they must search the horizon, getting input and help from customers, suppliers, and experts around the world. Tool providers are and will continue to emerge on all fronts, but if firms do not weave these tools and innovation sources into their processes, they will eventually be left behind. The key, of course, is not to hand over the reigns to outside innovators or customer whims but to incorporate the knowledge gained into your own expertise and discretion to mold it into real, profitable innovation.

Companies in the space:
Innocentive (Problems issued to recruited scientists)
NineSigma (Sends out RFPs to network of universities, inventors, businesses)
YourEncore (Posts projects to retired technical people)
yet2 (Matching and providing services/resources to IP buyers/sellers)

Further reading:
HBR: Getting Unusual Suspects to Solve R&D Puzzles
If You Have a Problem, Ask Everyone
Nine Sigma – Expert Response to Innocentive Post
Open Innovation Becoming Key to R&D Success

Next Page →


    Great Rates, No Banks. Borrow. Lend. Prosper.