The Core

Why We Are Here => Web Development => Topic started by: rcjordan on November 05, 2010, 10:23:12 PM

Title: If you're considering using the Mechanical Turk
Post by: rcjordan on November 05, 2010, 10:23:12 PM
it might be worthwhile to spend a few hours digging around in here

hxxp://turkers.proboards.com/
Title: Re: If you're considering using the Mechanical Turk
Post by: 4Eyes on November 06, 2010, 03:05:29 PM
:(

The Mechanical Turk is still not available outside the US.

I have to use non-mechanical Philippinos (not complaining, the ones that work with me do some damn fine work)
Title: Re: If you're considering using the Mechanical Turk
Post by: Rumbas on November 06, 2010, 04:46:29 PM
....blank. What is it? Do I need to check it out?

<added>
Ok, had to check it.
"The online market place for work"
</added>

>Philippinos

Seems to be a trend for a lot one man army's these days? How's it working for you guys and what do they actually do for you?
Title: Re: If you're considering using the Mechanical Turk
Post by: Drastic on November 06, 2010, 05:34:33 PM
In my experience it's a lot like hiring a regular bricks employee - a pita. However, when you get a good one, you can get full time employee for a few hundred bucks a month.

I'm using odesk per job this week. Got 4-5 hires on a contract.
Title: Re: If you're considering using the Mechanical Turk
Post by: 4Eyes on November 06, 2010, 06:14:16 PM
We have three working full time, one part time - share them between my main company work and my personal aff company work - roughly 50/50.


I got two of them via BestJobs.ph (not sure if they still let non-philippines folk use it the same way now) - got to check their CV, did a short Skype interview and then took them on a monthly contract.

The last one I got via Odesk - and then moved her out of Odesk to the same kind of contract as the others.

I pay them more than the going rate - which is still 'bugger-all' in our terms - and after a years good performance, gave them 10 days paid vacation and the 13th month payment that is the norm over there.

Points to consider:

Tasks mine are doing:

Elf 1: doing the article base link building for us. We tell her the sites to target, she commissions the articles locally (cheaper than TextBroker), checks the articles for plagiarism and grammar, does minimal spinning, creates an author box, submits them to article directories, blog networks etc - monitors results, bollocks us when we make mistakes or give her broken links etc
Irreplaceable  - or so we thought. She told us she is leaving us soon to get married - but 'not to worry', she has been training her sister up to take over for the last month.
I would like to clone her.

Elf 2: builds my ebay sites, I just give her the ftp details and the top level topic - she does the keyword research and builds the site. She also does a bit of blog commenting and profile spamming (both manual and automatic). A bit more of a handful to manage, but still well worth the salary.

Elf 3: 4 hours a day writing and spinning content for me. The only non-native-english speaker I have found who can spin to an acceptable level - not great, but at least it doesn't make me grind my teeth when I read it. Working out at about £4 for a spun article - cheaper than most non-spun articles - will try to move her to 8 hours a day next year if I can.

Elf 4: Paid less than others due to bad english - employed to do grunt work - eg. any link building that can't be automated, data manipulation etc


The first 2 have been with me for over 2 years now, the others just a month or so.  Tried a couple of others in the interim, but they failed to perform on the first few tasks, so let them go.

I think we got lucky at the start, but it still ranks as one of the best decisions we made.
Title: Re: If you're considering using the Mechanical Turk
Post by: PaulH on November 06, 2010, 06:39:47 PM
Wow Colin! paid holiday!!! you're too soft   ;D

Been using odesk for several months... Love it!!
Hired a dozen link builders for same contract; after a month i sacked 11 of them. The one good one has now built her own team of 4. I just speak with her she manages the rest, pay her $3, rest $2 (based in philippines, so probably paying too much)
(tried link builders making all sorts of claims and charging from $1 to $50 an hour - may have been unlucky, but paying more made no difference to quality)

Easiest was finding designers, guess you can tell a lot from their portfolios. Found some very talented designers. Paying about $100 for 3 page design with logos, standard is high. Probably paying too much but price im happy with.

Found some good content writers, star writer is Degree educated Brit who charges a shocking $1 to rewrite 500 words of content, $2 if she has to to research.

For coding, i've been unable to find anyone who doesn't seem to be talking out their rear. Just ask them to break the project down in to parts and put the hours along side - is laughable, and rates they ask for are on a par with UK - so hired myself a in house russian coder.

One thing i always do in my odesk job specs is ask a question and state i'll ignore anyone who applies who doesn't answer it. Straight away i can dismiss about 70% of the applicants who seem to apply for eveything.
Title: Re: If you're considering using the Mechanical Turk
Post by: 4Eyes on November 06, 2010, 07:39:58 PM
QuoteWow Colin! paid holiday!!! you're too soft

They have to use it for sick days, funerals etc - costs us bugger all in real financial terms, but really helps to keep them on board.
Title: Re: If you're considering using the Mechanical Turk
Post by: ukgimp on November 06, 2010, 08:12:07 PM
How to email virtual assistants and not get fooked over on one of my favourite dudes sites.

http://www.fourhourworkweek.com/blog/2010/11/02/virtual-assistants/

Title: Re: If you're considering using the Mechanical Turk
Post by: agerhart on November 07, 2010, 04:21:43 AM
We used Mechanical Turk (AMT) a LOT earlier this year. If anyone is considering using it let me know and I'd be happy to answer questions or offer guidance based on what we experienced. 
Title: Re: If you're considering using the Mechanical Turk
Post by: Travoli on November 07, 2010, 06:19:30 AM
agerhart,

Were there some tasks which were better completed than others?  IE:  They were great for job A but terrible for task B?
Title: Re: If you're considering using the Mechanical Turk
Post by: agerhart on November 07, 2010, 09:05:04 PM
QuoteWere there some tasks which were better completed than others?

Definitely.

We worked with a consultant to run tests and setup automated processes.  The consultant went out and talked to a variety of other companies that were using AMT at scale and most of them use it for content categorization.  For example, you have 200,000 products and need to categorize them for your database.  AMT is great for these simple tasks.  Or other items like view this photo and describe it. 

Even with these small tasks it is important to have a QA process.  But you can use the workers to QA the original work.  So the first task is describing a photo for $0.10 and the second task is verifying the original task for $0.05.   

We tried AMT for two things: 1) generate traffic, and 2) generate content. 

We were successful at the first one.  It is essentially arbitrage.  You pay users X amount to perform some task on your site and you make Y off those visits.  You tweak until the math is right.

The second one was difficult.  We were successful but a) often times the quality was questionable and b) the math never worked due to low return on investment. 

Regarding (a), we found the quality to go up with the amount we paid (obviously).  And we also found the quality to go up when we restricted the tasks to US workers (again, might be obvious).

Regarding (b), the problem was that we were using the content for new pages on our site and trying to back out the ROI from new traffic.  We may have had unrealistic expectations in such a short period of time.  We did see traffic go up, but in the advertising game it adds up slowly.  And with the consultants fees it didn't add up to enough to make our dough back.

AMT has an API that you can tap into to automate the processes so once we had all of this setup is was mostly set and forget.  Our tasks would automatically upload into the system when our levels went low enough and payments were made to the workers dynamically. 

Some items to remember:

- You can target specific demographics and locations.  We used this heavily as we only wanted people of a specific age, gender, and only from within the US for the traffic task.  But for the content task we didn't care where they were so we opened it up.
- Monitor the turk forums so you can manage your reputation.  Your rep is important among the people if you want them to complete your tasks.
- Pay quickly and fairly...the workers will love you.
- Keep the tasks interesting.  What we found is that boring tasks didn't get picked up as much.  The people completing the work were often well educated and had jobs...these people would pick up additional work at home or wherever for the cash but also for the challenge. 

I heard about a few people doing some really clever, and also very shady things using AMT.  I'll share over a beer at the next meet up :)
Title: Re: If you're considering using the Mechanical Turk
Post by: nffc on November 07, 2010, 09:17:14 PM
>Easiest was finding designers

Shall we have a whip round and get a new logo for this place.
Title: Re: If you're considering using the Mechanical Turk
Post by: Drastic on November 07, 2010, 09:48:00 PM
>equal spread of Apple fans & Non Apple fans

Equal spread? Apple logo is asking for a dos attack. hehe
Title: Re: If you're considering using the Mechanical Turk
Post by: Travoli on November 07, 2010, 10:29:47 PM
>I heard about a few people doing some really clever, and also very shady things using AMT.  I'll share over a beer at the next meet up

Thanks for the writeup AG, and yes I'd be happy to buy a round and hear those rumors. 
Title: Re: If you're considering using the Mechanical Turk
Post by: DrCool on November 07, 2010, 10:40:58 PM
Never done anything with AMT but we have worked with some Filipinos and other outsourcers also.

1.  I had a writer rewriting descriptions for products on datafeeds. The quality was OK. Not perfect but passable. I was paying per description and the lady doing it was pretty fast so she could probably earn $400-500 or so per month if she worked really, really hard. I found her through an ad on Craigslist in Manilla. I had about 80 applicants and maybe 20 of them completed a quick test I sent them to weed them out. Out of those 20 there were maybe 3 that were good enough.

2.  When we started with Traffic Geyser we got a programmer of Rent-a-Coder from Nepal and he built the first version of the system for around $500. He did such a great job we hired him on full time and had him bring in 3 more programmers. He is probably the best programmer I have ever worked with. Rather than just doing what we told him to do he would find ways to improve stuff on his own.

3.  We have had other programmers we have tried that just never panned out. Some we found through places like bestjobs.ph, 123employee.com, rentacoder, and other similar places. They would never get anything done, take way too long, and always have excuses.

4.  There are also tons of sob stories so you have to be ready. "Can I get paid in advance so I can pay for my mom's surgery" and stuff like that.

So it is really hit and miss. Mostly miss but if you can find a good worker it can be worth the hassle of finding them.
Title: Re: If you're considering using the Mechanical Turk
Post by: Drastic on November 09, 2010, 01:51:03 AM
>2nd bite from the other side

Heh, here's our old logo.