Amazon MTurk: worse and worse

18 replies
<rant>
It used to be that I could get usable articles at $5 from Mechanical Turk. Not great ones, but usable, if I posted 2 requests for each topic I wanted. Sometimes I even got 2 usable ones for a topic!

I just posted a batch of 16 requests yesterday, and I got just 4 usable ones back: of the others, 2 were simply too short but TEN were copy-and-pastes from Wikipedia and other sites. My specs said right up front that I would check for copies and not accept them, but did they care? Or even notice, probably? No.
</rant>

Time to train my daughter and open my own sweatshop
#amazon #mturk #worse
  • Profile picture of the author Colin Theriot
    We're hitting the wall right now in IM. Content has ALWAYS been king, and the people who figured it out were able to get it on the cheap from places with a favorable exchange. But now, those people are catching on, and pretty soon, the price of UNIQUE and FRESH content will stabilize across the globe. Quality will keep costing more and more, and it will put the focus BACK on making methods of INTELLIGENTLY spinning.

    In fact, try that. Get someone who KNOWS what they're doing to write ONE article for you on topic. Then outsource people to REWRITE that one article sentence by sentence. Get that done 4 or 5 times, then use some software to randomly sift together each sentence from the variants.

    You'll basically have a 60-70% unique article each time, and have a much higher incidence of quality, because they are all based on one GOOD article, and they can't copy.
    Signature

    Fair warning: It's possible I'm arguing with you because I have nothing better to do.
    Join my free copywriting group on Facebook: http://CultOfCopy.com

    {{ DiscussionBoard.errors[2276179].message }}
    • Profile picture of the author tecHead
      Originally Posted by Colin Theriot View Post

      We're hitting the wall right now in IM. Content has ALWAYS been king, and the people who figured it out were able to get it on the cheap from places with a favorable exchange. But now, those people are catching on, and pretty soon, the price of UNIQUE and FRESH content will stabilize across the globe. Quality will keep costing more and more, and it will put the focus BACK on making methods of INTELLIGENTLY spinning.

      In fact, try that. Get someone who KNOWS what they're doing to write ONE article for you on topic. Then outsource people to REWRITE that one article sentence by sentence. Get that done 4 or 5 times, then use some software to randomly sift together each sentence from the variants.

      You'll basically have a 60-70% unique article each time, and have a much higher incidence of quality, because they are all based on one GOOD article, and they can't copy.
      dayum, bro! I just had to point my content partner to this post. I might hafta stalk you, for a minute.
      Signature
      Learn Everything You Need to Know About CryptoCurrencies
      Automation is the primary conduit to successful relaxation
      {{ DiscussionBoard.errors[2276227].message }}
      • Profile picture of the author Colin Theriot
        Originally Posted by tecHead View Post

        dayum, bro! I just had to point my content partner to this post. I might hafta stalk you, for a minute.
        Feel free to PM me if you want more info, no stalking necessary.
        Signature

        Fair warning: It's possible I'm arguing with you because I have nothing better to do.
        Join my free copywriting group on Facebook: http://CultOfCopy.com

        {{ DiscussionBoard.errors[2276262].message }}
    • Profile picture of the author genetic
      Originally Posted by Colin Theriot View Post

      We're hitting the wall right now in IM. Content has ALWAYS been king, and the people who figured it out were able to get it on the cheap from places with a favorable exchange. But now, those people are catching on, and pretty soon, the price of UNIQUE and FRESH content will stabilize across the globe. Quality will keep costing more and more, and it will put the focus BACK on making methods of INTELLIGENTLY spinning.

      In fact, try that. Get someone who KNOWS what they're doing to write ONE article for you on topic. Then outsource people to REWRITE that one article sentence by sentence. Get that done 4 or 5 times, then use some software to randomly sift together each sentence from the variants.

      You'll basically have a 60-70% unique article each time, and have a much higher incidence of quality, because they are all based on one GOOD article, and they can't copy.
      I would disagree that IM is hitting any kind of "wall", in my experience you can still get usable articles for a decent price, you just need to look in the right place and above all qualify your perspective workers better.

      I believe that if, for some reason a project doesn't turn out the way I wanted it to, then I have to evaluate myself first before the poor sap I hired. Remember, you don't do data entry work because you're a nobel laureate, you do it for quick easy cash.

      Personally I think Mech Turk is trash, I prefer traditional routes like RAC etc. I still get dirt cheap content from there, but I take careful steps to make sure I pretty much always get what I want (and on the rare occasions when I don't - I've never lost an arbitration).

      Here's what I do quite simply (apart from all the obvious clear, detailed description blah, blah, blah):

      • I use a passcode in my bid requests. Simple, if I look at their response and there's no passcode, they haven't read the project properly, they will be sucky employees and they get ignored.
      • Nope, that it.
      Just that one thing improved my results dramatically. When they don't use your code you can assume it's a bot or just some fool that hasn't bothered to read your bid request. In normal conditions you wouldn't really be able to spot them, but this simple tweak lights them right up.

      Be warned for stuff like articles 80-95% of the respondents won't use your code, but when you find workers that do (especially since I bury the instruction to use the password in the body of my description) then you've struck gold.

      In one fell swoop they have both demonstrated that they are more thorough than the other 90% and have the ability to follow instructions

      Saved me a lot of time, money and headaches. Hopefully it'll do the same for you guys.
      {{ DiscussionBoard.errors[2277407].message }}
      • Profile picture of the author Colin Theriot
        Originally Posted by genetic View Post

        I would disagree that IM is hitting any kind of "wall", in my experience you can still get usable articles for a decent price, you just need to look in the right place and above all qualify your perspective workers better.
        You can still. But for how long? Maybe we haven't hit the wall yet, but we're approaching a homogenization as more people enter that workforce, and they all cover every website that offers jobs like this. Eventually, it'll just be low quality, low price.

        Originally Posted by genetic View Post

        I believe that if, for some reason a project doesn't turn out the way I wanted it to, then I have to evaluate myself first before the poor sap I hired. Remember, you don't do data entry work because you're a nobel laureate, you do it for quick easy cash.
        So wait, if I can't find good people to hire for the price I want to pay, it's my fault? I'm not sure what you're saying here. If it's that "you get what you pay for" no one's disputing that. We're saying the value for pay rate is sliding ever downward.

        Originally Posted by genetic View Post

        Personally I think Mech Turk is trash, I prefer traditional routes like RAC etc. I still get dirt cheap content from there, but I take careful steps to make sure I pretty much always get what I want (and on the rare occasions when I don't - I've never lost an arbitration).
        Good job, super awesome project manager!

        Originally Posted by genetic View Post

        Here's what I do quite simply (apart from all the obvious clear, detailed description blah, blah, blah):
        • I use a passcode in my bid requests. Simple, if I look at their response and there's no passcode, they haven't read the project properly, they will be sucky employees and they get ignored.
        • Nope, that it.
        Yeah, Robert Plant recommended this in a recent webinar. It's a net idea. But it's just an indicator of ability to follow directions. It's got nothing at all to do with how good a quality they will start out at or maintain.

        I predict people are going to get better at giving you the code and worse at giving you what you want, and you're going to be over there beating yourself up over it?

        Even down to the individual, that market can't do anything but burn itself out, because the worst provider with the cheapest price is going to begin to outnumber and outbid and outwork anyone who will try harder.

        At least that's exactly what it looked like in the web designer market a while back, and this is showing the same signs. I could be wrong, but I don't think I am.

        Originally Posted by genetic View Post

        Just that one thing improved my results dramatically. When they don't use your code you can assume it's a bot or just some fool that hasn't bothered to read your bid request. In normal conditions you wouldn't really be able to spot them, but this simple tweak lights them right up.
        Yeah, again, it's a trick that uses one factor as an incidental indicator, not a causal one.

        Originally Posted by genetic View Post

        Be warned for stuff like articles 80-95% of the respondents won't use your code, but when you find workers that do (especially since I bury the instruction to use the password in the body of my description) then you've struck gold.
        And once you find a provider, you keep em for life, right? You never find good ones who under bid to get the job and want to raise the price on you? Don't find that the quality slacks off over time anyway?

        If so, that would be amazing.

        Originally Posted by genetic View Post

        In one fell swoop they have both demonstrated that they are more thorough than the other 90% and have the ability to follow instructions

        Saved me a lot of time, money and headaches. Hopefully it'll do the same for you guys.
        Yeah, so ONLY 10% of people are capable of doing the BARE MINIMUM you require, and you don't think that's ever going to be a problem, even with how cutthroat content creation already is?
        Signature

        Fair warning: It's possible I'm arguing with you because I have nothing better to do.
        Join my free copywriting group on Facebook: http://CultOfCopy.com

        {{ DiscussionBoard.errors[2277518].message }}
        • Profile picture of the author Joe Mobley
          Larry,

          Good input. I haven't used RAC but I will keep them in mind going forward.

          I use oDesk. When, after some research, I do post a job, I limit the search to those with a feedback of 4.5 or greater (out of a possible 5.0 stars) among other things.

          A guarantee, certainly not. Stacking the odds in my, and ultimately the providers favor? I believe so.


          Hope this helps.

          Joe
          Signature

          .

          Follow Me on Twitter: @daVinciJoe
          {{ DiscussionBoard.errors[2277633].message }}
  • Profile picture of the author IdeasThatExcel
    I tried Mturk about a month ago for articles. I had the same problem - people cutting and pasting from Wiki even though I clearly stated I would check. Plus when I would ask for a 300 word article I would get a 150 word article half the time.
    Signature
    Excel Help - Excel Expert Available To Assist You!
    {{ DiscussionBoard.errors[2276294].message }}
  • Profile picture of the author jbode
    Yup, I noticed the same thing even when I clearly state unique articles only and will be check with copyscape...

    I'm guessing most people don't check for duplicates, the cool thing with MTurk is you can reject those articles that are not unique
    {{ DiscussionBoard.errors[2276481].message }}
  • Profile picture of the author VegasGreg
    Kinda like going to the dime store to buy a Mercedes.

    Doesn't work.
    Signature

    Greg Schueler - Wordpress Fanatic... Living The Offline Marketing Dream...

    {{ DiscussionBoard.errors[2276751].message }}
  • Profile picture of the author kevinw1
    Colin, I'd been considering that too, and might just try it out.

    Vegasgreg, LOL. I don't want a Merc tho... just a Matchbox model of a Merc would do, but can't even get that any more!
    {{ DiscussionBoard.errors[2277169].message }}
    • Profile picture of the author davidjames42973
      Yeah I've never been able to get one semi decent article on MTurk. I actually just gave up on using them for articles...
      {{ DiscussionBoard.errors[2277353].message }}
  • Profile picture of the author Lee Timothy
    I found Mturk to be effective in the premature development of gray hair! I got the heck outta there!
    {{ DiscussionBoard.errors[2277630].message }}
  • Profile picture of the author genetic
    Originally Posted by Colin Theriot

    You can still. But for how long? Maybe we haven't hit the wall yet, but we're approaching a homogenization as more people enter that workforce, and they all cover every website that offers jobs like this. Eventually, it'll just be low quality, low price.
    Nah, your looking at it one dimensionally. First off never underestimate the power of competition. Or how poor these people really are, the simple fact is that costs really haven't gone up and the quality you get is what you earn by preparing the workers properly before hand and doing the proper vetting. People will always try and get away with doing less work, you just need to be smarter than them.

    Originally Posted by Colin Theriot

    So wait, if I can't find good people to hire for the price I want to pay, it's my fault? I'm not sure what you're saying here. If it's that "you get what you pay for" no one's disputing that. We're saying the value for pay rate is sliding ever downward.
    Yeah, if you want it blunt. It's your fault. I don't mean to be harsh but we're all big boys and this is business - take responsibility, take ownership. I live by a philosophy where I always try to blame myself first, that way I never pass the buck for my cock ups. Sure sometimes people are just downright incompetent but you can't control that, you can only control what you do. You should ask yourself "did I do everything I could to prevent a bad outcome here". Forget about the worker, they are a variable you can't control, you can only manage them, this is the essence of what I was saying.

    Originally Posted by Colin Theriot

    Yeah, Robert Plant recommended this in a recent webinar. It's a net idea. But it's just an indicator of ability to follow directions. It's got nothing at all to do with how good a quality they will start out at or maintain.

    I predict people are going to get better at giving you the code and worse at giving you what you want, and you're going to be over there beating yourself up over it?

    Even down to the individual, that market can't do anything but burn itself out, because the worst provider with the cheapest price is going to begin to outnumber and outbid and outwork anyone who will try harder.

    At least that's exactly what it looked like in the web designer market a while back, and this is showing the same signs. I could be wrong, but I don't think I am.
    Your missing the point. This is to weed out all the crap in one fell swoop do that you can sift through the rubble and find the gems. It won't leave you with glittering candidates right off, but it gets rid of the worst so that you can concentrate on separating the mediocre from the best.

    Your predictions are wrong, at least based on my findings. This is something I do regularly so the advice I gave was based on actual fact, not armchair reasoning. It's very easy to sit there and pick holes in something, but you haven't tried it so I'm baffled that you are able to make such bold statements.

    Originally Posted by Colin Theriot

    And once you find a provider, you keep em for life, right? You never find good ones who under bid to get the job and want to raise the price on you? Don't find that the quality slacks off over time anyway?

    If so, that would be amazing.
    No, why would I do that? I pick the best provider for the job at hand I barely ever recycle because there is no need to the pool is deep and there are plenty of people desperate to work. And on the odd occasion that I have used the same worker twice I certainly have not experienced a price rise. Why would they? I mean, who do they think they are? They're just content writers, they're not special and I don't need them, they know they can be replaced in a heartbeat so they're not exactly in a very string position to go about jacking up prices.

    Has this been happening with you? I honestly can't imagine being treated that way by unskilled workers. If you strike the right tone when talking to these people they won't be giving you these hassles. I don't think my results are amazing.

    Originally Posted by Colin Theriot

    Yeah, so ONLY 10% of people are capable of doing the BARE MINIMUM you require, and you don't think that's ever going to be a problem, even with how cutthroat content creation already is?
    Yeah ONLY 10% as in my RAC project is viewed 200-300 times per day in the first few days, which gives me 60-90 people to pick from after a few days. That's plenty for me. Why would you want to sift through more people?

    I know your determined to cry gloomy days on the low level outsourcing but to be honest I have no idea where you have been getting your workers, whenever it is it has left you cynical and jaded. Just becauswe you're having a rough time of it doesn't mean we've hit a wall or that we're approaching some sort of wall.

    In fact as employers we have the upper hand in this climate as people are losing jobs globally, especially low skilled office jobs etc. Which is probably where a lot of these content writers come from. Sure they're literate but they ain't gonna spin you a best seller, but then again that's not what your paying them for.

    My final word on this is that there is no "wall", there is no crisis, in fact the future looks bright. I, for one am excited. Call me Mr Burns, but all these people out of work is excellent for me, it means I can pay them less and increase my bottom line. Because let's not kid ourselves here gentlemen, when all is said and done it come down to one thing above all else, profit.

    (cue rant/flame about how there's more to business that profit - tell that to your investors)

    Originally Posted by DigitalJoe

    Larry,

    Good input. I haven't used RAC but I will keep them in mind going forward.

    I use oDesk. When, after some research, I do post a job, I limit the search to those with a feedback of 4.5 or greater (out of a possible 5.0 stars) among other things.

    A guarantee, certainly not. Stacking the odds in my, and ultimately the providers favor? I believe so.


    Hope this helps.

    Joe
    Thanks, always glad to swap ideas and experiences.

    Originally Posted by Teeedo

    I found Mturk to be effective in the premature development of gray hair! I got the heck outta there!
    Testify!
    {{ DiscussionBoard.errors[2277635].message }}
    • Profile picture of the author Michael Shook
      No matter what my project, I have found that if I approach it with the idea that I want quality at a price I can afford, i usually get what I am looking for. People can zip through all these places like MTurk looking to make pennies. I am not much interested in buying or using work from someone whose goal is earning pennies.

      I invest time and money in long term projects rather than short term ones so maybe I have a different outlook on what kind of content I am willing to place on my sites or submit with the hopes of getting a click.
      Signature


      {{ DiscussionBoard.errors[2277673].message }}
      • Profile picture of the author genetic
        Originally Posted by JMichaelZ View Post

        No matter what my project, I have found that if I approach it with the idea that I want quality at a price I can afford, i usually get what I am looking for. People can zip through all these places like MTurk looking to make pennies. I am not much interested in buying or using work from someone whose goal is earning pennies.

        I invest time and money in long term projects rather than short term ones so maybe I have a different outlook on what kind of content I am willing to place on my sites or submit with the hopes of getting a click.
        I like that philosophy: not utilizing workers with low/zero goals or low self esteem. It's true that if a guy is only gonna charge you a dollar for an article, he is really making a statement as to what he thinks his work is worth.

        Much better to convince a guy to do a $50 piece of work for $30 than to make a deal with the guys who charges $10 right off the bat
        {{ DiscussionBoard.errors[2277690].message }}
    • Profile picture of the author Colin Theriot
      Originally Posted by genetic View Post

      Nah, your looking at it one dimensionally. First off never underestimate the power of competition. Or how poor these people really are, the simple fact is that costs really haven't gone up and the quality you get is what you earn by preparing the workers properly before hand and doing the proper vetting. People will always try and get away with doing less work, you just need to be smarter than them.
      I guess I'm just not smart enough then.

      Originally Posted by genetic View Post

      Yeah, if you want it blunt. It's your fault. I don't mean to be harsh but we're all big boys and this is business - take responsibility, take ownership.
      Yes, this is totally my problem.

      Originally Posted by genetic View Post

      I live by a philosophy where I always try to blame myself first, that way I never pass the buck for my cock ups.
      You are a man's man, sir.

      Originally Posted by genetic View Post

      Sure sometimes people are just downright incompetent but you can't control that, you can only control what you do.
      Wait, I thought you said if they do a bad job it's my fault, but now I can't control that... Hm.

      Originally Posted by genetic View Post

      You should ask yourself "did I do everything I could to prevent a bad outcome here". Forget about the worker, they are a variable you can't control, you can only manage them, this is the essence of what I was saying.
      Mnnnnnnnnn. Nope. Still don't get what you mean there. Maybe it's just a philosophical difference.

      Originally Posted by genetic View Post

      Your missing the point. This is to weed out all the crap in one fell swoop do that you can sift through the rubble and find the gems. It won't leave you with glittering candidates right off, but it gets rid of the worst so that you can concentrate on separating the mediocre from the best.
      No, I'm not missing it. I get why you do it.

      Originally Posted by genetic View Post

      Your predictions are wrong, at least based on my findings. This is something I do regularly so the advice I gave was based on actual fact, not armchair reasoning.
      Holy cow! You have a time machine!!! What's the future like!?

      Originally Posted by genetic View Post

      It's very easy to sit there and pick holes in something, but you haven't tried it so I'm baffled that you are able to make such bold statements.
      My laziness just makes me tell lies.

      Originally Posted by genetic View Post

      No, why would I do that? I pick the best provider for the job at hand I barely ever recycle because there is no need to the pool is deep and there are plenty of people desperate to work.
      Mmm hmm.

      Originally Posted by genetic View Post

      And on the odd occasion that I have used the same worker twice I certainly have not experienced a price rise. Why would they? I mean, who do they think they are? They're just content writers, they're not special and I don't need them, they know they can be replaced in a heartbeat so they're not exactly in a very string position to go about jacking up prices.
      Mmm hmm.

      Originally Posted by genetic View Post

      Has this been happening with you? I honestly can't imagine being treated that way by unskilled workers. If you strike the right tone when talking to these people they won't be giving you these hassles.
      Do you have an mp3 or something of you using the proper tone? Do you have to use an accent? I bet a British one would work good, because they sound all proper and authoritarian, right?

      Originally Posted by genetic View Post

      I don't think my results are amazing.
      Aw, don't be so hard on yourself.

      Originally Posted by genetic View Post

      Yeah ONLY 10% as in my RAC project is viewed 200-300 times per day in the first few days, which gives me 60-90 people to pick from after a few days. That's plenty for me. Why would you want to sift through more people?
      A serious question among the silliness: how frequently are you sifting through 60-90 people to replenish this pool of workers you move through project to project? Is it really 60-90 DIFFERENT people who bid every time?

      Originally Posted by genetic View Post

      I know your determined to cry gloomy days on the low level outsourcing but to be honest I have no idea where you have been getting your workers, whenever it is it has left you cynical and jaded.
      It's that oil slick. Just makes me lose faith in humanity, you know?

      Originally Posted by genetic View Post

      Just becauswe you're having a rough time of it doesn't mean we've hit a wall or that we're approaching some sort of wall.
      You're right. We should all just jump in there and talk to those workers like they should be talked to and make em keep doing whatever for cheap because dang it, they need work. And they'll never just have an overall quality drop due to crappy return on work put in. They'll just keep on striving to outdo each other because, you know competition in a race to the bottom just really drives people to over achieve.

      Originally Posted by genetic View Post

      In fact as employers we have the upper hand in this climate as people are losing jobs globally, especially low skilled office jobs etc.
      Yeah, it's us vs. them! We win!

      Originally Posted by genetic View Post

      Which is probably where a lot of these content writers come from.
      Yep.

      Originally Posted by genetic View Post

      Sure they're literate but they ain't gonna spin you a best seller, but then again that's not what your paying them for.
      I think the way you say you treat your writers kind of makes the above a self-fulfilling prophecy.

      Originally Posted by genetic View Post

      My final word on this is that there is no "wall", there is no crisis, in fact the future looks bright. I, for one am excited. Call me Mr Burns, but all these people out of work is excellent for me, it means I can pay them less and increase my bottom line.
      Yeah! Ha ha! You rule! High five!

      Originally Posted by genetic View Post

      Because let's not kid ourselves here gentlemen, when all is said and done it come down to one thing above all else, profit.
      When it all comes down to profit, it all comes down.

      Originally Posted by genetic View Post

      (cue rant/flame about how there's more to business that profit - tell that to your investors)
      Why is it that investors always get portrayed as this shadow cabinet of greed machines that only care about profit above all else? I think it's because a-hole CEO types like to have someone to blame their predatory behavior on.

      But hey, baby. Let's not fight. I work for myself so I wouldn't have to have bosses like you anymore.
      Signature

      Fair warning: It's possible I'm arguing with you because I have nothing better to do.
      Join my free copywriting group on Facebook: http://CultOfCopy.com

      {{ DiscussionBoard.errors[2277769].message }}
  • Profile picture of the author genetic
    You are a scholar and a gentleman. I see you thoroughly deserve "the Riot" title. You're a great laugh.

    Take care
    {{ DiscussionBoard.errors[2277777].message }}

Trending Topics