Disable RSS Feeds in Wordpress to Prevent Scrapers and Speed Up Performance Dramatically

14 replies
After running a search on Google, I found out my RSS feeds are being scraped and syndicated. This happened because Wordpress publishes an RSS feed.

Fact is, I have many sites that function as CMS "Static-style" sites. These sites don't need RSS feeds and no one reads or subscribes to these feeds.

Furthermore, I no longer care about backlinks from RSS Directories. If I do want a backlink from them, I'll create my own RSS Feed from RSSBot and upload the rss.xml file manually.

Finally, I don't feel like DMCA'ing each host to take down the content and I don't feel like dropping a backlink in the RSS feed. Most scrapers have grown advanced enough to remove anything with a "http://" prefix in the feed anyways.

So I permanently disabled my feed and comments feed by inserting the folllowing code at the very end of my functions.php file.

Code:
     function fb_disable_feed() {
    wp_die( __('No feed available,please visit our <a href="'. get_bloginfo('url') .'">homepage</a>!') );
    }
    add_action('do_feed', 'fb_disable_feed', 1);
    add_action('do_feed_rdf', 'fb_disable_feed', 1);
    add_action('do_feed_rss', 'fb_disable_feed', 1);
    add_action('do_feed_rss2', 'fb_disable_feed', 1);
    add_action('do_feed_atom', 'fb_disable_feed', 1);

function remove_comments_rss( $for_comments ) {
    return;
}
add_filter('post_comments_feed_link','remove_comments_rss');
Nice side benefit: all my Wordpress sites are blazing fast now.

Note, when inserting the code above, keep a close eye on the last line in the functions.php file. If you see a ?>, then insert the code BEFORE the ?>. If there isn't a ?>, go ahead and paste at the very end.

As always, save a backup of the functions.php file just in case.
#content scraper #content scraping #disable #disable rss feeds #dramatically #feeds #performance #plagiarism #prevent #rss #scrapers #speed #wordpress
  • Profile picture of the author netkid
    Hey Blue Squares,

    How can I find out if my RSS feeds are "actually" slowing my blog? Alexa.com has defined my blog as very slow and "warns" me of my slow speed will equal lower rankings because of it.

    I put WP Super Cache, which is supposed to speed things up but I still get a slow speed rating on my blog.
    {{ DiscussionBoard.errors[2493804].message }}
  • Profile picture of the author abednego
    Originally Posted by BlueSquares View Post

    ...
    Fact is, I have many sites that function as CMS "Static-style" sites. These sites don't need RSS feeds and no one reads or subscribes to these feeds.
    ...
    Thank you for the insightful post.

    I was under the impression that RSS feeds were commonly used, though...

    And what do you mean by static-style, that it doesn't get updated?
    Signature

    {{ DiscussionBoard.errors[2493845].message }}
  • Profile picture of the author Andyhenry
    That's weird - I love using RSS - I love it when people scrape my content, it brings me a ton of extra traffic and makes me more money.

    I've never understood why people don't want others to share their content with rss - it always links back to your site when it's scraped.

    Maybe I'm missing a problem somewhere but if you're marketing your site using rss and a few other basic methods it's not like someone else is going to get your traffic because they're scraping your content - I find the opposite, the more traffic they get - the more they send me from people liking my content and following the link back to me.
    Signature

    nothing to see here.

    {{ DiscussionBoard.errors[2494197].message }}
  • Profile picture of the author thebitbotdotcom
    Same here. I syndicate my entire blog post and fill it up with backlinks to my site in the articles and a custom footer that contains keywords of interest. I wish autobloggers would scrape it. I couldn't care less if they steal my articles this way. Google indexes my posts within 60 seconds of me posting them each and every time. SCRAPE AWAY! They don't get ANY credit for the content, but I get backlinks!!!
    Signature
    Do Your Copywriting Skills Suck?

    Let Us Help You Develop Your Writing Skills!

    Submit Guest Posts With [ TheBitBot.Com ]
    {{ DiscussionBoard.errors[2494478].message }}
  • Profile picture of the author sbucciarel
    Banned
    I get backlinks from scrapers ... I put out the welcome mat.
    {{ DiscussionBoard.errors[2494839].message }}
    • Profile picture of the author JayXtreme
      Originally Posted by sbucciarel View Post

      I get backlinks from scrapers ... I put out the welcome mat.
      When you are expanding websites into 50,000 pages+ and you are using an aggressive backlinking strategy, that isn't always the best option... for us anyway.

      Nobody wants links from "bad neighbourhoods" They have been proven to negatively affect the strength of a website.
      Signature

      Bare Murkage.........

      {{ DiscussionBoard.errors[2495199].message }}
  • Profile picture of the author goodmast3r
    I'm happy if other people use my content. It will give me backlink.
    Signature
    {{ DiscussionBoard.errors[2494924].message }}
  • Profile picture of the author ~kev~
    In your wordpress RSS feed setting, why not just set it to preview - instead of the full article?

    I use my rss feeds for too much, there is no way I could disable them. For one thing, their submitted to my google webmaster tools and yahoo site explorer as a sitemap. My feed goes to my facebook page, yahoo blog log,,,,,,, and a few other sites that I get keywords and backlinks from.
    {{ DiscussionBoard.errors[2495066].message }}
  • Profile picture of the author rjaf
    That's the whole point of RSS: get others to use your stuff and spread your URL/s all over the universe. Of course, if they use your stuff and strip-out the links to your site, then get all legal on their ass, etc.
    {{ DiscussionBoard.errors[2495085].message }}
  • Profile picture of the author Jesus Perez
    Again, this solution isn't for everyone. Some people want their content being scraped. I don't.

    Just keep in mind, your links aren't being posted to a lot of sites. These professional scrapers will remove all attribution and backlinks. Then, they'll potentially outrank you in the serps.

    Read this story: Scraped content ranking higher than the original source
    The original author was scraped manually, but the additional replies show how pro scrapers are outranking original sites.
    Signature

    {{ DiscussionBoard.errors[2495108].message }}
  • Profile picture of the author JayXtreme
    I'm sort of on the fence on this one.

    We can never really decide if we want to use RSS or not on most of our sites.

    On some of our sites, we are using similar code that is posted in the OP, and on others we don't use it.

    It's an ongoing test that we do...

    Great share, Jesus..

    I can see why some people might be happy having their content scraped, but there is also times when it is counter productive to your end goal with a website..

    Peace

    Jay
    Signature

    Bare Murkage.........

    {{ DiscussionBoard.errors[2495189].message }}
  • Profile picture of the author mojojuju
    Originally Posted by BlueSquares View Post



    Nice side benefit: all my Wordpress sites are blazing fast now.
    Have you run any tests (apache bench?) which gave results that support this statement?
    Signature

    :)

    {{ DiscussionBoard.errors[2495322].message }}
    • Profile picture of the author mojojuju
      Using a fresh Wordpress install fed with 10 posts, I ran apache bench to test the effects (if any) of disabling feeds on site performance.

      The exact command used to test was:


      ab -n 10000 -c 10 http://tempwordpresssite.com/?p=4

      That command was done with and without the modifications suggested in this thread.

      Here are the results:

      ===================
      Without Modifications
      ===================
      Code:
      This is ApacheBench, Version 2.3 <: 655654 $>
      Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
      Licensed to The Apache Software Foundation, http://www.apache.org/
      
      Benchmarking tempwordpresssite.com (be patient)
      Completed 1000 requests
      Completed 2000 requests
      Completed 3000 requests
      Completed 4000 requests
      Completed 5000 requests
      Completed 6000 requests
      Completed 7000 requests
      Completed 8000 requests
      Completed 9000 requests
      Completed 10000 requests
      Finished 10000 requests
      
      
      Server Software:        Apache/2.2.9
      Server Hostname:        tempwordpresssite.com
      Server Port:            80
      
      Document Path:          /?p=4
      Document Length:        12653 bytes
      
      Concurrency Level:      10
      Time taken for tests:   322.122 seconds
      Complete requests:      10000
      Failed requests:        13
         (Connect: 0, Receive: 0, Length: 13, Exceptions: 0)
      Write errors:           0
      Non-2xx responses:      13
      Total transferred:      130475040 bytes
      HTML transferred:       126375625 bytes
      Requests per second:    31.04 [#/sec] (mean)
      Time per request:       322.122 [ms] (mean)
      Time per request:       32.212 [ms] (mean, across all concurrent requests)
      Transfer rate:          395.56 [Kbytes/sec] received
      
      Connection Times (ms)
                    min  mean[+/-sd] median   max
      Connect:        0    0   0.2      0       9
      Processing:    12  322 166.6    275    4315
      Waiting:        7  320 166.2    272    4315
      Total:         12  322 166.6    275    4315
      
      Percentage of the requests served within a certain time (ms)
        50%    275
        66%    321
        75%    360
        80%    388
        90%    477
        95%    579
        98%    735
        99%    854
       100%   4315 (longest request)
      ===================
      With Modifications
      ===================
      Code:
      This is ApacheBench, Version 2.3 <: 655654 $>
      Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
      Licensed to The Apache Software Foundation, http://www.apache.org/
      
      Benchmarking tempwordpresssite.com (be patient)
      Completed 1000 requests
      Completed 2000 requests
      Completed 3000 requests
      Completed 4000 requests
      Completed 5000 requests
      Completed 6000 requests
      Completed 7000 requests
      Completed 8000 requests
      Completed 9000 requests
      Completed 10000 requests
      Finished 10000 requests
      
      
      Server Software:        Apache/2.2.9
      Server Hostname:        tempwordpresssite.com
      Server Port:            80
      
      Document Path:          /?p=4
      Document Length:        12481 bytes
      
      Concurrency Level:      10
      Time taken for tests:   334.120 seconds
      Complete requests:      10000
      Failed requests:        24
         (Connect: 0, Receive: 0, Length: 24, Exceptions: 0)
      Write errors:           0
      Non-2xx responses:      24
      Total transferred:      128627433 bytes
      HTML transferred:       124528483 bytes
      Requests per second:    29.93 [#/sec] (mean)
      Time per request:       334.120 [ms] (mean)
      Time per request:       33.412 [ms] (mean, across all concurrent requests)
      Transfer rate:          375.95 [Kbytes/sec] received
      
      Connection Times (ms)
                    min  mean[+/-sd] median   max
      Connect:        0    0   0.6      0      26
      Processing:    13  334 1775.6    238   64048
      Waiting:       13  332 1775.6    237   64048
      Total:         13  334 1775.6    239   64048
      
      Percentage of the requests served within a certain time (ms)
        50%    239
        66%    269
        75%    293
        80%    310
        90%    373
        95%    449
        98%    576
        99%    692
       100%  64048 (longest request)
      As you can see, the disabling of feeds in my situation did not result in a faster site in terms of mean requests per second or any other metric. In fact, the test site became slightly slower after disabling feeds.

      This is my setup though and your results may be different. Just make sure to test these things on your own server before implementing them.
      Signature

      :)

      {{ DiscussionBoard.errors[2495680].message }}
  • Profile picture of the author Mark Hess
    It's up to the blog owner to decide whether to have their RSS feed enabled/disabled...

    If you are not comfortable hacking the functions.php code, there is a free Wordpress plugin that will disable all feeds: WordPress › Disable RSS WordPress Plugins

    It appears to use the same code as BlueSquares pointed out....
    Signature
    {{ DiscussionBoard.errors[2495707].message }}

Trending Topics