[Tip] Scrape URLs from SERPs using FireFox
- SEO |
Recently I was playing around with Google search operators when I found a slew of sub domains that were present in Google's index for a client. They didn't know how they were created or why but they contained duplicate pages that were already present on their website. There were more than 1,300 of them and needless to say, something had to be done. I contacted their hosting company however they wanted a list of all the URLs so that they could proceed. No problem right? That's when I realized I would need to extract all those URL's from a Google SERP so that I could send them off to the hosting provider.
It used to be that you could export URLs from Google SERPs using the Mozbar however that feature is no longer available (at least for new downloads). I emailed back and forth with their team and they claim the feature will be back in the next versions of the browser add-on.
Scraping URLs and Much More Using SEO4Firefox
You can also get URLs out of SERPs using SEO4Firefox. You can also get a bunch of other data for each URL on the page. You can get the plugin at the link below.
1. Visit SEO for Firefox Extension: Free SEO Toolbar Firefox Browser Plugin
2. Install the tool.
3. Run your search in Google (note: it works for Bing too and that's the only other engine I tried so it may work for more).
4. At the top of the results page you will notice some links. One of them says CSV. Click it.
5. You will be prompted to name the file, name it and click save.
6. If you open the CSV file, you will see all the URLs that were on the page. Note that it will only export as many URLs as there were on the page so if you want more you will have to change your search settings to show you more.
If you haven't heard of this extension, it provides you tons of valuable data about the results in your SERPs among other cool features that you can use once on a page. For the export feature above, SEO4Firefox will pull all of this data off of the SERPs that is provided in the overlay like PR, links, etc, and include it in the spreadsheet for each URL. The only drawback is that the data is not provided by default in the SERP overlay from SEO4Firefox. The labels for the data appear however you have to click on the little question marks to get them to show up in your CSV file. Otherwise Outwit hub just pulls in all the question marks. Naturally that would be a lot of clicking if you were exporting thousands of URLs but if you are only doing a dozen or so its handy.
I know there are a lot of other tools out there that scrape pages of URLs but this is one of the most efficient and free ones if you are just interested in getting URLs off of an SERP.
Do you know of any other free tools to do this with?
It used to be that you could export URLs from Google SERPs using the Mozbar however that feature is no longer available (at least for new downloads). I emailed back and forth with their team and they claim the feature will be back in the next versions of the browser add-on.
Scraping URLs and Much More Using SEO4Firefox
You can also get URLs out of SERPs using SEO4Firefox. You can also get a bunch of other data for each URL on the page. You can get the plugin at the link below.
1. Visit SEO for Firefox Extension: Free SEO Toolbar Firefox Browser Plugin
2. Install the tool.
3. Run your search in Google (note: it works for Bing too and that's the only other engine I tried so it may work for more).
4. At the top of the results page you will notice some links. One of them says CSV. Click it.
5. You will be prompted to name the file, name it and click save.
6. If you open the CSV file, you will see all the URLs that were on the page. Note that it will only export as many URLs as there were on the page so if you want more you will have to change your search settings to show you more.
If you haven't heard of this extension, it provides you tons of valuable data about the results in your SERPs among other cool features that you can use once on a page. For the export feature above, SEO4Firefox will pull all of this data off of the SERPs that is provided in the overlay like PR, links, etc, and include it in the spreadsheet for each URL. The only drawback is that the data is not provided by default in the SERP overlay from SEO4Firefox. The labels for the data appear however you have to click on the little question marks to get them to show up in your CSV file. Otherwise Outwit hub just pulls in all the question marks. Naturally that would be a lot of clicking if you were exporting thousands of URLs but if you are only doing a dozen or so its handy.
I know there are a lot of other tools out there that scrape pages of URLs but this is one of the most efficient and free ones if you are just interested in getting URLs off of an SERP.
Do you know of any other free tools to do this with?
-
Abigo -
Thanks
{{ DiscussionBoard.errors[8730482].message }} -