Screaming Frog Blues
- SEO |
Hi Warriors!
I've recently taken on some SEO responsibilities with my company.
After doing some research on SEO basics I decided to use Screaming Frog to crawl me site and get a view of what's been indexed.
It's a big site (approx. 5,000 pages) but when I ran Screaming Frog it only came up with a handful of pages.
One was the homepage (which I typed in initially) and the others were non-customer facing pages.
I've done a few Google searches and found that other pages on the site are showing up on Google so I'm assuming Google has crawled them.
My question is why wouldn't they be showing up on Screaming Frog if that's the case?
As a comparison I ran a few competitors through SF and they all crawled perfectly.
If it's of any help I tried doing an SEO site audit in SEM Rush and when I set the crawl source as "Website" a similar thing happened - the tool only crawled the homepage and some sitemaps. When I switch the crawl source to "Sitemaps on Site" it worked fine.
Any help would be greatly appreciated. Any questions let me know.
Thanks
I've recently taken on some SEO responsibilities with my company.
After doing some research on SEO basics I decided to use Screaming Frog to crawl me site and get a view of what's been indexed.
It's a big site (approx. 5,000 pages) but when I ran Screaming Frog it only came up with a handful of pages.
One was the homepage (which I typed in initially) and the others were non-customer facing pages.
I've done a few Google searches and found that other pages on the site are showing up on Google so I'm assuming Google has crawled them.
My question is why wouldn't they be showing up on Screaming Frog if that's the case?
As a comparison I ran a few competitors through SF and they all crawled perfectly.
If it's of any help I tried doing an SEO site audit in SEM Rush and when I set the crawl source as "Website" a similar thing happened - the tool only crawled the homepage and some sitemaps. When I switch the crawl source to "Sitemaps on Site" it worked fine.
Any help would be greatly appreciated. Any questions let me know.
Thanks
We help sellers get the MAXIMUM amount for their websites and all buyers know that these sites are 100% vetted.