4 {{ upvoteCount | shortNum }}
4 {{ upvoteCount | shortNum }}

How do you sitemap a 300K pages site?

by debukur
3 replies
I have a 300K+ pages site that I'm having hardtime site-maping. We are using Unlimited XML, Text, HTML Sitemap Generator - XML Sitemaps Generator that is very slow and slow downs server badly. Any advice?
I heard some people use few site maps on big sites, and on each site. How would I approach that?
#300k #pages #site #sitemap
  • Profile picture of the author JayXtreme
    It is fairly easy to do with GSiteCrawler, but there is some manual tweaks that you need to make..

    Such as replacing underscores with / where GSiteCrawler puts them in when generating the sitemap.

    The program I mentioned, will crawl your site folders individually and give them all their individual sitemaps, linking from an index sitemap.

    It sounds a little complex, but I'm sure you'll get your head around it fairly easy. We've done a 200,000+ page site like this and it was eeeeasy (sort of)

    Peace

    Jay
    Signature

    Bare Murkage.........

    {{ DiscussionBoard.errors[1432019].message }}
    • Profile picture of the author debukur
      Originally Posted by JayXtreme View Post

      It is fairly easy to do with GSiteCrawler, but there is some manual tweaks that you need to make..

      Such as replacing underscores with / where GSiteCrawler puts them in when generating the sitemap.

      The program I mentioned, will crawl your site folders individually and give them all their individual sitemaps, linking from an index sitemap.

      It sounds a little complex, but I'm sure you'll get your head around it fairly easy. We've done a 200,000+ page site like this and it was eeeeasy (sort of)

      Peace


      Jay
      Thanks Jay it does sound complex, but I'll ask my guys too look into it. We though something automated...
      Any ideas about writing a site map script?

      Chris, no I did not write it all :rolleyes: Majority of it is a directory with couple pages for each company
      Signature
      Dream like you will live forever, live like you will die tomorrow
      NEEDED: virtual assistant (VA)
      {{ DiscussionBoard.errors[1438264].message }}
  • Profile picture of the author Chris Stigson
    Dang, that's a lot of pages =) Is it an article directory or something like that? You cannot have created all those pages yourself, 1000 per day for a year Nice.
    {{ DiscussionBoard.errors[1432032].message }}

Trending Topics