22 {{ upvoteCount | shortNum }}
22 {{ upvoteCount | shortNum }}

HELP WANTED: How can I create a list of every url on my site?

by magentawave
17 replies
I need a list of ALL the urls on my site. Does anyone know of a simple way to get the urls ONLY and then paste all of them into notepad? I just want the urls without the titles, etc. I know I can see those pages on my XML sitemap but copying and pasting that into notepad requires a lot of editing after to remove all the NON-url text.

Thanks
Steve
#create #list #site #url #wanted
  • Profile picture of the author John Romaine
    Not sure about this one - but have a look at Google's Webmaster tools.
    Signature

    BS free SEO services, training and advice - SEO Point

    {{ DiscussionBoard.errors[1870323].message }}
    • Profile picture of the author magentawave
      Yeah I checked and couldn't find anything there. Thanks though!


      Originally Posted by ramone_johnny View Post

      Not sure about this one - but have a look at Google's Webmaster tools.
      {{ DiscussionBoard.errors[1870490].message }}
  • Profile picture of the author George Wright
    Hi,

    Don't know if this will help but I found these on Google. free extract urls from web page - Google Search=
    Signature
    "The first chapter sells the book; the last chapter sells the next book." Mickey Spillane
    {{ DiscussionBoard.errors[1870514].message }}
  • Profile picture of the author PowerWealth247
    You can "view source" to see your HTML then under "edit" use the find function to find "www." and it will highlight all the URLs.
    Signature

    marketing online for offline businesses
    http://www.Facebook.com/ColumbusLocalMarketing
    www.TerryPower.com

    {{ DiscussionBoard.errors[1870549].message }}
  • Profile picture of the author W2L
    Mmmm I know SocialBot has a crawler feature that gets all the links from your site and puts them in a txt or xml format.... don't know what other tool may have something similar...
    {{ DiscussionBoard.errors[1870559].message }}
  • Profile picture of the author Will Edwards
    Provided they are linked together, you could use Xenu - Google 'Xenu Download' or similar. The program is called Link Sleuth and it's free.

    Will
    {{ DiscussionBoard.errors[1870565].message }}
    • Profile picture of the author patey88
      The tool here produces a urllist.txt file. If all your pages can be reached by following links from the main page, and if your site doesn't exceed the tool's max of 500 pages, then this should do the trick:

      www (dot) xml-sitemaps (dot) com
      {{ DiscussionBoard.errors[1870585].message }}
      • Profile picture of the author Dennis Gaskill
        Originally Posted by patey88 View Post

        The tool here produces a urllist.txt file. If all your pages can be reached by following links from the main page, and if your site doesn't exceed the tool's max of 500 pages, then this should do the trick:

        www (dot) xml-sitemaps (dot) com
        You beat me to it.

        Seriously, if your site is online just generate a sitemap and choose the url list. Here's the link since patey88 can make links yet.

        Create your Google Sitemap Online - XML Sitemaps Generator
        Signature

        Just when you think you've got it all figured out, someone changes the rules.

        {{ DiscussionBoard.errors[1873614].message }}
        • Profile picture of the author VanessaB
          Guess I'm the only still copying and pasting into spreadsheets

          -Dani
          Signature
          The Recon Report
          Reliable Results, Predictable Profits
          {{ DiscussionBoard.errors[1873643].message }}
          • Profile picture of the author magentawave
            Yes they are all wordpress sites. No I don't know how to use phpMyAdmin. I saw some code I'm supposed to use for what you're thinking of on another forum but they didn't give step by step instructions on how to use it. I'm not a retard but then I'm not a techie either so blew off that idea.

            Originally Posted by KirkMcD View Post

            Are these wordpress sites? Do you know how to use phpMyAdmin?
            I'll write a sql query that can pull out the info for you.



            Thanks Dennis but that was the first thing I did (click on my xml sitemap) but it includes a bunch of other stuff besides just the urls and that will require a lot of editing for 125+ post sites X 10 sites (like editing close to 1300 or 1400 urls in the lists to remove everything but the urls).

            Originally Posted by Dennis Gaskill View Post

            You beat me to it.

            Seriously, if your site is online just generate a sitemap and choose the url list. Here's the link since patey88 can make links yet.

            Create your Google Sitemap Online - XML Sitemaps Generator




            Gawd, I'm hoping we don't have to do that! :p

            Originally Posted by DanielleS View Post

            Guess I'm the only still copying and pasting into spreadsheets

            -Dani
            {{ DiscussionBoard.errors[1874518].message }}
            • Profile picture of the author magentawave
              I'm trying out SocialBot right now but it seems to be adding tons of pages to the list like Propeller, Newsvine, etc. that aren't on my site. We shall see.

              Originally Posted by W2L View Post

              Mmmm I know SocialBot has a crawler feature that gets all the links from your site and puts them in a txt or xml format.... don't know what other tool may have something similar...


              I think Xenu will be my next choice - but will Xenu give me a list of urls ONLY, or does it include other text in that list?

              Originally Posted by Will Edwards View Post

              Provided they are linked together, you could use Xenu - Google 'Xenu Download' or similar. The program is called Link Sleuth and it's free.

              Will
              {{ DiscussionBoard.errors[1874568].message }}
            • Profile picture of the author Dennis Gaskill
              Originally Posted by magentawave View Post

              Thanks Dennis but that was the first thing I did (click on my xml sitemap) but it includes a bunch of other stuff besides just the urls and that will require a lot of editing for 125+ post sites X 10 sites (like editing close to 1300 or 1400 urls in the lists to remove everything but the urls).
              You must not have chosen the url list text file. I tried it and didn't have any extra stuff. Just the raw url and nothing else. Notepad may show a hollow square in place of the carriage return at the end of each URL, but open the file in any other text editor and you'll have just the URL's, one per line.

              You want to choose the last link in the list of options once it's completed. It will read:

              Download Sitemap in Text Format
              http:// www . xml-sitemaps . com/download/yourDomain.com/urllist.txt
              Signature

              Just when you think you've got it all figured out, someone changes the rules.

              {{ DiscussionBoard.errors[1874622].message }}
              • Profile picture of the author magentawave
                You were absolutely right Dennis! I must have done it the wrong way the first time. Anyway, I just ran it (Create your Google Sitemap Online - XML Sitemaps Generator) for one of my sites and saved it as a text file and that is exactly what I needed!

                THANK YOU VERY MUCH DENNIS --- AND EVERYONE ELSE THAT PARTICIPATED TOO!

                Steve


                Originally Posted by Dennis Gaskill View Post

                You must not have chosen the url list text file. I tried it and didn't have any extra stuff. Just the raw url and nothing else. Notepad may show a hollow square in place of the carriage return at the end of each URL, but open the file in any other text editor and you'll have just the URL's, one per line.

                You want to choose the last link in the list of options once it's completed. It will read:

                Download Sitemap in Text Format
                http:// www . xml-sitemaps . com/download/yourDomain.com/urllist.txt
                {{ DiscussionBoard.errors[1874803].message }}
  • Profile picture of the author Deepak Media
    If your site is a wordpress blog, install google XML sitemaps plugin and it will generate all the URLs for you!
    Signature
    Digital Marketing Author | Speaker | Consultant

    Read my Blog: DigitalDeepak.com

    @ Bangalore, India.
    {{ DiscussionBoard.errors[1870732].message }}
    • Profile picture of the author magentawave
      Thanks but that was the first thing I tried and besides the urls, it also shows Priority, Change Frequency and Last Change which means lots and lots of editing for 100+ post/page sites X 10.

      Thanks
      Steve



      Originally Posted by Deepak Media View Post

      If your site is a wordpress blog, install google XML sitemaps plugin and it will generate all the URLs for you!
      {{ DiscussionBoard.errors[1872639].message }}
  • Profile picture of the author KirkMcD
    Are these wordpress sites? Do you know how to use phpMyAdmin?
    I'll write a sql query that can pull out the info for you.
    {{ DiscussionBoard.errors[1873592].message }}
  • Profile picture of the author AllanWard
    I've often wondered if there was a Wordpress plugin that could do this. It would be great if there was something that gave you a list of the post / page titles and URL's. Then when I'm writing a new post and want to link to an older post, it's easier to find.
    {{ DiscussionBoard.errors[1874584].message }}

Trending Topics