finding all replicated sites in a domain
I'm not sure if programming is the right forum but I thought that it is more of a programming problem than anything else.
I want to find a program or tool that will identify all the replicated sites in an mlm domain. Typically the replicated sites are subdomains with a unique id in front of the domain name, like # or * dot mlm dot com or they are extensions like mlm dot com/id name or number (/# or *). I have mostly just lurked in the forum instead of posting so the rules won't let me use actual urls.
The process would probably be to use the search engines to find the sites and then be able to crawl through all the sites and scrape for the wanted information. The key information would be phone numbers, but names and email addresses would be a huge plus.
Does anyone know of a program that combines the crawling and scraping in one package?
I want to find a program or tool that will identify all the replicated sites in an mlm domain. Typically the replicated sites are subdomains with a unique id in front of the domain name, like # or * dot mlm dot com or they are extensions like mlm dot com/id name or number (/# or *). I have mostly just lurked in the forum instead of posting so the rules won't let me use actual urls.
The process would probably be to use the search engines to find the sites and then be able to crawl through all the sites and scrape for the wanted information. The key information would be phone numbers, but names and email addresses would be a huge plus.
Does anyone know of a program that combines the crawling and scraping in one package?
www.OFFLINEMARKETINGFORUM.com (Register Now)
:)