Criteria by which google will index a page

3 years ago

Your on-page SEO must be on point, also other websites that link to your page is called a backlink.

Having lots of high quality backlinks assures that your website gets a good index position in google.

[ 1 ] Thanks

Signature

Got my first Clickbank Sale within 24 hours from this Twitter Method. After struggling for 3 months with no success!

{{ DiscussionBoard.errors[11125266].message }}

zorbee 3 years ago

[DELETED]

{{ DiscussionBoard.errors[11125542].message }}

Risktaker89

3 years ago

Include a few pages such as contact, about me / about us, privacy policy, income disclaimer to your website.

Make sure your website is not infected with virus (mine was and it was the main reason it was not indexed)

To help get your new post indexed, ping your blog post, share your link on twitter and google plus, submit your url to Google. This usually gets your post indexed on the same day.

[ 1 ] Thanks

Signature

Finally Succeed With This 100% Free Money Making System Which Does All The Selling For You! 100% Newbie Friendly!

{{ DiscussionBoard.errors[11126312].message }}

nishagarg

3 years ago

The best way to get your page indexed is to check your website's whois on several whois checker websites. They naturally crawl your website and gives Google a reason to index.

Another way is to request indexing under the Google Console tool. It takes 1-5 hours for them to index a page. However, what I have discovered in this case is, since you forced to index a page, the ranking show no or inaccurate results.

Even better, give your page a link from sites like Quora, LinkedIn, and WarriorForum (if it is niche related).

Have tried all the 3 methods and all did work

[ 1 ] Thanks
2 replies

Signature

Need help with building an online business?
Private message me so I can guide you further.

{{ DiscussionBoard.errors[11126391].message }}

nishagarg 3 years ago

Also, as Risktaker89 mentioned, you should have 4 common pages which are about, contact, privacy policy, and disclaimer.
- [ 1 ] Thanks
Signature

Need help with building an online business?
Private message me so I can guide you further.
{{ DiscussionBoard.errors[11126395].message }}

mattbarrie

3 years ago

Would the reason that the whois server triggers google be that it creates a page with a backlink and then google thinks it's worth visiting?

Thanks
2 replies

{{ DiscussionBoard.errors[11126400].message }}

nishagarg 3 years ago

Yes, that's the case. I tested different ways for my different websites, and this can be the possible reason.
- Thanks
- 1 reply
Signature

Need help with building an online business?
Private message me so I can guide you further.
{{ DiscussionBoard.errors[11126402].message }}
- andre james 3 years ago
  
  I've heard this before but does it actually work? Also the backlink that is created does it remain or get deleted?
  
  Thanks
  
  {{ DiscussionBoard.errors[11128123].message }}

MikeFriedman

3 years ago

Originally Posted by mattbarrie

Would the reason that the whois server triggers google be that it creates a page with a backlink and then google thinks it's worth visiting?

Matt,

It doesn't really do that. Whatever results nishagarg saw was just a coincidence.

Plus, even if it did, it won't help you get a large site indexed or new pages of content.

And the Search Console thing he mentioned is fine for a small site. For a very large one, it won't do as much good. You are limited how many times you can request Google to crawl pages on the site. Plus, it is very time consuming if you have say thousands of pages to enter in.

Thanks
2 replies

Signature

{{ DiscussionBoard.errors[11126761].message }}

mattbarrie

3 years ago

Mike, I've read somewhere that despite being a sitemap, Google will only index a page if it has another page pointing to it with enough authority. Thus I think when nishagarg used these whois sites, perhaps one of them made a page (e.g. domaintools) and linked back to the domain which kicked off the indexing process? Alternately could be a coincidence as you mention.

Thanks
1 reply

{{ DiscussionBoard.errors[11126778].message }}

MikeFriedman

3 years ago

Originally Posted by mattbarrie

Mike, I've read somewhere that despite being a sitemap, Google will only index a page if it has another page pointing to it with enough authority. Thus I think when nishagarg used these whois sites, perhaps one of them made a page (e.g. domaintools) and linked back to the domain which kicked off the indexing process? Alternately could be a coincidence as you mention.

If that were true, brand new sites would almost never, ever get indexed.

New Tumblr blogs would struggle to get indexed.

I could go on and on with examples, but I think you get the idea.

Thanks
2 replies

Signature

{{ DiscussionBoard.errors[11126838].message }}

mattbarrie 3 years ago

New Tumblr blogs would be linked to from a directory or "new sites" page somewhere on Tumblr.
- Thanks
{{ DiscussionBoard.errors[11127128].message }}
mattbarrie 3 years ago

Also perhaps google checks the root / of all domains it knows about periodically?
- Thanks
{{ DiscussionBoard.errors[11127129].message }}

AlphaAlpha 3 years ago

Are you saying that we should all just disregard everything in nishararg's post?
- Thanks
- 1 reply
Signature

Internet Marketing newb since 2015
{{ DiscussionBoard.errors[11127027].message }}
- MikeFriedman 3 years ago
  
  Originally Posted by AlphaAlpha
  
  Are you saying that we should just disregard everything in nishararg's post?
  
  Pretty much.
  
  The Search Console thing works if you are just concerned with indexing a small static website.
  
  Building links from popular sites has a chance to bring the search engine spiders faster, but there is no guarantee on that. It depends on the crawling schedule, how deep they crawl, and when the last time was they crawled the site. And even then, just crawling a page doesn't mean Google is going to index it.
  
  If you are having trouble getting pages indexed, there is most likely a problem with the site.
  
  Thanks
  
  1 reply
  
  Signature
  
  {{ DiscussionBoard.errors[11127045].message }}
  
  nishagarg 3 years ago
  
  @MikeFriedman, I agree that this can be a coincidence. This is true that my websites are WordPress based and have 20-100 pages. Perhaps a small website.
  
  Another best thing you can do mattbarrie is add small unique posts at sites like SteemIt, OnMogul, Medium and Tumblr.
  
  Thanks
  
  Signature
  
  Need help with building an online business?
  Private message me so I can guide you further.
  
  {{ DiscussionBoard.errors[11127175].message }}

luxazan

3 years ago

Try to Have Unique TITLE and DESCRIPTION and of course worthy content. that is all google needs to index a page.

[ 1 ] Thanks
1 reply

{{ DiscussionBoard.errors[11126715].message }}

mattbarrie 3 years ago

This could be a reason why some pages are not being indexed.
- Thanks
{{ DiscussionBoard.errors[11126779].message }}

topcoder

3 years ago

Even if google indexes your page, it doesn't mean you'll get traffic from the search giant. That's where SERP comes into play.

Thanks
1 reply

Signature

Test Your Email - Send an email to mailtest@unlocktheinbox.com - Results in minutes.
ZeroBounce - The ultimate email validation system.

{{ DiscussionBoard.errors[11127063].message }}

mattbarrie 3 years ago

Yes of course but I am just talking about basic indexability at this point
- Thanks
{{ DiscussionBoard.errors[11127133].message }}

yukon

Banned 3 years ago

Google will index a page without any links pointing at the page/URL.

An xml sitemap alone will get a page indexed.

Strictly talking indexing here, not ranking.

Thanks
1 reply

Signature

Hi

{{ DiscussionBoard.errors[11127141].message }}

mattbarrie

3 years ago

Google will not index everything in the sitemap, I am trying to figure out the conditions by which Google will decide to index something from that sitemap or not

Thanks
2 replies

{{ DiscussionBoard.errors[11127143].message }}

yukon

Banned 3 years ago

Originally Posted by mattbarrie

Google will not index everything in the sitemap, I am trying to figure out the conditions by which Google will decide to index something from that sitemap or not

Google will index URLs in an xml sitemap.

Impossible to know why your sitemap is failing without seeing the file.

You also need to include the xml sitemap URL/s in the robots.txt file.

Thanks
2 replies

Signature

Hi

{{ DiscussionBoard.errors[11127837].message }}

Mike Anthony

3 years ago

Originally Posted by yukon

Impossible to know why your sitemap is failing without seeing the file.

I might have got it wrong from this or some other related thread but I got that Matt was talking about a fairly large site.

Thanks
1 reply

Signature

{{ DiscussionBoard.errors[11127840].message }}

yukon

Banned 3 years ago

Originally Posted by Mike Anthony

I might have got it wrong from this or some other related thread but I got that Matt was talking about a fairly large site.

I read the other thread, 40 million pages.

Big project but doable. That's roughly 800 child sitemaps considering Google will process files up to 50MB and 50,000 URLs.

Actually this could be OPs problem If he's got files larger than the Google max. Maybe Google hits the max limit on OPs file/s and bails before processing the entire file/s.

Again, hard to help without seeing OPs xml files.

Thanks
2 replies

Signature

Hi

{{ DiscussionBoard.errors[11127851].message }}

Mike Anthony

3 years ago

Originally Posted by yukon

I read the other thread, 40 million pages.

Big project but doable. That's roughly 800 child sitemaps considering Google will process files up to 50MB and 50,000 URLs.

Actually this could be OPs problem If he's got files larger than the Google max. Maybe Google hits the max limit on OPs file/s and bails before processing the entire file/s.

Possible but I think this gets down tot he area of disagreement between the two of you. I don't see anywhere where Google has promised to keep pages indexed merely because they are submitted in a sitemap. In fact not even using my own anecdotal evidence its been a widely reported phenomenon across the web that google drops lots of pages from larger sites.

With so many pages some may not be considered to be worthy of indexing (like duplicates etc).

I'd definitely agree its impossible to figure out without seeing it.

Thanks
1 reply

Signature

{{ DiscussionBoard.errors[11127872].message }}

yukon

Banned 3 years ago

Originally Posted by Mike Anthony

Possible but I think this gets down tot he area of disagreement between the two of you. I don't see anywhere where Google has promised to keep pages indexed merely because they are submitted in a sitemap. In fact not even using my own anecdotal evidence its been a widely reported phenomenon across the web that google drops lots of pages from larger sites.

With so many pages some may not be considered to be worthy of indexing (like duplicates etc).

I'd definitely agree its impossible to figure out without seeing it.

If it's duplicate pages for OP it's another story which could be getting tripped up in an entirely different algo. (indexed by one algo... deindexed by another algo.).

Sure nothing is guaranteed with Google but there's best practices, well, reverse engineering other sites to get what you want.

As far as finding sitemaps, it needs to be easily accessible...

Build valid sitemap/s
Add sitemap URL to robots.txt
Submit robots.txt to Google

Thanks

Signature

Hi

{{ DiscussionBoard.errors[11127881].message }}

mattbarrie 3 years ago

For those that want to look at the sitemap, they are listed in here: https://www.freelancer.com/robots.txt
- Thanks
{{ DiscussionBoard.errors[11129786].message }}

mattbarrie 3 years ago

Google will not index everything in a sitemap, they say so explicitly and they also show the % in webmaster tools.
- Thanks
{{ DiscussionBoard.errors[11129785].message }}

Mike Anthony

3 years ago

Originally Posted by mattbarrie

Google will not index everything in the sitemap, I am trying to figure out the conditions by which Google will decide to index something from that sitemap or not

I think there is some communicating at cross purposes here. When people usually say "getting a page indexed" they are usually talking about their first page/pages of a site. Yukon is right. I can't think of any time I have started any blog that it has not been indexed and often with no authoritative backlink (or even a sitemap). Personally I have never understood the great angst with indexing....indexed on page 20 for a term someone searches is just as lost in space as not being indexed so I focus on ranking.

However what you are talking about is getting hundreds/thousands of pages indexed. Thats a different dynamic. A good while ago Google admitted it reserves a certain amount of crawl time for each site/entry point. The bot goes through indexes and then depending on how authoritative the site is (by incoming links - and whatever secret sauce google has) it decides to leave at a certain point. Makes perfect sense as crawl time is a resource Google is eating up and someone could create a 100 million long page gibberish site and appreciably slow Google's operations.

So if you have a smallish site you are likely to get all the pages indexed and fairly easily. The bot will come in through some link (really any generic link of low authority or not) do a minimum depth crawl and be on its way getting everything. If your site is huge and more time and resources are going to be used than the minimum then there has to be factors to that site (or entry point) that makes it hang around longer and go deep.

Authoritative link would do it. New content is another factor. Google crawls sites that change often more frequently than those that don't change at all.

So I think both your points are valid - you are just talking about slightly different things.

Thanks

Signature

{{ DiscussionBoard.errors[11127838].message }}

Anand Kumar Kush

3 years ago

create backlinks for your website

Thanks

{{ DiscussionBoard.errors[11127319].message }}

mrjackpowers

3 years ago

I've had plenty of pages indexed which were never on a site map nor connected to other pages. Although, I do recommend both. The quickest way to get your pages indexed is to submit your site to Google AND start getting clicks from offsite links as soon as possible.

Thanks

Signature

Your New Backup Copywriter - Good, Fast Affordable

{{ DiscussionBoard.errors[11127336].message }}

shipwrecked

3 years ago

The sitemap helps Google better understand the structure of the website, but it might not help indexing it... In the past they said that placing AdSense ads on a site helps get indexed faster!

Thanks

{{ DiscussionBoard.errors[11127455].message }}

Rhyan Higgins Orshalev

3 years ago

I personally have had success buy submitting my blog entries and sites to pingomatic and adding quality back links. Personally I found the safest and easiest backlinks to be from multiple niche related social profiles: pintrest, facebook, twitter, tumblr. Also reddit is good too....

Thanks

Signature

My Traffic Jacker Review

{{ DiscussionBoard.errors[11127620].message }}

TrafficQueen

Banned 3 years ago

I would be placing as many backlinks as i could.

Thanks

{{ DiscussionBoard.errors[11127960].message }}

netgeek

3 years ago

Google doesn't index a page automatically.

But, your posted page does get indexed within minutes because your website url is indexed on other website pages like whois database websites.

When Google bots crawl those whois database pages, they come to your website to crawl, gather the data and index.

Increase Indexability?
Any new page posted, you can submit google index using your webmaster tools, and indexing happens within seconds, may take minutes in some cases though.

Let me know if you still need any info.

Thanks

{{ DiscussionBoard.errors[11127986].message }}

Alireza Safiri

Banned 3 years ago

[DELETED]

{{ DiscussionBoard.errors[11128530].message }}

mattbarrie

3 years ago

For those that want to look at the sitemap, they are listed in here: https://www.freelancer.com/robots.txt

Thanks

{{ DiscussionBoard.errors[11129787].message }}

Criteria by which google will index a page

Trending Topics

New Site Graphics Won't Show.

Cell Phone Solar Power Chargers & Adapters & Affiliate Programs

Is shared hosting acceptable for SMTP?

No Astronaut Has Ever Left The Earths Atmosphere, According To Science

Diversifying Your Traffic Sources Beyond SEO