Solved - Does Google Crawl Angularjs website pages? If not is there any solution?

Some suggestions after looking at your site:
- your sitemap.xml doesnt contain dates for when pages get updated
- you load *ALL* your content via ajax. Your page w/o javascript is *completely* empty

Have you looked into doing even *some* server side rendering?
You may also want to check this out Does google execute javascript?

Update: From Google (ca. 2015) Deprecating our AJAX crawling scheme

AH I FIGURED IT OUT.

Your angular app, fetches content from api.crushprice.com to render the items on your store.


Your robots.txt however contains this.


Your robots.txt blocks the resources that google needs to render your page correctly!

I suggest removing api.crushprice.com from your robots.txt, add date updated to your sitemap entries and then resubmitting sitemap to google webmaster tools.

If it works, please keep us updated .

As far I know, if you resubmit a sitemap it will trigger and crawler of your site..
To submit your sitemap:
1. Select your site on your Google Search Console home page.
2. Click Crawl.
3. Click Sitemaps.
4. Click ADD/TEST SITEMAP.
5. Type sitemap.xml.
6.Click Submit Sitemap.

There's 4,720 pages indexed, what's the problem?

• site:crushprice.com

If the on-page text is in javascript (I didn't look) you're pretty much hiding the content on purpose because the Google algo. is based on plain text. Google can crawl some basic javascript but you should never depend on it for showing up in the SERPs.

Matter of fact I've used javascript for years to hide content from Google, stuff like text/links I did't want skewing my optimized plain text/links.

Hey Warriors,

Thank you for the amazing information. I have removed api.crushprice.com and using prerender software to render all the content for spiders and crawlers.
Checking Checking
Thank you

Hi,
you can resubmit the sitemap.

Rajender Smts Any updates? I see that your page now has 11k pages indexed in google

Thought I'd add some observations about Google crawling Ajax-rendered content that I have seen (disclaimer - this site is my first Ajax-heavy site and still has not been indexed despite upwards of 5000 pages a day being crawled, I am just saying what I have seen):

1) Google definitely SOMETIMES executes abritrary JS - I have a custom-made site stats routine that is triggered asyncronously via Ajax code on the page (the Ajax basically fetches a PHP file which doesn't actually return anything, just logs the visit in the dbase). The GoogleBot triggers this code maybe once every 30 visits (comparing the number of logged visits with actual visits from my Apache access_log), but evidently recognises it as something not worth crawling regularly. The asynchronous stuff on my page is deferred until the static content has fully rendered, so the bot may be executing the deferral routine, and not just the Ajax call.

2) Google definitely fetches JSON data that is loaded via Ajax in order to fully render a page, and it does so ALMOST as frequently as it fetches the static page that the Ajax was called from (again, I can see this from my Apache logs). What is weird is that at times (more rarely) it will fetch the Ajax content within seconds of crawling the static page, at other times (more commonly) it will fetch the JSON content completely independently (at a different time). E.g like this:

It seems like it has a pretty robust approach to crawling and processing that data, and given its interest in that content one would imagine that it will all contribute to the ranking of the given page, so it's all getting put back together somewhere behind the scenes.

3) About Google not actually rendering pages, well, it is rare to see in the logs that the bot has fetched CSS, but then it doesn't need to do it often, I am sure it is cached. Many have discussed how Google does appear to fully render pages (including JS and CSS) when indexing them. Maybe full rendering of pages is reserved for "important" sites, but we sure as hell know the crawler can do it, because you can Fetch pages via Google's various tools and see them fully rendered. E.g. run a random JS/html5 demo through the PageSpeed tool and you will see it rendered: https://developers.google.com/speed/...ge.sh%2F&hl=en

It has long been theorised that the Chrome browser in fact IS the technology that is used to render pages on the back end of Google.

So my feeling is that in this day and age it is no longer true that Google is crawling only flat HTML and that the technology is very much able to render even very complicated web apps - it's going to have to be with HTML5/JS running the modern web. Which doesn't explain the problems of OP, I just thought I'd add this as a point of discussion.

I do still agree with Yukon that your static content is a big deal, especially in terms of page speed, since it's no good making your visitors wait 5 seconds before anything is rendered at all, and that is something I addressed on my own site after we had a discussion about that in another thread last month and saw not only big increases in my pagespeed score (maybe Google only scoring the static part?!) but also a huge increase in crawl rate, from a few hundred pages a day to thousands. Now, if only it would actually index them, lol.