how do i get google to index my website? | MissionFound

How Do I Get Google to Index My Website?

Jill Van Nostran Analytics

Once your website is optimized for search engines and launched, your next question should be: “How do I get Google to index my website?” In order for anyone to find your Christian organization’s website via Google search, it first has to be indexed by Google.

Google uses content on a website to understand what the site is about. It does this by using Googlebot, its main crawler (also called a spider) to “crawl” websites to find new pages and content to add to its index. It then processes information gathered by Googlebot to determine if it is high-enough quality to add to its searchable index.

But with billions of pages on the web, Googlebot uses an algorithm to determine which sites and pages to crawl. This is why it’s important that you tell Google about your new or updated site.

But first, there are two very necessary administrative steps you need to take (hang on, I’m about to get a little technical, but I’ve tried to make it easy):

1. Set Up A Search Console Account. Search Console is a free service that helps you manage your site’s presence in Google search results. I always think of it as the method by which your website and Google “talk” to each other. Search Console will tell you if Googlebot is unable to crawl your site and why, if your site’s performance is optimal or if it contains errors, and how Google displays your website to the world. It also tells you what search terms people use to find your website and which websites are linking to your website (thus, driving traffic to your site). In other words, Search Console is a critical tool for making sure your website is search-engine optimized.

Follow these steps to set up Search Console:

      • Sign up for a Google account if you don’t already have one. Your Google email address is how you will “link” your Google properties, like Search Console and Google Analytics.
      • Sign into Search Console with your Google account.
      • Verify you are your website’s owner:  You have four choices for doing this including adding a DNS record; uploading a file to your server; adding a meta tag to your HTML or; the easiest, best and least technical option, in my opinion:  use your Google Analytics account. You need a Google Analytics account anyway.
        • When you sign up for Google Analytics using your Google email, you’ll get a tracking ID. You can find the tracking ID from within Google Analytics listed in Admin (look at the top menu) → Property (“property” is the name of the website you set up when you signed up for Google Analytics) → Tracking Info → Tracking Code.
        • You’ll have to make sure the Google Analytics tracking code is installed on your website. You can manually add the code to your website’s code. Or, if your website is on a content management system, you can install a plugin to just handle this for you (Yoast for WordPress is an example).
      • Back in Search Console, select “Add A Property” which will be your website URL.
      • You need to actually do this twice: Add A Property for the www and Add A Property for the non-www version of your website (for example, in Search Console, I added one property for https://missionfound.com and a separate property for just https://missionfound.com.routing.yourhost.co).
      • In both properties, select the “gear” drop down in the top far right corner of Search Console (next to the “Help” dropdown) and select “Site Settings.” You need to choose a preferred domain here so your data is funneled through just one Search Console property. Typically, the preferred domain will just be the non-www version of your site (most people will just type in yourdomain.com into their browser vs the full www.yourdomain.com). Whichever you choose as your preferred domain, make sure you select the same preferred domain in each property. This will be the property you’ll use in Search Console.

2. Link Search Console Account to Google Analytics. Google Analytics is uber important so that you can effectively measure your Christian organization’s website performance. So please don’t go with out it. But to measure properly, you need to make sure that Search Console is linked to Google Analytics. Take the following steps:

      • Sign into Google Analytics → Admin (look at the top menu) → Property → Product Linking → All Products. Then look to the right under “Google Analytics Product Linking.” You’ll see a list of unlinked products. Scroll all the way to the bottom to find Search Console and select “Link Search Console.”
      • You can adjust Search Console settings if you like. But your main priority is to select “Adjust Search Console” then select “Edit” and associate your Google Analytics account / website with your Search Console account. Hint: make sure your URL in Google Analytics Property Settings is set to the non-www version of your site (or whichever version you specific in Search Console as your preferred domain).
      • Select “Done” then “Save.”

Still with me? Now that these administrative tasks are done, take the following steps to alert Google to your new (or updated) site:

3. Submit A Sitemap to Search Console. A sitemap is a document that lists all your web pages. It tells search engines when pages have been updated. If you are using a plugin like Yoast, it’s easy to generate a sitemap through that plugin. Another method is to use a sitemap generator. Once you download the sitemap, upload it to the domain root folder of your website. Then go to Search Console → Crawl → Sitemaps → Add/Test Sitemap and append it to your URL (http://your domain/sitemap_index.xml). You can first test it to make sure Google finds no errors. But make sure you actually submit it.

4. Submit A /robots.txt File to Search Console. A /robots.txt file gives instructions to search engine spiders (like Googlebot) about which pages it is allowed to crawl and which should be ignored. It is installed in the document root folder of your domain. To see your site’s /robot.txt file, go to yourdomain.com/robots.txt. You can use the file to instruct spiders to only crawl certain areas of your website and to ignore others. To allow all robots complete access to your site, it should say something like this:

      • User-agent:*
      • Disallow:

Once you’ve created (or checked) your /robots.txt file, submit it to Google via Search Console. Just open Search Console and go to Crawl → robots.txt Tester → Submit which will open a dialog box for instructions on submitting an updated file. If you’re not exactly sure how to create or submit this file, it’s best to ask a professional web developer for help here.

5. Submit Your Website to Google (and Bing). This is easy. Just go here to tell Google about your website. Go here to tell Bing and Yahoo about your website.

6. Implement an Online Strategy that Includes Content Marketing. Make sure you have a solid idea of what you want to accomplish with your website. Develop an online strategy for your Christian organization that 1) identifies your key audience(s) are, 2) spells out your digital marketing goals, 3) which channels you want to use to achieve those goals (your website, your social media properties, perhaps other online media properties) and 4) which tactics you plan to employ, like content marketing (blogging and social media marketing are examples). At the least, make sure you are creating online content in some form and linking your website to that content.

7. Wait. After you take these steps to help Google find your website, you’ll have to give it time for Googlebot to crawl your site and then start indexing pages. After I launched this website, it took 10 days for Google to index it. Be patient.

Hopefully you’ve stayed with me through this fairly technical post. It’s important to ask “How do I get Google to index my website?” and just as important to follow through on these steps. If Google or Bing can’t find your site, they can’t index it–and then your key audiences can’t find you either.

Your 5-minute homework: Although I think you should take half a day to implement ALL of these recommendations (more time than that of course to develop an online strategy), if you only have 5 minutes, then do this: Check your /robots.txt file (go to yourdomain/robot.txt). If you see nothing at all or something like “Disallow: /” that means you either don’t have one set up or your file is instructing spiders not to crawl your site. Either way, both are bad. Call a professional web developer or even just your web host and get a proper /robots.txt file in your root domain stat. It shouldn’t take more than a few minutes.

We’re listening. Share your thoughts with us on Facebook or Twitter and use #GetFoundGiveHope.