Originally Published on SEMrush June 10, 2020 (Jason Barnard)
If you have just launched a new website, on a brand new domain, you are not going to see it ranking on Google straight away. Until Google and other search engines know that your website exists, you won’t even see it indexed.
For those of you reading this guide because you have published a new website and can’t figure out why you can’t find it anywhere, let’s quickly explain how search engines work.
Let’s turn to Google’s own ‘how search works‘ guide to help with this…
Even before you search, Google organises information about webpages in our Search index. The index is like a library, except that it contains more info than in all the world’s libraries put together.
If your website isn’t in Google’s index, it won’t be able to be found when a user makes a search. Google needs to know that your site exists to be able to crawl it and include it in its index.
In this guide, you will learn how to submit your site to Google and other search engines, as well as understand other ways that they can discover it.
Do I Need to Manually Add a New URL or My Website to Google?
The good news is that submitting your website isn’t a specific requirement for your website to appear in Google’s index, but, for a new website, it is often the fastest way.
Google just needs to know that your website exists. From there, it can crawl the site and index its pages.
Google’s crawlers find new URLs (and websites) by following links from other pages and websites. As long as your site is linked to from somewhere else on the web, Google will eventually find and index your site, but you can speed this up by manually submitting a new site.
When it comes to a new URL (page or post) that has been published on an existing website, you certainly don’t need to go through the process of manually submitting it, but there are steps that you can take to speed up seeing the page in Google’s index.
You don’t need to manually submit your site or page to Google as long as it’s linked to from somewhere else on the web, but doing so can speed up the process of the search engines finding your content.
How Long Will It Take Google to Index My Website or URL?
There is no set length of time that it takes for Google to index your website or URL. That said, what we can all be confident about is that this is a lot faster than it once was.
A study by HubSpot found that, without submitting a new URL to Google through a sitemap, it took Google an average of 1,375 minutes to crawl the page (that is 23 hours). However, when submitting an updated sitemap to Google Search Console, this dropped to just 14 minutes.
Leaving Google to find new content on its own can result in delays where your page isn’t indexed, but it takes just minutes when you manually inform Google.
On the other hand, the time taken to crawl and index a completely new domain can differ significantly, depending on whether or not any external links exist and how frequently these themselves are crawled.
At the very least, you need to make sure you submit a new site to Google, and doing so for a new page can speed up the indexation.
How to Submit a URL to Google
The general consensus is that yes, you should be submitting your URL or new website to Google – if only to speed up seeing it in the index.
https://www.youtube.com/embed/pSLmtnrVV4Y?start=86&feature=oembed&enablejsapi=1&origin=https%3A%2F%2Fwww.semrush.com
There are different ways that you can do this, depending on the circumstances, and we will walk you through these options below.
How to Check if a URL is Indexed
Before you go ahead and submit your URL to Google, it makes sense to run a quick check to establish whether or not it is already indexed.
You can do this by using Google Search Console’s URL Inspection Tool.
(If you have not set up Google Search Console for your site, you can learn how to do so here.)
Use the ‘Inspect a URL’ search box at the top of the dashboard and enter the URL whose index status you want to check.
Once the data has been retrieved from the index, you will either see a confirmation that the page is on Google:
Or that the page is not on Google:
You will also be able to clearly see any coverage issues with that specific URL below this.
There is also a quick and easy way to check, without using Search Console – use the “site:” modifier in Google search like this site:example.com/url-of-the-page. This will show the page and any children of that page. I discuss this in more detail in the How to Check if a Website is Indexed section of this article.
Your Options For Submitting a URL to Google
When it comes to submitting a URL to Google, you have a number of options. But first, it is important to understand that not all of these involve actually ‘submitting’ your site to the search engine.
Instead, you need to think of these methods as informing Google that your page (or site) exists. Let’s look at these options for getting your URL indexed (we will look at submitting a new website separately).
Request Indexing With the Inspect a URL Tool in Search Console
There is a good chance that you have just used the Inspect a URL tool to check whether or not your URL is in Google’s index. Perhaps the quickest way to get your URL in Google’s index is to do so through this tool.
Regardless of whether the URL is, or isn’t, in Google’s index, you will see a ‘REQUEST INDEXING’ link at the bottom of the box.
Go ahead and click this and your page will be added to a queue for indexing.
If there are any issues, you will be notified of them.
You will be able to check the index status of the page using the same tool, as we showed before.
You used to be able to use Google’s ‘fetch as Google‘ tool, but this was discontinued in the new version of Search Console and no longer exists.
Submit an Updated Sitemap to Google Search Console
When you submit an updated sitemap to Search Console and include new URLs, you are informing Google that there has been a change and that these pages should be crawled.
For reference, we are talking about an XML sitemap here, not an HTML sitemap.
If you are submitting for an existing site and want to see new URLs indexed as quickly as possible, you have likely already submitted a sitemap previously.
But once you have added a sitemap, you might be surprised to find that you can’t actually go in and ‘resubmit’ in the new Search Console.
As Google Search Console help says:
Google doesn’t check a sitemap every time a site is crawled; a sitemap is checked only the first time that we notice it, and thereafter only when you ping us to let us know that it’s changed. You should alert Google about a sitemap only when it’s new or updated; do not submit or ping unchanged sitemaps multiple times.
— Google Search Console Help
The good news is that if you are using a platform such as WordPress combined with an SEO plugin, your sitemap will automatically update and ping Google when you publish a new page or post.
If you are not using WordPress, or another CMS where the sitemap automatically pings Google when updated, you can use the ‘ping’ functionality to ask for this to happen.
Send an HTTP GET request like this:
As a side note, your XML sitemap should be referenced in your site’s robots.txt file.
Use Internal Links
As we mentioned before, Google doesn’t actually need you to submit a URL for it to be indexed, it just needs to be informed that it exists. It is a good time to point out here that one of the main ways Google finds new pages is through links.
If you add an internal link into another page on your site that Google already has in its index, this will help the new URL to be discovered.
Of course, you should only add links where it makes sense to do so in topically relevant pages, but you can learn more about internal linking best practices in this guide.
Link from an External Source
Just as Google finds new content by crawling internal links, the same goes for inbound links from external sources.
Of course, earning links from a third-party website isn’t as simple or as quick as adding internal links, updating your sitemap or inspecting the URL with Search Console, but given that links are a top ranking factor, it is recommended that you consider the various link building strategies that you could use to get other people to link to your new page.
How to Submit a Website to Google
If you are launching a brand new website for the first time, you will undoubtedly be wondering how you can get it included in Google’s index as quickly as possible. Let’s look at your options.
How to Check if a Website is Indexed
You can quickly check to see whether or not a website is indexed by Google straight on the search engine using the “site:” search operator.
Run a search for: site:[Your Domain]
e.g., site:yourdomain.com
If your website is indexed, you will see results returned when using this search operator.
As an example:
Notice both the number of returned results as well as the indexed URLs displayed? In the event that there are no URLs indexed for the domain, you will see:
When You Need To Submit a Website
Most commonly, you only need to submit your website to Google when you launch a site for the first time (because Google is not aware that it exists) or when you move your website to a new domain.
If you are working on an existing site, you shouldn’t need to submit the whole site if it is already been indexed.
There are, however, instances when you may need to do so as the result of an error; let’s say a developer accidentally added a rel=”noindex” tag across the site and you saw the site drop from the index.
How to Submit a Website to Google
The quickest and most effective way to submit a website to Google is to add an XML sitemap to Google Search Console.
You can do so by heading to the Sitemaps tab of Search Console.
You will now see the ‘Add a new sitemap’ box. Go ahead and enter the extension of your site’s XML sitemap.
Once you have done this, you will see a list of submitted sitemaps and the number of discovered URLs:
How To Submit a Website or URL to Other Search Engines
It is important to remember that Google isn’t the only search engine, and you want to submit to other popular search engines people use, like Bing, Yahoo, Yandex, Baido, and DuckDuckGo.
Next, we will look at how you can submit your site or website to each of them.
How to Submit a URL or Website to Bing
To submit your site or URL to Bing, you need to head over to Bing Webmaster Tools.
Once there, you first need to add your site, if you haven’t done so already. The good news is that you can skip verification and import straight from Google Search Console.
To submit a whole website, you can add your site’s XML sitemap, just as you did with Google.
Head to the sitemaps tab:
You will then see a ‘Submit Sitemap’ button in the top right of your screen that opens a popup. From here, you can enter the URL of your sitemap:
Unlike Google, you can click into a submitted sitemap and you will see a re-submit button in the top right of the page.
If you just want to submit a URL, you can do so using Bing’s URL submission tool that you will find as part of the left-hand menu.
Simply enter the full URL and hit submit.
How to Submit a URL or Website to Yahoo
If you have already followed the steps to submit your website to Bing, you have done everything that you need to do in order to submit it to Yahoo. Since 2010, Bing has powered Yahoo’s search results and the method of submitting to the search engine is simply to submit to Bing.
There is nothing else to do.
How to Submit a URL or Website to Yandex
Yandex is the leading search engine in Russia, with a market share of over 60%.
Not everyone will need to submit their site to Yandex, but if you are serving customers in Russia, it makes sense to do.
Here are the steps to submitting your URL or website to Yandex:
- Head to Yandex Webmaster Tools.
- If you have not added and verified your site, you need to follow the steps to do that first.
- To submit your site, you will see a ‘Sitemap files’ link on the left-hand menu. Click this and you will be taken to a page where you can add your sitemap in the same way as the other search engines.
If you want to get a new URL indexed, head over to the ‘Reindex pages’ tab on the left-hand menu.
Here, you can submit up to 20 URLs per day that are the highest priority for indexing:
How to Submit a URL or Website to DuckDuckGo
As of May 2020, DuckDuckGo has an estimated market share of 1.35% in the US.
This is still way behind Bing’s 6.5%, Yahoo’s 3.6% and Google’s 88%, but an increasing number of people are using the privacy-first search engine.
The good news is that you don’t need to submit your URL or website to DuckDuckGo. The search engine uses more than 400 sources for its results, including Bing’s search results, meaning that if you have submitted there, you don’t need to do anything else.
How to Avoid Getting De-Indexed
We have already covered how to get your website indexed by Google and other search engines, but it is important to quickly raise the reasons why your website could be de-indexed (you were in the search engine and now you are not).
But first, you need to understand that a website being de-indexed is rare. You certainly aren’t going to see this happening frequently.
When this does happen, the cause is usually one of the reasons below.
Your Site Has Been Blocked With ‘NoIndex’
Perhaps the most common reason why sites are de-indexed is that a developer accidentally left a ‘noindex’ directive in place when updating the code.
This will usually either be in the form of a noindex meta tag or by returning a ‘noindex’ header in the HTTP request. You can learn more about this here.
If your site has been de-indexed because of accidentally added noindex directives, you need to remove these and resubmit your site.
You Have Been Served A Manual Action
Also known as a manual penalty, occasionally you may find that your site has been de-indexed because a member of the webspam team has taken the decision to remove your site from the search results as a result of it violating Google’s webmaster guidelines.
You can see whether or not your site has been affected by a manual action in Google Search Console, and learn more about the different types here.
To reiterate; it is rare that this happens and unless a site has been specifically violating the webmaster guidelines. This is not something to worry about if you are following the rules.
Glossary of Terms
- Bing Webmaster Tools: A free service supplied by Bing that allows webmasters to add their website to the search engine and see the performance.
- De-Indexed: When a site that has previously been in a search engine’s index is removed; either accidentally or following action being taken to do so.
- Google Search Console: Previously known as Google Webmaster Tools, this free tool allows webmasters to see and manage their site’s performance on the search engine.
- Google’s Index: ImagineGoogle’s index to be similar to an index in a library, but instead of books, this lists the web pages available to find on Google.
- Inbound Links: Links from other websites to your own.
- Internal Linking: Links between two pages on your own website.
- Manual Action: An action (or penalty) that someone at Google places on your website to negatively affect the performance when it is deemed that the site is in violation of the Webmaster Guidelines.
- Noindex: A directive added to request that a site or web page is de-indexed.
- Re-Index Pages: A tool available in Yandex to request prioritized indexing of a page.
- Site: Search Operator: A simple way to return a list of URLs available on Google for a particular domain.
- URL Inspection Tool: A tool available in Google Search Console that you can use to inspect the index status of a URL.
- URL Submission Tool: Bing’s tool that allows you to submit new URLs to the index.
- XML Sitemap: An XML sitemap that is submitted to a search engine informs the key pages that should be indexed.
- Yandex Webmaster Tools: An alternative to Google Search Console and Bing Webmaster Tools, but for Russian search engine Yandex.
Submitting your site and pages to Google and other search engines doesn’t need to be complex or take long, and so long as you follow a few simple steps you’ll see your URLs indexed in no time at all.
