By 2004, search engines had incorporated a wide range of undisclosed factors in their ranking algorithms to reduce the impact of link manipulation. In June 2007, The New York Times' Saul Hansell stated Google ranks sites using more than 200 different signals.[26] The leading search engines, Google, Bing, and Yahoo, do not disclose the algorithms they use to rank pages. Some SEO practitioners have studied different approaches to search engine optimization, and have shared their personal opinions.[27] Patents related to search engines can provide information to better understand search engines.[28] In 2005, Google began personalizing search results for each user. Depending on their history of previous searches, Google crafted results for logged in users.[29]

Website-specific crawlers, or software that crawls one particular website at a time, are great for analyzing your own website's SEO strengths and weaknesses; they're arguably even more useful for scoping out the competition's. Website crawlers analyze a website's URL, link structure, images, CSS scripting, associated apps, and third-party services to evaluate SEO. Not unlike how a website monitoring tool scans for a webpage's overall "health," website crawlers can identify factors such as broken links and errors, website lag, and content or metadata with low keyword density and SEO value, while mapping a website's architecture. Website crawlers can help your business improve website user experience (UX) while identifying key areas of improvement to help pages rank better. DeepCrawl is, by far, the most granular and detailed website crawler in this roundup, although Ahrefs and Majestic also provide comprehensive domain crawling and website optimization recommendations. Another major crawler we didn't test is Screaming Frog, which we'll soon discuss in the section called "The Enterprise Tier."
Searching Google.com in an incognito window will bring up that all-familiar list of autofill options, many of which can help guide your keyword research. The incognito ensures that any customized search data Google stores when you’re signed in gets left out. Incognito may also be helpful to see where you truly rank on a results page for a certain term.
For example, within the HubSpot Blogging App, users will find as-you-type SEO suggestions. This helpful inclusion serves as a checklist for content creators of all skill levels. HubSpot customers also have access to the Page Performance App, Sources Report, and the Keyword App. The HubSpot Marketing Platform will provide you with the tools you need to research keywords, monitor their performance, track organic search growth, and diagnose pages that may not be fully optimized.
When using the Keyword Explorer, Ahrefs will also produce the "parent topic" of the keyword you looked up, as you can see in the screenshot above, underneath the Keyword Difficulty meter. A keyword's parent topic is a broader keyword with higher search volume than your intended keyword, but likely has the same audience and ranking potential -- giving you more a valuable SEO opportunity when optimizing a particular blog post or webpage.
For the purposes of our testing, we standardized keyword queries across the five tools. To test the primary ad hoc keyword search capability with each tool, we ran queries on an identical set of keywords. From there we tested not only the kinds of data and metrics the tool gave, but how it handled keyword management and organization, and what kind of optimization recommendations and suggestions the tool provided.
This broken-link checker makes it easy for a publisher or editor to make corrections before a page is live. Think about a site like Wikipedia, for example. The Wikipedia page for the term "marketing" contains a whopping 711 links. Not only was Check My Links able to detect this number in a matter of seconds, but it also found (and highlighted) seven broken links.
The emphasis on tools, meaning plural, is important because there's no one magical way to plop your website atop every single search results page, at least not organically, though there are best practices to do so. If you want to buy a paid search ad spot, then Google AdWords will happily take your money. This will certainly put your website at the top of Google's search results but always with an indicator that yours is a paid position. To win the more valuable and customer-trusted organic search spots (meaning those spots that start below all of those marked with an "Ad" icon), you must have a balanced and comprehensive SEO strategy in place.
Difficulty scores are the SEO market's answer to the patchwork state of all the data out there. All five tools we tested stood out because they do offer some version of a difficulty metric, or one holistic 1-100 score of how difficult it would be for your page to rank organically (without paying Google) on a particular keyword. Difficulty scores are inherently subjective, and each tool calculates it uniquely. In general, it incorporates PA, DA, and other factors, including search volume on the keyword, how heavily paid search ads are influencing the results, and how the strong the competition is in each spot on the current search results page.
Write a description that would both inform and interest users if they saw your description meta tag as a snippet in a search result. While there's no minimal or maximal length for the text in a description meta tag, we recommend making sure that it's long enough to be fully shown in Search (note that users may see different sized snippets depending on how and where they search), and contains all the relevant information users would need to determine whether the page will be useful and relevant to them.
That's why PA and DA metrics often vary from tool to tool. Each ad hoc keyword tool we tested came up with slightly different numbers based on what they're pulling from Google and other sources, and how they're doing the calculating. The shortcoming of PA and DA is that, even though they give you a sense of how authoritative a page might be in the eyes of Google, they don't tell you how easy or difficult it will be to position it for a particular keyword. This difficulty is why a third, newer metric is beginning to emerge among the self-service SEO players: difficulty scores.
Difficulty scores are the SEO market's answer to the patchwork state of all the data out there. All five tools we tested stood out because they do offer some version of a difficulty metric, or one holistic 1-100 score of how difficult it would be for your page to rank organically (without paying Google) on a particular keyword. Difficulty scores are inherently subjective, and each tool calculates it uniquely. In general, it incorporates PA, DA, and other factors, including search volume on the keyword, how heavily paid search ads are influencing the results, and how the strong the competition is in each spot on the current search results page.

After the Panda update rolled out, the latent risk in such a strategy could (typically would) vastly exceed the direct cost of the content, as poor pages on one part of a site could drag down the ranking of other pages on the site which targeted different keywords. Some sites have seen their search traffic fall over 90% with their ad revenues falling even faster. Demand Media went from being worth a couple billion to tens of millions of Dollars. ArticlesBase.com sold on Flippa for $80,000, but was making over $500,000 PER MONTH in profit before getting hit by Panda. Many other Panda-torched sites like Suite101 have simply went offline.
For example, within the HubSpot Blogging App, users will find as-you-type SEO suggestions. This helpful inclusion serves as a checklist for content creators of all skill levels. HubSpot customers also have access to the Page Performance App, Sources Report, and the Keyword App. The HubSpot Marketing Platform will provide you with the tools you need to research keywords, monitor their performance, track organic search growth, and diagnose pages that may not be fully optimized.
SEO platforms are leaning into this shift by emphasizing mobile-specific analytics. What desktop and mobile show you for the same search results is now different. Mobile results will often pull key information into mobile-optimized "rich cards," while on desktop you'll see snippets. SEMrush splits its desktop and mobile indexes, actually providing thumbnails of each page of search results depending on the device, and other vendors including Moz are beginning to do the same.

If you own, manage, monetize, or promote online content via Google Search, this guide is meant for you. You might be the owner of a growing and thriving business, the webmaster of a dozen sites, the SEO specialist in a Web agency or a DIY SEO ninja passionate about the mechanics of Search : this guide is meant for you. If you're interested in having a complete overview of the basics of SEO according to our best practices, you are indeed in the right place. This guide won't provide any secrets that'll automatically rank your site first in Google (sorry!), but following the best practices outlined below will hopefully make it easier for search engines to crawl, index and understand your content.

The emphasis on tools, meaning plural, is important because there's no one magical way to plop your website atop every single search results page, at least not organically, though there are best practices to do so. If you want to buy a paid search ad spot, then Google AdWords will happily take your money. This will certainly put your website at the top of Google's search results but always with an indicator that yours is a paid position. To win the more valuable and customer-trusted organic search spots (meaning those spots that start below all of those marked with an "Ad" icon), you must have a balanced and comprehensive SEO strategy in place.
This keyword tool was built on a custom database we have compiled over the past four years. We researched data from the (now defunct) Google Search-Based Keyword Tool and also looked at a few more recent data snapshots to refresh the database and enhance our keyword coverage. Our database contains 28,527,279 keywords representing 13,762,942,253 monthly searches. Our database is primarily composed of English language keywords.
When referring to the homepage, a trailing slash after the hostname is optional since it leads to the same content ("https://example.com/" is the same as "https://example.com"). For the path and filename, a trailing slash would be seen as a different URL (signaling either a file or a directory), for example, "https://example.com/fish" is not the same as "https://example.com/fish/".
Getting the most out of your optimization efforts means understanding the data you’re collecting, from analytics implementation to report setup to analysis techniques. In this session, Krista walks you through several tips for using analytics data to empower your optimization efforts, and then takes it further to show you how to level-up your efforts to take advantage of personalization from mass scale all the way down to individual user actions.

The tools we tested in this round of reviews were judged on which do the best job of giving you the research-driven investigation tools to identify SEO opportunities ripe for growth, along with offering enterprise-grade functionality at a reasonable price. Whether one of these optimization tools is an ideal fit for your business, or you end up combining more than one for a potent SEO tool suite, this roundup will help you decide what makes the most sense for you. There's a wealth of data out there to give your business an edge and boost pages higher and higher in key search results. Make sure you've got the right SEO tools in place to seize the opportunities.
You may not want certain pages of your site crawled because they might not be useful to users if found in a search engine's search results. If you do want to prevent search engines from crawling your pages, Google Search Console has a friendly robots.txt generator to help you create this file. Note that if your site uses subdomains and you wish to have certain pages not crawled on a particular subdomain, you'll have to create a separate robots.txt file for that subdomain. For more information on robots.txt, we suggest this Webmaster Help Center guide on using robots.txt files13.
Getting the most out of your optimization efforts means understanding the data you’re collecting, from analytics implementation to report setup to analysis techniques. In this session, Krista walks you through several tips for using analytics data to empower your optimization efforts, and then takes it further to show you how to level-up your efforts to take advantage of personalization from mass scale all the way down to individual user actions.
We expect advertisements to be visible. However, you should not let the advertisements distract users or prevent them from consuming the site content. For example, advertisements, supplement contents, or interstitial pages (pages displayed before or after the content you are expecting) that make it difficult to use the website. Learn more about this topic.38
On October 17, 2002, SearchKing filed suit in the United States District Court, Western District of Oklahoma, against the search engine Google. SearchKing's claim was that Google's tactics to prevent spamdexing constituted a tortious interference with contractual relations. On May 27, 2003, the court granted Google's motion to dismiss the complaint because SearchKing "failed to state a claim upon which relief may be granted."[68][69]

The Java program is fairly intuitive, with easy-to-navigate tabs. Additionally, you can export any or all of the data into Excel for further analysis. So say you're using Optify, Moz, or RavenSEO to monitor your links or rankings for specific keywords -- you could simply create a .csv file from your spreadsheet, make a few adjustments for the proper formatting, and upload it to those tools.
Companies that employ overly aggressive techniques can get their client websites banned from the search results. In 2005, the Wall Street Journal reported on a company, Traffic Power, which allegedly used high-risk techniques and failed to disclose those risks to its clients.[15] Wired magazine reported that the same company sued blogger and SEO Aaron Wall for writing about the ban.[16] Google's Matt Cutts later confirmed that Google did in fact ban Traffic Power and some of its clients.[17]
×