In a world where content has long been king, a search crawler’s ability to understand relevance and rank pages based on their perceived ability to help a searcher can easily be considered queen.  While the search crawler’s capabilities and algorithms have evolved greatly over the past five years, the analysis tools available have evolved as well.  Machine learning analysis can help bridge the gap between great content writing, and the perceived analysis by search crawlers.

One of our favorite tools at Mad Fish for on-page content analysis is the IBM Watson Platform.  More specifically the Natural Language Understanding tool set (formerly known as Alchemy API).

By using natural language, our strategists can analyze top ranking pages and determine content (or “relevance”) gaps between a ranking page, and any of our client’s pages.  This specific toolset is based on the concept of the “Deep Learning” algorithm, which was a paper from 2006 by Hinton, Osindero, and Teh.

More than just a simple keyword analyzer, the toolset can read a block of HTML or text, and structure it into Entities, Categories, Concepts, Emotion, Sentiment, Relations and more.  The big caveat is that while the toolset is available through demos, and an expensive platform from IBM, it is also available for pennies through an API.  At Mad Fish, we baked in the API to our analysis and management platform, Mad Fish Elements.

There are several data points that Mad Fish utilizes from IBM Watson’s Natural Language discovery, however, these five are my favorites:


Entities are people, places, events, and other types of things that are mentioned in content.  Entity analysis looks at all words on a given page, places them into know types and sub-types, then gives you a score on each entities relevance (a decimal value between 0 and 1), emotion (Joy, Anger, Disgust, Sadness and Fear), and Sentiment (Positive or Negative) for how it was used in your web page’s content.  You can view the complete list of entities here.


We look at concepts as a representation for how a search crawler, or machine will understand the combination of words that are used on a page.  For example, if you are optimizing a page for local SEO, will crawlers see your page as representing concepts around the city, state and niche that you are targeting.


We use category analysis to classify content using a five-level hierarchy.  You can view the complete list of standardized categories here.


Analyzing a page for emotions is just what it sounds like.  These are the emotions conveyed by specific target phrases on your web page.  For example, let’s say the content we analyze contained the two phrases: “I miss eating donuts, but really hate kale salads” and “I love camping with my Airstream trailer.”  Our analysis would tell us that donuts = “sadness”, kale salads = “anger”, and airstream trailers = “joy”.  Understanding the keywords in this manner helps to systematically analyze our copy quickly and efficiently. Emotional analysis can also be helpful for brands wanting to measure the conversations taking place about their products and services on social media sites such as Twitter and Facebook.  At Mad Fish, we use the emotional analysis to determine the level of Sadness, Joy, Fear, Disgust, and Anger, that our content will be perceived as. The analysis helps us to make sure that the messaging in our client’s content will align with the messaging and tone of their brand.


Our keyword analysis finds all of the keywords that a document contains, and scores them based on overall relevance to the page’s content. We then benchmark a client’s page against a competing web page’s keyword analysis, to see where there is cross over, and where we need to expand our client’s content.

When Natural Language Understanding Can Help

While knowing these data points is great, understanding when to use them is pivotal.

Here are a few areas where they come in handy.

  • Comparing a draft of updated page copy for a client to that of a competing page from the SERPs
  • Analysis of Pay-Per-Click landing pages to understand a page’s relevance to the keywords that we will be bidding on in Adwords
  • Analysis of a page’s relevance to local city/keyword terms
  • Analysis of a web page in a test environment to a competing page in the SERPs
  • Analysis of twitter conversations specific to a brand, product, or service

Got thoughts on using machine learning for SEO? Hit us up on twitter @madfishdigital, and let us know how you use AI and machine learning in your processes.