Internal Site Search Analysis: Simple, Effective, Life Altering!

Understanding of your site visitors’ intent is one of the most delightful parts of web data analysis. In this article, we’ll learn five ways to analyze your internal site-search data—data that’s easy to get, to understand, and to act on.

But let’s take a step back. Why should you care about this in the first place? Good question.

In the good old days, people dutifully used site navigation at the left, right, or top of a website. But, two websites have fundamentally altered how we navigate the web: Amazon, because the site is so big, sells so many things, and is so complicated that many of us go directly to the site search box on arrival. And Google, which has trained us to show up, type what we want, and hit the search button.

Now when people show up at a website, many of them ignore our lovingly crafted navigational elements and jump to the site search box. The increased use of site search as a core navigation method makes it very important to understand the data that site search generates.

There’s also one more (really important) reason, just in case you need a bit more convincing.

All the search and clickstream data you have (from Google Analytics, Omniture, WebTrends, etc.) is missing one key ingredient: Customer intent. You have all the clicks, the pages people viewed, and where they bailed, but not why people came to the site, except where your referral logs contain information from search engines. For example, you can look at the “top ten pages viewed” report in your web analytics tool and know what people saw, but how do you know what they wanted to see?

Your internal site-search data contains that missing ingredient: intent. Internal search queries contain, in your customers’ own words, what they want and why they’re there.

Once you understand intent, you can easily figure out whether your website has the content your users need, and, if it does, where they can actually find it.

How to read minds#section2

Internal site-search data is easy to access and analyze, no matter which web analytics tool you use. Most modern web analytic tools use JavaScript-based data collection mechanisms to capture and report internal site-search data. Usually, you’ll be able to simply log into your tool’s admin area and identify which URL parameter contains the search string.

For example if you do a search on my blog, Occam’s Razor, for the phrase “engagement metrics,” the search results page you get has this URL: http://www.kaushik
.net/avinash/index.php?s=engagement+metrics&submit=Search.

A quick look at the URL tells you that the s parameter shows the user query string. (Notice the s=engagement+metrics.) Your site search may be set up differently, so just search your website and look at how the URL displays the search parameter. Once you have it, type the URL into your web analytics tool’s admin settings for internal site search.

Now you’re all set to do some pretty sweet quantitative analysis of your data. Let’s go.

Step 1: understand site-search usage#section3

Basics first. Most people hugely underestimate how many visitors use internal search, so the first step is simple: just get that number.

In the following examples, I will show reports from Google Analytics, but equivalent reports are found in all modern web analytics tools.

The Site Search Overview shows you, by day, what percentage of visitors used site search on your website.

Fig. 1. For this website, the site-search overview shows that 18.75% of all visitors used the internal site-search feature.

The site-search usage number for your website might be higher or lower, depending on the type of website it is. Anything more than say 5% makes internal site search one of the most used navigational aids on your website. In our example, we had an average of 18.57% for this time period.

The next thing to do is learn what all those people are searching for.

Fig. 2. A list of the top ten search terms site visitors use the most.

There are so few places in your website’s data where you can read visitors’ minds, but with this simple report, you are doing exactly that. This is where the important insights and questions begin to arise:

• I didn’t realize people really care about new campaigns that much!
• We explain cancellations right on the home page, so why is “cancel” the eighth most searched keyword?
• We don’t have any content on our site about quality score, and people are really looking for it.

You can also dive deeper into interesting results and get more details.

Fig. 3. Detailed metrics for the internal site search query “bounce rate” by visitors to your site.

The reports show detailed metrics such as Percent Search Exits, Results Pageviews/Search, Search Depth (more on these metrics later). Each of these tells you more about site performance in response to visitor queries.

By spending just five minutes with your summary site-level data, you have learned how important site search is to your website, what your visitors are looking for (as indicated by their queries), and what the trends are for specific queries (going up, going down?).

Now let’s explore the more detailed insights available to you.

Step 2: Where visitors search and what they do next#section4

We all love our beloved home pages, but the sad truth is that site visitors increasingly use search engines (and e-mail campaigns and Twitter links and so on) to go directly to internal pages, bypassing the home page.

That evolution in customer behavior raises this question: what first impression does your website produce?

You can answer that question by looking at the “Pages Searched On” report. As the name indicates, this report lists the pages on which visitors used internal site search.

Fig. 4. The ten pages on your website on which the internal site search feature is used the most.

This information can tell you how well page content works. Ideally, if the page is working, people shouldn’t need to use your internal search.

Take the figure above as an example. The answer.py page is intended to answer visitor questions. The fact that that page is second in the list of pages people start searches on is disappointing. But at least now, we know.

The next step is to go to the pages from which searches are initiated, see what people are searching for, and figure out how to improve the content accordingly.

This next report, the “Search Navigation” report, is my favorite. For a particular keyword, it shows you the web pages on which people did the internal search AND where they ended up on your website. Put another way, it shows you the top pages viewed after people type a search query.

For example, you would think no matter where I search for the keyword “new campaign,” I would end up on the same page from search results, but no!

Fig. 5. Search Navigation report for the internal site search query “new campaign,” where the visitor performed the search and what page they ended up on after seeing search results.

This report can give you an in-depth understanding of visitor behavior. The box on the left shows where they start looking, in the middle is the search query they typed in, and the right box shows where they ended up.

In the screenshot above, people searched from the page search.py, saw the results page, and only managed to land on the correct page (first row on the right) 24.05% of the time. So what happened to the other 75.95%? Time to go back to the start page, try to understand customer intent, and then improve search performance. As an example, the link to the correct page might be third on the search results page; if so, you want to make sure it’s number one.

It’s obvious how the report above can measure the quality of your search results. Let’s look at some other reports that do so even more directly.

Step 3: Measure site search quality#section5

Here’s a quick introduction to one of my absolute favorite web analytics metrics: the bounce rate.

Some people come to your site, take one look at it, and leave right away. This phenomenon (“I came, I puked, I left”) is measured by your web analytics tool using a metric called bounce rate, which analyzes the number of visits to your site that contain only one page view.

Bounce rate helps you measure the quality of traffic that each source—Google, e-mail campaigns, Facebook, etc.,—sends to your website. It’s also useful in understanding the quality of your landing pages: If they stink, people leave right away. If your bounce rate is high, it means something’s amiss.

In Google Analytics, “% Search Exits” is a valuable metric you can use to measure the quality of your site-search results. It measures how many visitors leave the site as soon as they see the results of your internal site search engine of all those who do a search on your website.

Fig. 6. This report shows the most popular searches by your website visitors and the % Search Exit Rate for each keyword.

Look for keywords with a high % Search Exit rate such as numbers two, four, and seven above. This is how to interpret it: 343 visitors searched your website using the keyword “ad” and of those visitors, 71.43% left the site as soon they saw the search results. Clearly, this is a problem. Now you can tune your internal search engine for these keywords.

Another way to measure search quality is to measure the number of search result pages the visitor views. We are all trained by major search engines to expect the most relevant result on top of the first page, so we can hypothesize that if we provide the most relevant results at the top of the first results page, visitors will click on them and be on their merry way.

Fig. 7. The top searched keywords on your website and the Results Pageviews seen by your visitors.

The report shows you the search queries where visitors had to dig deeper into site search results to find what they were looking for.

Consider the search query “gears,” which has an average of 1.92 results pageviews. Two pages of search results, with an average of 30 to 40 links to look through to find what you’re looking for, is too much. It’s time to look at what the results are on page one and fix them.

Step 4: Segment, segment, segment#section6

In analytics, “segmentation” is the process of breaking your data into smaller pieces (segments) to understand the behavior of a smaller group of people, or different sources, or different outcomes. For example, you might look at all visitors, or only at visitors from search engines, visitors from e-mail campaigns, and so on. 

If you want truly useful insights from your search data, you’ll need to segment like crazy.

Fig. 8. Default segmentation options in Google Analytics.

In the view above I am about to segment something cool: what was the keyword that brought visitors to my website (from Google, Yahoo, Ask, etc.), and what did people who used that external search keyword search for on my site?

Segmentation helps me better understand customer intent and search behavior, because now I know what these visitors did before they came to my site and what they looked for once they arrived.

The next report is a more straightforward way to compare site-search usage, exits, refinements, and—this is new—time spent on the site after doing an internal site search.

Fig. 9. Key Metrics reported by source of your site-search visitors.

In most internal search reports, you can compute the increase or “lift” that site search provides. If you run a content site, does your internal search deliver relevant results and encourage visitors to spend time on your site?

In the screenshot above, visitors who came from Google spent an average of 39 minutes after site search vs. just 25 minutes for visitors from Yahoo. For content sites (such as cnn.com or huffingtonpost.com) this could be a very important piece of information, since those sites need visitors to spend as much time on the site as possible so they can see more ads.

Step 5: Measure outcomes#section7

This is what your bosses care about.

Look for a report called Usage. In this report, click on the Goal Conversion tab to get the Outcomes (conversion) data. (Or an equivalent path in whatever tool you’re using.)

“Conversion” is a term commonly associated with e-commerce websites, but in web analytics, it denotes any type of measurable positive user interaction with the site. For example, the report below shows “conversions” for a blog, which means leads submitted, RSS subscriptions, and other measurable outcomes.

Fig. 10. Goal Conversion report for visitors who used site search vs. those who didn’t.

This website does internal search so well that for people who search on the site, the conversion rate is 11%, compared to 5% for those who don’t search on the site!

What better way to show a return on the time and money you’ve invested implementing and improving an internal site search engine?

If you are running an e-commerce website, you can also measure increased revenue (or lack thereof) for visitors who use site search, or you can measure the average order size. For example, you might find that your internal search engine is so good that it helps people find the products and services they’re looking for quickly, which increases conversions and results in higher revenue and order size.

I can’t stress how important it is to measure conversion. Often, we’re so enamored with other easily available metrics that we forget to show how site search influences the bottom line. But, if you want senior management’s attention or your clients’ money, measure results.

That’s it—five simple ways to comprehensively understand your site’s search engine performance. You get a better understanding of customer intent, your customers get to find what they want faster, and your company or client leadership gets improved outcomes. Everybody wins.

Good luck!