Back
SharePoint

Hybrid Search in SharePoint 2013

One of the key guiding principles for us in designing this release of SharePoint was to provide customers with a consistent cloud experience. We know the cloud can provide huge benefits, but at different rates for individual customers and in different ways. And, we build so our customers won’t sacrifice the rich capabilities they’ve come to expect from Office to take advantage of the cloud.

Our customers use SharePoint in wildly different ways, and have built up complex ecosystems around our tools to meet the needs of their businesses. Whether it’s line-of-business applications, ERP system integration, or other systems and solutions, we know that the choice of a productivity suite is a lot more complex than deciding which tool you’re going to use. A “rip and replace” strategy simply isn’t going to work. So, we placed a huge emphasis on ensuring that customers can move to the cloud on their own terms. There are three key principles we used here:

  1. Some customers will not fully move into the cloud —we believe that a mix of some cloud and some on-premises solutions will be the norm for the foreseeable future. Cloud and on-premises solutions must co-exist.
  2. The cloud is not a closed system—much of the value customers get from SharePoint comes from the way we interact with other systems that customer businesses depend on—CRM, ERP, Big Data, and so on. No solution that is cut off from those systems can be very successful.
  3. Customers have to see rapid value from the cloud, whether existing systems move to the cloud or not. We must enable experimentation and exploration of new capabilities for customers within the cloud, even if the bulk of their existing infrastructure remains on-premises.

All Office 365 services can handle a mix of cloud and on-premises instances. For example, a customer can move all or some mailboxes to the cloud, and leave all their SharePoint infrastructure on-premises. This degree of flexibility is a key design point for us.

Onboarding to the cloud was also a key focus area. In Exchange, for example, we provide a rich toolset for migrating mailboxes to and from the cloud, and for handling a mix of mailboxes in either environment.

For SharePoint, the challenge of handling mixed environments is tricky. In Exchange you have a single container, the mailbox, owned by a single user. Routing to the correct locale for the mailbox is enough to make it all work. SharePoint sites and services are about multiple people collaborating at a single site, integrating data from multiple sources, and searching across an entire collection of documents. The boundary between cloud and on-premises is by nature not at all clear. So the challenge for us was deciding how to approach these considerations—go breadth first and enable hybrid support everywhere, or focus in on the scenarios where we saw the highest value? In the end, we chose the latter. In particular, we focused on building an oAuth layer to enable service-to-service authentication and communication, and enabling rich hybrid support for search.

When you have SharePoint investments on-premises and in the cloud, one of the first needs of your users is to find content, regardless of where it lives. Users want to find what they are looking for quickly and easily, without having to learn new concepts or perform additional steps. We solved this problem by adding hybrid functionality for search. More specifically, we enable users to execute one search and see the most relevant results from SharePoint Online and SharePoint 2013 on-premises locations.

Along the way, we had to tackle a few challenging design problems:

  • How to retrieve fresh results from a remote system in a scalable way
  • How to integrate multiple sets of results in a single search experience
  • How to deliver great query performance

For the first problem, we evaluated both crawling/indexing the remote system and run-time query federation. We settled on the latter approach for a few reasons, including the effectiveness of fresh results, and more confidence in our ability to deliver a fast, predictable experience. (Imagine tens of thousands of customers’ search crawlers all pinging SharePoint Online simultaneously, requesting changes.)

We then had to decide on the most efficient way to present results to the user. By using the new Query Rules capabilities introduced in SharePoint 2013, we present a result block containing the top n search results from the remote system inline in the search results page. Users see the top results in that location, and can click on the result block to see more results. The search system also learns over time—if results in that block are useful and users click on them, the entire block “moves up” the search results page over time. If users don’t click on results in that block for a given query, the block is pushed further down.

Finally, we focused on how to streamline execution of these remote queries to optimize performance. With the move to client-side rendering of search results (after the initial page load) in SharePoint 2013, we started with a solid foundation to build on. We modeled these remote queries as closely as possible to regular queries against a local search index, with a minimum of additional logic to enable secure traversal of the corporate firewall (when presenting on-premises results in SharePoint Online).

Example of hybrid search results
By modeling hybrid search queries as closely as possible to regular search queries, we were able to minimize the complexity of the underlying code and ensure that the entire search experience—from graphical refiners to rich Office document previews in the hover panel—worked smoothly in hybrid scenarios.

Let’s close by tackling a few frequently asked questions. Thanks for your time, and let us know how you plan to use hybrid by leaving comments on the blog!

What does hybrid search enable me to do?

  • From a SharePoint Online site, get search results for content in the cloud and on-premises
  • From an on-premises SharePoint 2013 site, get search results for content on-premises and in the cloud
  • Conditionally enable “hybrid” results for specific types of queries

I want to try out hybrid search. What are the prerequisites?

  • Any SharePoint Online tenancy that’s been upgraded to SharePoint 2013 (upgrades will happen on a rolling schedule through Spring 2013)
  • SharePoint Server 2013 deployed on-premises
  • User profiles from your on-premises Active Directory synchronized to Office 365, using the Microsoft Online Services Directory Synchronization tool
  • • Reverse Proxy or F5 device to securely expose your SharePoint 2013 on-premises farm to SharePoint Online (required only to enable users in SharePoint Online to find content on-premises)

Where can I find instructions?

Our writers are working hard to capture the full set of steps necessary to configure trust between on-premises instances and SharePoint Online, and the steps to configure search-specific concepts (result source, query rule, result block, Secure Store credentials) that are necessary to enable the hybrid search scenarios. We hope to have guidance published on TechNet in time for the SharePoint Conference (November 12–15, 2012).