Module 7: Pagination & Result Shaping¶

Overview¶

Pagination and result shaping are crucial for creating user-friendly search experiences. This module covers how to implement efficient pagination strategies, control result composition, and optimize the presentation of search results in Azure AI Search applications.

Learning Objectives¶

By the end of this module, you will be able to:

Implement various pagination techniques (skip/top, range-based)
Control which fields are returned in search results
Add result counting and metadata
Implement hit highlighting for better user experience
Handle large result sets efficiently
Optimize pagination performance
Troubleshoot common pagination issues

Key Concepts¶

Pagination¶

Pagination divides large result sets into manageable pages: - Skip/Top Pattern: Traditional offset-based pagination - Range-Based Pagination: Using filters for large datasets - Cursor-Based Pagination: For real-time data scenarios - Search After Pattern: For deep pagination scenarios

Result Shaping¶

Control what information is returned: - Field Selection: Choose specific fields to return - Result Counting: Get total result counts - Hit Highlighting: Emphasize search terms in results - Search Scores: Include relevance scores - Metadata: Add execution and diagnostic information

Performance Considerations¶

Deep Pagination: Challenges with large offsets
Result Set Size: Impact on performance and costs
Field Selection: Reducing payload size
Caching Strategies: Improving response times

Prerequisites¶

Before starting this module, ensure you have: - Completed Module 1 (Introduction & Setup) - Completed Module 2 (Basic Search) - Completed Module 4 (Simple Queries) - Understanding of search result structure - Basic knowledge of REST API parameters

Module Structure¶

This module is organized into the following sections:

Prerequisites - Required setup and knowledge
Best Practices - Guidelines for effective pagination and result shaping
Practice & Implementation - Hands-on exercises and examples
Troubleshooting - Common issues and solutions
Code Samples - Practical examples in multiple programming languages

What You'll Build¶

Throughout this module, you'll create: - Basic pagination implementations with skip/top patterns - Advanced range-based pagination strategies for large datasets - Field selection and result shaping configurations - Hit highlighting implementations with custom tags - Result counting strategies with performance optimization - Performance-optimized pagination solutions for production use

Pagination Techniques¶

Skip/Top Pagination¶

GET /indexes/hotels/docs?search=*&$skip=20&$top=10

- Simple to implement - Works well for small to medium result sets - Performance degrades with large skip values

Range-Based Pagination¶

GET /indexes/hotels/docs?search=*&$filter=id gt '50'&$orderby=id&$top=10

- Better performance for large datasets - Requires sortable unique field - Handles concurrent data changes better

Search After Pattern¶

GET /indexes/hotels/docs?search=*&$orderby=score desc,id&searchAfter=0.85,hotel123&$top=10

- Optimal for deep pagination - Consistent performance regardless of page depth - Requires compound sorting

Result Shaping Parameters¶

Field Selection¶

GET /indexes/hotels/docs?search=*&$select=hotelName,rating,location

- Reduces response payload - Improves performance - Controls data exposure

Result Counting¶

GET /indexes/hotels/docs?search=*&$count=true

- Returns total result count - Useful for pagination UI - May impact performance for large result sets

Hit Highlighting¶

GET /indexes/hotels/docs?search=luxury&highlight=description,amenities

- Emphasizes search terms in results - Improves user experience - Configurable highlighting tags

Performance Optimization¶

Efficient Pagination¶

Use appropriate page sizes (10-50 items typically)
Avoid deep pagination with skip/top
Consider range-based pagination for large datasets
Implement caching for frequently accessed pages

Result Set Optimization¶

Select only necessary fields
Use appropriate data types
Consider result caching strategies
Monitor query performance metrics

Memory Management¶

Limit maximum page sizes
Implement timeout handling
Use streaming for large responses
Monitor resource utilization

Common Patterns¶

Infinite Scroll¶

// Load more results as user scrolls
function loadMoreResults(lastId) {
    return searchClient.search('*', {
        filter: `id gt '${lastId}'`,
        orderBy: ['id'],
        top: 20,
        select: ['id', 'title', 'summary']
    });
}

Faceted Navigation with Pagination¶

GET /indexes/products/docs?search=*&facet=category&facet=brand&$skip=0&$top=20

Search Result Preview¶

GET /indexes/documents/docs?search=query&$select=title,summary&highlight=content&$top=10

Best Practices¶

User Experience¶

Provide clear pagination controls
Show total result counts when appropriate
Implement loading states
Handle empty result sets gracefully

Performance¶

Use appropriate page sizes
Implement result caching
Monitor pagination performance
Optimize for common use cases

Accessibility¶

Ensure keyboard navigation
Provide screen reader support
Use semantic HTML elements
Include skip links for large result sets

Next Steps¶

After completing this module, you'll be ready to: - Build sophisticated search interfaces with pagination - Implement efficient result presentation - Move on to Module 8: Search Explorer & Portal Tools - Explore advanced search features in intermediate modules

Code Samples Available¶

This module includes comprehensive code samples in multiple programming languages:

🐍 Python Examples (8 samples)¶

01_basic_pagination.py - Skip/top pagination with performance monitoring
02_field_selection.py - Field selection optimization with context-based presets
03_hit_highlighting.py - Hit highlighting with custom tags and analysis
04_result_counting.py - Result counting with caching and performance comparison
05_range_pagination.py - Range-based pagination for large datasets
06_search_scores.py - Search scores and relevance analysis
07_large_result_sets.py - Efficient handling of large datasets
08_performance_optimization.py - Comprehensive performance optimization

🟨 JavaScript Examples (8 samples)¶

01_basic_pagination.js - Basic pagination with error handling
02_field_selection.js - Field selection with validation and presets
03_hit_highlighting.js - Hit highlighting implementation
04_result_counting.js - Smart counting strategies and caching
05_range_pagination.js - Range pagination implementation
06_advanced_range_pagination.js - Advanced range pagination with hybrid strategies
07_search_scores.js - Search scores and relevance analysis
08_large_result_sets.js - Large dataset handling with streaming

🔷 C# Examples (8 samples)¶

01_BasicPagination.cs - Comprehensive pagination with async patterns
02_FieldSelection.cs - Field selection with strongly-typed models
03_HitHighlighting.cs - Hit highlighting for enhanced search results
04_ResultCounting.cs - Result counting and metadata management
05_RangePagination.cs - Range-based pagination for large datasets
06_SearchScores.cs - Search scores and relevance analysis
07_LargeResultSets.cs - Efficient large dataset handling
08_PerformanceOptimization.cs - Production-ready performance optimization

🌐 REST API Examples (8 samples)¶

01_basic_pagination.http - Skip/top pagination examples
02_field_selection.http - Field selection parameter usage
03_hit_highlighting.http - Hit highlighting configuration
04_result_counting.http - Count parameter usage and optimization
05_range_pagination.http - Range-based pagination with filters
06_search_scores.http - Search scores and relevance
07_large_result_sets.http - Large dataset handling
08_performance_optimization.http - Performance optimization techniques

📓 Interactive Notebooks (6 comprehensive notebooks)¶

01_pagination_fundamentals.ipynb - Interactive pagination exploration
02_result_shaping_techniques.ipynb - Result shaping experimentation
03_advanced_pagination_patterns.ipynb - Advanced patterns workshop
04_performance_analysis_workshop.ipynb - Performance testing framework
05_comprehensive_pagination_workshop.ipynb - Complete hands-on workshop
06_pagination_troubleshooting_guide.ipynb - Diagnostic and troubleshooting toolkit

All samples are production-ready and include comprehensive error handling, performance monitoring, and best practices based on Microsoft's official documentation.