Skip to content

Module 10: Analyzers & Scoring - Prerequisites

Required Knowledge

Before starting this module, you should have a solid understanding of:

Core Azure AI Search Concepts

  • Search Index Structure: Understanding of fields, data types, and field attributes
  • Indexing Process: How documents are processed and stored in the search index
  • Query Fundamentals: Basic search queries and result processing
  • Search Service Management: Creating and configuring Azure AI Search services

Previous Module Completion

Technical Skills

Text Processing Fundamentals

  • Tokenization: Understanding how text is broken into individual terms
  • Normalization: Concepts like case folding, accent removal, and character mapping
  • Linguistic Processing: Basic knowledge of stemming, lemmatization, and stop words
  • Regular Expressions: Pattern matching for custom tokenization and filtering

JSON Configuration

  • JSON Syntax: Ability to read and write JSON configuration files
  • Schema Validation: Understanding of JSON schema requirements
  • Nested Objects: Working with complex JSON structures for analyzer definitions

Search Relevance Concepts

  • TF-IDF: Term Frequency-Inverse Document Frequency scoring basics
  • Field Weighting: How different fields contribute to relevance scores
  • Boosting: Concepts of term and document boosting
  • Ranking Factors: Understanding what influences search result ordering

Technical Prerequisites

Azure AI Search Service

  • Service Tier: Standard tier or higher (Basic tier has limited analyzer support)
  • Admin Access: Admin API key required for creating custom analyzers and scoring profiles
  • Index Permissions: Ability to create, modify, and delete search indexes
  • Service Capacity: Sufficient storage and search units for testing

Development Environment

Required Tools

  • REST Client: Postman, VS Code REST Client, or curl for API testing
  • Text Editor: VS Code, Visual Studio, or similar with JSON syntax highlighting
  • Web Browser: For accessing Azure Portal and Search Explorer
  • Command Line: Terminal or PowerShell for running scripts

Programming Language (Choose One)

  • Python 3.7+: With azure-search-documents SDK
  • JavaScript/Node.js 14+: With @azure/search-documents SDK
  • C# .NET 6+: With Azure.Search.Documents NuGet package
  • REST API: Direct HTTP client usage

Sample Data Requirements

Test Content

You should have access to:

  • Multi-language Text: Content in different languages for testing language analyzers
  • Rich Text Content: HTML content for testing character filters
  • Structured Data: Documents with multiple fields for scoring profile testing
  • Time-series Data: Documents with date fields for freshness scoring
  • Numeric Data: Documents with rating, view count, or similar numeric fields
{
  "documents": [
    {
      "id": "1",
      "title": "Introduction to Machine Learning",
      "content": "<p>Machine learning is a <b>powerful</b> subset of artificial intelligence...</p>",
      "category": "Technology",
      "publishDate": "2024-01-15T10:00:00Z",
      "rating": 4.5,
      "viewCount": 1250,
      "tags": ["AI", "ML", "Technology", "Education"]
    },
    {
      "id": "2",
      "title": "Advanced Data Science Techniques",
      "content": "<div>Data science combines <em>statistics</em>, programming, and domain expertise...</div>",
      "category": "Data Science",
      "publishDate": "2024-02-20T14:30:00Z",
      "rating": 4.8,
      "viewCount": 2100,
      "tags": ["Data Science", "Analytics", "Python", "Statistics"]
    }
  ]
}

Knowledge Validation

Self-Assessment Questions

Before proceeding, ensure you can answer these questions:

  1. Text Analysis: What happens to text during the indexing process in Azure AI Search?
  2. Analyzers: What's the difference between a tokenizer and a token filter?
  3. Scoring: How does TF-IDF scoring work in search engines?
  4. JSON: Can you create and modify JSON configuration objects?
  5. APIs: How do you make REST API calls to Azure AI Search?

Practical Prerequisites

1. Index Management Experience

You should be comfortable with: - Creating search indexes via REST API or SDK - Understanding field definitions and attributes - Modifying index schemas and configurations

2. Query Experience

You should have experience with: - Basic search queries (search=*, search=term) - Query parameters ($select, $filter, $orderby) - Understanding search results and scoring

3. API Usage

You should be familiar with: - Making HTTP requests to Azure AI Search - Using API keys for authentication - Reading and interpreting API responses

Setup Verification

Environment Check

Before starting the module, verify your setup:

1. Azure AI Search Service

# Test service connectivity
curl -X GET \
  "https://[your-service].search.windows.net/indexes?api-version=2024-07-01" \
  -H "api-key: [your-admin-key]"

2. Development Tools

  • [ ] REST client installed and configured
  • [ ] Text editor with JSON syntax highlighting
  • [ ] Access to Azure Portal
  • [ ] Command line tools available

3. Sample Data

  • [ ] Test documents prepared with varied content
  • [ ] Documents include different field types (text, dates, numbers)
  • [ ] Content includes HTML tags for character filter testing
  • [ ] Multi-language content available (if applicable)

If you're missing any prerequisites, consider this learning sequence:

  1. Text Processing Basics: Learn about tokenization, stemming, and normalization
  2. JSON Fundamentals: Practice creating and modifying JSON objects
  3. Regular Expressions: Basic pattern matching for custom analyzers
  4. Azure AI Search Basics: Complete earlier modules if not done
  5. API Testing: Practice making REST API calls

Common Prerequisites Issues

Issue 1: Limited Service Tier

Problem: Basic tier doesn't support custom analyzers Solution: Upgrade to Standard tier or use built-in analyzers only

Issue 2: Insufficient Permissions

Problem: Cannot create or modify analyzers Solution: Ensure you have admin API key, not query key

Issue 3: Missing Test Data

Problem: No suitable content for testing analyzers Solution: Use provided sample datasets or create representative test content

Issue 4: JSON Configuration Errors

Problem: Syntax errors in analyzer definitions Solution: Use JSON validator and review Azure AI Search schema documentation

Next Steps

Once you've verified all prerequisites:

  1. Review Module Objectives: Understand what you'll learn
  2. Prepare Test Environment: Set up your development tools
  3. Gather Sample Data: Prepare representative content for testing
  4. Start Module Content: Begin with text analysis fundamentals

Additional Preparation Resources

Documentation

Tools

Sample Datasets

Completing these prerequisites ensures you have the foundation needed to successfully work with analyzers and scoring profiles in Azure AI Search.