Skip to content

Module 8: Search Explorer & Portal Tools - Python Examples

This directory contains Python code samples that demonstrate how to replicate Azure AI Search portal functionality programmatically using the Azure SDK for Python.

Overview

The Azure portal provides powerful tools for working with Azure AI Search, but many operations can be automated using Python scripts. These examples show how to achieve the same functionality as portal tools using code.

Directory Structure

python/
├── 01_search_explorer_equivalent.py      # Replicate Search Explorer functionality
├── 02_import_data_wizard_equivalent.py   # Automate Import Data Wizard workflow
├── 03_portal_management_operations.py    # Service management and monitoring
├── 04_index_management_portal.py         # Index operations via code
├── 05_indexer_monitoring_automation.py   # Automated indexer monitoring
├── 06_query_testing_framework.py         # Systematic query testing
├── 07_performance_monitoring.py          # Performance analysis and monitoring
├── 08_configuration_export_import.py     # Export/import portal configurations
├── requirements.txt                      # Python dependencies
├── config.py                            # Configuration management
├── utils.py                             # Utility functions
└── README.md                            # This file

Prerequisites

Python Environment

  • Python 3.8 or higher
  • pip package manager

Required Packages

Install the required packages using pip:

pip install -r requirements.txt

Azure Resources

  • Azure AI Search service
  • Admin API key for management operations
  • Query API key for search operations
  • Sample data indexed in your search service

Configuration

Environment Variables

Set up your environment variables:

# Required configuration
export AZURE_SEARCH_SERVICE_NAME="your-search-service-name"
export AZURE_SEARCH_ADMIN_KEY="your-admin-api-key"
export AZURE_SEARCH_QUERY_KEY="your-query-api-key"
export AZURE_SEARCH_INDEX_NAME="hotels-sample"

# Optional configuration
export AZURE_SEARCH_API_VERSION="2024-07-01"
export AZURE_STORAGE_CONNECTION_STRING="your-storage-connection-string"
export AZURE_SQL_CONNECTION_STRING="your-sql-connection-string"

Configuration File

Alternatively, create a config.json file:

{
  "search_service_name": "your-search-service-name",
  "admin_key": "your-admin-api-key",
  "query_key": "your-query-api-key",
  "index_name": "hotels-sample",
  "api_version": "2024-07-01"
}

Code Examples Overview

1. Search Explorer Equivalent (01_search_explorer_equivalent.py)

Purpose: Replicate Search Explorer functionality for interactive query testing.

Features: - Basic text search with various parameters - Advanced filtering and sorting - Faceted search implementation - Hit highlighting configuration - Query performance measurement - Result analysis and formatting

Key Functions: 

def basic_search(search_client, query, **kwargs)
def advanced_search_with_filters(search_client, query, filters, **kwargs)
def faceted_search(search_client, query, facets, **kwargs)
def search_with_highlighting(search_client, query, highlight_fields, **kwargs)

Usage Example: 

from azure.search.documents import SearchClient
from search_explorer_equivalent import SearchExplorerEquivalent

# Initialize
explorer = SearchExplorerEquivalent(service_name, api_key, index_name)

# Basic search
results = explorer.basic_search("luxury hotel")

# Advanced search with filters
results = explorer.advanced_search_with_filters(
    query="hotel",
    filters="Rating gt 4.0 and Category eq 'Luxury'",
    order_by="Rating desc",
    select_fields=["HotelName", "Rating", "Category"]
)

2. Import Data Wizard Equivalent (02_import_data_wizard_equivalent.py)

Purpose: Automate the complete Import Data Wizard workflow programmatically.

Features: - Data source creation (Blob Storage, SQL Database, Cosmos DB) - Index schema generation and customization - Skillset creation with cognitive skills - Indexer configuration and execution - Monitoring and validation - Error handling and retry logic

Key Functions: 

def create_data_source(search_admin_client, data_source_config)
def create_index_from_data(search_admin_client, index_config)
def create_skillset(search_admin_client, skillset_config)
def create_indexer(search_admin_client, indexer_config)
def monitor_indexer_execution(search_admin_client, indexer_name)

Usage Example: 

from import_data_wizard_equivalent import ImportDataWizardEquivalent

# Initialize
wizard = ImportDataWizardEquivalent(service_name, admin_key)

# Create complete solution
solution = wizard.create_complete_solution(
    data_source_type="blob_storage",
    connection_string=storage_connection_string,
    container_name="sample-data",
    index_name="automated-index",
    enable_cognitive_skills=True
)

3. Portal Management Operations (03_portal_management_operations.py)

Purpose: Implement portal management and monitoring features programmatically.

Features: - Service statistics and health monitoring - Index management and operations - Indexer status monitoring - Performance metrics collection - Resource usage tracking - Automated health checks

Key Functions: 

def get_service_statistics(search_admin_client)
def list_indexes_with_stats(search_admin_client)
def monitor_indexer_status(search_admin_client, indexer_name)
def collect_performance_metrics(search_client, queries)
def generate_health_report(search_admin_client)

4. Index Management Portal (04_index_management_portal.py)

Purpose: Comprehensive index management operations.

Features: - Index creation, modification, and deletion - Schema analysis and validation - Field attribute management - Index statistics and monitoring - Bulk operations and batch processing

Key Functions: 

def create_index_with_validation(search_admin_client, index_definition)
def analyze_index_schema(search_admin_client, index_name)
def update_index_fields(search_admin_client, index_name, field_updates)
def get_index_statistics(search_admin_client, index_name)

5. Indexer Monitoring Automation (05_indexer_monitoring_automation.py)

Purpose: Automated monitoring and management of indexers.

Features: - Real-time indexer status monitoring - Execution history analysis - Error detection and alerting - Performance trend analysis - Automated retry mechanisms

Key Functions: 

def monitor_indexer_real_time(search_admin_client, indexer_name)
def analyze_execution_history(search_admin_client, indexer_name)
def detect_indexer_issues(search_admin_client, indexer_name)
def setup_indexer_alerts(search_admin_client, alert_config)

6. Query Testing Framework (06_query_testing_framework.py)

Purpose: Systematic query testing and validation framework.

Features: - Automated query testing suites - Performance benchmarking - Result validation and comparison - Regression testing capabilities - Test report generation

Key Functions: 

def run_query_test_suite(search_client, test_cases)
def benchmark_query_performance(search_client, queries)
def validate_search_results(search_client, test_cases)
def generate_test_report(test_results)

7. Performance Monitoring (07_performance_monitoring.py)

Purpose: Comprehensive performance monitoring and analysis.

Features: - Query performance tracking - Resource utilization monitoring - Trend analysis and reporting - Performance optimization recommendations - Automated alerting

Key Functions: 

def track_query_performance(search_client, queries)
def monitor_resource_usage(search_admin_client)
def analyze_performance_trends(performance_data)
def generate_optimization_recommendations(analysis_results)

8. Configuration Export/Import (08_configuration_export_import.py)

Purpose: Export and import portal configurations for automation.

Features: - Index definition export/import - Indexer configuration backup - Skillset template management - Environment migration tools - Configuration validation

Key Functions: 

def export_index_configuration(search_admin_client, index_name)
def import_index_configuration(search_admin_client, config_file)
def backup_service_configuration(search_admin_client)
def migrate_configurations(source_service, target_service)

Running the Examples

Individual Examples

Run individual examples with specific parameters:

# Search Explorer equivalent
python 01_search_explorer_equivalent.py --query "luxury hotel" --filter "Rating gt 4.0"

# Import Data Wizard equivalent
python 02_import_data_wizard_equivalent.py --data-source blob --container sample-data

# Portal management operations
python 03_portal_management_operations.py --operation health-check

# Index management
python 04_index_management_portal.py --index hotels-sample --operation stats

# Indexer monitoring
python 05_indexer_monitoring_automation.py --indexer hotels-indexer --monitor

# Query testing framework
python 06_query_testing_framework.py --test-suite basic-queries.json

# Performance monitoring
python 07_performance_monitoring.py --duration 3600 --interval 60

# Configuration export/import
python 08_configuration_export_import.py --export --index hotels-sample

Batch Execution

Run all examples in sequence:

python run_all_examples.py

Utility Functions

Configuration Management (config.py)

def load_config(config_file=None)
def get_search_client(config)
def get_search_admin_client(config)
def validate_configuration(config)

Common Utilities (utils.py)

def format_search_results(results)
def measure_execution_time(func, *args, **kwargs)
def log_operation(operation, status, details)
def handle_search_exception(exception)

Error Handling

All examples include comprehensive error handling:

try:
    # Search operation
    results = search_client.search(query)
except Exception as e:
    logger.error(f"Search operation failed: {e}")
    # Handle specific error types
    if "authentication" in str(e).lower():
        # Handle authentication errors
    elif "not found" in str(e).lower():
        # Handle resource not found errors
    else:
        # Handle general errors

Performance Considerations

Optimization Tips

  • Use connection pooling for multiple operations
  • Implement caching for frequently accessed data
  • Use async operations for better performance
  • Monitor and log execution times
  • Implement retry logic with exponential backoff

Resource Management

# Use context managers for proper resource cleanup
with SearchClient(endpoint, credential) as search_client:
    results = search_client.search(query)
    # Client automatically closed

Testing

Unit Tests

Run unit tests for individual components:

python -m pytest tests/test_search_explorer.py
python -m pytest tests/test_import_wizard.py
python -m pytest tests/test_portal_management.py

Integration Tests

Run integration tests against actual Azure AI Search service:

python -m pytest tests/integration/ --slow

Logging and Monitoring

Logging Configuration

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('search_operations.log'),
        logging.StreamHandler()
    ]
)

Performance Metrics

All examples collect and log performance metrics: - Execution time - Result count - Memory usage - API call frequency

Best Practices

Code Organization

  • Separate configuration from code
  • Use type hints for better code documentation
  • Implement proper error handling
  • Follow PEP 8 style guidelines

Security

  • Never hardcode API keys in source code
  • Use environment variables or secure key storage
  • Implement proper authentication and authorization
  • Log security-related events

Performance

  • Use async operations when possible
  • Implement connection pooling
  • Cache frequently accessed data
  • Monitor and optimize query performance

Troubleshooting

Common Issues

  1. Authentication Errors
  2. Verify API keys are correct
  3. Check service name and endpoint

  4. Ensure proper permissions

  5. Connection Issues

  6. Check network connectivity
  7. Verify firewall settings

  8. Test with simple operations first

  9. Query Errors

  10. Validate query syntax
  11. Check field names and types

  12. Test with minimal queries first

  13. Performance Issues

  14. Monitor query complexity
  15. Check service tier and capacity
  16. Optimize query parameters

Debug Mode

Enable debug mode for detailed logging:

export AZURE_SEARCH_DEBUG=true
python 01_search_explorer_equivalent.py --debug

Contributing

To contribute to these examples:

  1. Fork the repository
  2. Create a feature branch
  3. Add or improve examples
  4. Include comprehensive tests
  5. Update documentation
  6. Submit a pull request

License

These examples are provided under the MIT License. See LICENSE file for details.

Module Documentation

Other Code Examples

External Resources

By using these Python examples, you can automate Azure AI Search portal operations and integrate them into your applications and workflows effectively.