MarketAlly.AIPlugin.Extensions/MarketAlly.AIPlugin.Context/API_REFERENCE.md

26 KiB
Executable File

MarketAlly Context Plugin - API Reference

Overview

This document provides comprehensive API documentation for the MarketAlly Context Management Plugin suite. The API consists of five main plugins with enhanced capabilities for enterprise-grade context management.

Table of Contents

Authentication & Configuration

Base Configuration

All plugins support the following configuration options:

var configuration = new ContextConfiguration
{
    StoragePath = ".context",
    MaxContextSize = 50000,
    EnableCompression = true,
    Retention = new RetentionPolicy
    {
        RetentionDays = 90,
        MaxEntriesPerFile = 1000,
        CompressionAgeInDays = 30
    },
    Search = new SearchConfiguration
    {
        EnableSemanticSearch = true,
        EnableFuzzyMatching = true,
        OpenAIApiKey = "your-openai-key"
    },
    Security = new SecurityConfiguration
    {
        EnableEncryption = true,
        EnableSensitiveDataDetection = true
    }
};

Core Plugins

ContextStoragePlugin

Stores context entries with enhanced security and performance features.

Plugin Name

ContextStorage

Parameters

Parameter Type Required Default Description
contextType string Yes - Type of context: conversation, decision, codechange, insight, milestone, documentation
content string Yes - The content to store
summary string Yes - Brief summary or title
tags string No null Comma-separated tags for categorization
projectPath string No Current directory Project path to associate with
priority string No "medium" Priority level: low, medium, high, critical
metadata string No null Additional metadata as JSON string

Request Example

{
  "tool": "ContextStorage",
  "parameters": {
    "contextType": "decision",
    "content": "We decided to implement OAuth 2.0 with PKCE for mobile authentication instead of JWT tokens due to better security for mobile apps and reduced token exposure risk.",
    "summary": "Mobile authentication: OAuth 2.0 with PKCE decision",
    "tags": "authentication, oauth, mobile, security, architecture",
    "priority": "high",
    "projectPath": "./mobile-app",
    "metadata": "{\"decisionDate\": \"2025-06-24\", \"participants\": [\"team-lead\", \"security-expert\"], \"alternatives\": [\"JWT\", \"Session-based\"]}"
  }
}

Response Format

{
  "success": true,
  "message": "Successfully stored decision context: Mobile authentication decision",
  "data": {
    "Success": true,
    "EntryId": "550e8400-e29b-41d4-a716-446655440000",
    "StoredAt": "./mobile-app/.context",
    "Type": "decision",
    "Summary": "Mobile authentication: OAuth 2.0 with PKCE decision",
    "Timestamp": "2025-06-24T10:30:00Z",
    "Message": "Context stored successfully"
  }
}

Enhanced Features

  • Automatic Encryption: Sensitive content is automatically encrypted
  • Data Validation: Content is validated for sensitive information
  • Thread-Safe Storage: Concurrent operations are handled safely
  • Compression: Older entries are automatically compressed
  • Indexing: Automatic index updates for fast searching

ContextRetrievalPlugin

Retrieves context with streaming support for large datasets.

Plugin Name

ContextRetrieval

Parameters

Parameter Type Required Default Description
contextType string Yes - Type: conversation, codebase, changes, project, all
projectPath string No Current directory Project directory to analyze
conversationLimit int No 10 Number of recent conversation entries
includeFileSummaries bool No true Include file content summaries
includeGitHistory bool No true Include recent git changes
maxContextSize int No 50000 Maximum context size in characters

Request Example

{
  "tool": "ContextRetrieval",
  "parameters": {
    "contextType": "all",
    "projectPath": "./mobile-app",
    "conversationLimit": 5,
    "includeFileSummaries": true,
    "includeGitHistory": true,
    "maxContextSize": 75000
  }
}

Response Format

{
  "success": true,
  "message": "Retrieved all context successfully. Context size: 45231 characters",
  "data": {
    "ConversationHistory": {
      "Entries": [
        {
          "Timestamp": "2025-06-24T10:00:00Z",
          "Type": "assistant",
          "Content": "I'll help you implement OAuth 2.0 authentication...",
          "Context": "Mobile app authentication discussion"
        }
      ],
      "Source": ".context/conversation.json"
    },
    "CodebaseInfo": {
      "RootPath": "./mobile-app",
      "LastAnalyzed": "2025-06-24T10:30:00Z",
      "ProjectFiles": [
        {
          "Path": "package.json",
          "LastModified": "2025-06-24T09:15:00Z",
          "Size": 2048
        }
      ],
      "SourceFiles": [
        {
          "Path": "src/auth/oauth.js",
          "LastModified": "2025-06-24T10:20:00Z",
          "Size": 5120,
          "Summary": "OAuth 2.0 implementation with PKCE support"
        }
      ]
    },
    "RecentChanges": {
      "ModifiedFiles": [
        {
          "Path": "src/auth/oauth.js",
          "ModifiedDate": "2025-06-24T10:20:00Z",
          "ChangeType": "Modified"
        }
      ],
      "GitCommits": [
        {
          "Hash": "abc123",
          "Date": "2025-06-24",
          "Message": "Implement OAuth 2.0 with PKCE",
          "Author": "developer"
        }
      ]
    },
    "ProjectInfo": {
      "Path": "./mobile-app",
      "Name": "mobile-app",
      "ConfigurationFiles": {
        "package.json": "{\"name\": \"mobile-app\", \"version\": \"1.0.0\"}"
      },
      "DirectoryStructure": ["src", "tests", "docs"]
    }
  }
}

Enhanced Features

  • Streaming Processing: Handle large files without memory issues
  • Smart Caching: Cached results for improved performance
  • Selective Loading: Only load requested context types
  • Git Integration: Automatic git history analysis
  • File Summarization: AI-powered file summaries

ContextSearchPlugin

Advanced search with semantic understanding and fuzzy matching.

Plugin Name

ContextSearch

Parameters

Parameter Type Required Default Description
query string Yes - Search terms or keywords
contextType string No "all" Filter by type: all, decision, conversation, etc.
projectPath string No Current directory Project path to search in
maxResults int No 10 Maximum number of results
priority string No "all" Filter by priority: all, high, medium, low
daysBack int No 0 Search within last N days (0 = all time)
tags string No null Filter by tags (comma-separated)
includeContent bool No true Include full content or just summaries

Request Example

{
  "tool": "ContextSearch",
  "parameters": {
    "query": "OAuth authentication mobile security",
    "contextType": "decision",
    "projectPath": "./mobile-app",
    "maxResults": 5,
    "priority": "high",
    "daysBack": 30,
    "includeContent": true,
    "tags": "authentication, security"
  }
}

Response Format

{
  "success": true,
  "message": "Found 3 context entries matching 'OAuth authentication mobile security'",
  "data": {
    "Query": "OAuth authentication mobile security",
    "Results": [
      {
        "Id": "550e8400-e29b-41d4-a716-446655440000",
        "Type": "decision",
        "Summary": "Mobile authentication: OAuth 2.0 with PKCE decision",
        "Content": "We decided to implement OAuth 2.0 with PKCE...",
        "Tags": ["authentication", "oauth", "mobile", "security"],
        "Priority": "high",
        "Timestamp": "2025-06-24T10:30:00Z",
        "ProjectPath": "./mobile-app",
        "Relevance": 4.2,
        "MatchedTerms": ["OAuth", "authentication", "mobile", "security"],
        "Metadata": {
          "decisionDate": "2025-06-24",
          "participants": ["team-lead", "security-expert"]
        }
      }
    ],
    "TotalFound": 3,
    "SearchParameters": {
      "ContextType": "decision",
      "Priority": "high",
      "DaysBack": 30,
      "Tags": "authentication, security",
      "MaxResults": 5,
      "IncludeContent": true
    },
    "Message": "Found 3 relevant context entries"
  }
}

Enhanced Search Features

  • Semantic Search: Uses OpenAI embeddings for intelligent understanding
  • Fuzzy Matching: Handles typos and variations with Levenshtein and Jaro-Winkler algorithms
  • Multi-Dimensional Scoring: Combines keyword, semantic, context, and recency scores
  • Relevance Breakdown: Detailed scoring information for transparency
  • Performance Optimization: Intelligent caching and index-based search

Relevance Scoring

The search engine uses a sophisticated scoring algorithm:

  • Keyword Relevance (40%): Exact matches in summary, tags, and content
  • Fuzzy Relevance (20%): Similarity matching for typos and variations
  • Semantic Relevance (25%): AI-powered understanding of meaning
  • Context Relevance (10%): Priority and type-based scoring
  • Recency Boost (5%): More recent entries get higher scores

ContextDeletionPlugin

Secure deletion with bulk operations and confirmation requirements.

Plugin Name

ContextDeletion

Parameters

Parameter Type Required Default Description
entryId string Yes - ID of entry to delete (or criteria for bulk)
projectPath string No Current directory Project path where context is stored
deletionType string No "single" Type: single, bulk, by_tag, by_type, by_date_range
deletionCriteria string No null JSON criteria for bulk deletion
confirm bool No false Must be true to proceed with deletion

Single Entry Deletion

{
  "tool": "ContextDeletion",
  "parameters": {
    "entryId": "550e8400-e29b-41d4-a716-446655440000",
    "projectPath": "./mobile-app",
    "deletionType": "single",
    "confirm": true
  }
}

Bulk Deletion by Criteria

{
  "tool": "ContextDeletion",
  "parameters": {
    "entryId": "bulk-operation",
    "projectPath": "./mobile-app",
    "deletionType": "bulk",
    "deletionCriteria": "{\"type\": \"conversation\", \"priority\": \"low\", \"olderThan\": \"2025-05-01T00:00:00Z\"}",
    "confirm": true
  }
}

Deletion by Tag

{
  "tool": "ContextDeletion",
  "parameters": {
    "entryId": "deprecated",
    "projectPath": "./mobile-app",
    "deletionType": "by_tag",
    "confirm": true
  }
}

Response Format

{
  "success": true,
  "message": "Successfully deleted entry 550e8400-e29b-41d4-a716-446655440000",
  "data": {
    "Success": true,
    "EntryId": "550e8400-e29b-41d4-a716-446655440000",
    "DeletedFrom": "context-2025-06.json",
    "RemainingEntries": 45,
    "Operation": "single_deletion"
  }
}

Bulk Deletion Response

{
  "success": true,
  "message": "Successfully deleted 12 entries matching criteria",
  "data": {
    "Success": true,
    "TotalDeleted": 12,
    "FilesProcessed": 3,
    "DeletedEntries": ["id1", "id2", "id3"],
    "Criteria": {
      "Type": "conversation",
      "Priority": "low",
      "OlderThan": "2025-05-01T00:00:00Z"
    },
    "Operation": "bulk_deletion"
  }
}

Safety Features

  • Confirmation Required: All deletions require explicit confirm: true
  • Atomic Operations: All-or-nothing deletion for bulk operations
  • Index Updates: Automatic index cleanup after deletion
  • Audit Trail: Detailed logging of deletion operations

ConversationContinuityPlugin

High-level orchestrator combining all context operations.

Plugin Name

ConversationContinuity

Available Actions

Action Description Required Parameters
initialize Start a new conversation session topic, projectPath
store_decision Store important decisions information, summary
find_relevant Find related context searchQuery or topic
summarize_session End session with summary sessionSummary
get_project_context Get comprehensive project overview projectPath

Initialize Session

{
  "tool": "ConversationContinuity",
  "parameters": {
    "action": "initialize",
    "topic": "mobile app OAuth implementation",
    "projectPath": "./mobile-app"
  }
}

Store Decision

{
  "tool": "ConversationContinuity",
  "parameters": {
    "action": "store_decision",
    "information": "We chose OAuth 2.0 with PKCE over JWT tokens for mobile authentication due to enhanced security and better token management.",
    "summary": "OAuth 2.0 with PKCE for mobile authentication",
    "priority": "high",
    "tags": "oauth, mobile, security, authentication"
  }
}

Find Relevant Context

{
  "tool": "ConversationContinuity",
  "parameters": {
    "action": "find_relevant",
    "searchQuery": "authentication security mobile",
    "projectPath": "./mobile-app"
  }
}

Summarize Session

{
  "tool": "ConversationContinuity",
  "parameters": {
    "action": "summarize_session",
    "sessionSummary": "Completed OAuth 2.0 implementation with PKCE for mobile app. Implemented authorization flow, token refresh, and secure storage. Next: implement logout and token revocation.",
    "topic": "mobile authentication implementation"
  }
}

Get Project Context

{
  "tool": "ConversationContinuity",
  "parameters": {
    "action": "get_project_context",
    "projectPath": "./mobile-app"
  }
}

Enhanced Orchestration Features

  • Intelligent Context Retrieval: Automatically finds relevant context for topics
  • Session Management: Tracks conversation flow and maintains state
  • Cross-Plugin Coordination: Seamlessly integrates all context operations
  • Context Optimization: Automatically manages context size and relevance

Configuration API

ContextConfiguration

Complete configuration object for all plugins:

public class ContextConfiguration
{
    public string StoragePath { get; set; } = ".context";
    public int MaxContextSize { get; set; } = 50000;
    public bool EnableCompression { get; set; } = true;
    public RetentionPolicy Retention { get; set; } = new();
    public SearchConfiguration Search { get; set; } = new();
    public PerformanceConfiguration Performance { get; set; } = new();
    public SecurityConfiguration Security { get; set; } = new();
    public MonitoringConfiguration Monitoring { get; set; } = new();
}

RetentionPolicy

public class RetentionPolicy
{
    public int MaxEntriesPerFile { get; set; } = 1000;
    public int RetentionDays { get; set; } = 90;
    public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
    public int CompressionAgeInDays { get; set; } = 30;
    public bool EnableAutoCleanup { get; set; } = true;
}

SearchConfiguration

public class SearchConfiguration
{
    public bool EnableSemanticSearch { get; set; } = false;
    public bool EnableFuzzyMatching { get; set; } = true;
    public double FuzzyMatchingThreshold { get; set; } = 0.7;
    public int MaxSearchResults { get; set; } = 50;
    public bool EnableCaching { get; set; } = true;
    public int CacheExpirationMinutes { get; set; } = 30;
    public string? OpenAIApiKey { get; set; }
    public string OpenAIEmbeddingModel { get; set; } = "text-embedding-3-small";
}

SecurityConfiguration

public class SecurityConfiguration
{
    public bool EnableEncryption { get; set; } = true;
    public string? EncryptionKey { get; set; }
    public bool EnableSensitiveDataDetection { get; set; } = true;
    public bool AutoEncryptSensitiveData { get; set; } = true;
    public List<string> SensitiveDataPatterns { get; set; } = new()
    {
        @"\b[A-Za-z0-9+/]{40,}\b",        // API keys
        @"\b[\w\.-]+@[\w\.-]+\.\w+\b",    // Email addresses
        @"\b\d{3}-\d{2}-\d{4}\b",         // SSN patterns
        @"\b(?:\d{4}[-\s]?){3}\d{4}\b",   // Credit card numbers
        @"\bbearer\s+[A-Za-z0-9\-\._~\+\/]+=*\b", // Bearer tokens
        @"\bpassword[:=]\s*[^\s]+\b"      // Password patterns
    };
}

Security Features

Automatic Sensitive Data Detection

The system automatically detects and protects sensitive data:

{
  "detectedTypes": ["Email", "APIKey", "CreditCard"],
  "protectionApplied": "encryption",
  "patternsMatched": 3
}

Supported Sensitive Data Types

  1. Email Addresses: user@example.com
  2. API Keys: Long base64 strings (40+ characters)
  3. Social Security Numbers: 123-45-6789
  4. Credit Card Numbers: 4532 1234 5678 9012
  5. Bearer Tokens: Bearer eyJhbGciOiJIUzI1NiIs...
  6. Password Fields: password: mySecretPass123

Encryption Details

  • Algorithm: AES-256-CBC
  • Key Derivation: PBKDF2 with SHA-256 (10,000 iterations)
  • Initialization Vector: Random IV per encryption operation
  • Encoding: Base64 encoding for storage

Monitoring & Health

Health Check Endpoints

# Basic health check
GET /health
{
  "status": "healthy",
  "timestamp": "2025-06-24T10:30:00Z",
  "duration": "15ms"
}

# Detailed health check
GET /health/detailed
{
  "status": "healthy",
  "timestamp": "2025-06-24T10:30:00Z",
  "duration": "45ms",
  "components": {
    "storage": { "status": "healthy", "message": "Storage accessible" },
    "memory": { "status": "healthy", "usage": "125MB" },
    "diskSpace": { "status": "healthy", "available": "15.2GB" },
    "permissions": { "status": "healthy", "message": "All permissions OK" }
  }
}

Metrics Endpoints

# Prometheus metrics
GET /metrics
# Returns OpenTelemetry/Prometheus format metrics

# Custom metrics
GET /api/metrics/summary
{
  "totalOperations": 15420,
  "totalErrors": 23,
  "averageResponseTime": "125ms",
  "cacheHitRatio": 0.85,
  "activeConnections": 5
}

Performance Metrics

Available metrics include:

  • Operation Metrics: Duration, throughput, success/failure rates
  • Cache Metrics: Hit ratio, cache size, eviction rate
  • Storage Metrics: File sizes, compression ratios, I/O operations
  • Security Metrics: Encryption operations, sensitive data detections
  • System Metrics: Memory usage, CPU utilization, disk I/O

Error Handling

Standard Error Response Format

{
  "success": false,
  "message": "Error description for user",
  "error": {
    "type": "ValidationError",
    "code": "INVALID_PARAMETER",
    "details": "The 'query' parameter is required for search operations",
    "timestamp": "2025-06-24T10:30:00Z",
    "requestId": "550e8400-e29b-41d4-a716-446655440000"
  }
}

Common Error Codes

Code Description Resolution
INVALID_PARAMETER Required parameter missing or invalid Check parameter names and types
STORAGE_ERROR File system access error Check permissions and disk space
ENCRYPTION_ERROR Encryption/decryption failure Verify encryption key configuration
SEARCH_ERROR Search operation failed Check search parameters and index integrity
RATE_LIMIT_EXCEEDED Too many concurrent operations Reduce request rate or increase limits
CONFIGURATION_ERROR Invalid configuration settings Validate configuration file

Error Recovery

The system implements several error recovery mechanisms:

  • Retry Logic: Automatic retries for transient failures
  • Circuit Breaker: Prevents cascade failures during outages
  • Graceful Degradation: Core functionality continues during partial failures
  • Error Isolation: Errors in one operation don't affect others

Examples

Complete Workflow Example

// 1. Initialize conversation
var initResult = await registry.CallFunctionAsync("ConversationContinuity", new Dictionary<string, object>
{
    ["action"] = "initialize",
    ["topic"] = "payment system refactoring",
    ["projectPath"] = "./ecommerce-api"
});

// 2. Find relevant previous context
var searchResult = await registry.CallFunctionAsync("ContextSearch", new Dictionary<string, object>
{
    ["query"] = "payment processing security stripe",
    ["contextType"] = "decision",
    ["projectPath"] = "./ecommerce-api",
    ["daysBack"] = 60
});

// 3. Store new decision
var storeResult = await registry.CallFunctionAsync("ContextStorage", new Dictionary<string, object>
{
    ["contextType"] = "decision",
    ["content"] = "After reviewing security requirements and PCI compliance needs, we decided to migrate from custom payment processing to Stripe Payment Intents API. This provides better security, reduces PCI scope, and offers built-in fraud protection.",
    ["summary"] = "Payment system migration: Stripe Payment Intents",
    ["tags"] = "payment, stripe, security, pci-compliance, migration",
    ["priority"] = "critical",
    ["projectPath"] = "./ecommerce-api"
});

// 4. Summarize session
var summaryResult = await registry.CallFunctionAsync("ConversationContinuity", new Dictionary<string, object>
{
    ["action"] = "summarize_session",
    ["sessionSummary"] = "Analyzed payment system requirements and decided on Stripe migration. Identified security benefits and compliance improvements. Next steps: create migration plan and timeline.",
    ["topic"] = "payment system refactoring"
});

Batch Operations Example

// Store multiple related decisions
var decisions = new[]
{
    new { content = "Database: PostgreSQL for better ACID compliance", summary = "Database selection", tags = "database,postgresql" },
    new { content = "Caching: Redis for session storage and rate limiting", summary = "Caching strategy", tags = "cache,redis,performance" },
    new { content = "API: GraphQL for flexible client queries", summary = "API technology choice", tags = "api,graphql,client" }
};

foreach (var decision in decisions)
{
    await registry.CallFunctionAsync("ContextStorage", new Dictionary<string, object>
    {
        ["contextType"] = "decision",
        ["content"] = decision.content,
        ["summary"] = decision.summary,
        ["tags"] = decision.tags,
        ["priority"] = "high",
        ["projectPath"] = "./api-redesign"
    });
}

Advanced Search Example

// Multi-criteria search with semantic understanding
var advancedSearch = await registry.CallFunctionAsync("ContextSearch", new Dictionary<string, object>
{
    ["query"] = "database performance optimization indexing",
    ["contextType"] = "all",
    ["projectPath"] = "./api-redesign",
    ["maxResults"] = 10,
    ["priority"] = "high",
    ["daysBack"] = 90,
    ["tags"] = "database,performance",
    ["includeContent"] = true
});

// Process results with relevance scores
var searchData = (JsonElement)advancedSearch.Data;
var results = searchData.GetProperty("Results").EnumerateArray();

foreach (var result in results)
{
    var relevance = result.GetProperty("Relevance").GetDouble();
    var summary = result.GetProperty("Summary").GetString();
    var matchedTerms = result.GetProperty("MatchedTerms").EnumerateArray()
                             .Select(t => t.GetString()).ToList();
    
    Console.WriteLine($"Relevance: {relevance:F2} - {summary}");
    Console.WriteLine($"Matched: {string.Join(", ", matchedTerms)}");
}

Security Configuration Example

// Configure enhanced security
var securityConfig = new SecurityConfiguration
{
    EnableEncryption = true,
    EncryptionKey = Environment.GetEnvironmentVariable("CONTEXT_ENCRYPTION_KEY"),
    EnableSensitiveDataDetection = true,
    AutoEncryptSensitiveData = true,
    SensitiveDataPatterns = new List<string>
    {
        @"\bsk-[a-zA-Z0-9]{48}\b",  // OpenAI API keys
        @"\bghp_[a-zA-Z0-9]{36}\b", // GitHub tokens
        @"\bAKIA[0-9A-Z]{16}\b",    // AWS access keys
        // ... custom patterns
    }
};

var configuration = new ContextConfiguration
{
    Security = securityConfig,
    Monitoring = new MonitoringConfiguration
    {
        EnableDetailedLogging = true,
        EnableMetrics = true,
        EnableTracing = true
    }
};

Version Information

  • API Version: 2.0.0
  • Plugin Framework: MarketAlly.AIPlugin 1.0.0+
  • Minimum .NET Version: 8.0
  • OpenTelemetry: 1.6.0+
  • Security: AES-256-CBC encryption

Change Log

Version 2.0.0 (Current)

  • Added semantic search with OpenAI embeddings
  • Implemented fuzzy matching algorithms
  • Enhanced security with automatic encryption
  • Added comprehensive monitoring and health checks
  • Implemented thread-safe concurrent operations
  • Added streaming JSON processing for large files
  • Enhanced configuration management
  • Added context deletion plugin with bulk operations

Version 1.0.0 (Legacy)

  • Basic context storage and retrieval
  • Simple keyword search
  • File-based storage with monthly partitioning
  • Basic error handling and logging

For more information, see the README and Configuration Guide.