VoiceRun — Enterprise voice, deployed securely.

This example demonstrates an advanced A/B testing agent that automatically optimizes conversion rates with sophisticated stop conditions and notifications.

Features

Advanced A/B testing with multiple variants
Automatic test stopping based on confidence
Conversion rate tracking
Email notifications when tests complete
Fallback to default variant

Code Example

async def handler(event: Event, context: Context):
    if isinstance(event, StartEvent):
        # A/B test for welcome message
        context.add_test("welcome_variant", {
            "A": 0.5,  # Control
            "B": 0.5   # Treatment
        }, stop={
            "max_iterations": 1000,
            "max_confidence": 95,
            "target_outcome": "conversion_rate",
            "stop_on": 2,
            "default": "A",
            "notify": ["analytics@company.com"]
        })

        context.add_outcome("conversion_rate", "float")

        variant = context.get_test("welcome_variant")

        if variant == "A":
            yield TextToSpeechEvent(text="Welcome! How can I help?", voice="nova")
        else:
            yield TextToSpeechEvent(text="Hi! What can I do for you today?", voice="nova")

    if isinstance(event, TextEvent):
        user_message = event.data.get("text", "").lower()

        if "buy" in user_message or "purchase" in user_message:
            current_rate = context.get_outcome("conversion_rate")
            context.set_outcome("conversion_rate", current_rate + 0.1)

        yield TextToSpeechEvent(text="I'm here to help!", voice="nova")

How It Works

Test Configuration

The test is configured with equal distribution between control (A) and treatment (B) variants, with sophisticated stop conditions.

Stop Conditions

The test automatically stops when it reaches 1000 iterations, achieves 95% confidence, or when a clear winner emerges (2x better performance).

Conversion Tracking

Conversion rates are tracked when users express purchase intent, allowing the system to measure which welcome message drives more sales.

Notifications

When the test completes, email notifications are sent to the analytics team for review and action.

Stop Condition Parameters

Parameter	Description	Value
max_iterations	Maximum test iterations	1000
max_confidence	Statistical confidence threshold (%)	95
target_outcome	Outcome to optimize for	conversion_rate
stop_on	Stop when winner is X times better	2
default	Fallback variant	A
notify	Email addresses to notify	["analytics@company.com"]

This advanced example demonstrates how to build sophisticated A/B testing agents that automatically optimize performance and notify stakeholders.

Documentation