A/B Test Calculator

Statistical power calculations & AI insights for your experiments

Overview

A/B Test Calculator is a powerful Chrome extension that brings statistical expertise directly to your browser. Whether you're designing experiments or analyzing results, get instant calculations and AI-powered insights without leaving your workflow.

📊

Statistical Calculations

Sample size, test duration, significance testing, and confidence intervals

🤖

AI Insights

Hypothesis analysis and results interpretation with your own API keys

🔒

Privacy First

All calculations run locally in your browser. No data sent to external servers

⚡

Instant Results

No waiting for server responses. Get calculations in milliseconds

Installation

Recommended

Chrome Web Store

The easiest way to install and stay updated automatically.


Add to Chrome

Developer Installation

For advanced users who want to try the latest features.


Download Source

Manual Installation Steps

  1. Download the extension files from GitHub
  2. Open Chrome and navigate to chrome://extensions/
  3. Enable "Developer mode" in the top right corner
  4. Click "Load unpacked" and select the extension folder
  5. Pin the extension to your toolbar for easy access

Quick Start Guide

1. Access the Extension

Click the A/B Test Calculator icon in your browser toolbar. If you don't see it, click the puzzle piece icon and pin it.

2. Validate Your Experiment Setup

  • Hypothesis: Describe what you're changing and expected impact
  • Baseline Rate: Current conversion rate (e.g., 5%)
  • Minimum Detectable Effect: Smallest change you want to detect (e.g., 0.5%)
  • Traffic: Daily visitors and allocation percentage

3. Get Instant Results

The extension calculates:

  • Required sample size per variant
  • Estimated test duration
  • Statistical power analysis
  • Trade-off scenarios

4. Analyze Your Results

Once your test is complete, use the "Analyze Results" tab to:

  • Test statistical significance
  • Calculate confidence intervals
  • Get AI-powered interpretation (with API key)

Features & Capabilities

Statistical Calculations

  • Sample Size Calculator: Determines required participants per variant
  • Power Analysis: Ensures your test can detect meaningful changes
  • Duration Estimation: Predicts how long your test needs to run
  • Significance Testing: Two-proportion z-test for result analysis
  • Confidence Intervals: 95% confidence bounds for your results
  • Effect Size: Relative and absolute lift calculations

AI-Powered Insights (Optional)

  • Hypothesis Analysis: AI evaluates your experiment design
  • Results Interpretation: Plain-English explanations of statistical findings
  • Recommendations: Next steps based on your results
  • Risk Assessment: Identifies potential issues with your experiment

Data Management

  • Local Storage: All data saved securely in your browser
  • Export Options: Download results as CSV or JSON
  • Form Memory: Automatically saves your inputs
  • Settings Backup: Export/import your configuration

AI Setup (Optional)

Note: AI features are completely optional. The extension works perfectly for statistical calculations without any API keys.

Google Gemini (Recommended)

  1. Visit Google AI Studio
  2. Sign in with your Google account
  3. Click "Create API Key" and copy it
  4. Open A/B Test Calculator → Settings
  5. Paste your key in the Gemini API field
  6. Click "Test Connection" to verify

Anthropic Claude

  1. Visit Anthropic Console
  2. Create an account or sign in
  3. Navigate to API Keys and create a new key
  4. Copy the key (it starts with "sk-ant-")
  5. Open A/B Test Calculator → Settings
  6. Paste your key in the Claude API field
  7. Click "Test Connection" to verify
Security: Your API keys are stored locally in Chrome's encrypted storage and never sent to our servers. They're only used for direct communication with AI services.

Frequently Asked Questions

Do I need an internet connection to use the extension?
No! All statistical calculations run locally in your browser. You only need internet for optional AI insights.
How accurate are the statistical calculations?
Very accurate. Our calculations use the same formulas as industry-standard tools like SciPy and match server-side implementations within numerical precision limits.
Is my data secure?
Yes. All data is stored locally in your browser using Chrome's secure storage API. We don't collect, store, or transmit any of your experiment data.
What's the difference between this and online calculators?
Speed, privacy, and AI insights. No waiting for page loads, no data sent to servers, and optional AI analysis of your experiments.
Can I use this for non-web experiments?
Absolutely! The statistical methods work for any A/B test - mobile apps, email campaigns, offline experiments, etc.
Why do I need to provide my own AI API keys?
This ensures your data privacy and gives you control over AI costs. We don't have access to your experiment data or AI conversations.
Does this work in other browsers?
It works in most Chromium-based browsers (Edge, Brave, Opera) but is optimized for Chrome. Firefox support may come in the future.
How do I export my results?
After running calculations, look for the export buttons. You can download results as CSV (for spreadsheets) or JSON (for other tools).

Troubleshooting

Extension Not Loading

  • Refresh the extension at chrome://extensions/
  • Make sure Chrome is up to date (version 88+)
  • Try disabling and re-enabling the extension

AI Features Not Working

  • Verify your API key is correct and has credits
  • Use "Test Connection" in settings to diagnose issues
  • Check if your API key has the necessary permissions
  • Try switching between Gemini and Claude if one isn't working

Calculations Seem Wrong

  • Double-check your input values (percentages should be decimals like 0.05 for 5%)
  • Ensure your baseline rate is realistic for your context
  • Consider if your MDE is too small to detect reliably

Performance Issues

  • Clear the extension's data in settings if it becomes slow
  • Restart Chrome if the extension becomes unresponsive
  • Check if other extensions are conflicting

Support & Contact

Need help? We're here to assist you!

Self-Service Options

  • Built-in Help: Click the "?" tooltips throughout the extension
  • Settings Guide: Detailed explanations in the Settings page
  • This Documentation: Comprehensive guide you're reading now

Get in Touch

Community

Connect with other Product Managers using A/B testing tools:

  • LinkedIn group discussions
  • Product Manager forums
  • GitHub discussions and issues
Response Time: We typically respond to support requests within 24 hours on business days.

Privacy & Security

Data Handling

  • Local Storage Only: All your experiment data stays in your browser
  • No Analytics: We don't track usage, clicks, or any user behavior
  • No External Servers: Calculations happen entirely in your browser
  • API Keys: Stored encrypted in Chrome's secure storage system

Permissions Explained

  • Storage: Save your preferences and form data locally
  • Notifications: Alert you when calculations complete (optional)
  • Host Permissions: Only for AI APIs when you provide keys

Third-Party Services

When you provide API keys for AI features:

  • Data is sent directly to Google/Anthropic (not through our servers)
  • Subject to their respective privacy policies
  • You control what data is shared through your input