A/B Testing

Feature Flags for Experimentation

By Denys Pankov · March 30, 2026 · 9 min read

How Feature Flags Replaced the “Big Bang” Release

Five years ago, shipping a new checkout flow meant a Friday deploy, a war room, and three engineers refreshing dashboards until midnight. Today the same change ships behind a flag at 1% on Tuesday, ramps to 10% on Wednesday, and reaches 100% by Friday — with a kill switch one click away.

Feature flags decouple deployment from release. That single architectural shift is the reason high-velocity teams can run 8–15 experiments a month without breaking production.

2–10× Faster release cadence with feature flags
<60s Time to roll back a bad release via kill switch
80% Of top-velocity teams use flags + experimentation together
5 Major platforms worth evaluating

This guide covers the major platforms, when to use each, and how to combine flag-based rollouts with proper A/B tests.


What Feature Flags Actually Are

A feature flag is a runtime conditional. Instead of:

showNewCheckout = true;  // hardcoded

You write:

showNewCheckout = flags.isOn('new-checkout', { userId });

Now the decision happens at request time, against a configuration you can change without a deploy. That config can target by user ID, geography, plan, device, session — or simply by a percentage rollout.

Three jobs a flag does, often at once:

  1. Release management — gradual rollout, kill switch
  2. Experimentation — randomized assignment, conversion measurement
  3. Targeting — entitlements, beta access, segmentation

A flag platform that does only one of these will eventually push you to add a second tool. Pick one that handles all three.


The Five Platforms Worth Comparing

PlatformBest forPricing modelNotable strength
LaunchDarklyEnterprise eng teamsSeat + MAUMature, deep targeting, audit log
Split.ioExperimentation-firstMAU-basedBuilt-in stats engine, attribution
Optimizely RolloutsTeams already on OptimizelyFree tier, then enterpriseTight integration with their experiment platform
UnleashSelf-hosted / open sourceOpen source + cloudFull control, no vendor lock-in
GrowthBookCost-conscious, SQL-savvy teamsOpen source + cloudBayesian engine, warehouse-native

LaunchDarkly

The market default for engineering-led teams. Polished SDKs for every major language, mature audit trail, fine-grained targeting rules, percentage rollouts down to 0.01%. The experimentation module is solid but is sold as an add-on — you’ll pay enterprise pricing fast.

Use it when: engineering owns flags, you need SOC 2 audit trails, and budget isn’t the constraint.

Split.io

Built around experimentation from day one. The stats engine handles sample ratio mismatch detection, sequential testing, and proper segment analysis out of the box. Attribution is data-warehouse-friendly.

Use it when: product and engineering share flag ownership and you want experimentation as a first-class citizen, not a bolt-on.

Optimizely Rollouts

Free tier for unlimited flags (no experimentation in the free tier). If you’re already on Optimizely Web/Full Stack for A/B testing, Rollouts integrates cleanly — same SDK, same dashboard.

Use it when: you’ve already standardized on Optimizely’s experiment stack.

Unleash

Open source, self-hosted. Full control over data residency, no per-MAU pricing surprises. Smaller ecosystem of integrations, and you’ll spend engineering time on operations.

Use it when: data residency requirements, hostile to vendor lock-in, or you’ve got platform engineers with capacity.

GrowthBook

Open source with a hosted option. Warehouse-native — connects to BigQuery, Snowflake, Redshift, or Postgres and runs experiments against your existing data. Bayesian stats engine. Significantly cheaper than the enterprise options.

Use it when: you have a data warehouse, want experimentation tied to your existing event data, and want to avoid four-figure monthly bills. See the AI experimentation platforms comparison for how GrowthBook stacks up on the analysis side.


Gradual Rollouts: The 1/10/50/100 Pattern

Almost every team converges on the same release ramp:

Stage% of trafficDurationWhat you watch
Canary1%1–24 hoursError rates, latency, crash logs
Early rollout10%1–3 daysConversion metrics, support tickets
Broad rollout50%2–7 daysFull statistical comparison vs control
Full release100%Long-term metric drift

The 1% stage catches infrastructure issues — null pointer exceptions in a code path you forgot existed. The 10% stage catches UX issues — the support team starts hearing things. The 50% stage is when you actually know whether the feature is working from a conversion standpoint.

Skipping the canary saves a day and costs you a quarter the first time it fails.


Kill Switches: The Real ROI of Flags

The argument that finally sells feature flags to skeptical engineering leaders isn’t experimentation — it’s incident response. Mean time to recovery (MTTR) drops from hours to seconds.

A kill switch lets you turn off any feature without a deploy, a rollback, or a war room. That changes the risk calculus for every change. Teams that ship behind flags ship more aggressively because the cost of a bad ship is bounded.


Combining Feature Flags with A/B Tests

This is where most teams fumble. A feature flag and an A/B test are not the same thing — but they share infrastructure.

A flag rollout answers: “Did the new code break anything?” You watch errors, latency, crashes. You don’t need statistical rigor — you need a kill switch.

An A/B test answers: “Is the new variant better than the control on the primary metric?” You need a pre-registered sample size, statistical significance, and a single primary metric.

The clean pattern: ship behind a flag at 1% → 10% to validate stability → then start the A/B test at 50/50 with a pre-registered sample size. Don’t confuse “10% rollout looks fine” with “the test won.” Stability ≠ effectiveness.

A common mistake is treating a gradual rollout as a test: “We rolled out to 50%, conversion went up 8%, we’re shipping it.” That’s not an experiment — it’s a before/after with no control group. The 8% might be a Tuesday-vs-Wednesday effect. Run the actual test. See A/B testing mistakes for why this fails.


Targeting Rules: Where Power and Pain Live

Modern flag platforms support targeting rules of arbitrary complexity:

  • 100% of users in EU AND on plan Pro AND signed up after 2026-01-01
  • 25% of users where cart_value > $200 AND device_type = mobile
  • All users in the early_access cohort, regardless of other rules

Targeting unlocks legitimate use cases — beta programs, regional rollouts, plan-based entitlements. It also tempts you into post-hoc segmentation that wrecks your test validity. If you’re running an A/B test, your randomization must happen on a stable, consistent unit (usually user ID or anonymous ID, hashed). Don’t use targeting rules to “look at just the mobile segment” mid-test. See segmentation in A/B testing for why pre-registration matters.


Flags and CI/CD: The Connection

Feature flags only work if they integrate with how you ship code. Two requirements:

  1. Default-off, fail-safe. If the flag service is unreachable, the SDK returns a safe default. No one should ever see a feature because the platform crashed.
  2. Flag cleanup as a CI step. Long-lived flags become technical debt. Most platforms support flag age tracking; pipe it into your linter so flags older than 90 days break the build until they’re either promoted to permanent config or deleted.

The teams that get the most out of flags treat them as ephemeral. Ship, validate, ramp, clean up. Flags that stay in the codebase for six months become accidental config — and the next engineer to touch them won’t know what they were for.


When You Don’t Need Feature Flags

Flags aren’t free. They add latency (single-digit milliseconds with caching, but real), code complexity, and another service to monitor.

Skip them when:

  • Your release cadence is monthly or slower — the overhead doesn’t pay back
  • You don’t have engineering capacity for the cleanup discipline
  • Your changes are pure marketing copy and you’re using a visual editor anyway

The break-even point is roughly: 3+ releases per week, or any business with revenue at risk during a deploy window.


Frequently Asked Questions

Should marketing or engineering own feature flags?

Both, with clear scopes. Engineering owns release flags and kill switches. Marketing/product owns experiment flags and targeting rules for campaigns. The platform should support role-based access so a marketer can’t accidentally kill production.

How do I prevent feature flag debt?

Treat flags as ephemeral. Tag every flag with a creation date, owner, and expected lifespan. Pipe flag age into CI — flags older than 90 days fail the build until they’re cleaned up or promoted to permanent configuration. Run a quarterly flag review.

Can I run A/B tests without a feature flag platform?

Yes — most CRO tools (Optimizely Web, VWO, Convert) handle randomization without a separate flag platform for client-side tests. Flag platforms become important when tests touch backend logic, pricing, or anything that can’t be done in a visual editor. The A/B testing tools comparison covers the trade-offs.

What’s the cheapest path to a flag-based experimentation setup?

GrowthBook (open source, self-hosted) connected to your existing data warehouse. Total cost is hosting (single-digit dollars per month) plus the engineering time to set up SDK integration. Compared to LaunchDarkly Enterprise (often $50K+/year), the savings fund a year of CRO program investment.

See where your store is leaking revenue

Our AI-powered audit analyzes your pages against 48 behavioral science heuristics and shows you exactly what to fix first – in minutes, not weeks.

Get Instant CRO Audit → Book Strategy Call