Back to all writing
Identity

Identity at Scale: Consolidating Four Login Systems Onto One Standard

Architecture11 min read

At tens of millions of users, the hard part of identity wasn't building a login system. It was that we already had four. Years of growth and acquisitions had left four different authentication mechanisms wired into the product: each minting its own tokens, each covering a different slice of the experience, and none of them fully following a standard. Every feature that touched sign-in had to be built four times and reconciled by hand. The job was to make it one, on a real standard, without logging anyone out.

Four front doors

None of this was anyone's plan. It was a byproduct of how the company grew. Organic features added one auth path, each acquisition brought another, and over a decade they built up into four mechanisms that overlapped without agreeing. They used different token formats, different session models, and different user stores. Each covered only the functions it was originally built for. One knew about guest flows, another about a particular brand's SSO, another about an acquired app's accounts. No single system had the whole picture.

The cost compounded steadily. Anything that touched identity had to be implemented in up to four places and kept in sync by coordination, which is the most expensive kind of synchronization there is. The security surface was four times larger than it needed to be, with four sets of token-handling and session code to audit. And any capability that required a single identity for a person, social login most obviously, was simply impossible, because there was no single identity to attach it to.

Standardize on protocols, not products

The first decision was to anchor on open standards before choosing any vendor. We standardized authentication on OIDC and token issuance and authorization on OAuth 2.0: Authorization Code flow with PKCE for the clients, short-lived signed JWT access tokens, ID tokens carrying only the claims the relying parties actually needed, rotating refresh tokens, and scopes for authorization decisions. To prove it end to end we built a thin, standards-conformant login and token service and ran the real flows through it: login, consent, issuance, validation, refresh. We did that before betting the platform on it.

Standards first, because a protocol is portable in a way a product never is. If every relying party integrates against OIDC, the identity provider underneath becomes an implementation detail you can change your mind about later. It also collapsed four bespoke security models into one that is well understood, documented, and externally reviewed. That matters a great deal when the thing you're standardizing is the front door to every account in the product.

Standards are a hedge against your own future decisions. Build to OIDC and the identity provider becomes a detail you're allowed to be wrong about.

Choosing the system of record

With the protocol fixed, the provider was a procurement decision rather than an ideological one, and we ran it that way. We evaluated the field, including Okta, Auth0, and Shibboleth, against the constraints that actually bit at our scale: cost at tens of millions of identities, where per-active-user pricing models break down; horizontal scaling and raw throughput on the auth path; and the specific feature set we needed around federation, custom flows, and token customization.

We chose AWS Cognito, which at the time was genuinely in its infancy, and partnered closely with the AWS team to harden the gaps as we adopted it. That was a deliberate trade: a less-finished service we could shape and run economically at this scale, over a more polished one whose pricing and ceilings didn't survive contact with a user base this size. Cognito became the system of record for identity and credentials, sitting behind the standard OIDC surface the rest of the platform had been built against.

A facade that spoke every dialect

We could not ask the entire estate to cut over to Cognito at once, and it would have been reckless to try. So we introduced an SDK and a facade, an anti-corruption layer for authentication, that plugged into the whole ecosystem. The facade understood each of the four legacy variations, normalized them, and translated to the standard target implementation. Relying parties kept calling what they had always called; behind the SDK, those calls were mapped onto OIDC flows and Cognito.

That single integration surface is what turned a single coordinated cutover into an incremental migration. There was one place to reason about behavior, one place to enforce the new security model, one place to instrument, and, most important, the freedom to move callers onto the standard on their own schedule instead of in one high-risk release.

Flipping the token

The real pivot was the token itself. We updated the SDK to stop minting legacy tokens and start minting the standard target tokens, and, just as important, we taught the shared infrastructure to accept them. The central session store and the shared single-sign-on cookie were updated to understand the target token format. During the transition the SDK could validate an incoming legacy token and exchange it for a standard one, so sessions that were already in flight never broke and no one was forced to sign in again.

Once the central storage and the shared cookie spoke the target token, every system was carrying the same standard, signed token, no matter which of the four doors a user had come through. Single sign-on stopped being four overlapping approximations and became one fact. The security mechanics underneath were deliberately conventional, because conventional is auditable: JWTs signed with rotating keys published through JWKS so relying parties validate locally without a network hop; short access-token lifetimes with rotating refresh tokens and reuse detection; PKCE on public clients; the SSO cookie scoped, HttpOnly, Secure, and SameSite, bound to the session; MFA available on the standard flow; and platform-level rate limiting, brute-force and credential-stuffing protection, and anomaly detection on the authentication path.

What one token made possible

A single standard token everywhere changed the economics of the roadmap. Anything that touched identity got built once instead of four times and reconciled. The clearest payoff was social login. Letting someone sign in with their Google, Facebook, or Apple identity only works if that identity resolves to one account that behaves the same everywhere in the product. With four fragmented auth systems it was a non-starter. You would have had to onboard each social provider into each system and keep four mappings consistent forever. On the consolidated platform it became a federation configuration: the social providers federated in through OIDC into the unified identity, and a person's social login worked across the entire product because there was finally one identity for it to attach to. Social login shipped on top of the consolidation, not alongside it. The consolidation is what made it possible at all.

Migrating the whole estate without a logout

All of this had to happen while people were signed in, so the migration had no maintenance window and no acceptable amount of forced re-authentication. We moved every identity transparently, with zero downtime, using the same disciplined cutover we used elsewhere on the platform. Dual sync kept the legacy stores and Cognito in agreement in both directions through the transition. On-demand, just-in-time migration moved a user the first time they authenticated: verify against the legacy credential, materialize them in Cognito, mint the standard token. A background backfill swept the dormant tail. And a strangler-fig rollout behind feature flags routed authentication traffic from the legacy mechanisms to the target a cohort at a time, each ramp gated on health and instantly reversible.

We advanced on metrics, not on a calendar: migration coverage, reconciliation drift held near zero, and authentication success rate and latency inside their budgets under real traffic. A legacy door was only retired once its traffic had genuinely gone to zero. The non-functional bar here is higher than almost anywhere else in a system, because authentication is tier-zero. If login is down, everything is down. So it was designed for peak rather than average load, ran multi-AZ with cached JWKS and graceful degradation so a single dependency hiccup couldn't lock out the entire user base, kept token validation stateless and local for latency, made every migration step idempotent and reversible, and was audited and observable end to end, because the auth path is also the first place an attack shows up.

The point

You get roughly one chance to consolidate identity, and the only acceptable outcome is that no one notices it happened. Four front doors became one, on open standards, on infrastructure we could actually afford at this scale. The real dividend wasn't the cutover itself. It was that the day it finished, the interesting work, starting with social login and everything after it, turned from a four-team coordination project into a configuration change.

Related: Strangling the Monolith Get in touch