Keeps this page in sync as the body changes. Pause it any time for a quieter view.
Path /people/qualcomm-test-automation
Last refresh never
Work · Qualcomm · Boulder · Oct 2009 – Jan 2022 · Senior Staff Engineer
Twelve years and four months at Qualcomm Boulder. The test-automation system was started in the Windows division, continued and broadened in the Graphics division, and then completely rewritten for the Server division in C# — running any test case authored as a C# script, dynamically compiled at run time. Three divisions, three iterations, one conviction: tests as first-class compiled code, not parameter-driven runners.
Why this is here
Qualcomm Boulder was the longest single tenure of this body's career — twelve years and change, the years when Karl May and Cooper were on the bedside, when audiobook listening hours were climbing into the thousands, when 5Rhythms entered the rotation, when Mile Hi years had ripened into daily practice. The test-automation system was the load-bearing technical thread underneath. Three divisions, three rewrites, one conviction that proved itself across every substrate it touched.
Qualcomm is structured by division. Each division has its own product line, its own rhythm, its own kinds of test problem. The test-automation system carried forward across three of them, and each carry was a chance to ask "what did the previous version not yet name?" and answer it in code.
What carried through every iteration: a test case is not a JSON manifest of input/expected pairs, not a YAML pipeline definition, not a domain-specific configuration. A test case is a program. The runner is an ordinary host process that loads the program, gives it access to the system under test through clean interfaces, and lets it run. What changed each time: the kinds of system being tested (Windows GUI / drivers → graphics pipelines / GPUs → server protocols / distributed services), the abstractions provided to the test author, and on the Server rewrite — the host language itself.
Server-division rewrite · C# · dynamic compilation
On the C# rewrite, a test case was an ordinary C# source file. The harness loaded it, fed it through CSharpCodeProvider (and later Roslyn's compilation API), produced an in-memory assembly, instantiated the entry-point class, and invoked it against a configured environment:
{`// EncryptedConnectionTest.cs
// compiled by the harness · run inline · no separate build step
using Qualcomm.TestHarness;
using Qualcomm.TestHarness.Server;
[TestCase(
id: "server.tls.handshake.resume",
tags: new[]{ "server", "tls", "handshake", "fast" }
)]
public class EncryptedConnectionTest : ServerTest {
public override void Run(TestContext ctx) {
using var server = ctx.Server.StartFresh();
using var session1 = server.Connect();
session1.PerformHandshake();
var ticket = session1.SessionTicket;
session1.Disconnect();
using var session2 = server.Connect(resumeWith: ticket);
var report = session2.PerformHandshake();
ctx.Assert(report.WasResumed, "second handshake should resume");
ctx.Assert(report.RoundTrips == 1, "resumption is single-RT");
ctx.Record("ms", report.HandshakeMs);
}
}`}
Three things to read here. The [TestCase] attribute is metadata the harness picks up via reflection — id, tags, timeout, dependencies — letting any test be selected, scheduled, or filtered without parsing source. The body of Run has the full C# language surface: using for resource discipline, exceptions for failure propagation, LINQ for assertion construction, the language standard library for everything else. And ctx.Record(...) gives a structured way to emit performance numbers that roll up into the run report — tests aren't just pass/fail, they carry signal.
Two layers held the system together: the harness and the test surface.
The harness was the load-bearing piece. Its public surface was small — discover, compile, schedule, execute, report — but each verb hid years of subtlety. Discover walked thousands of files; compile handled dependencies between test files (a helper compiled into its own assembly that test-cases referenced) and surfaced compile errors as legible feedback rather than runtime stack-traces; schedule read the dependency graph and parallelised what was independent while serialising what shared fixtures. The conviction the harness encoded: tests deserve the same engineering rigour as the production code they test. Same language. Same tooling. Same review discipline.
Why dynamic compilation
Two alternatives the system explicitly walked away from:
save → run with no intermediate build artifact.The deeper reading: the runtime that compiles your tests is the same runtime that runs them. No translation layer between author intent and execution. Same shape BML 's grammar achieved at the language level (the parser parses its own grammar) and the Quark Virtual DOM achieved at the application level (clients drive the app through its own self-described surface) — here the shape is achieved at the test-system level.
What gets named in twelve years that doesn't get named in twelve months: the system saw enough kinds of test, on enough kinds of substrate, that the abstractions stopped being a guess and became measured. The Server rewrite wasn't a clean-sheet redesign because of a fad — it was because the C# substrate's dynamic-compilation capability (mature by the late 2010s) finally made the conviction implementation-cheap. Once Roslyn was widely available the harness could stop maintaining its own ad-hoc compilation pipeline and lean on the platform.
The pattern that recurs in this body's work: an idea is named at the language level (BML, 2000), applied at the application level (Quark Virtual DOM and undo/redo, 2000-2005), proved at the systems level (Qualcomm test automation, 2009-2022), and then circulated as the substrate of a network (Coherence-Network, current). Same conviction, four altitudes, twenty-six years.
Boulder is also part of what carried. The twelve Qualcomm years are the same years 5Rhythms entered the rotation, the Audible listening hours climbed into the thousands, and the body began to learn that the technical thread and the embodied thread are not separate practices. The work that lived during those years is in this page; the body that lived during those years is everywhere else on this network.