Building a Data and Event Tracking Component: Architecture, Stack Selection, and MVP Delivery

Every AdTech and MarTech platform is different.

Demand-side platforms (DSPs), for example, help advertisers purchase inventory from publishers on an impression-by-impression basis via real-time bidding (RTB). Customer data platforms (CDPs), on the other hand, collect first-party data from a range of sources, create single customer views (SCVs), and push audiences to other systems and tools.

Although the functionality and goals of AdTech and MarTech platforms vary considerably, they share a common requirement: every platform needs a component that collects and delivers data from different sources (e.g., websites) to different downstream systems (e.g., DSPs). That component is known as a tracker.

This article walks through the scoping, architecture decisions, stack selection, and sprint-by-sprint delivery of a tracker MVP — covering the technical patterns and tradeoffs that apply broadly to any team building event collection infrastructure for AdTech or MarTech.

Tracker component diagram

The Scenario

The goal was to build a tracker with core functionality, example extensions, example deployment scripts, and documentation — and to design it so it could be integrated into other platform components such as analytics tools and reporting databases.

While the initial scope targeted a DSP use case, the design was intentionally general enough to support any AdTech or MarTech platform that needs to collect event data, including impressions, clicks, video metrics (e.g., watch time and video completion rates), and conversions.

A well-designed reusable tracker component of this kind can save significant development time when building or extending programmatic advertising infrastructure.

Development team working session

Defining the Requirements

Story Mapping

The scoping phase began with story mapping sessions to:

• Define the key components and features of the tracker.
• Identify and address the main technical challenges — performance, speed, and scalability.
• Decide what events the tracker should collect and how it would do so.

Functional Requirements

The tracker would need to:

• Receive and handle multiple types of requests (impressions, clicks, conversions, etc.).
• Process requests and generate events with proper dimensions and metrics.
• Allow request types and event preprocessing to be configured.
• Extend core functionality via plugins through an exposed API.
• Expose generated events to specified collectors (plugins) and allow those collectors to be attached to specific event types through configuration.
• Include built-in log and queue collectors.
• Include a built-in plugin for a budget management component (referred to internally as "banker").
• Support request redirects.

Non-Functional Requirements

The non-functional requirements addressed performance, security, deployability, and resilience:

• High availability: 99.999% uptime target.
• Throughput: 2,000 requests per second.
• Latency: average request processing time of 15 milliseconds.
• Security and privacy: no exposure of personal data to third parties when sharing data with other components.
• DDoS protection: the platform must not be impacted by distributed denial-of-service attacks originating from malformed requests.
• Multi-cloud deployability: support for AWS, GCP, and Azure.
• Plugin integration: ability to integrate with custom plugins and other DSP components such as the banker.
• Scalability: the tracker must handle large spikes in event volume.
• Idempotency: the system must be able to process the same logs multiple times in the event of errors or temporary unavailability.

Selecting the Architecture and Technology Stack

Choosing the right stack started with benchmarking. The primary metrics evaluated were:

• Request latency
• Requests per second
• Error ratio
• Percentile latency distribution

Technology Stack Comparison

Four options were evaluated:

• Golang (Go) — growing in popularity and well-understood within the AdTech development community.
• Rust — used by other teams to build trackers, known for its raw performance characteristics.
• Python — highly familiar, though not typically associated with high-throughput request handling.
• OpenResty (Nginx + Lua) — enables tracker construction using just an Nginx HTTP server, combining well-known infrastructure with lightweight scripting.

Initial benchmarks were run across all four technologies, with deeper testing conducted using wrk2, gatling, and locust — all three configured for maximum elasticity.

Results and Stack Decision

The two best-performing options were Nginx + Lua and Golang, which produced similar results across all benchmarking tools. Golang was ultimately selected based on current market demand and its growing ecosystem, despite the performance parity with Nginx + Lua.

The MVP Scoping Phase — covering story mapping, requirements definition, benchmarking, and stack selection — took 1 sprint (2 weeks) to complete.

Tracker development environment

MVP Development Phase

With scoping complete and the stack decided, development of the tracker MVP proceeded over 5 sprints (10 weeks). Here is what each sprint delivered.

Sprint 1

• Implemented basic tracker functionality.
• Configured event types.
• Created benchmarks as a continuous integration (CI) step.
• Produced visual benchmark results for ongoing performance monitoring.

Sprint 2

• Implemented and tested request redirect functionality.
• Ran general tests on the tracker.

Sprint 3

• Built a plugin for the log storage component.
• Enhanced configuration to support any custom tracking paths.
• Produced initial tracker documentation.

Sprint 4

• Built a plugin for the banker (budget management component).
• Configured event extraction.
• Ran end-to-end tests.

Sprint 5

• Produced auto-generated documentation.
• Created a quickstart guide.
• Built Docker images and pushed them to an internal registry.

Architecture Highlights

Several design decisions stand out as broadly applicable patterns for tracker development in this space:

Plugin-based extensibility: By exposing an API for plugins, the tracker's core remains lean while custom behaviours — log storage, budget management integrations, queue forwarding — are added without modifying the core codebase. This is a standard approach for teams that need to serve multiple platform configurations from a single tracker foundation.

Built-in idempotency: Processing the same log events multiple times without side effects is essential for fault-tolerant event pipelines. Any tracker operating at scale will encounter retry scenarios, and the architecture must account for them from the start.

Multi-cloud deployment support: Supporting AWS, GCP, and Azure simultaneously requires careful abstraction of infrastructure dependencies. Containerized deployment via Docker, as used here, is the practical path to cloud portability.

DDoS and malformed request handling: At 2,000 requests per second with a 15 ms latency target, the tracker sits on the hot path of the ad serving stack. Protecting against floods of malformed requests is a non-negotiable security requirement, not an afterthought.

Outcomes and Tradeoffs

A tracker built to these specifications — Golang-based, plugin-extensible, multi-cloud deployable, and tested against the 99.999% availability and 2,000 RPS targets — provides a solid foundation for any AdTech or MarTech platform that needs reliable event collection.

The decision to favour Golang over Nginx + Lua reflects a pragmatic tradeoff: both perform comparably, but Go offers a more accessible developer experience, a richer ecosystem for building out integrations, and broader adoption in modern programmatic infrastructure. Teams evaluating the same choice today would likely reach the same conclusion, though Nginx + Lua remains a valid option in resource-constrained or operationally conservative environments.

The plugin architecture ensures the component doesn't need to be re-architected when new event types, storage backends, or downstream integrations are introduced — a critical property for any platform component intended to serve multiple use cases over time.