Embed Workers

Embed workers are dedicated compute resources that process queries originating from embedded analytics contexts, separate from the queries generated by internal users. They ensure that customer-facing dashboards and internal reporting operate on independent processing queues with independent resource limits.

The problem they solve is simple: a traffic spike on your customer-facing analytics must avoid slowing your internal dashboards, and a heavy internal ad-hoc query must avoid degrading the experience for your customers.

How embed workers function

When a query enters the analytics platform, the system identifies its origin. Queries from embedded contexts – customer-facing dashboards, partner portals, white-labeled analytics – route to one pool of workers. Queries from internal users – analysts running reports, executives viewing dashboards – route to another.

Each pool has its own concurrency limits, queue depth, and timeout thresholds. If the embedded pool is fully saturated, new embedded queries queue or degrade gracefully within that pool. Internal queries continue executing at full speed on their own pool. The isolation is at the processing layer rather than the data layer – both pools can query the same warehouse, but they compete for warehouse resources through separate channels.

Some implementations go further and assign different worker pools to different tenant tiers. A premium customer with an SLA guarantee gets queries routed to a high-priority pool. Free-tier tenants share a separate pool with lower concurrency limits. This tiered approach maps compute allocation directly to contract commitments.

Why separation matters

Without embed workers, all queries share a single processing queue. This creates several failure modes.

Customer-facing latency spikes. An analyst runs a complex exploratory query that consumes significant warehouse resources. Meanwhile, 200 customers are loading dashboards. Those dashboard queries wait behind the analyst's query in the same queue. Customers experience slow load times. They don't know why, and they don't care – they just see a slow product.

Internal analysis blocked by customer traffic. The reverse scenario. A product launch drives a surge in customer dashboard usage. The shared queue fills with embedded queries. Internal analysts can't get their reports to run. Decision-making stalls because the analytics platform is busy serving external traffic.

Unpredictable performance. Without separation, response times for both audiences depend on the other's behavior. Performance becomes a function of coincidence – what queries happen to be running when yours arrives. This makes SLA commitments impossible to guarantee.

Capacity planning

Sizing embed workers requires understanding four variables.

Peak concurrent users. Customer-facing analytics traffic follows patterns – spikes at market open, at the start of business hours, after email report deliveries. The embed worker pool must handle peak concurrency without queuing delays. Measure this from access logs rather than averages.

Query complexity. A dashboard with six simple aggregation queries and a dashboard with two complex joins across billion-row tables place very different demands on workers. Profile the actual queries your embedded dashboards generate rather than hypothetical ones.

Warehouse throughput. Embed workers are only as fast as the warehouse behind them. If the warehouse has its own concurrency limits – and most do – the worker pool size must align with what the warehouse can actually serve in parallel.

Growth rate. Customer count grows. Dashboard usage grows. Query complexity tends to grow as customers adopt more features. Plan capacity based on projected load at 6 and 12 months rather than current load.

SLA implications

Embed workers are what make SLA commitments credible. Without dedicated processing, a promise of "sub-2-second dashboard load times" depends on the assumption that internal users won't be running heavy queries at the same time. That assumption fails regularly.

With embed workers, the SLA applies to a known, isolated resource pool. You can measure its capacity, monitor its saturation, and scale it independently. This moves performance guarantees from aspirational to enforceable.

Embed workers also enable usage tracking through embed events – when queries are processed through a dedicated pool, logging query origin, execution time, and queue wait time becomes clean. That data feeds back into capacity planning and SLA reporting.

For any multi-tenant analytics deployment where customer-facing performance matters, embed workers are a requirement, beyond mere optimization.