Microservices

Chapter 11 of 27 · Part 5: Data management

Queries: API composition and CQRS

How to answer queries that need data from many services: compose at read time, or keep a CQRS view.

key idea / coordinatorservice / good pathin progress / watch outproblem / failureneutral

The query problem

Each service keeps its data private, so a query that spans services cannot use a JOIN.

One screen, four ownersfindOrder(orderId)Order Status screenOrder svcbasic info + statusKitchen svcticket, ready ETADelivery svccourier, ETA, placeAccounting svcpayment statusIn the monolith this was one SELECT with JOINs. Now each piece lives in a private database.
Two patterns for queries across servicesAPI compositionAsk each owner, join in memorySimple: try this firstWeak for big joins and filtersCQRSKeep a ready-made read copyFast, any query shapeMore moving parts, a little lagRule of thumb: use API composition when you can, CQRS when you must.

API composition

An API composer calls every service that owns part of the answer, then merges the results.

findOrder() with API compositionClientAPI composerOrder svcKitchen svcGET /order/{id}GET /orders/{id}GET /tickets?orderIdDelivery + Accounting called tooorderticketjoin on orderIdOrderDetailsCalls that do not depend on each other run in parallel, so the wait is the slowest call, not the sum.
Who plays the composer?ClientWeb appon the same fast LANAPI gatewayGatewayexternal API, mobileOwn serviceComposer svcinternal or complex
Benefits and drawbacks of compositionBenefitSimple and easy to understandNo extra databaseDrawbacksMore overheadmany calls, many DBsLower availability0.995^5 = 97.5%No consistencyorder CANCELLED, ticket notKeep it available by returning cached or partial data when one provider is down.

When composition is not enough

Some queries need big in-memory joins, or a kind of database the owner does not use.

Motivations for CQRSWhere composition failsfindOrderHistory()filter + sort + pagefindAvailable...geospatial queryWrong ownerowner should not run itOrder history filters by keyword, but Delivery and Accounting do not even store menu items.

When composition hurts

API composition is fine for small, centred data, such as an order detail page. It struggles when a screen filters and sorts across services. Take a product list: clothing under $50, in stock.

Why filters across services multiply callsProduct svc6 clothing itemsPricing svconly 2 under $50Product svcask for the next 6Repeatuntil the page fillsAdd a stock filter and sorting and it gets worse. Deep paging also eats the composer's memory.

The event-driven answer is a read model that handles the events of all the services and keeps a denormalized view ready for the query.

A read model built from event streamsevents on a durable logProduct servicePricing serviceInventory serviceRead modeldenormalized, searchableListing pageAdd a new filter or screen? Build another view from the same streams.

A product service can publish one whole document per product, keyed by id, on a compacted topic. The read model needs far less code than handling six partial events. If its data is damaged, or a new search need appears, consume the topic from the start with the same code, adding instances to go faster. There is no downtime and no special migration tool.

Compared with polling services. A read model that polls the services adds load, creates dependencies and widens the staleness window. Reacting to events avoids all three. The cost is that the view is eventually consistent, and it should hold little or no business logic, which stays with the owning service.

From Practical Event-Driven Microservices Architecture (ch. 4 and ch. 9).

CQRS

Command Query Responsibility Segregation: one model changes the data, a separate model answers queries.

CQRS: split writes from readsCommand sidecreate, update, delDomain eventsOrderCreated...Event handlersupdate the viewQuery sideread-only view DBThe view is a replica shaped for one set of queries, kept fresh by events.
Order History ServiceA query-only service subscribes to events from four services and answers findOrderHistory() and findOrder().Order svcOrder eventsKitchen svcTicket eventsDelivery svcDelivery eventsAccounting svcAccounting eventsOrder History svcevent handlersView DBpre-joinedAnswers findOrderHistory() and findOrder()with one fast read, no in-memory join.
A query-only service builds its own view from other services' events.
Benefits and drawbacks of CQRSBenefitsEfficient queries across servicesBest database for each viewQueries for event-sourced appsSimpler command and query codeDrawbacksMore code and datastoresReplication lag: views may be stale

Choosing a view database

Let the queries choose the store. NoSQL fits many views because they only need simple updates by id.

If the view needs…Use a…Examples
Look up a JSON object by idDocument or key-value storeMongoDB, DynamoDB, Redis
Query JSON objects by fieldsDocument storeMongoDB, DynamoDB
Text searchSearch engineElasticsearch
Graph queries (fraud)Graph databaseNeo4j
SQL reporting / BIRelational databasePostgreSQL, MySQL

Keeping the view correct

Events can arrive twice and the view always trails the writes a little.

Duplicate events must not corrupt the viewBrokerEvent handlerViewDeliveryPickedUpstatus = PICKED_UPDeliveryDeliveredstatus = DELIVEREDPickedUp (again!)seen id: skip itRecord the highest event id per aggregate and ignore anything older.
Designing the view's data access moduleData access concernAnswerConcurrent updatesoptimistic or pessimistic lockDuplicate eventsidempotent handlers, track idsEventual consistencyreturn a token; client waitsNew or changed viewrebuild from archived events
Handling replication lagClientCommand svcQuery svcrevise orderOK + token (event id)GET order + tokenview not caught up yetretry laterOr the client simply updates its own local model from the command's response.

New views and a real example

Plan how to build a view that did not exist before.

Building a new view incrementallyArchived eventse.g. in S3Batch jobe.g. SparkSnapshotview at time TNewer eventsapplied on topBrokers do not keep events forever, so archive them and build views from snapshot + new events.
Case study: the view on DynamoDBEvent handlerOrderCreatedDAOOrderHistoryDaoUpdateItemconditionalDynamoDBone item per orderA secondary index on consumerId + creation time sorts each consumer's history.

Caching reads

Another way to cheapen queries is to cache. Newman gives three reasons.

Why cachePerformanceSkip network hopsLess load downstreamScaleReads hit copiesOrigin less contendedRobustnessServe if origin is downKeep stale, do not evictCaching for robustness favours availability over consistency.
Where to cache the top-ten best sellersClientSales serviceCatalog serviceRequest cache: top-10 resultClient-side cache: namesServer-side cacheSales needs album names from CatalogThe request cache is fastest but helps only that one request.
InvalidationIdeaWatch out
Time to live (TTL)Entry is valid for a set timeBlunt: data can be stale for the whole TTL
Conditional GET (ETag)Ask for the resource only if changed; 304 if notStill a round trip, but saves rebuilding it
NotificationOrigin sends an event when data changesNeeds a broker. Send heartbeats so silence is noticed
Write-throughUpdate cache and origin togetherEasy only on the server side
Write-behindUpdate cache first, origin laterData loss risk; unclear source of truth. Rare
The golden rule. Do not cache in too many places. Two nested 1-minute TTL caches can serve data up to 2 minutes old while the client believes it is 1. The ideal number of caches is zero; add each one as a deliberate optimization.

From Building Microservices (ch. 13, Caching).

Recap

Quick check

If each of five services is 99.5% available, how available is findOrder() with API composition?
Show answer

About 97.5%, because all five (composer plus four providers) must be up: 0.995^5 ≈ 0.975. Cached or partial answers help.

Sales caches album names with a 1-minute TTL, and Catalog caches them server-side with a 1-minute TTL. How stale can a name be in Sales?
Show answer

Up to about 2 minutes, because Sales may refill its cache from a Catalog entry that is already nearly 1 minute old. Nested caches add up, so cache in as few places as possible.