Kafka partitions: ordering has a boundary

I had reduced Kafka’s ordering behaviour to one confident sentence:

Kafka keeps events in order.

That sounded useful.

It was also missing the boundary around the promise.

Suppose an order topic contains:

• OrderCreated
• StockReserved
• PaymentAuthorised
• OrderConfirmed

I might imagine one topic-wide sequence:

1. OrderCreated
2. StockReserved
3. PaymentAuthorised
4. OrderConfirmed

But a Kafka topic is divided into partitions.

ORDER EVENTS TOPIC
│
├── partition 0
├── partition 1
└── partition 2

Each partition is its own ordered log.

Kafka preserves order within each partition.

It does not create one total order across all three.

That changes the claim:

Kafka orders records within a partition. The partitioning strategy decides which records receive that shared ordering guarantee.

The partition is not merely a storage bucket.

It is the boundary around ordering and one of the boundaries around parallel processing.

A topic contains several ordered logs

A Kafka topic is a named stream of records.

For example:

bfstore.order.events.v1

The topic may have several partitions:

bfstore.order.events.v1
│
├── partition 0
├── partition 1
├── partition 2
└── partition 3

Records are appended to one of those partitions.

PRODUCER
   │
   ├──► partition 0
   ├──► partition 1
   ├──► partition 2
   └──► partition 3

Within a partition, the records form an ordered sequence.

PARTITION 2

offset 18  OrderCreated
offset 19  OrderConfirmed
offset 20  OrderCancelled

A later appended record receives a later offset in that partition.

The partition behaves like an append-only log:

oldest                                  newest
  │                                        │
  ▼                                        ▼

offset 18 ── offset 19 ── offset 20 ── offset 21

A topic containing four partitions is therefore not one log with four shelves.

It is four ordered logs grouped under one topic name.

Offsets belong to partitions

An offset identifies a record’s position within one partition.

For example:

The full position is:

topic + partition + offset

Offset 19 in partition 2 is unrelated to offset 19 in partition 3.

PARTITION 2               PARTITION 3

offset 19                 offset 19
OrderConfirmed            OrderCancelled

Neither record is globally “the nineteenth event” in the topic.

Each is the nineteenth position after offset zero within its own partition.

This means offsets cannot be compared across partitions to create a topic-wide chronology.

partition 0, offset 200

is not automatically earlier or later than:

partition 1, offset 170

The numbers describe different logs.

There is no global topic order

Suppose two records are written to different partitions:

Kafka does not provide one authoritative statement about which event came first across the topic.

A consumer might fetch partition 1 before partition 0.

Another consumer group might process partition 0 first.

Network and scheduling differences can change when each record is observed.

Both consumers can be respecting Kafka’s partition ordering.

The records live in different ordered histories.

The question should therefore not be:

Does this topic need ordering?

It should be:

Which records must be ordered relative to one another?

That answer helps choose the partition key.

The key defines the ordering scope

A Kafka record can contain:

• key
• value
• headers
• timestamp

For example:

When no partition is supplied explicitly, Kafka’s default producer partitioner uses the record key to select a partition.

Conceptually:

partition = hash(key) mapped into available partitions

Records with the same serialised key are routed to the same partition when three things remain compatible:

• producers serialise the key into the same bytes;
• producers use compatible partitioning logic;
• the topic’s partition count remains stable.

A custom partitioner, a changed key serializer, or a different partition count can change the mapping even when the application-level identifier still looks the same.

order-5001
    │
    ├── OrderCreated
    ├── StockReserved
    ├── PaymentAuthorised
    └── OrderConfirmed

becomes:

PARTITION 2

offset 18  OrderCreated
offset 19  StockReserved
offset 20  PaymentAuthorised
offset 21  OrderConfirmed

The key gives all events for order-5001 one ordered lane.

Another order may use another partition.

order-9002
    │
    ▼
PARTITION 0

The system preserves order per order while allowing different orders to be processed concurrently.

The partition key defines the ordering scope

Suppose the topic carries order events.

Possible keys include:

• order_id
• customer_id
• payment_id
• no key

Each creates a different ordering boundary.

Key by order ID

Key: order-5001

All events for one order share a partition.

• OrderCreated
• OrderConfirmed
• OrderCancelled

can be observed in partition order for that order.

Different orders may be processed concurrently.

This is often a sensible choice for an order lifecycle.

Key by customer ID

Key: customer-72

All events for one customer share a partition.

This preserves order across that customer’s orders.

It can also place a busy customer’s complete workload into one lane.

The ordering scope is wider.

Key by event ID

Key: event-91a3

Every event has a different key.

Related order events may be distributed across partitions.

The key provides little useful ordering for the order lifecycle.

No key

Without a key or explicit partition, the producer may distribute records for batching and load distribution.

Two events for one order are not guaranteed to reach the same partition.

OrderCreated order-5001
    │
    └──► partition 0


OrderConfirmed order-5001
    │
    └──► partition 3

The topic can carry both events.

It no longer provides per-order ordering.

The key is therefore not a decorative routing field.

It defines which records are forced into one sequence.

Choose the entity whose transitions must remain ordered

Suppose an event stream describes changes to products.

• ProductCreated
• ProductPriceChanged
• ProductPublished
• ProductDiscontinued

If consumers need the history of each product in order, the key should usually identify the product:

Product Id: chair-42

Then:

PARTITION 1

chair-42 ProductCreated
chair-42 ProductPriceChanged
chair-42 ProductPublished
chair-42 ProductDiscontinued

A different product may land elsewhere:

PARTITION 3

table-17 ProductCreated
table-17 ProductPublished

The system does not need to decide whether:

chair-42 ProductPublished

occurred before:

table-17 ProductCreated

unless the business has a real cross-product invariant requiring that comparison.

Most domains need ordering within one entity or aggregate rather than across the entire organisation.

Business ordering and Kafka ordering are different

Suppose the producer writes:

• OrderCreated
• OrderConfirmed

to one partition in that order.

Kafka can preserve that log order.

It cannot prove that confirming the order was valid.

PARTITION 2

offset 18  OrderCreated
offset 19  OrderConfirmed

The application still needs rules such as:

• order must exist
• payment must be accepted
• stock must be reserved
• cancelled order cannot be confirmed

Kafka ordering says:

The consumer sees record 18 before record 19.

It does not say:

Record 19 represents a valid domain transition.

A beautifully ordered stream of incorrect events remains incorrect.

The broker preserves sequence.

The application preserves meaning.

The partition log records arrival, not universal time

Records may contain timestamps such as:

14:30:01.100

14:30:01.080

It is tempting to sort events by time and reconstruct one global order.

That introduces several problems:

• producer clocks may differ
• network delay may differ
• events may be published late
• several events may share one timestamp
• clock adjustments can move time backwards
• occurrence time may differ from publication time

Suppose:

The smaller timestamp does not prove that service B’s business event happened first in a globally meaningful sense.

Kafka’s partition log provides a definite order within the partition.

Application timestamps provide useful time information.

They should not be mistaken for one universal sequencing authority.

Several producers can write to one partition

A partition may receive records from more than one producer.

PRODUCER A ──┐
             ├──► PARTITION 2
PRODUCER B ──┘

The broker appends records into one partition log.

Consumers read that final log order.

However, the application should not assume that independently running producers share one causal clock.

Suppose producer A begins sending one record before producer B begins another.

Network delay may cause B’s record to reach the broker first.

PRODUCER A

send EventA
    │
    │ delayed
    ▼


PRODUCER B

send EventB
    │
    └────────────► broker first

The partition may contain:

offset 40  EventB
offset 41  EventA

Kafka has created one log order.

It has not reconstructed the producers’ intended real-world chronology.

Where one entity requires one authoritative sequence, it is often safer for one owning service to control that entity’s event publication.

Producer configuration can preserve or disturb order

A producer sends records in batches.

Suppose two batches target the same partition.

The producer sends both while requests are in flight.

Batch A fails temporarily.

Batch B succeeds.

If the producer retries batch A without idempotent ordering protection, the partition could receive:

records 11 to 20

then

records 1 to 10

The application produced A before B.

The retry caused B to be appended first.

Kafka producer settings therefore matter to the ordering guarantee.

When this article was first published in 2022, producer idempotence had to be considered explicitly alongside acknowledgements, retries and the number of in-flight requests.

In current Kafka clients, idempotence is enabled by default when the producer configuration does not conflict with it. The effective configuration still matters: idempotence requires acks=all, retries greater than zero, and no more than five in-flight requests per connection. If idempotence is disabled while several requests are in flight, a failed earlier batch can be retried after a later batch has already been appended.

The durable lesson is to verify the producer’s effective configuration rather than assume that calling send() establishes the intended order.

The safe conclusion is not:

Kafka always orders whatever I call send().

It is:

Ordering also depends on how the producer handles concurrent requests, acknowledgement and retries.

Idempotent production protects the partition log

An idempotent Kafka producer associates producer identity and sequence information with its batches.

This allows the broker to recognise retries of data it has already appended.

Conceptually:

If the same batch is retried, the broker can avoid appending another copy.

FIRST DELIVERY

sequence 41
    │
    └── append


RETRY

sequence 41
    │
    └── already handled

Idempotent production protects Kafka records within the producer session and supported configuration.

It does not make the consumer’s business side effects idempotent.

A consumer may still:

• update its database
• send an email
• call another service

and repeat that work after redelivery or restart.

Producer idempotence protects the log write.

Consumer idempotence protects the consumer’s effect.

Kafka transactions protect a narrower consume-transform-produce boundary

Kafka transactions can extend this protection when an application consumes records, produces new Kafka records, and commits the consumed offsets as one Kafka transaction.

Within that boundary, either the output records and offsets become visible together or neither does.

That is useful for a Kafka-to-Kafka pipeline such as:

consume OrderConfirmed
        │
        ▼
produce FulfilmentRequested
        │
        ▼
commit consumed offset

It still does not atomically include an ordinary database, an email provider, a payment processor, or an arbitrary RPC.

Kafka transactions strengthen one defined boundary. They do not turn every effect around Kafka into one universal transaction.

Consumer groups turn partitions into parallel lanes

Consumers can join a consumer group.

Consumer Group: order-projection

Kafka assigns topic partitions among the group’s members.

For a topic with four partitions and two consumers:

Within one consumer group, a partition is assigned to one consumer member at a time.

That allows the consumer to read the partition in order while several partitions are processed in parallel.

partition 0 ──► consumer A

partition 1 ──► consumer A

partition 2 ──► consumer B

partition 3 ──► consumer B

Kafka’s parallelism comes from partitions.

The consumer group does not split one partition among several consumers simultaneously for ordinary consumption.

Current note: share groups choose different semantics

This article describes the traditional Kafka consumer-group model.

Kafka 4.2 made share groups production-ready. A share group can allow several consumers to cooperate on records from the same partitions using per-record acknowledgement and delivery-attempt tracking.

That model is designed for queue-like work distribution rather than strict partition-order processing. Choosing it deliberately trades away the traditional relationship between one assigned partition owner and ordered consumption.

Partition count limits active consumer parallelism

Suppose a topic has:

3 partitions

and the consumer group has:

5 consumer instances

Only three instances can receive partition assignments.

Adding more consumers does not create more partition-level parallelism once every partition already has an owner.

This gives the partition count several jobs:

• storage distribution
• producer throughput
• consumer parallelism
• ordering scope

Increasing the partition count can improve parallelism.

It also changes the partitioning surface and can affect keyed ordering.

Partition count is therefore an architectural decision, not simply a dial labelled “faster”.

One consumer may own several partitions

Suppose a group has one consumer and four partitions.

CONSUMER A
│
├── partition 0
├── partition 1
├── partition 2
└── partition 3

The consumer receives records from all four logs.

It should preserve order within each partition.

It does not need to create one global sequence across them.

Conceptually:

partition 0:
    A1 → A2 → A3


partition 1:
    B1 → B2 → B3

A valid processing sequence could be:

A1
B1
B2
A2
A3
B3

provided:

A1 precedes A2 precedes A3

and

B1 precedes B2 precedes B3

The records from different partitions may interleave.

That is expected.

Kafka can deliver in order while the application completes out of order

Suppose one consumer receives:

offset 18  ProductCreated
offset 19  ProductPublished
offset 20  ProductDiscontinued

The consumer then starts one goroutine for each record.

offset 18 ──► worker A
offset 19 ──► worker B
offset 20 ──► worker C

Worker C finishes first.

completion order:

offset 20
offset 18
offset 19

Kafka delivered the records in partition order.

The application processed them concurrently and completed them out of order.

This distinction matters:

Kafka controls the first.

Consumer design affects the other two. Partition order controls the sequence in which records become available to the consumer; it does not automatically serialise database commits, outgoing requests, callbacks or other asynchronous effects.

If business effects must follow partition order, the consumer must process that partition accordingly.

Parallelise across partitions before parallelising within one

A natural Kafka processing shape is:

partition 0 ──► ordered worker

partition 1 ──► ordered worker

partition 2 ──► ordered worker

Each partition retains sequential processing.

Several partitions run concurrently.

          ┌──► partition 0 worker
CONSUMER ─┼──► partition 1 worker
          └──► partition 2 worker

This preserves the ordering relationship encoded by the partition key.

Parallel processing inside one partition can be safe where records are independent.

But if they are independent, the partition key may be grouping them more tightly than necessary.

The application should not casually destroy the ordering guarantee it paid for through partitioning.

Consumer progress is not business completion

A consumer tracks how far it has progressed through each partition.

Suppose the consumer has successfully processed through offset 20.

Kafka’s committed offset normally identifies the next record to consume, not the last record successfully processed. In this example, resumption begins at offset 21.

After restart or reassignment, consumption can resume from the appropriate stored position according to the consumer’s configuration and commit behaviour.

The offset does not record every business effect automatically.

Suppose the consumer does this:

1. Commit the offset.
2. Update the database.

If it crashes after step one, Kafka may consider the record handled while the database update never occurred.

If the consumer instead does:

1. Update the database.
2. Commit the offset.

and crashes between the steps, the record may be delivered again.

That produces duplicate processing risk.

The consumer must choose how its own state and Kafka progress are coordinated.

Redelivery does not mean the log was reordered

Suppose a consumer processes:

offset 18

but crashes before its offset progress is committed.

Another consumer later receives the partition and processes offset 18 again.

The partition log remains:

offset 18
offset 19
offset 20

Kafka did not reorder the log.

The application observed a repeated attempt at offset 18.

Ordering and duplicate delivery are separate properties.

A consumer may receive:

18, 18, 19, 20

across recovery attempts while the underlying log still contains one record at offset 18.

The consumer needs idempotent handling where repeating the local effect would be harmful.

Rebalancing changes partition ownership

A consumer group can rebalance when:

• a consumer joins
• a consumer leaves
• a consumer fails
• topic partitioning changes

Partition ownership moves between group members.

BEFORE

consumer A:
    partitions 0 and 1

consumer B:
    partitions 2 and 3


AFTER

consumer A:
    partition 0

consumer B:
    partition 2

consumer C:
    partitions 1 and 3

The new owner continues from the group’s recorded position.

The application needs to handle work that was in progress during revocation.

Once a partition is revoked, the old owner should stop starting new work for it and safely finish, cancel, or record any in-flight work before the new owner proceeds. If consumer A is still processing partition 1 while ownership moves to consumer C, careless local concurrency can create overlapping work.

Consumer lifecycle, worker ownership and offset management need one coordinated design.

The broker preserves partition order.

The application must preserve safe handover.

Slow processing can threaten consumer-group membership

A consumer must continue interacting with the group and poll within its configured processing expectations.

If record processing blocks the consumer loop for too long, the group may treat the consumer as unable to make progress and reassign its partitions.

This can cause:

• partition revocation;
• consumer rebalance;
• record redelivery;
• duplicate local work.

Long-running work may therefore need:

• bounded batch sizes
• separate worker management
• careful pause and resume
• appropriate poll configuration
• asynchronous job handoff

The correct response is not simply to increase every timeout until the consumer is allowed to disappear for an afternoon.

The processing architecture should make ownership and progress visible.

When processing is delegated to worker threads, the poll loop can pause the affected partition, continue polling to maintain group membership, and resume the partition only after the delegated work has completed and its progress can be committed safely.

That arrangement needs bounded queues and explicit ownership. Otherwise polling can remain healthy while an unbounded worker backlog quietly grows.

Partitioning creates capacity and evolution trade-offs

Suppose most order traffic is spread evenly.

Then one unusually active customer generates a large proportion of events.

If the topic is keyed by customer ID:

customer-72
    │
    ▼
partition 3

all of that customer’s events use partition 3.

The partition becomes hot.

Other partitions and consumers may remain underused.

The ordering guarantee prevents the hot key from being divided across partitions without changing the key or processing model.

This exposes a trade:

wider ordering scope
    │
    └── less parallelism for that key


narrower ordering scope
    │
    └── more parallelism,
        weaker shared ordering

The key should represent the smallest business scope that genuinely requires one ordered sequence.

Adding a random suffix or otherwise salting the key can distribute the load, but it also weakens or relocates the original ordering guarantee. The application then needs another mechanism to combine, sequence, or reconcile the divided effects.

Hot-key mitigation is therefore not a free partitioning trick. It is a redesign of the ordering boundary.

Ordering can become a throughput constraint

Suppose the business demands:

Every event in the company must be processed in one exact order.

One topic with one partition could provide a single partition log.

ALL EVENTS
    │
    ▼
ONE PARTITION
    │
    ▼
ONE CONSUMER PER GROUP

That gives one total Kafka order for the topic.

It also limits partition-level parallelism.

All producers target one partition.

Each consumer group has only one active partition owner.

One slow record can delay every record behind it.

slow event
    │
    ▼
everything later waits

Global ordering is expensive because it creates a global queue.

Most systems do not need it.

They need:

• per order ordering
• per account ordering
• per product ordering
• per payment attempt ordering

Those narrower scopes allow many independent sequences to run concurrently.

Increasing partitions can change keyed routing

Suppose a topic has four partitions.

The default key-based partitioning maps:

order-5001
    │
    ▼
partition 2

The topic is later expanded to eight partitions.

The existing records remain in their original partitions.

Kafka does not redistribute them automatically.

The same key may now map differently under a partitioning calculation that depends on the partition count.

BEFORE

order-5001 → partition 2


AFTER PARTITION EXPANSION

order-5001 → partition 6

The order’s history is now split.

There is no single partition containing the complete ordered history.

Apache Kafka’s administrative API explicitly warns that increasing partitions for keyed topics can affect partition logic and ordering.

Partition expansion should therefore be planned before treating the existing key-to-partition mapping as permanent.

Existing data is not automatically repartitioned

Adding partitions gives new records more possible destinations.

It does not move old records into a newly balanced arrangement.

BEFORE

partition 0:
    many records

partition 1:
    many records


AFTER ADDING PARTITIONS

partition 0:
    same old records

partition 1:
    same old records

partition 2:
    empty initially

partition 3:
    empty initially

New records begin filling the new partitions according to producer routing.

Historical data remains where it was written.

This can matter for:

• key locality
• stream joins
• state rebuilding
• ordering assumptions
• consumer load

Topic partition count is easier to increase than to treat as though it had always been larger.

Capacity planning should happen before a topic becomes a long-lived contract.

Changing the key can split the history too

Suppose order events initially use:

Key: customer_id

Later, the producer changes to:

Key: order_id

Events for an existing order may move to another partition.

Even if the topic’s partition count remains unchanged, the entity’s history can split.

Changing the key serializer can have the same effect: an identifier that looks unchanged in application code may produce different bytes and therefore a different partition.

A key or serializer change is therefore a message-contract and operational migration.

It may require:

• a new topic
• a controlled cutover
• consumer support for both arrangements
• rebuilding derived state
• explicit versioning

The key deserves the same design discipline as the event payload.

A migration also needs an overlap policy. The design should answer:

• Is there a cutover timestamp, schema version, or producer epoch?
• Can consumers read both topics or key schemes during the transition?
• How are duplicate events across the old and new paths recognised?
• Which path is authoritative if both publish?
• When can the old path be retired safely?

For a significant change, a new topic and an explicit cutover are often clearer than silently changing the key beneath existing consumers.

One workflow may cross several ordering boundaries

Suppose order publishes:

OrderConfirmed

to:

bfstore.order.events.v1

and payment publishes:

PaymentAuthorised

to:

bfstore.payment.events.v1

These are different topics and partitions.

A consumer subscribing to both may observe:

OrderConfirmed

then

PaymentAuthorised

even if payment authorisation caused the order confirmation.

Network delay, publication delay and separate partition consumption can affect observation order.

Correlation and causation metadata can express the relationship:

event:
    OrderConfirmed

causation_id:
    PaymentAuthorised event ID

The consumer may also need a state machine that tolerates events arriving in either order.

Kafka does not create one distributed transaction log across independently produced topics merely because the records share a correlation ID.

Causal order may need explicit modelling

Suppose a workflow expects:

PaymentAuthorised
      │
      ▼
OrderConfirmed

A projection receives OrderConfirmed first.

It can:

• reject the event
• buffer it temporarily
• record order confirmation independently
• query authoritative state
• wait for the causal event
• process using an entity version

The right choice depends on what the projection needs.

A robust consumer should not always assume that every related event arrives in the ideal storytelling order.

Useful fields may include:

• entity version
• correlation ID
• causation ID
• operation ID

These help the consumer reason about relationships that partition order alone does not cover.

Entity versions can protect derived state

Suppose product events contain:

Product Version: 18

then:

Product Version: 19

A consumer has already applied version 19.

It later receives a delayed or replayed version 18.

It can reject the older update.

18 < 19
    │
    ▼
do not overwrite current state

The version provides domain-level sequencing for one entity.

Kafka partition order may already deliver the events correctly under normal operation.

The version remains useful for:

• replay
• repair
• cross-topic processing
• migrations
• defence against stale updates

The owner must define what the version orders.

A number without scope is simply a smaller mystery.

The harder case is a gap:

The version reveals that something is missing, but it does not decide the repair. Depending on the projection, the consumer may:

• pause and wait for version 19;
• fetch an authoritative snapshot;
• rebuild from an earlier offset;
• apply version 20 because only the latest state matters;
• quarantine the event and trigger reconciliation.

Version numbers detect stale or missing transitions. The consumer contract must define what happens next.

Partition order does not replace idempotency

Suppose the partition contains:

offset 40  OrderConfirmed

The consumer processes it and sends one confirmation email.

Before committing its offset, the consumer crashes.

The record is delivered again.

offset 40  OrderConfirmed

The ordering is perfect.

The customer receives two emails.

Ordering answered:

Which record comes before which other record?

It did not answer:

Has this consumer already applied this record?

The consumer may use:

• event ID
• order ID plus notification type
• unique database constraint
• processed-message table

to protect its effect.

Ordering and idempotency solve different problems.

A system often needs both.

Partition order does not replace transaction boundaries

Suppose the consumer handles OrderConfirmed:

1. Update the order projection.
2. Create the notification job.
3. Commit the offset.

If step two fails after step one commits, the local state is partial.

The fact that the event appeared in the correct partition position does not make those local writes atomic.

The consumer may need one local database transaction:

BEGIN
  │
  ├── update order projection
  ├── create notification job
  └── mark event processed
COMMIT

Then offset progress can be coordinated according to the chosen delivery model.

Kafka orders the input.

The consumer still needs a reliable local state transition.

Where the complete pipeline remains inside Kafka, a Kafka transaction may coordinate produced records with consumed offsets. Where the effect crosses into another system, the consumer still needs a local transaction, an inbox or deduplication record, an outbox, or another explicit recovery design.

The key may differ by topic purpose

Suppose bfstore has an order-event topic.

Key: order_id

That preserves each order lifecycle.

Inventory may have a stock-change topic.

Possible key:

product_id

That preserves stock changes for each product.

Payment may use:

payment_attempt_id

to preserve one payment attempt’s lifecycle.

The same business workflow therefore crosses several ordering boundaries.

No one key is correct for every topic.

The partition key should follow the invariant and ownership represented by that stream.

One event may participate in several projections

Consider:

StockReserved

Inventory may publish it keyed by:

product_id

because inventory needs stock changes for one product to remain ordered.

Order wants to process reservations by:

order_id

The source topic’s partitioning does not match the order projection’s preferred grouping.

A stream-processing application may repartition records using a new key.

SOURCE TOPIC

key = product_id
      │
      ▼
REPARTITION
      │
      ▼
DERIVED TOPIC

key = order_id

Repartitioning writes records into another partitioned topic.

That creates another ordering boundary.

Order within the derived partition follows the order in which records are written to that repartitioned log.

The design should understand which stream owns which grouping instead of assuming one original key satisfies every consumer.

Partition selection should follow the invariant

A useful question is:

Which pieces of work must not be handled concurrently?

For order events:

two state transitions for one order

may need serialisation.

For inventory:

stock changes for one product

may need serialisation.

For customer email preferences:

updates to one customer’s preferences

may need serialisation.

The partition key can group those operations.

But Kafka partitions alone do not create a full distributed lock.

Other write paths may exist:

• synchronous RPC
• database administration
• another topic
• batch import

The authoritative owner must still enforce its invariants in its own state.

Partitioning can simplify ordered processing.

It should not become the only thing preventing invalid concurrent updates.

Design framework and bfstore example

The important word in almost every ordering claim is:

within

A possible bfstore order topic

Suppose bfstore publishes:

Topic: bfstore.order.events.v1

The events include:

• OrderCreated
• OrderConfirmed
• OrderCancelled
• OrderDispatchRequested

The partition key is:

order_id

Example records:

Kafka routes the stable key to one partition.

PARTITION 2

offset 90  order-5001 OrderCreated
offset 91  order-5001 OrderConfirmed
offset 92  order-5001 OrderDispatchRequested

Another order can be processed elsewhere.

PARTITION 0

offset 44  order-9002 OrderCreated
offset 45  order-9002 OrderCancelled

The fulfilment consumer group may have three instances:

FULFILMENT GROUP
│
├── consumer A:
│      partition 0
│
├── consumer B:
│      partition 1
│
└── consumer C:
       partition 2

Each consumer processes its assigned partitions in order.

The design still needs:

• idempotent event handling;
• safe offset management;
• schema compatibility;
• entity-version checks;
• failed-event recovery;
• rebalance handling.

The partition provides one useful guarantee.

It does not operate the entire consumer.

A partitioning checklist

If the only answer is “Kafka handles ordering”, the boundary has not yet been designed.

The mental model I am keeping

My old model was:

TOPIC

one ordered stream of events

The stronger model is:

                              TOPIC
                                │
              ┌─────────────────┼─────────────────┐
              │                 │                 │
              ▼                 ▼                 ▼
         PARTITION 0       PARTITION 1       PARTITION 2
              │                 │                 │
              ▼                 ▼                 ▼
        ordered log       ordered log       ordered log

The record key chooses an ordering lane:

entity key
    │
    ▼
partition
    │
    ▼
ordered entity events

The partitions also define consumer-group parallelism:

partition 0 ──► consumer A

partition 1 ──► consumer B

partition 2 ──► consumer C

This creates a deliberate trade:

So when somebody says:

Kafka guarantees order,

I should ask:

• Within which partition?
• Chosen by which key?
• Produced by which owner?
• Under which producer settings?
• Processed with what consumer concurrency?
• Preserved across which topic changes?
• Required by which business invariant?

Kafka provides a powerful ordering primitive.

It does not provide one universal timeline.

The partition is where the promise begins and where it ends.

Everything that must stay in the same story needs to enter through the same door.

References and further reading

The article was originally published in August 2022. The following references use current Apache Kafka documentation where the underlying behaviour remains relevant, with an explicit current note for share groups.

Topics, partitions, offsets and consumer position

Apache Kafka documentation
Introduces topics, partitions, records, keys, consumer groups and Kafka’s partitioned-log model.

Apache Kafka design: consumer position
Explains that a traditional consumer group’s position is tracked per partition as the offset of the next record to consume, and describes Kafka-to-Kafka transactional processing.

Producer partitioning, retries and transactions

Apache Kafka producer configuration
Documents key handling, producer idempotence, acknowledgements, retries, in-flight requests and the reordering risk when idempotence is disabled.

Apache Kafka 4.3 ProducerConfig API
Lists current producer configuration fields, including serializers, partitioner behaviour, acknowledgements, idempotence and transactional settings.

Consumers, rebalancing and share groups

Apache Kafka consumer configuration
Documents consumer-group assignment, polling, committed offsets and consumer processing configuration.

Apache Kafka 4.2 upgrade guide
Describes production-ready share groups as an alternative, queue-like consumption model that does not preserve the traditional one-partition-owner ordering relationship.

Partition-count and key migrations

Apache Kafka basic operations
Explains partition administration and warns that increasing a keyed topic’s partition count can change routing for new records without redistributing existing data.

Kafka Streams repartitioning

Apache Kafka Streams documentation
Documents stream processing, grouping, repartitioning and stateful operations that depend on compatible record keys.