All notes

The platform should remove toil, not understanding

A developer platform should automate repetitive work and contain unnecessary infrastructure complexity without hiding the behaviour, risks and ownership engineers need to operate their services safely.

about 12 minutes min read

A developer creates a new service through the internal platform.

They provide:

name: review-service
runtime: go
protocol: grpc
exposure: internal

The platform supplies:

  • a repository
  • a build pipeline
  • a container image
  • workload identity
  • a Kubernetes deployment
  • network policy
  • telemetry
  • deployment automation

This is useful abstraction.

The developer does not need to recreate the organisation’s standard infrastructure by hand.

Then the service fails in production.

The developer asks:

Where does it run?

Which identity does it use?

How does traffic reach it?

Where are its logs?

What happens when a Pod is replaced?

Who retries failed messages?

The platform’s answer is:

Those details are managed by the platform.

Open a support request
if you need help diagnosing the failure.

That response reveals a problem.

The platform has removed the work required to create the infrastructure, but it has also removed the knowledge required to reason about the service.

A platform should remove toil, not understanding.

A good abstraction reduces the machinery a developer must operate directly.

It still provides enough visibility for them to reason about:

  • behaviour
  • limits
  • failure
  • security
  • dependencies
  • cost
  • recovery
  • ownership

The aim is not to make infrastructure disappear.

It is to place construction work behind stable interfaces without pretending the consequences have vanished with it.

Remove repeated work, not engineering judgement

Not every difficult task is toil.

Toil is usually operational work that:

  • must be repeated frequently
  • follows a predictable process
  • provides little lasting improvement
  • grows with the number of services
  • can often be automated

Examples include:

copying CI workflows

creating similar IAM roles

registering standard dashboards

manually creating deployment manifests

These tasks may require care, but they do not require a new design decision every time.

A platform can encode the organisation’s supported answer once and reuse it.

REPEATED TASK
      │
      ▼
PLATFORM AUTOMATION
      │
      ▼
CONSISTENT OUTCOME

The developer expresses the requirement.

The platform performs the routine work.

Understanding is different.

An engineer does not need to configure every load balancer field to understand how traffic reaches a service.

They do not need to create every IAM policy manually to understand which resources the service may access.

They do not need to operate the telemetry backend to understand where logs, metrics and traces go.

The platform should reduce manual operation while preserving the ability to reason about behaviour, boundaries and consequences.

A developer may not need to write the workload-identity policy.

They should still understand:

This service receives temporary cloud credentials.

Those credentials permit access
only to approved resources.

The credentials expire.

The application must handle
credential acquisition and renewal failure.

The implementation belongs to the platform.

The behavioural contract belongs to everyone depending on it.

Hide construction, expose behaviour

Suppose the platform exposes:

database:
  profile: transactional

Behind that request, it creates:

  • a MySQL instance
  • private networking
  • encrypted storage
  • credentials
  • backups
  • monitoring

The developer does not need to understand every provider argument.

They do need to know the properties that influence application design and operation:

  • whether the database is shared or dedicated
  • how connections are limited
  • what durability and availability are promised
  • how credentials rotate
  • how backup and restoration work
  • who initiates recovery
  • which recovery targets are supported
  • what the service costs

A useful capability should therefore expose:

behaviour

guarantees and limits

failure and recovery

ownership

An abstraction should hide construction work, not properties that affect the service outcome.

This distinction also changes the questions developers need to answer.

Without a platform, they may ask:

Which subnet identifier should I use?

Which security-group rule should I create?

Which cloud annotation enables private access?

With a platform, those questions can become:

Should this service be public or internal?

Which dependencies may it reach?

Which data classification applies?

Which availability profile is required?

The second group is closer to product and operating intent.

The platform has removed repeated implementation decisions while preserving those that change behaviour, risk or cost.

The interface should teach the operating model

A platform API communicates how the organisation expects software to be operated.

Consider:

spec:
  exposure: internal

  availability: standard

  data:
    classification: confidential

These fields show that:

  • exposure is a deliberate decision
  • availability has supported levels
  • data classification affects the resulting infrastructure

The documentation can then explain the consequences.

exposure: internal

creates private service discovery

rejects public ingress

applies internal network policy

The interface becomes a map of meaningful platform concepts.

A weaker interface may conceal those concepts:

spec:
  useDefaults: true

That is easy to submit, but it teaches very little.

Defaults should remain easy to inherit, but their purpose and consequences should remain visible:

GUARDRAIL

no permanent cloud credentials


PLATFORM IMPLEMENTATION

workload identity


OPERATING CONSEQUENCE

credentials are temporary,
scoped and tied to the workload

The platform automates compliance.

The explanation builds judgement that engineers can apply elsewhere.

Documentation should follow the same principle. It does not need every provider field. It should explain the capability, defaults, meaningful choices, limits, failure modes, recovery and ownership.

The user does not need every implementation detail. They need a usable mental model.

Failure and status should reveal system boundaries

A platform can appear wonderfully simple while everything works.

The real test begins when the abstraction fails.

Suppose the review service cannot connect to its database.

An unhelpful platform reports:

Provisioning failed.

A more useful platform exposes:

Database:
    ready

Credential:
    ready

Network access:
    denied

Reason:
    review-service is not authorised
    to reach the database endpoint

The second message reveals the system boundary.

The user can identify whether the failure belongs to:

  • the application
  • platform configuration
  • network policy
  • identity
  • the database

Self-service without understandable failure becomes ticket automation.

The user can initiate the work but cannot participate in recovering it.

Status should also show progress at the level users need.

The platform does not need to expose every controller event. It should reveal the dependency chain:

Application:
    review-service

Identity:
    ready

Database:
    provisioning

Deployment:
    waiting for database

Telemetry:
    pending

A single spinner hides work.

Structured status explains what is happening, what is blocked and where the user should look next.

Understanding should be progressive

Not every user needs the same depth immediately.

A platform can reveal information in layers.

First layer: the supported journey

Create an internal Go service
with workload identity and telemetry.

Second layer: operating behaviour

The service runs on Kubernetes,
uses private networking
and receives temporary credentials.

Third layer: implementation detail

The platform uses a service account,
federated identity,
network policy
and GitOps reconciliation.

Fourth layer: platform internals

The capability is implemented
through Crossplane compositions,
provider identities
and GitOps controllers.
SIMPLE ENTRY
     │
     ▼
OPERATING MODEL
     │
     ▼
IMPLEMENTATION DETAIL
     │
     ▼
PLATFORM INTERNALS

A developer can begin with the first layer and move deeper when debugging, designing or learning.

The platform remains approachable without becoming a black box.

This balance matters because both extremes fail.

Too much abstraction creates users who are productive only inside the expected path. Every unusual condition requires the platform team.

Too little abstraction exposes cloud identifiers, Kubernetes annotations and provider-specific wiring as the user interface. Every application team must then recreate infrastructure expertise.

The stronger design separates knowledge required to use the capability safely from implementation work that can be owned centrally.

Ownership should be visible before failure

A service team cannot own a production service if it does not understand:

  • what the service depends on
  • how it fails
  • which data it owns
  • how it recovers
  • which alerts matter
  • which changes are safe

The platform may operate the shared machinery.

The application team still owns the service outcome.

PLATFORM TEAM

owns shared capabilities


APPLICATION TEAM

owns service behaviour


BOTH

share the operating contract

Without an explicit boundary, the application team may assume:

The platform owns everything below the code.

The platform team may assume:

The service team owns everything
created for the service.

Incidents then become live negotiations about responsibility.

The platform team should not be the only source of truth. A healthy platform publishes capability documentation, architecture boundaries, status, runbooks, common failure modes, service-level objectives, change notices and ownership records.

Backstage can make this information discoverable.

Crossplane can expose status.

OpenTelemetry can provide operational evidence.

None of those tools creates understanding automatically.

The explanation must be designed as deliberately as the automation.

A bfstore service example

In this series, bfstore is the example commerce platform used to make the design concrete.

Suppose a developer creates the review service:

apiVersion: platform.bfstore.dev/v1alpha1
kind: Application
metadata:
  name: review-service
spec:
  runtime: go
  protocol: grpc
  exposure: internal

  database:
    profile: transactional-mysql

  messaging:
    consumes:
      - order.completed

The platform removes toil by providing:

  • repository scaffolding
  • CI configuration
  • image publication
  • Kubernetes resources
  • workload identity
  • database provisioning
  • Kafka permissions
  • telemetry
  • GitOps deployment

The developer should not need to create each resource manually.

The platform should still explain:

RUNTIME

Kubernetes Deployment


IDENTITY

temporary workload credentials;
the application must tolerate
credential renewal failure


DATABASE

private MySQL instance;
backup retention, recovery targets
and restore ownership are documented


MESSAGING

at-least-once event consumption;
duplicate delivery is possible
and the service owns safe handling


TELEMETRY

logs, metrics and traces
sent through OpenTelemetry


RECOVERY

deployment configuration is restored
from Git;

database recovery follows
the documented restore procedure

This gives the service owner a workable mental model.

They can reason about duplicate Kafka events, know that a Pod restart does not restore database state, and understand that Git can recreate deployment configuration but cannot recover lost data.

The platform has removed construction toil without removing engineering responsibility.

The questions I now ask

Repetition

Which work is routine enough
to automate centrally?

Decision

Which remaining choices affect
behaviour, risk or cost?

Contract

What does the capability promise,
and where do those promises stop?

Failure

Can the user identify
which system boundary failed?

Recovery

Does the user know how service
and data are restored?

Status

Does the platform explain
progress and blocked conditions?

Ownership

Which responsibilities belong
to the platform and service teams?

Learning

Can users move from a simple journey
to deeper understanding when needed?

These questions help distinguish useful simplification from concealed complexity.

The mental model I am keeping

My earlier model was:

DEVELOPER PLATFORM

hide infrastructure complexity
from developers

The stronger model is:

USER INTENT
     │
     ▼
PLATFORM API
     │
     ├── safe defaults
     ├── automation
     └── guardrails
     │
     ▼
SUPPORTED OUTCOME
     │
     ├── visible status
     ├── documented behaviour
     └── explicit ownership
     │
     ▼
ENGINEERING UNDERSTANDING

The model asks three things:

Which repeated work
can the platform perform?

How does the capability behave,
where does it fail
and who is responsible?

How can the user learn more
without confronting everything at once?

A platform should make the common journey easier.

It should remove repeated setup, private rituals and ticket-driven hand-offs, encode safe defaults, and let application teams focus more attention on their users and domain.

But hiding infrastructure is not the same as removing its consequences.

The network still fails.

Credentials still expire.

Databases still need recovery.

Messages can still be delivered more than once.

Deployments still change production behaviour.

The platform should contain the machinery while exposing the operating truth.

The best platform abstractions reduce the work required to use a system without reducing the understanding required to own it.

Not every developer needs to build the control plane.

Every service owner should understand the promises that control plane makes on their behalf, where those promises end and what happens when reality refuses to cooperate.