Skip to content
Data Forge

What we do, plane by plane.

A platform is six kinds of work that have to agree with each other. We do all six, in one codebase, and we run the result. Jump to a plane, or read down.

Six planes, exploded

2026-09-09

The six planes of the estate, exploded into layers. auth-engine · Rust auth-idp · Rust authz-projector · Go code-intel-toolchain · Go config-vault · Rust config-vault-cli · Rust · in the codebase, no lane config-vault-reconciler · Rust telemetry-gateway · Go analytics-etl · Go analytics-service · Python data-pipeline · Python dataforge-pg-migrations · Go · in the codebase, no lane graph-sync · Go intel-clickhouse-consumer · Go intel-event-pipeline · Go intel-graph-sync · Go intel-migrations · Go · in the codebase, no lane intel-orchestrator · Python agent-api · Go agent-kb-embedder · Go agent-kb-mcp · Go · in the codebase, no lane agent-orchestrator · Go · in the codebase, no lane agent-runtime · Go clickhouse-mcp · Go · in the codebase, no lane code-intel-mcp · Go defect-embedder · Go defect-ingest · Python defects-search · Go embedding-gateway · Go forge-context · Go knowledge-graph · Go openproject-mcp · Go product-search · Rust promstack-mcp · Go property-intelligence · Python qdrant-mcp · Go rag-memory-mcp · Go redpanda-mcp · Go · in the codebase, no lane scip-loader · Go · in the codebase, no lane search-api · Rust sure-pg-mcp · Go · in the codebase, no lane cashflow-api · Go core-api · Go deal-checklist · Go document-processor · Python finance-api · Go image-processor · Rust integrations-api · Go manifest-api · Go ops-api · Go procurement · Go quote-calculator · Rust quotes-api · Go relationship-engine · Rust report-generator · Python solar-api · Go surebi-api · Go surebi-report-intake · Python surebi-reports · Go video-processor · Rust cms-analytics-sync · Go growth-os-api · Go growth-studio-api · Go intel-activation · Go intel-analysers · Go intel-analysers · Rust intel-capi-dispatcher · Go intel-capture-services · Go intel-collector-api · Go intel-compliance · Go intel-lead-scoring · Go intel-profile-api · Go command-centre-api · Go csv-import-worker · Rust lead-assignment-worker · Rust notification-handler · Go realtime-gateway · Go scheduling-optimizer · Rust workflow-engine · Rust

Platform & infrastructure

The layer nothing above it should have to think about.

Kubernetes on immutable, API-managed nodes. GitOps reconciliation, one image lane with provenance gates, and a release train so a deploy is a decision rather than an accident. Identity, secrets, certificates, storage and backups are part of the build, not a follow-up ticket.

  • Talos Linux Kubernetes, nine nodes in Perth
  • Argo CD GitOps with pinned manifests and a release train
  • One image lane with provenance and entrypoint gates
  • OIDC identity: auth-engine sessions, auth-idp provider, SpiceDB
  • config-vault secrets with provenance and audit
  • Longhorn block storage, MinIO object storage, offsite backup lanes
  • cert-manager, Traefik ingress, Tailscale access
  • Prometheus, Loki, Tempo, Grafana, Sentry

8 in the estate

  • auth-engine
  • auth-idp
  • authz-projector
  • code-intel-toolchain
  • config-vault
  • config-vault-cli
  • config-vault-reconciler
  • telemetry-gateway
The k8-forge fleet, drawn to the hardware One Dell PowerEdge R740xd 2U server above a 4 by 2 grid of eight HP EliteDesk 800 minis, each with a status LED. U2 U1 node-9 Dell PowerEdge R740xd node-1 6c / 16 GB node-2 6c / 16 GB node-3 6c / 16 GB node-4 6c / 16 GB node-5 12c / 32 GB node-6 12c / 32 GB node-7 12c / 32 GB node-8 12c / 32 GB
Nine machines, drawn to the hardware facts; arrangement schematic.

Data platform

One store, one job. Nothing quietly doing three.

Most data problems are placement problems: events in the transactional database, relationships in a JSON column, embeddings in whatever was nearest. We put each kind of data in the store built for it and keep every derived layer rebuildable from its source.

  • Postgres as the transactional system of record
  • ClickHouse for append-only events and analytics, queried in SQL
  • Redpanda as the transport between services, with declared topics and dead-letter queues
  • FalkorDB graphs for identity and code relationships, queried in Cypher
  • Qdrant vector search for embeddings, tenancy enforced as a payload filter
  • Dagster pipelines for identity, conversions and analytics assets
  • Versioned SQL migrations, end to end
  • Raw event export to BigQuery

10 in the estate

  • analytics-etl
  • analytics-service
  • data-pipeline
  • dataforge-pg-migrations
  • graph-sync
  • intel-clickhouse-consumer
  • intel-event-pipeline
  • intel-graph-sync
  • intel-migrations
  • intel-orchestrator

Applied AI & agents

Agents wired into real systems, not a chat box on the side.

We build agent runtimes and tool servers that read and write the actual estate: the work ledger, the code graph, the event bus, the warehouse. Retrieval runs over your own corpus with tenancy and consent already enforced, so an agent can be useful without being dangerous. Nothing gets write access before it has been evaluated.

  • Agent runtime and orchestrator services
  • MCP tool servers over Postgres, ClickHouse, Redpanda, Qdrant, Prometheus and OpenProject
  • A code-intelligence graph for impact analysis before a change ships
  • Knowledge graphs for entities and relationships
  • An embedding gateway that routes to Gemini or a local Ollama model
  • Semantic search over ten thousand building defects
  • A registry router so agents look facts up instead of guessing them
  • Guardrails, idempotency and rate limits on every write path

23 in the estate

  • agent-api
  • agent-kb-embedder
  • agent-kb-mcp
  • agent-orchestrator
  • agent-runtime
  • clickhouse-mcp
  • code-intel-mcp
  • defect-embedder
  • defect-ingest
  • defects-search
  • embedding-gateway
  • forge-context
  • knowledge-graph
  • openproject-mcp
  • product-search
  • promstack-mcp
  • property-intelligence
  • qdrant-mcp
  • rag-memory-mcp
  • redpanda-mcp
  • scip-loader
  • search-api
  • sure-pg-mcp

Product engineering

The right language for the job, in one coherent codebase.

Go for domain services, Rust where the work is CPU-bound, Python where the work is data and machine learning, typed React on the front and Astro for the sites. One monorepo, one build graph, and shared packages for the things every surface needs, so a capability is built once and used everywhere.

  • Go domain APIs on a standard service contract
  • Rust workers for quoting, scheduling, search and media
  • Python services for documents, reports and property intelligence
  • Vite and React operator consoles and customer apps
  • Astro marketing sites built for Core Web Vitals
  • Shared packages: auth, API clients, tenancy, logging, UI
  • Moon-orchestrated builds with affected-set CI gates
  • Directus CMS so marketing teams publish without engineers

19 in the estate

  • cashflow-api
  • core-api
  • deal-checklist
  • document-processor
  • finance-api
  • image-processor
  • integrations-api
  • manifest-api
  • ops-api
  • procurement
  • quote-calculator
  • quotes-api
  • relationship-engine
  • report-generator
  • solar-api
  • surebi-api
  • surebi-report-intake
  • surebi-reports
  • video-processor

Growth & measurement

Numbers you can defend when the spend is questioned.

Client-side analytics quietly lies. Consent gating, ad blockers and platform attribution all bend the shape of the truth. We collect server-side, score the bots out, deduplicate conversions into Meta and Google, export raw events to the warehouse, and run ad accounts against real jobs rather than platform-reported conversions.

  • Server-side event collection with browser fingerprint capture
  • Bot scoring before an event counts
  • An identity graph and recency-weighted lead scoring per brand
  • Meta and Google Conversion APIs with event deduplication
  • GA4 with raw export to BigQuery
  • Google Ads operations: budgets, guards, zone attribution, incrementality
  • Growth Studio: campaign editor and ad-spend analytics
  • Consent, suppression and erasure under the Australian Privacy Act

12 in the estate

  • cms-analytics-sync
  • growth-os-api
  • growth-studio-api
  • intel-activation
  • intel-analysers
  • intel-analysers
  • intel-capi-dispatcher
  • intel-capture-services
  • intel-collector-api
  • intel-compliance
  • intel-lead-scoring
  • intel-profile-api

Automation for operators

The repetitive work handled, permanently.

Lead routing, notifications with a delivery record per lead, franchise and territory workflows, scheduling, accounting and CRM integrations, and internal consoles so an owner can see the whole business without asking anyone. This is the original promise of Data Forge: technology that gives people their time back.

  • A durable notification lane with a delivery record per lead
  • Round-robin lead assignment and territory routing
  • A workflow engine for work-item transitions and triggers
  • Schedule optimisation for field availability and conflicts
  • Xero and Pipedrive integrations
  • A realtime gateway streaming events to connected consoles
  • Command Centre: fleet, deploy and work-ledger views in one place
  • Email journeys and franchisee notifications

7 in the estate

  • command-centre-api
  • csv-import-worker
  • lead-assignment-worker
  • notification-handler
  • realtime-gateway
  • scheduling-optimizer
  • workflow-engine

Open a conversation.

Tell us what you run today and what keeps breaking. We will tell you, plainly, what we would do about it and whether we are the right people to do it.