Data Layer gRPC Service Reference

Package platform.data.v1 contains the DataLayerService — the gRPC interface for tenant-scoped CRUD operations over module-defined collections.

All mutations run inside an RLS-guarded PostgreSQL transaction: the tenant_id field is enforced at the Row-Level Security policy layer, not only by the application, preventing any cross-tenant data access.

See the gRPC Overview for buf configuration, Go package conventions, deadlines, mTLS, and error mapping.

Proto file: proto/platform/data/v1/data.proto

Go package: kernel.internal/platform-kernel/gen/go/platform/data/v1;datav1

---

DataLayerService — All RPCs

// DataLayerService provides tenant-scoped CRUD operations.
// All mutations run inside an RLS-guarded transaction.
service DataLayerService {
  rpc CreateRecord(CreateRecordRequest) returns (CreateRecordResponse);
  rpc GetRecord(GetRecordRequest)       returns (GetRecordResponse);
  rpc ListRecords(ListRecordsRequest)   returns (ListRecordsResponse);
  rpc UpdateRecord(UpdateRecordRequest) returns (UpdateRecordResponse);
  rpc DeleteRecord(DeleteRecordRequest) returns (DeleteRecordResponse);
}

RPC	Description
CreateRecord	Inserts a new record into a collection. Returns the created Record.
GetRecord	Retrieves a single record by ID.
ListRecords	Paginated, filterable list of records in a collection.
UpdateRecord	Full replace of a record's data payload.
DeleteRecord	Removes a record by ID (soft-delete at DB layer via RLS).

---

Shared Message — Record

message Record {
  string                   id         = 1;
  string                   collection = 2;
  google.protobuf.Struct   data       = 3;
  google.protobuf.Timestamp created_at = 4;
  google.protobuf.Timestamp updated_at = 5;
}

Field	Type	Description
id	string	UUID v7. Time-sortable, generated by the service.
collection	string	Collection name in the format {module}.{model} (e.g., crm.contacts).
data	google.protobuf.Struct	Schema-free JSON payload. Max 1 MB total, max JSONB field 256 KB.
created_at	Timestamp	Server-set creation timestamp.
updated_at	Timestamp	Server-set update timestamp.

google.protobuf.Struct is the canonical Protobuf representation of a JSON object. In Go it maps to map[string]interface{}; in TypeScript via Protobuf-ES it maps to JsonObject. The schema is defined by the module's manifest — the Data Layer enforces size limits but not field types at the gRPC layer. Field-level validation is the module's responsibility.

---

CreateRecord

Inserts a new record. The tenant_id used in the RLS policy is injected by the Gateway from the JWT — it is not accepted from callers bypassing the Gateway.

Request — CreateRecordRequest

message CreateRecordRequest {
  string                 tenant_id  = 1; // Required. UUID v7. Injected by Gateway from JWT.
  string                 collection = 2; // Required. e.g. "crm.contacts".
  google.protobuf.Struct data       = 3; // Required. Record payload.
}

Field	Required	Notes
tenant_id	✅	Must match the JWT claim. Gateway enforces this.
collection	✅	Format: {module}.{model}. Module must be registered and active.
data	✅	JSON object. Max size: 1 MB. Max JSONB field: 256 KB.

Response — CreateRecordResponse

message CreateRecordResponse {
  Record record = 1; // The created record, including server-generated id and created_at.
}

gRPC errors:

gRPC status	Condition
INVALID_ARGUMENT	Missing tenant_id or collection, or data exceeds size limit
NOT_FOUND	collection module is not registered for this tenant
PERMISSION_DENIED	Tenant is SUSPENDED or TERMINATED — billing limit
RESOURCE_EXHAUSTED	Tenant's records limit exceeded (CheckLimit returns allowed: false)

---

GetRecord

Retrieves a single record by ID. The RLS policy automatically scopes the lookup to the calling tenant's rows.

Request — GetRecordRequest

message GetRecordRequest {
  string tenant_id  = 1; // Required. UUID v7.
  string collection = 2; // Required.
  string record_id  = 3; // Required. UUID v7 of the record.
}

Response — GetRecordResponse

message GetRecordResponse {
  Record record = 1;
}

gRPC errors:

gRPC status	Condition
NOT_FOUND	Record ID does not exist in this collection for this tenant

---

ListRecords

Paginated retrieval with optional filtering and sorting. Uses page token pagination (AIP-158 style) — not cursor-based like other services, because Data Layer uses page_size / page_token from the proto directly rather than platform.common.v1.PaginationRequest.

Request — ListRecordsRequest

message ListRecordsRequest {
  string tenant_id  = 1; // Required.
  string collection = 2; // Required.
  int32  page_size  = 3; // Max items per page. Default: 20. Max: 100.
  string page_token = 4; // Opaque token from previous ListRecordsResponse.next_page_token.
  string filter     = 5; // Optional. Filter expression (see Filter Syntax below).
  string order_by   = 6; // Optional. Sort field (see Order Syntax below).
}

Response — ListRecordsResponse

message ListRecordsResponse {
  repeated Record records         = 1;
  string          next_page_token = 2; // Empty string = no more pages.
  int32           total_count     = 3; // Total matching records (before pagination).
}

Filter Syntax

The filter field uses a PostgREST-inspired expression language. The Gateway translates these to parameterized PostgreSQL WHERE clauses:

Operator	Example	Meaning
eq	status=eq.active	Exact match
neq	status=neq.deleted	Not equal
gt	price=gt.100	Greater than
gte	price=gte.100	Greater than or equal
lt	price=lt.50	Less than
lte	price=lte.50	Less than or equal
ilike	name=ilike.john*	Case-insensitive prefix/contains (* = wildcard)
in	status=in.(active,pending)	IN list
or	or=(price.lt.50,status.eq.sale)	Logical OR

Limits: Max 10 filters per request. Max depth of OR/AND nesting: 2 levels.

Order Syntax

order_by = "price.desc,name.asc"

Multiple fields separated by commas. Direction: asc (default) or desc. Max 3 order fields.

---

UpdateRecord

Full replace of the data payload. The id, collection, created_at are immutable — only data is replaced and updated_at is refreshed.

Request — UpdateRecordRequest

message UpdateRecordRequest {
  string                 tenant_id  = 1; // Required.
  string                 collection = 2; // Required.
  string                 record_id  = 3; // Required. UUID v7.
  google.protobuf.Struct data       = 4; // Required. New payload (full replace, not patch).
}

Response — UpdateRecordResponse

message UpdateRecordResponse {
  Record record = 1; // Updated record with refreshed updated_at.
}

gRPC errors:

gRPC status	Condition
NOT_FOUND	Record ID does not exist in this collection for this tenant
INVALID_ARGUMENT	data exceeds size limit

Full replace semantics: UpdateRecord replaces the entire data object. For partial updates, callers should read the current record with GetRecord, merge fields client-side, then send the merged object via UpdateRecord. The REST layer exposes PATCH semantics on top of this via the Gateway.

---

DeleteRecord

Soft-deletes a record. The record is marked as deleted via the RLS policy and is no longer returned by GetRecord or ListRecords for this tenant.

Request — DeleteRecordRequest

message DeleteRecordRequest {
  string tenant_id  = 1; // Required.
  string collection = 2; // Required.
  string record_id  = 3; // Required. UUID v7.
}

Response — DeleteRecordResponse

message DeleteRecordResponse {} // Empty — success signals deletion.

gRPC errors:

gRPC status	Condition
NOT_FOUND	Record ID does not exist in this collection for this tenant

---

CDC Pipeline

Every mutation to a Data Layer collection is propagated to ClickHouse for analytics via the CDC pipeline:

PostgreSQL (OLTP) → Debezium (WAL capture) → Kafka (platform.data.events) → ClickHouse (OLAP)

Parameter	Value
Lag	< 5 seconds end-to-end
Delivery guarantee	At-least-once (Kafka)
Deduplication	ReplacingMergeTree + SELECT … FINAL in ClickHouse
Kafka topic	platform.data.events

Analytics queries (e.g., aggregations, time-series) run against ClickHouse, not PostgreSQL. The REST GET /api/v1/data/{module}/{model} route routes simple CRUD to PostgreSQL and analytical queries to ClickHouse based on query complexity heuristics.

---

Limits Reference

Limit	Value	Env variable
Max record size	1 MB	DATA_MAX_RECORD_SIZE_BYTES=1048576
Max JSONB field	256 KB	DATA_MAX_JSONB_SIZE_BYTES=262144
Max tables per module	50	DATA_MAX_TABLES_PER_MODULE=50
Max relations depth	2 levels	—
Max page_size	100	—
Max filter expressions	10	—

---

Security — RLS Enforcement

The Data Layer enforces Row-Level Security at the PostgreSQL level for every query:

Layer	Mechanism
PostgreSQL RLS	WHERE tenant_id = current_setting('app.tenant_id') policy on every table
Application layer	tenant_id injected from JWT via gRPC metadata — never from client payload
ClickHouse analytics	Dual enforcement: Row Policy + application-level WHERE tenant_id = ?

No code path in DataLayerService accesses data without setting the app.tenant_id session variable. This is verified by the integration test suite (testcontainers with RLS tests on every repository method).