Every LLM observability tool has a setting that captures full prompt and completion content. It is enormously useful for debugging. It is also the fastest way I have seen a compliant organisation become non-compliant without anyone filing a change request.
Think about what is actually in a prompt in a Malaysian business context. A hospital assistant handling a patient enquiry. A bank's support agent processing an account query. An HR tool screening a CV. Those prompts contain NRIC numbers, account details, medical history, salary figures — the exact categories that PDPA treats as personal data, and in the medical case, sensitive personal data.
Enable full capture without a redaction layer and you have created a second copy of all of it, sitting in a system that likely has looser access controls than the database you were so careful about.
The tiers: always, conditionally, never
Always log
These carry no personal data and you cannot operate without them:
- Trace and span structure — which steps ran, in what order, nested correctly.
- Timings — per-span latency, so you can find the slow step.
- Model identity and version — which model answered. Essential when a provider updates silently and your quality shifts.
- Token counts — input and output, per call. Your cost attribution depends on this.
- Tool and function names invoked — the names, not necessarily the arguments.
- Outcome status — success, refusal, escalation, error, guardrail block.
- Prompt template version or hash — so you can correlate a quality change to a specific prompt change.
- Quality scores — groundedness, relevance and similar metrics attached to the trace.
This tier alone answers most operational questions: what is slow, what is expensive, what is failing, did quality move after Tuesday's prompt change.
Log conditionally — with redaction and a documented basis
- Prompt and completion content — genuinely valuable for debugging quality problems, and genuinely risky. Route it through PII detection and redaction before it is written. Open-source options like Microsoft Presidio handle detection and anonymisation; for Malaysian deployments you will want custom recognisers for NRIC format, local phone formats, and MyKad-adjacent identifiers, because off-the-shelf recognisers are tuned for US and EU formats.
- Retrieved document chunks — in a RAG system these often contain the most sensitive material in the entire pipeline. Log document identifiers and relevance scores by default; log chunk text only in a restricted environment.
- Tool call arguments — frequently contain the customer identifier that triggered the lookup.
Never log
- Credentials of any kind — API keys, tokens, passwords, session identifiers. This sounds obvious and it still happens, usually via a tool argument or an error payload that includes an auth header.
- Full payment card data — PCI-DSS scope is not somewhere you want your observability stack.
- Unredacted sensitive personal data — health, biometric, and similar categories, unless you have an explicit documented lawful basis and matching controls.
- Anything you cannot delete on request — if a data subject exercises deletion rights, you must be able to honour it in your traces too. If your observability store has no deletion path, do not put personal data in it.
Retention: the question nobody asks until an audit
Most teams enable tracing and never set a retention period. Two years later there is a searchable archive of every customer interaction, with no documented justification for keeping it.
A defensible pattern separates retention by tier: operational metadata for 12–24 months (cheap, low risk, useful for trend analysis); redacted content for 30–90 days (long enough to debug an incident); and unredacted content, where a basis exists at all, for 7–14 days in a restricted store with separate access control and its own audit log.
Under PDPA's Retention Principle you should not keep personal data longer than necessary for the purpose. "We might want to look at it someday" is not a purpose.
Regulated sectors: the extra layer
For Malaysian financial institutions, BNM's RMiT policy expectations around technology risk, data residency, and third-party arrangements apply to your observability stack as much as to your core systems — which is a substantial part of why self-hosted tooling is common in that sector.
In healthcare, patient information handling has its own constraints on top of PDPA. The safe default in both cases is that prompt content stays inside infrastructure you control.
NIST's Generative AI Profile (NIST AI 600-1) is a useful structuring document here — its MEASURE and MANAGE functions treat monitoring and incident disclosure as core risk controls rather than optional engineering hygiene, which is a helpful framing when you need to justify the work to a risk committee.
A practical starting policy
- Enable metadata tracing everywhere. No redaction needed; immediate operational value.
- Keep content capture off in production until redaction is deployed and tested.
- Deploy PII redaction with Malaysian-format recognisers. Test it against real NRIC and phone patterns, not just the defaults.
- Set retention per tier and configure it in the tool, not in a policy document nobody enforces.
- Restrict access to content-bearing traces to a named group, and log access to that store.
- Document the lawful basis and review it when the use case changes.
None of this is exotic. It is the same data governance you already apply elsewhere — the failure mode is simply that observability tooling gets adopted by engineers under delivery pressure, and the governance conversation happens after the data is already flowing.
Our AI Security programme covers PDPA-aligned AI logging, redaction pipelines and governance for regulated Malaysian environments — HRD Corp SBL-KHAS claimable.
