Glossary
Log Retention
How long you keep logs, and why. Retention is where regulatory minimums (keep it), data protection law (do not keep it longer than needed) and storage cost meet.
General definition
Log Retention is the rule that says how long each class of log is kept and what happens to it afterwards: deletion, archival to cheaper storage, or transfer to immutable storage. Logs are not one thing. Application logs, access logs, security event logs, audit logs and message archives serve different purposes and are subject to different rules, so a retention policy is usually a table of log types against periods rather than a single number.
- HIPAA: documentation of policies, procedures and required records must be kept for six years (45 CFR 164.316); many covered entities apply the same window to audit logs
- PCI DSS: at least twelve months of audit log history, with the most recent three months immediately available for analysis
- SEC 17a-4 and FINRA: communications and records of broker-dealers for three to six years, on non-rewriteable media or with an audit-trail alternative
- GDPR: no fixed period; Article 5(1)(e) storage limitation means logs containing personal data are kept only as long as a documented purpose justifies
Those requirements pull in different directions. A regulated healthcare or finance business must keep records for years; GDPR and similar laws (UK GDPR, CCPA) penalise keeping personal data without a purpose. The usual resolution is to separate the log types: keep security and audit logs for the regulatory period, often in WORM storage, and give shorter windows to verbose application and access logs. Where a person exercises a right to erasure, the archive keeps what a legal obligation requires and removes the rest.
Operationally, retention is enforced with lifecycle rules on object storage, index rollover in log platforms, and time-to-live settings in databases. The policy should be written down, mapped to each regulation, tested (does deletion actually happen?), and reviewed when a new region or product line is added, because data residency and retention are decided together.
In the Ethora ecosystem
Ethora keeps chat history in a server-side message archive with configurable retention, so a deployment can be set to keep records for the period a regulator expects, or to shorter windows where data minimisation matters more. Per-user erasure handles GDPR requests, and Trust and Safety rules can remove accidentally shared sensitive data immediately or after a delay. The compliance audit trail is exported as immutable logs to S3, where the customer applies its own lifecycle and Object Lock settings.
On dedicated and self-hosted deployments those settings, the audit exports and the operational logs from Grafana and Prometheus all live in the customer’s own account, so one retention policy can cover the whole stack. A backup policy is defined alongside it, and the same choices apply to the healthcare, finance and insurance SDK deployments where retention periods are a contract term.
