269 lines
5.0 KiB
Markdown
269 lines
5.0 KiB
Markdown
# ccdn_architecture_upgrades.md
|
|
|
|
# CCDN Architecture Upgrade Ideas
|
|
|
|
This document lists potential improvements to the CCDN architecture that were identified during a design review. None of these are considered essential for the initial release; they are intended as future enhancements once the core system is stable.
|
|
|
|
The philosophy remains unchanged:
|
|
|
|
* Keep the system simple.
|
|
* Prefer proven Unix tools.
|
|
* Avoid unnecessary dependencies.
|
|
* Preserve the stateless nature of edge nodes.
|
|
|
|
---
|
|
|
|
## 1. Atomic Content Deployment
|
|
|
|
**Priority:** High
|
|
|
|
Instead of synchronising directly into the live content directory, use a staging directory.
|
|
|
|
Suggested workflow:
|
|
|
|
1. Download the signed control file.
|
|
2. Verify its signature.
|
|
3. Synchronise content into a staging directory.
|
|
4. Perform validation.
|
|
5. Atomically rename the staging directory into production.
|
|
6. Reload nginx if required.
|
|
|
|
Benefits:
|
|
|
|
* Prevents partially updated content from being served.
|
|
* Makes interrupted synchronisations harmless.
|
|
* Provides clean rollback behaviour if validation fails.
|
|
|
|
---
|
|
|
|
## 2. Content Manifest
|
|
|
|
**Priority:** High
|
|
|
|
Publish a manifest alongside the content containing:
|
|
|
|
* filename
|
|
* size
|
|
* SHA-256 checksum
|
|
* modification time
|
|
|
|
After synchronisation, each node can verify downloaded files against the manifest.
|
|
|
|
Benefits:
|
|
|
|
* Detects storage corruption.
|
|
* Detects incomplete synchronisations.
|
|
* Simplifies troubleshooting.
|
|
* Provides confidence that mirrors contain identical content.
|
|
|
|
---
|
|
|
|
## 3. Configuration Versioning
|
|
|
|
**Priority:** Medium
|
|
|
|
Include a version number or timestamp in the signed control file.
|
|
|
|
Example:
|
|
|
|
```json
|
|
{
|
|
"version": 14
|
|
}
|
|
```
|
|
|
|
or
|
|
|
|
```json
|
|
{
|
|
"generated": "2026-08-01T12:00:00Z"
|
|
}
|
|
```
|
|
|
|
Benefits:
|
|
|
|
* Easier troubleshooting.
|
|
* Prevents accidental rollback.
|
|
* Makes monitoring simpler.
|
|
|
|
---
|
|
|
|
## 4. Configuration Expiry
|
|
|
|
**Priority:** Medium
|
|
|
|
Include an expiry timestamp in the signed control file.
|
|
|
|
Example:
|
|
|
|
```json
|
|
{
|
|
"valid_until": "2026-09-01T00:00:00Z"
|
|
}
|
|
```
|
|
|
|
Nodes should continue serving existing content if the configuration expires, but generate warnings or alerts so administrators know updates are no longer being received.
|
|
|
|
Benefits:
|
|
|
|
* Detects long-term communication failures.
|
|
* Helps identify replay attacks using stale configurations.
|
|
|
|
---
|
|
|
|
## 5. Health Endpoint
|
|
|
|
**Priority:** Medium
|
|
|
|
Expose a small JSON document such as:
|
|
|
|
```
|
|
/health.json
|
|
```
|
|
|
|
Example information:
|
|
|
|
* configuration version
|
|
* last successful synchronisation
|
|
* current origin
|
|
* disk usage
|
|
* software version
|
|
* node identifier
|
|
|
|
Benefits:
|
|
|
|
* Simplifies monitoring.
|
|
* Easy integration with Prometheus or external monitoring.
|
|
* Useful during troubleshooting.
|
|
|
|
---
|
|
|
|
## 6. Disk Space Protection
|
|
|
|
**Priority:** Medium
|
|
|
|
Before synchronisation, verify sufficient free disk space exists.
|
|
|
|
If available space falls below a configured threshold, abort the update and generate an alert.
|
|
|
|
Benefits:
|
|
|
|
* Prevents failed deployments.
|
|
* Protects nodes from filling the filesystem.
|
|
|
|
---
|
|
|
|
## 7. SSH Restrictions
|
|
|
|
**Priority:** Medium
|
|
|
|
Restrict the rsync account so it cannot obtain an interactive shell.
|
|
|
|
Possible approaches include:
|
|
|
|
* `rrsync`
|
|
* `ForceCommand`
|
|
* `command=` restrictions in `authorized_keys`
|
|
|
|
Benefits:
|
|
|
|
* Reduces the impact of a compromised SSH key.
|
|
* Limits access strictly to file synchronisation.
|
|
|
|
---
|
|
|
|
## 8. Signing Key Rotation
|
|
|
|
**Priority:** Low
|
|
|
|
Support publishing both the current and next public signing keys.
|
|
|
|
This allows new keys to be distributed before they become active.
|
|
|
|
Benefits:
|
|
|
|
* Simplifies planned key rotation.
|
|
* Avoids emergency replacement procedures.
|
|
|
|
---
|
|
|
|
## 9. Split Configuration Files
|
|
|
|
**Priority:** Low
|
|
|
|
If the control file becomes large, consider splitting it into independently signed files.
|
|
|
|
For example:
|
|
|
|
* origins.json
|
|
* sync.json
|
|
* nginx.json
|
|
* security.json
|
|
|
|
Benefits:
|
|
|
|
* Easier maintenance.
|
|
* Smaller updates.
|
|
* Simpler reviews.
|
|
|
|
This is not recommended until the configuration grows significantly.
|
|
|
|
---
|
|
|
|
## 10. Declarative Configuration Only
|
|
|
|
**Priority:** Ongoing
|
|
|
|
The signed configuration should describe desired system state rather than commands to execute.
|
|
|
|
For example:
|
|
|
|
Prefer:
|
|
|
|
* sync interval
|
|
* fail2ban settings
|
|
* nginx options
|
|
* origin list
|
|
|
|
Avoid:
|
|
|
|
* arbitrary shell commands
|
|
* remote script execution
|
|
|
|
Benefits:
|
|
|
|
* Smaller attack surface.
|
|
* Easier auditing.
|
|
* More predictable behaviour.
|
|
|
|
---
|
|
|
|
# Future Enhancements
|
|
|
|
These ideas are intentionally outside the scope of the first implementation but may become useful if CCDN grows.
|
|
|
|
* Delta manifests for large libraries.
|
|
* Geographic origin selection.
|
|
* Optional peer-to-peer mirror synchronisation.
|
|
* Compression for text-based metadata.
|
|
* Signed content release tags.
|
|
* Read-only content mounts between updates.
|
|
|
|
---
|
|
|
|
# Deliberately Out of Scope
|
|
|
|
The following technologies are intentionally excluded from the CCDN design:
|
|
|
|
* Kubernetes
|
|
* Docker Swarm
|
|
* Redis
|
|
* PostgreSQL
|
|
* Elasticsearch
|
|
* Message queues
|
|
* Dynamic service discovery
|
|
* Distributed databases
|
|
* Complex orchestration systems
|
|
|
|
The goal is to keep CCDN easy to understand, easy to operate, and easy to recover by relying on mature Unix tooling rather than additional infrastructure.
|