214 lines
5.4 KiB
Markdown
214 lines
5.4 KiB
Markdown
# Community Content Delivery Network Docker nodes
|
|
|
|
## Background
|
|
|
|
We wish to deploy a Community CDN that is easy to deploy and maintain.
|
|
|
|
This can be achieved by providing a Community Managed docker nodes that is regularly updated.
|
|
|
|
Changes can be applied to the GitTea, and the mirror nodes should pick up the changes.
|
|
|
|
## Removal of node
|
|
|
|
- Remove from central DNS
|
|
- Remove from monitoring
|
|
- Remove from allow lists
|
|
- Remove from authorized_keys
|
|
|
|
##
|
|
|
|
- Addition of new nodes
|
|
- Removal of old nodes
|
|
|
|
|
|
|
|
|
|
Path Validation Strategy
|
|
|
|
This is where your design differs from a normal web server.
|
|
|
|
You know:
|
|
|
|
episode numbers
|
|
valid extensions
|
|
exact file list
|
|
|
|
Therefore every request can be validated.
|
|
|
|
Option 1 (recommended)
|
|
|
|
Allow nginx to serve files normally.
|
|
|
|
If file does not exist:
|
|
|
|
error_page 404 = @invalid_request;
|
|
|
|
location @invalid_request {
|
|
access_log /var/log/nginx/invalid-paths.log;
|
|
return 404;
|
|
}
|
|
|
|
This scales well even with tens of thousands of files.
|
|
|
|
Prevent Directory Browsing
|
|
|
|
autoindex off;
|
|
|
|
location / {
|
|
try_files $uri =404;
|
|
}
|
|
|
|
No listings.
|
|
|
|
No traversal.
|
|
|
|
No guessing directories.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
thank you - I have refined the requirements now so ignore what I said earlier.
|
|
|
|
We are building a private community content delivery network, with nodes that serves files that typically change once a day.
|
|
|
|
## Hardware Requirements
|
|
|
|
Requirements for Hosting
|
|
|
|
- 24/7 Home Service
|
|
- fixed IP address
|
|
- unlimited bandwidth
|
|
- fast > 500mb/sec upload
|
|
- large > 5T of storage
|
|
- permission from your ISP to run a web server
|
|
- Contact information know to the Janitors
|
|
- Optional: UPS
|
|
|
|
## Software updates
|
|
|
|
|
|
The nodes should ideally run on both [Docker](https://en.wikipedia.org/wiki/Docker_%28software%29) and [Podman](https://en.wikipedia.org/wiki/Podman)
|
|
|
|
It should run Debian `debian:stable-slim`
|
|
|
|
All software MUST be running verifiable Free Libre Open Source software with links to the license.
|
|
|
|
The nodes should have as little utilities as possible to do it's job and in order to reduce the attack surface.
|
|
|
|
The amount of traffic is about 5Mb/sec with a load of 2 TPS
|
|
|
|
There will be many nodes, that will come and go over time.
|
|
|
|
The nodes will be added to DNS so the load can be shared.
|
|
|
|
The origin server requirements will be dictated by the needs of the nodes.
|
|
|
|
Nodes will monitor a rss control channel from the origin server for instructions.
|
|
|
|
The RSS file will contain a link to a json file.
|
|
|
|
The json file will contain
|
|
|
|
- the fail2ban settings, list of ip addresses, maxretry, findtime and bantime
|
|
- force full rsync Boolean flag - false
|
|
- hours between rsync eg every 3 hours
|
|
- list of origin servers
|
|
- list of Admin IP addresses
|
|
- list of IP addresses to ban immediately
|
|
- list of useragents to ban immediately
|
|
|
|
If the origin server is not available then the nodes should connect to the next origin server in the list.
|
|
|
|
nodes will synchronize the files from a origin server using rsync over ssh.
|
|
|
|
A full rsync is sufficient to ensure files are copied correctly.
|
|
|
|
A full rsync will be done several times a day, eg every 3 hours - from settings
|
|
|
|
atomic updates are not required as only new files are added.
|
|
|
|
There are thousands of files
|
|
|
|
The files will come from a well known directory eg `public_html/eps/hpr${ep_num}/hpr${ep_num}*.${extension}`
|
|
|
|
The `ep_num` will be a digit from 0001 to 9999
|
|
|
|
The `extension` will be from a well know list from the origin updated perhaps once a year
|
|
|
|
nodes will then serve the files using `nginx`
|
|
|
|
`nginx` must restrict access to files outside the well known files
|
|
|
|
A list of well known files that clients require (favorite icon, robots.txt, etc) will be available from the origin server.
|
|
|
|
The files we wish to serve are all Creative Commons Licensed so can be shared.
|
|
|
|
Random browsing is not allowed.
|
|
|
|
All file paths are known, and are provided by the origin server.
|
|
|
|
There should never be any request for a file outside the well known paths.
|
|
|
|
Any attempt to access any file outside the known paths should be logged.
|
|
|
|
Based on a configuration provided by the origin server, the server will run failtoban
|
|
|
|
repeat offenders will be banned for 7 days.
|
|
|
|
The Admin IP addresses should never be banned
|
|
|
|
traffic should be secured with let's encrypt of equivalent tls transport
|
|
|
|
clients can request the files without tls
|
|
|
|
The nodes public ip address will be placed on a allow list for ssh
|
|
|
|
Monitoring should be available using Prometheus or similar.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
+----------------+
|
|
| Origin Server A|
|
|
+--------+-------+
|
|
|
|
|
RSS Control Feed
|
|
|
|
|
JSON Configuration
|
|
|
|
|
+------------------+------------------+
|
|
| |
|
|
v v
|
|
|
|
+----------------+ +----------------+
|
|
| Edge Node 1 | | Edge Node 2 |
|
|
| | | |
|
|
| nginx | | nginx |
|
|
| rsync | | rsync |
|
|
| fail2ban | | fail2ban |
|
|
| node-agent | | node-agent |
|
|
| prometheus exp.| | prometheus exp.|
|
|
+--------+-------+ +--------+-------+
|
|
| |
|
|
+----------------+--------------------+
|
|
|
|
|
DNS RR
|
|
|
|
|
Clients
|
|
|
|
|