app refactored
This commit is contained in:
@@ -24,34 +24,98 @@ The SQLite database is stored in `./data/urlshort.db` on the host — it survive
|
|||||||
|
|
||||||
## Configuration (`config.json`)
|
## Configuration (`config.json`)
|
||||||
|
|
||||||
| Key | Required | Default | Description |
|
| Key | Required | Default | Description |
|
||||||
|----------------|----------|----------------------|----------------------------------------------|
|
|----------------------|----------|----------------------|-------------------------------------------------------------------|
|
||||||
| `base_url` | ✅ | — | Public base URL used in generated short URLs |
|
| `base_url` | ✅ | — | Public base URL. The path component (e.g. `/s` in `http://example.com/s`) is automatically used as the server's routing prefix. |
|
||||||
| `short_length` | ✅ | — | Character length of auto-generated codes |
|
| `api_key` | ✅ | — | Secret key to protect write/read operations |
|
||||||
| `api_key` | ✅ | — | Secret key to protect write/read operations |
|
| `host` | | `"0.0.0.0"` | Bind address |
|
||||||
| `host` | | `"0.0.0.0"` | Bind address |
|
| `port` | | `8080` | Bind port |
|
||||||
| `port` | | `8080` | Bind port |
|
| `db_path` | | `"data/urlshort.db"` | Path to the SQLite database file |
|
||||||
| `db_path` | | `"data/urlshort.db"` | Path to the SQLite database file |
|
| `retention_days` | | `0` | Default retention period in days for new URLs. `0` means never expire. |
|
||||||
|
| `min_short_length` | | `6` | Minimum character length for generated short codes |
|
||||||
|
| `max_short_length` | | `32` | Maximum character length for generated short codes |
|
||||||
|
| `max_url_length` | | `2048` | Maximum allowed length for original URLs |
|
||||||
|
| `max_retention_days` | | `3650` | Maximum allowed retention_days value per URL |
|
||||||
|
| `rate_limit_requests`| | `60` | Max requests per IP per rate-limit window |
|
||||||
|
| `rate_limit_window` | | `60` | Rate-limit window in seconds |
|
||||||
|
|
||||||
|
> **Backward compat:** If an old config contains `short_length`, it is automatically used as `min_short_length`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deploying behind a sub-path (e.g. `http://example.com/s`)
|
||||||
|
|
||||||
|
Just set `base_url` to include the desired path prefix — the server derives its routing prefix automatically from it:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"base_url": "http://example.com/s",
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The path component `/s` is extracted at startup. The server will only respond to requests whose path starts with `/s`; everything else returns 404.
|
||||||
|
|
||||||
|
| `base_url` | Derived routing prefix | Short URL example |
|
||||||
|
|---|---|---|
|
||||||
|
| `http://example.com` | *(none — root)* | `http://example.com/aB3xYz` |
|
||||||
|
| `http://example.com/s` | `/s` | `http://example.com/s/aB3xYz` |
|
||||||
|
| `http://example.com/go/links` | `/go/links` | `http://example.com/go/links/aB3xYz` |
|
||||||
|
|
||||||
|
### With Docker Compose
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set base_url in config.json, then:
|
||||||
|
docker compose up --build
|
||||||
|
# Service is now available at http://localhost:8080/s/
|
||||||
|
```
|
||||||
|
|
||||||
|
### With an existing nginx vhost
|
||||||
|
|
||||||
|
`nginx.conf` contains **location blocks only** — drop them into an existing `server { }` block.
|
||||||
|
The app handles the base_path prefix internally; nginx proxies API/redirect requests and serves frontend static files.
|
||||||
|
|
||||||
|
```
|
||||||
|
Browser ──► nginx /s/ ──► static/index.html
|
||||||
|
Browser ──► nginx /s/static/… ──► static files (CSS/JS)
|
||||||
|
Browser ──► nginx /s/api/… ──► urlshort :8080 (proxy)
|
||||||
|
Browser ──► nginx /s/<code> ──► urlshort :8080 (proxy → 302)
|
||||||
|
```
|
||||||
|
|
||||||
|
To change the prefix, update `base_url` in `config.json` **and** the `location /s` blocks in `nginx.conf`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Authentication
|
## Authentication
|
||||||
|
|
||||||
All `/api/*` endpoints require the API key.
|
All endpoints marked with 🔒 require the API key.
|
||||||
Pass it as a **Bearer token** in the `Authorization` header or as a query parameter:
|
Pass it as a **query parameter** or in the **JSON request body**:
|
||||||
|
|
||||||
```
|
```
|
||||||
Authorization: Bearer <api_key>
|
|
||||||
# or
|
|
||||||
?api_key=<api_key>
|
?api_key=<api_key>
|
||||||
|
# or in JSON body
|
||||||
|
{"api_key": "<api_key>", ...}
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Retention
|
||||||
|
|
||||||
|
URLs can have a retention period (`retention_days`). When set to a positive integer, the URL will be automatically deleted after that many days. If `0` or not set, the URL never expires.
|
||||||
|
|
||||||
|
- The **default** retention is set in `config.json` (`retention_days` key, default `0`).
|
||||||
|
- Each URL can override the default at creation time via the `retention_days` field.
|
||||||
|
- Expired URLs are cleaned up on startup and lazily on each incoming request.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## API Reference
|
## API Reference
|
||||||
|
|
||||||
### `GET /`
|
### `GET /`
|
||||||
Health check.
|
Serves the frontend page (if `static/index.html` exists), otherwise returns health check JSON.
|
||||||
|
|
||||||
|
### `GET /api/health`
|
||||||
|
Health check endpoint.
|
||||||
|
|
||||||
**Response `200`**
|
**Response `200`**
|
||||||
```json
|
```json
|
||||||
@@ -63,22 +127,21 @@ Health check.
|
|||||||
### `POST /api/shorten` 🔒
|
### `POST /api/shorten` 🔒
|
||||||
Create a new short URL.
|
Create a new short URL.
|
||||||
|
|
||||||
**Request body**
|
Fields can be passed as **query parameters** (URL-encoded) or in a **JSON request body**.
|
||||||
```json
|
Query parameters take precedence over body fields.
|
||||||
{
|
|
||||||
"url": "https://example.com/very/long/path",
|
|
||||||
"custom_code": "mycode" // optional
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response `201`**
|
| Field | Required | Description |
|
||||||
```json
|
|------------------|----------|-----------------------------------------------------------|
|
||||||
{
|
| `api_key` | ✅ | API key for authentication |
|
||||||
"short_code": "aB3xYz",
|
| `url` | ✅ | The URL to shorten (must start with `http://` or `https://`) |
|
||||||
"short_url": "http://localhost:8080/aB3xYz",
|
| `retention_days` | | Override the default retention period for this URL |
|
||||||
"original_url": "https://example.com/very/long/path",
|
|
||||||
"created_at": 1710000000
|
Short code length is determined automatically: the server starts at `min_short_length` and
|
||||||
}
|
progressively tries longer codes on collision, up to `max_short_length`.
|
||||||
|
|
||||||
|
**Response `201`** — plain text containing only the short URL:
|
||||||
|
```
|
||||||
|
http://localhost:8080/s/aB3xYz
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -93,10 +156,11 @@ List all short URLs, newest first.
|
|||||||
"urls": [
|
"urls": [
|
||||||
{
|
{
|
||||||
"short_code": "aB3xYz",
|
"short_code": "aB3xYz",
|
||||||
"short_url": "http://localhost:8080/aB3xYz",
|
"short_url": "http://localhost:8080/s/aB3xYz",
|
||||||
"original_url": "https://example.com",
|
"original_url": "https://example.com",
|
||||||
"created_at": 1710000000,
|
"created_at": 1710000000,
|
||||||
"visit_count": 5
|
"visit_count": 5,
|
||||||
|
"retention_days": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -104,21 +168,44 @@ List all short URLs, newest first.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### `GET /api/urls/<code>` 🔒
|
### `GET /api/urls/<code>`
|
||||||
Get metadata for a single short code.
|
Get metadata for a single short code. No API key required.
|
||||||
|
|
||||||
**Response `200`** — same shape as one item from the list above.
|
**Response `200`**
|
||||||
**Response `404`** — code not found.
|
```json
|
||||||
|
{
|
||||||
|
"short_code": "aB3xYz",
|
||||||
|
"short_url": "http://localhost:8080/s/aB3xYz",
|
||||||
|
"original_url": "https://example.com",
|
||||||
|
"created_at": 1710000000,
|
||||||
|
"visit_count": 5,
|
||||||
|
"retention_days": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response `404`** — code not found (empty body).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `GET /api/lookup`
|
||||||
|
Look up a URL by its **original URL**. No API key required.
|
||||||
|
Used by the frontend to check if a URL has already been shortened.
|
||||||
|
|
||||||
|
| Parameter | Required | Description |
|
||||||
|
|-----------|----------|-------------------------------|
|
||||||
|
| `url` | ✅ | The original URL to look up |
|
||||||
|
|
||||||
|
**Response `200`** — same metadata JSON as `GET /api/urls/<code>`.
|
||||||
|
**Response `404`** — no short URL exists for this original URL.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### `DELETE /api/urls/<code>` 🔒
|
### `DELETE /api/urls/<code>` 🔒
|
||||||
Delete a short URL entry.
|
Delete a short URL entry.
|
||||||
|
|
||||||
**Response `200`**
|
**Response `204`** — success (empty body).
|
||||||
```json
|
**Response `404`** — code not found (empty body).
|
||||||
{ "message": "Deleted 'aB3xYz'" }
|
**Response `403`** — not authorized (empty body).
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -131,21 +218,81 @@ Increments `visit_count` on each hit.
|
|||||||
## Example with `curl`
|
## Example with `curl`
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Shorten a URL
|
# Shorten a URL (JSON body)
|
||||||
curl -X POST http://localhost:8080/api/shorten \
|
curl -X POST http://localhost:8080/s/api/shorten \
|
||||||
-H "Authorization: Bearer change-this-secret-key" \
|
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d '{"url": "https://github.com"}'
|
-d '{"api_key": "change-this-secret-key", "url": "https://github.com"}'
|
||||||
|
|
||||||
|
# Shorten a URL (query parameters)
|
||||||
|
curl -X POST "http://localhost:8080/s/api/shorten?api_key=change-this-secret-key&url=https%3A%2F%2Fgithub.com&retention_days=30"
|
||||||
|
|
||||||
# Follow the redirect
|
# Follow the redirect
|
||||||
curl -L http://localhost:8080/aB3xYz
|
curl -L http://localhost:8080/s/aB3xYz
|
||||||
|
|
||||||
|
# Get metadata for a short URL (no API key needed)
|
||||||
|
curl http://localhost:8080/s/api/urls/aB3xYz
|
||||||
|
|
||||||
# List all URLs
|
# List all URLs
|
||||||
curl http://localhost:8080/api/urls \
|
curl "http://localhost:8080/s/api/urls?api_key=change-this-secret-key"
|
||||||
-H "Authorization: Bearer change-this-secret-key"
|
|
||||||
|
|
||||||
# Delete a URL
|
# Delete a URL
|
||||||
curl -X DELETE http://localhost:8080/api/urls/aB3xYz \
|
curl -X DELETE "http://localhost:8080/s/api/urls/aB3xYz?api_key=change-this-secret-key"
|
||||||
-H "Authorization: Bearer change-this-secret-key"
|
|
||||||
|
# Lookup by original URL
|
||||||
|
curl "http://localhost:8080/s/api/lookup?url=https%3A%2F%2Fexample.com"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend
|
||||||
|
|
||||||
|
A clean single-page frontend is included in `static/`. It provides:
|
||||||
|
|
||||||
|
- **URL shortening** — paste a URL and press Enter to create a short URL (requires API key)
|
||||||
|
- **Existing URL lookup** — as you type a URL, the frontend checks if it already exists and shows its metadata
|
||||||
|
- **Admin table** — enter a valid API key to see all shortened URLs in a sortable, paginated table
|
||||||
|
- **Copy & Delete** — per-row copy and delete buttons (delete on hover only)
|
||||||
|
- **Theme switching** — matches system dark/light preference, with a manual toggle
|
||||||
|
|
||||||
|
### Accessing the frontend
|
||||||
|
|
||||||
|
- **Via nginx** (production): browse to `/s/`
|
||||||
|
- **Via Python backend** (local dev): browse to `http://localhost:8080/s/`
|
||||||
|
|
||||||
|
### Fonts
|
||||||
|
|
||||||
|
The CSS includes a Google Fonts `@import` that works out of the box. For strict local-serve deployments, replace it with locally-hosted font files (use navpage's `fetch_fonts.py` as a reference).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security & Hardening
|
||||||
|
|
||||||
|
### CORS
|
||||||
|
All responses include `Access-Control-Allow-Origin: *` headers. `OPTIONS` preflight requests are handled automatically.
|
||||||
|
|
||||||
|
### Rate limiting
|
||||||
|
A per-IP sliding-window rate limiter protects all endpoints. Default: 60 requests per 60-second window (configurable via `rate_limit_requests` and `rate_limit_window`). Behind a reverse proxy, the real client IP is extracted from `X-Real-IP` / `X-Forwarded-For` headers. Returns `429 Too Many Requests` when exceeded.
|
||||||
|
|
||||||
|
### Field validation
|
||||||
|
All limits are configurable via `config.json`.
|
||||||
|
|
||||||
|
| Field | Constraint (defaults) |
|
||||||
|
|------------------|------------------------------------------------|
|
||||||
|
| `url` | Max `max_url_length` (2048) chars, valid http(s) |
|
||||||
|
| `retention_days` | 0–`max_retention_days` (3650) |
|
||||||
|
| `short_code` | Alphanumeric only, max `max_short_length` (32) chars |
|
||||||
|
|
||||||
|
### Security headers
|
||||||
|
Every response includes: `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `X-XSS-Protection: 1; mode=block`.
|
||||||
|
|
||||||
|
### SQL injection prevention
|
||||||
|
All database queries use parameterized statements (`?` placeholders).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Nginx configuration
|
||||||
|
|
||||||
|
`nginx.conf` contains **location blocks only** — drop them into an existing `server { }` block. Adjust the `alias` paths to match your deployment layout.
|
||||||
|
|
||||||
|
When running without nginx (Docker Compose or local dev), the Python backend serves the frontend directly at the base URL.
|
||||||
|
|
||||||
|
|||||||
@@ -45,8 +45,8 @@ I'd like to implement a URL shortener, mocking the de-facto `urlshortener` proje
|
|||||||
6. GET `<base_url>/<short_id>` - redirects to the original URL (302). Increments the visit count on each hit.
|
6. GET `<base_url>/<short_id>` - redirects to the original URL (302). Increments the visit count on each hit.
|
||||||
|
|
||||||
# Simple Frontend
|
# Simple Frontend
|
||||||
* Implement a clean and simple frontend, follow the same design and style requirements for the single page HTML located in the `navpage` folder.
|
* Implement a clean and simple frontend, follow the same design and style requirements for the single page HTML located in the `navpage` folder, as described in its TASK md file.
|
||||||
* Add an optional one-line API key input field to top left corner, so that admin can access to the restricted APIs.
|
* Add an optional one-line API key input field to top right corner, so that admin can access to the restricted APIs.
|
||||||
* There is a long search input bar on top center of the page, where user can input a URL to shorten once enter is hit. if shortening is successful, show all its metadata below in a table at center of the page below the search bar;
|
* There is a long search input bar on top center of the page, where user can input a URL to shorten once enter is hit. if shortening is successful, show all its metadata below in a table at center of the page below the search bar;
|
||||||
* If the input URL is existing (even without hitting enter), also show all its metadata in the table.
|
* If the input URL is existing (even without hitting enter), also show all its metadata in the table.
|
||||||
* If the input URL is malformed on hitting enter, display nothing below the search bar.
|
* If the input URL is malformed on hitting enter, display nothing below the search bar.
|
||||||
|
|||||||
+7
-2
@@ -1,10 +1,15 @@
|
|||||||
{
|
{
|
||||||
"base_url": "http://localhost:8080/s",
|
"base_url": "http://localhost:8080/s",
|
||||||
"short_length": 6,
|
|
||||||
"api_key": "change-this-secret-key",
|
"api_key": "change-this-secret-key",
|
||||||
"host": "0.0.0.0",
|
"host": "0.0.0.0",
|
||||||
"port": 8080,
|
"port": 8080,
|
||||||
"db_path": "data/urlshort.db",
|
"db_path": "data/urlshort.db",
|
||||||
"retention_days": 0
|
"retention_days": 0,
|
||||||
|
"min_short_length": 6,
|
||||||
|
"max_short_length": 32,
|
||||||
|
"max_url_length": 2048,
|
||||||
|
"max_retention_days": 3650,
|
||||||
|
"rate_limit_requests": 60,
|
||||||
|
"rate_limit_window": 60
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+264
-41
@@ -4,24 +4,43 @@ URL Shortener — stdlib-only, Python 3.8+
|
|||||||
Usage: python urlshort.py <config.json>
|
Usage: python urlshort.py <config.json>
|
||||||
|
|
||||||
API (all routes are prefixed with base_path, e.g. /s):
|
API (all routes are prefixed with base_path, e.g. /s):
|
||||||
GET / Health check
|
GET / Frontend (or health check if no static/)
|
||||||
|
GET /api/health Health check
|
||||||
POST /api/shorten Create a short URL (API key required)
|
POST /api/shorten Create a short URL (API key required)
|
||||||
GET /api/urls List all short URLs (API key required)
|
GET /api/urls List all short URLs (API key required)
|
||||||
GET /api/urls/<code> Get info for a code (no API key required)
|
GET /api/urls/<code> Get info for a code (no API key required)
|
||||||
|
GET /api/lookup?url=<url> Look up by original URL (no API key required)
|
||||||
DELETE /api/urls/<code> Delete a short URL (API key required)
|
DELETE /api/urls/<code> Delete a short URL (API key required)
|
||||||
GET /<code> Redirect to original URL
|
GET /<code> Redirect to original URL
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import logging
|
||||||
import os
|
import os
|
||||||
import random
|
import random
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import string
|
import string
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
|
import collections
|
||||||
|
import mimetypes
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||||
from urllib.parse import parse_qs, urlparse
|
from urllib.parse import parse_qs, urlparse
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Logging — outputs to console (stdout); Docker captures it automatically.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s %(levelname)s %(message)s",
|
||||||
|
datefmt="%Y-%m-%d %H:%M:%S",
|
||||||
|
stream=sys.stdout,
|
||||||
|
)
|
||||||
|
log = logging.getLogger("urlshort")
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Config
|
# Config
|
||||||
@@ -31,19 +50,26 @@ def load_config(path: str) -> dict:
|
|||||||
with open(path, "r", encoding="utf-8") as fh:
|
with open(path, "r", encoding="utf-8") as fh:
|
||||||
cfg = json.load(fh)
|
cfg = json.load(fh)
|
||||||
|
|
||||||
for key in ("base_url", "short_length", "api_key"):
|
for key in ("base_url", "api_key"):
|
||||||
if key not in cfg:
|
if key not in cfg:
|
||||||
raise ValueError(f"Missing required config key: '{key}'")
|
raise ValueError(f"Missing required config key: '{key}'")
|
||||||
|
|
||||||
|
# Backward compat: old 'short_length' → 'min_short_length'
|
||||||
|
if "short_length" in cfg and "min_short_length" not in cfg:
|
||||||
|
cfg["min_short_length"] = cfg["short_length"]
|
||||||
|
|
||||||
cfg.setdefault("host", "0.0.0.0")
|
cfg.setdefault("host", "0.0.0.0")
|
||||||
cfg.setdefault("port", 8080)
|
cfg.setdefault("port", 8080)
|
||||||
cfg.setdefault("db_path", "data/urlshort.db")
|
cfg.setdefault("db_path", "data/urlshort.db")
|
||||||
cfg.setdefault("retention_days", 0)
|
cfg.setdefault("retention_days", 0)
|
||||||
|
cfg.setdefault("min_short_length", 6)
|
||||||
|
cfg.setdefault("max_short_length", 32)
|
||||||
|
cfg.setdefault("max_url_length", 2048)
|
||||||
|
cfg.setdefault("max_retention_days", 3650)
|
||||||
|
cfg.setdefault("rate_limit_requests", 60)
|
||||||
|
cfg.setdefault("rate_limit_window", 60)
|
||||||
|
|
||||||
# Derive base_path from the path component of base_url.
|
# Derive base_path from the path component of base_url.
|
||||||
# e.g. "http://example.com/s" → "/s"
|
|
||||||
# "http://example.com/s/" → "/s"
|
|
||||||
# "http://example.com" → ""
|
|
||||||
raw = urlparse(cfg["base_url"]).path.strip("/")
|
raw = urlparse(cfg["base_url"]).path.strip("/")
|
||||||
cfg["base_path"] = f"/{raw}" if raw else ""
|
cfg["base_path"] = f"/{raw}" if raw else ""
|
||||||
|
|
||||||
@@ -118,6 +144,53 @@ def is_valid_url(url: str) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Validation helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_CODE_RE = re.compile(r'^[A-Za-z0-9]+$')
|
||||||
|
|
||||||
|
|
||||||
|
def is_valid_code(code: str, max_length: int = 32) -> bool:
|
||||||
|
"""Short codes must be alphanumeric and within length limits."""
|
||||||
|
return bool(code) and len(code) <= max_length and bool(_CODE_RE.match(code))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Static file serving
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
STATIC_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "static")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Rate limiter (sliding window, per IP)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class RateLimiter:
|
||||||
|
"""Simple in-memory sliding-window rate limiter."""
|
||||||
|
|
||||||
|
def __init__(self, max_requests: int = 60, window: int = 60):
|
||||||
|
self.max_requests = max_requests
|
||||||
|
self.window = window
|
||||||
|
self._hits: dict = collections.defaultdict(list)
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
def is_allowed(self, ip: str) -> bool:
|
||||||
|
now = time.time()
|
||||||
|
cutoff = now - self.window
|
||||||
|
with self._lock:
|
||||||
|
hits = self._hits[ip]
|
||||||
|
self._hits[ip] = hits = [t for t in hits if t > cutoff]
|
||||||
|
if len(hits) >= self.max_requests:
|
||||||
|
return False
|
||||||
|
hits.append(now)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
_rate_limiter = RateLimiter()
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# HTTP Handler
|
# HTTP Handler
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -132,6 +205,20 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
# Routing helpers
|
# Routing helpers
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _client_ip(self) -> str:
|
||||||
|
"""Get the real client IP, checking reverse-proxy headers first.
|
||||||
|
|
||||||
|
nginx adds X-Real-IP and X-Forwarded-For via proxy_set_header.
|
||||||
|
Without these headers, falls back to the TCP connection source.
|
||||||
|
"""
|
||||||
|
ip = self.headers.get("X-Real-IP", "").strip()
|
||||||
|
if ip:
|
||||||
|
return ip
|
||||||
|
xff = self.headers.get("X-Forwarded-For", "").strip()
|
||||||
|
if xff:
|
||||||
|
return xff.split(",")[0].strip()
|
||||||
|
return self.client_address[0]
|
||||||
|
|
||||||
def _local_path(self):
|
def _local_path(self):
|
||||||
"""Return the request path with base_path prefix stripped.
|
"""Return the request path with base_path prefix stripped.
|
||||||
|
|
||||||
@@ -155,11 +242,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
def log_message(self, fmt, *args): # noqa: N802 – stdlib override
|
def log_message(self, fmt, *args): # noqa: N802 – stdlib override
|
||||||
sys.stderr.write(
|
log.info("%s - %s", self._client_ip(), fmt % args)
|
||||||
f"[{self.log_date_time_string()}] {self.address_string()} - "
|
|
||||||
+ (fmt % args)
|
|
||||||
+ "\n"
|
|
||||||
)
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Low-level response helpers
|
# Low-level response helpers
|
||||||
@@ -195,6 +278,20 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
self.send_header("Content-Length", "0")
|
self.send_header("Content-Length", "0")
|
||||||
self.end_headers()
|
self.end_headers()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# CORS & Security headers (injected into every response)
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def end_headers(self):
|
||||||
|
self.send_header("Access-Control-Allow-Origin", "*")
|
||||||
|
self.send_header("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS")
|
||||||
|
self.send_header("Access-Control-Allow-Headers", "Content-Type")
|
||||||
|
self.send_header("Access-Control-Max-Age", "86400")
|
||||||
|
self.send_header("X-Content-Type-Options", "nosniff")
|
||||||
|
self.send_header("X-Frame-Options", "DENY")
|
||||||
|
self.send_header("X-XSS-Protection", "1; mode=block")
|
||||||
|
super().end_headers()
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Auth — purely api_key based (query param or JSON body)
|
# Auth — purely api_key based (query param or JSON body)
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -232,12 +329,31 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
def do_GET(self): # noqa: N802
|
def do_GET(self): # noqa: N802
|
||||||
cleanup_expired(self.cfg["db_path"])
|
cleanup_expired(self.cfg["db_path"])
|
||||||
|
|
||||||
|
if not _rate_limiter.is_allowed(self._client_ip()):
|
||||||
|
self._error(429, "Too many requests")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Redirect bare base path to base path + / for correct relative URLs
|
||||||
|
raw = urlparse(self.path).path
|
||||||
|
base = self.cfg.get("base_path", "")
|
||||||
|
if base and raw == base:
|
||||||
|
self._redirect(base + "/")
|
||||||
|
return
|
||||||
|
|
||||||
path = self._local_path()
|
path = self._local_path()
|
||||||
if path is None:
|
if path is None:
|
||||||
self._send_empty(404)
|
self._send_empty(404)
|
||||||
return
|
return
|
||||||
|
|
||||||
if path == "/":
|
if path == "/":
|
||||||
|
# Serve frontend if static/index.html exists, else health check
|
||||||
|
index_path = os.path.join(STATIC_DIR, "index.html")
|
||||||
|
if os.path.isfile(index_path):
|
||||||
|
self._serve_static("index.html")
|
||||||
|
else:
|
||||||
|
self._send_json(200, {"status": "ok", "service": "url-shortener"})
|
||||||
|
|
||||||
|
elif path == "/api/health":
|
||||||
self._send_json(200, {"status": "ok", "service": "url-shortener"})
|
self._send_json(200, {"status": "ok", "service": "url-shortener"})
|
||||||
|
|
||||||
elif path == "/api/urls":
|
elif path == "/api/urls":
|
||||||
@@ -247,6 +363,13 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
code = path[len("/api/urls/"):]
|
code = path[len("/api/urls/"):]
|
||||||
self._handle_get_url(code)
|
self._handle_get_url(code)
|
||||||
|
|
||||||
|
elif path == "/api/lookup":
|
||||||
|
self._handle_lookup()
|
||||||
|
|
||||||
|
elif path == "/static" or path.startswith("/static/"):
|
||||||
|
rel = path[len("/static"):].lstrip("/") or "index.html"
|
||||||
|
self._serve_static(rel)
|
||||||
|
|
||||||
else:
|
else:
|
||||||
code = path.lstrip("/")
|
code = path.lstrip("/")
|
||||||
self._handle_redirect(code)
|
self._handle_redirect(code)
|
||||||
@@ -254,6 +377,10 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
def do_POST(self): # noqa: N802
|
def do_POST(self): # noqa: N802
|
||||||
cleanup_expired(self.cfg["db_path"])
|
cleanup_expired(self.cfg["db_path"])
|
||||||
|
|
||||||
|
if not _rate_limiter.is_allowed(self._client_ip()):
|
||||||
|
self._error(429, "Too many requests")
|
||||||
|
return
|
||||||
|
|
||||||
path = self._local_path()
|
path = self._local_path()
|
||||||
if path is None:
|
if path is None:
|
||||||
self._send_empty(404)
|
self._send_empty(404)
|
||||||
@@ -267,6 +394,10 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
def do_DELETE(self): # noqa: N802
|
def do_DELETE(self): # noqa: N802
|
||||||
cleanup_expired(self.cfg["db_path"])
|
cleanup_expired(self.cfg["db_path"])
|
||||||
|
|
||||||
|
if not _rate_limiter.is_allowed(self._client_ip()):
|
||||||
|
self._error(429, "Too many requests")
|
||||||
|
return
|
||||||
|
|
||||||
path = self._local_path()
|
path = self._local_path()
|
||||||
if path is None:
|
if path is None:
|
||||||
self._send_empty(404)
|
self._send_empty(404)
|
||||||
@@ -278,6 +409,14 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
else:
|
else:
|
||||||
self._send_empty(404)
|
self._send_empty(404)
|
||||||
|
|
||||||
|
def do_OPTIONS(self): # noqa: N802
|
||||||
|
"""Handle CORS preflight requests."""
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Length", "0")
|
||||||
|
self.send_header("Allow", "GET, POST, DELETE, OPTIONS")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.flush()
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Handlers
|
# Handlers
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -303,19 +442,10 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
if not is_valid_url(original_url):
|
if not is_valid_url(original_url):
|
||||||
self._error(400, "Invalid URL — must start with http:// or https://")
|
self._error(400, "Invalid URL — must start with http:// or https://")
|
||||||
return
|
return
|
||||||
|
max_url = self.cfg["max_url_length"]
|
||||||
# short_length: query param, then body, then config default
|
if len(original_url) > max_url:
|
||||||
raw_sl = qs.get("short_length", [None])[0]
|
self._error(400, f"URL too long (max {max_url} characters)")
|
||||||
if raw_sl is None:
|
return
|
||||||
raw_sl = body.get("short_length")
|
|
||||||
if raw_sl is not None:
|
|
||||||
try:
|
|
||||||
short_length = int(raw_sl)
|
|
||||||
except (ValueError, TypeError):
|
|
||||||
self._error(400, "Invalid short_length")
|
|
||||||
return
|
|
||||||
else:
|
|
||||||
short_length = self.cfg["short_length"]
|
|
||||||
|
|
||||||
# retention_days: query param, then body, then config default
|
# retention_days: query param, then body, then config default
|
||||||
raw_rd = qs.get("retention_days", [None])[0]
|
raw_rd = qs.get("retention_days", [None])[0]
|
||||||
@@ -330,8 +460,13 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
else:
|
else:
|
||||||
retention_days = self.cfg.get("retention_days", 0)
|
retention_days = self.cfg.get("retention_days", 0)
|
||||||
|
|
||||||
|
max_ret = self.cfg["max_retention_days"]
|
||||||
|
if not (0 <= retention_days <= max_ret):
|
||||||
|
self._error(400, f"retention_days must be 0–{max_ret}")
|
||||||
|
return
|
||||||
|
|
||||||
with db_connect(self.cfg["db_path"]) as conn:
|
with db_connect(self.cfg["db_path"]) as conn:
|
||||||
short_code = self._unique_code(conn, short_length)
|
short_code = self._unique_code(conn)
|
||||||
if short_code is None:
|
if short_code is None:
|
||||||
self._error(500, "Could not generate a unique short code — try again")
|
self._error(500, "Could not generate a unique short code — try again")
|
||||||
return
|
return
|
||||||
@@ -367,11 +502,16 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
"original_url": r["original_url"],
|
"original_url": r["original_url"],
|
||||||
"created_at": r["created_at"],
|
"created_at": r["created_at"],
|
||||||
"visit_count": r["visit_count"],
|
"visit_count": r["visit_count"],
|
||||||
|
"retention_days": r["retention_days"],
|
||||||
})
|
})
|
||||||
self._send_json(200, {"count": len(urls), "urls": urls})
|
self._send_json(200, {"count": len(urls), "urls": urls})
|
||||||
|
|
||||||
def _handle_get_url(self, code: str) -> None:
|
def _handle_get_url(self, code: str) -> None:
|
||||||
# No API key required
|
# No API key required
|
||||||
|
if not is_valid_code(code, self.cfg["max_short_length"]):
|
||||||
|
self._send_empty(404)
|
||||||
|
return
|
||||||
|
|
||||||
with db_connect(self.cfg["db_path"]) as conn:
|
with db_connect(self.cfg["db_path"]) as conn:
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT * FROM urls WHERE short_code = ?", (code,)
|
"SELECT * FROM urls WHERE short_code = ?", (code,)
|
||||||
@@ -388,6 +528,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
"original_url": row["original_url"],
|
"original_url": row["original_url"],
|
||||||
"created_at": row["created_at"],
|
"created_at": row["created_at"],
|
||||||
"visit_count": row["visit_count"],
|
"visit_count": row["visit_count"],
|
||||||
|
"retention_days": row["retention_days"],
|
||||||
})
|
})
|
||||||
|
|
||||||
def _handle_delete_url(self, code: str) -> None:
|
def _handle_delete_url(self, code: str) -> None:
|
||||||
@@ -395,6 +536,9 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
if not self._check_api_key():
|
if not self._check_api_key():
|
||||||
self._send_empty(403)
|
self._send_empty(403)
|
||||||
return
|
return
|
||||||
|
if not is_valid_code(code, self.cfg["max_short_length"]):
|
||||||
|
self._send_empty(404)
|
||||||
|
return
|
||||||
|
|
||||||
with db_connect(self.cfg["db_path"]) as conn:
|
with db_connect(self.cfg["db_path"]) as conn:
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
@@ -411,6 +555,10 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
self._send_empty(204)
|
self._send_empty(204)
|
||||||
|
|
||||||
def _handle_redirect(self, code: str) -> None:
|
def _handle_redirect(self, code: str) -> None:
|
||||||
|
if not is_valid_code(code, self.cfg["max_short_length"]):
|
||||||
|
self._send_empty(404)
|
||||||
|
return
|
||||||
|
|
||||||
with db_connect(self.cfg["db_path"]) as conn:
|
with db_connect(self.cfg["db_path"]) as conn:
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT original_url FROM urls WHERE short_code = ?", (code,)
|
"SELECT original_url FROM urls WHERE short_code = ?", (code,)
|
||||||
@@ -426,20 +574,85 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
|
|
||||||
self._redirect(row["original_url"])
|
self._redirect(row["original_url"])
|
||||||
|
|
||||||
|
def _handle_lookup(self) -> None:
|
||||||
|
"""Look up a URL by its original URL. No API key required."""
|
||||||
|
qs = parse_qs(urlparse(self.path).query)
|
||||||
|
url = qs.get("url", [""])[0].strip()
|
||||||
|
if not url:
|
||||||
|
self._error(400, "Missing required parameter: url")
|
||||||
|
return
|
||||||
|
max_url = self.cfg["max_url_length"]
|
||||||
|
if len(url) > max_url:
|
||||||
|
self._error(400, f"URL too long (max {max_url} characters)")
|
||||||
|
return
|
||||||
|
|
||||||
|
with db_connect(self.cfg["db_path"]) as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT * FROM urls WHERE original_url = ? ORDER BY created_at DESC LIMIT 1",
|
||||||
|
(url,),
|
||||||
|
).fetchone()
|
||||||
|
|
||||||
|
if row is None:
|
||||||
|
self._send_empty(404)
|
||||||
|
return
|
||||||
|
|
||||||
|
base = self.cfg["base_url"].rstrip("/")
|
||||||
|
self._send_json(200, {
|
||||||
|
"short_code": row["short_code"],
|
||||||
|
"short_url": f"{base}/{row['short_code']}",
|
||||||
|
"original_url": row["original_url"],
|
||||||
|
"created_at": row["created_at"],
|
||||||
|
"visit_count": row["visit_count"],
|
||||||
|
"retention_days": row["retention_days"],
|
||||||
|
})
|
||||||
|
|
||||||
|
def _serve_static(self, rel_path: str) -> None:
|
||||||
|
"""Serve a static file from STATIC_DIR (for local dev; nginx in prod)."""
|
||||||
|
if not rel_path:
|
||||||
|
rel_path = "index.html"
|
||||||
|
# Prevent directory traversal
|
||||||
|
safe = os.path.normpath(rel_path)
|
||||||
|
if safe.startswith("..") or os.path.isabs(safe):
|
||||||
|
self._send_empty(403)
|
||||||
|
return
|
||||||
|
fpath = os.path.join(STATIC_DIR, safe)
|
||||||
|
if not os.path.isfile(fpath):
|
||||||
|
self._send_empty(404)
|
||||||
|
return
|
||||||
|
mime, _ = mimetypes.guess_type(fpath)
|
||||||
|
if not mime:
|
||||||
|
mime = "application/octet-stream"
|
||||||
|
with open(fpath, "rb") as f:
|
||||||
|
data = f.read()
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", mime)
|
||||||
|
self.send_header("Content-Length", str(len(data)))
|
||||||
|
self.send_header("Cache-Control", "public, max-age=3600")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(data)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Internal utils
|
# Internal utils
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
def _unique_code(self, conn: sqlite3.Connection, length: int = None, attempts: int = 10):
|
def _unique_code(self, conn: sqlite3.Connection, attempts_per_length: int = 10):
|
||||||
if length is None:
|
"""Generate a unique short code with progressive length increment.
|
||||||
length = self.cfg["short_length"]
|
|
||||||
for _ in range(attempts):
|
Starts at min_short_length and tries `attempts_per_length` random
|
||||||
code = generate_code(length)
|
codes at each length. On exhaustion, increments the length by 1
|
||||||
exists = conn.execute(
|
and repeats, up to max_short_length. Returns None only when the
|
||||||
"SELECT 1 FROM urls WHERE short_code = ?", (code,)
|
entire range is exhausted (extremely unlikely).
|
||||||
).fetchone()
|
"""
|
||||||
if not exists:
|
min_len = self.cfg["min_short_length"]
|
||||||
return code
|
max_len = self.cfg["max_short_length"]
|
||||||
|
for length in range(min_len, max_len + 1):
|
||||||
|
for _ in range(attempts_per_length):
|
||||||
|
code = generate_code(length)
|
||||||
|
exists = conn.execute(
|
||||||
|
"SELECT 1 FROM urls WHERE short_code = ?", (code,)
|
||||||
|
).fetchone()
|
||||||
|
if not exists:
|
||||||
|
return code
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
@@ -448,29 +661,39 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
|
global _rate_limiter
|
||||||
|
|
||||||
if len(sys.argv) != 2:
|
if len(sys.argv) != 2:
|
||||||
print(f"Usage: {sys.argv[0]} <config.json>", file=sys.stderr)
|
log.error("Usage: %s <config.json>", sys.argv[0])
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
cfg = load_config(sys.argv[1])
|
cfg = load_config(sys.argv[1])
|
||||||
init_db(cfg["db_path"])
|
init_db(cfg["db_path"])
|
||||||
cleanup_expired(cfg["db_path"])
|
cleanup_expired(cfg["db_path"])
|
||||||
|
|
||||||
|
# Re-initialize rate limiter with config values
|
||||||
|
_rate_limiter = RateLimiter(
|
||||||
|
max_requests=cfg["rate_limit_requests"],
|
||||||
|
window=cfg["rate_limit_window"],
|
||||||
|
)
|
||||||
|
|
||||||
Handler.cfg = cfg
|
Handler.cfg = cfg
|
||||||
|
|
||||||
host, port = cfg["host"], int(cfg["port"])
|
host, port = cfg["host"], int(cfg["port"])
|
||||||
server = HTTPServer((host, port), Handler)
|
server = HTTPServer((host, port), Handler)
|
||||||
|
|
||||||
print(f"URL Shortener listening on http://{host}:{port}", file=sys.stderr)
|
log.info("URL Shortener listening on http://%s:%s", host, port)
|
||||||
print(f"Base URL : {cfg['base_url']}", file=sys.stderr)
|
log.info("Base URL : %s", cfg["base_url"])
|
||||||
print(f"Base path : {cfg['base_path'] or '/'}", file=sys.stderr)
|
log.info("Base path : %s", cfg["base_path"] or "/")
|
||||||
print(f"DB path : {cfg['db_path']}", file=sys.stderr)
|
log.info("DB path : %s", cfg["db_path"])
|
||||||
print(f"Retention days : {cfg['retention_days']}", file=sys.stderr)
|
log.info("Short codes : %s–%s chars", cfg["min_short_length"], cfg["max_short_length"])
|
||||||
|
log.info("Retention days : %s", cfg["retention_days"])
|
||||||
|
log.info("Rate limit : %s req/%ss per IP", cfg["rate_limit_requests"], cfg["rate_limit_window"])
|
||||||
|
|
||||||
try:
|
try:
|
||||||
server.serve_forever()
|
server.serve_forever()
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
print("\nShutting down.", file=sys.stderr)
|
log.info("Shutting down.")
|
||||||
server.shutdown()
|
server.shutdown()
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user