cabbage/ushort
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rewrite URL shortener in Rust

Browse Source
This commit is contained in:
cabbage
2026-07-10 04:46:34 +00:00
parent a250ce4c1d
commit a6e8be7553
16 changed files with 2906 additions and 1102 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+161 -271
View File
@@ -1,321 +1,211 @@
# URL Shortener
# ushort
A minimal URL shortener written in pure Python 3.8+ (zero third-party dependencies) backed by SQLite.
`ushort` is a compact URL shortener written in Rust and backed by SQLite. It is
a behavior-compatible replacement for the original stdlib-only Python service:
the routes, base-path handling, response shapes, database schema, retention,
API-key checks, throttling, redirects, and frontend are preserved.
The frontend under `static/` is embedded into the executable at compile time.
The final Linux image contains only one statically linked executable and runs
without Python, a shell, a package manager, or external static files.
## Quick start
### Run locally
```bash
python urlshort.py config.json
cp config.example.toml config.toml
# Edit base_url and api_key, then:
docker compose up --build -d
```
### Run with Docker Compose
The service listens on `127.0.0.1:18082` on the host. Its SQLite database is
stored at `./data/urlshort.db` and survives container replacement.
Edit `config.json` first (especially `api_key` and `base_url`), then:
The container runs as UID/GID `1001:1001` by default. Override
`USHORT_UID`/`USHORT_GID` if needed, and ensure `./data` is writable by that
identity.
## Configuration
TOML is the preferred format; see `config.example.toml`. Existing JSON files
remain supported, including the old `short_length` alias for
`min_short_length`.
```bash
docker compose up --build
ushort config.toml
ushort legacy-config.json
```
The SQLite database is stored in `./data/urlshort.db` on the host — it survives container restarts.
| Key | Required | Default | Description |
|---|---:|---:|---|
| `base_url` | yes | — | Public URL. Its path becomes the routing prefix. |
| `api_key` | yes | — | Secret used by list and delete operations. |
| `host` | no | `0.0.0.0` | Bind address. |
| `port` | no | `8080` | Bind port. |
| `db_path` | no | `data/urlshort.db` | SQLite database path. |
| `retention_days` | no | `0` | Default lifetime for new URLs; zero never expires. |
| `min_short_length` | no | `6` | Initial generated code length. |
| `max_short_length` | no | `32` | Maximum generated and accepted code length. |
| `max_url_length` | no | `2048` | Maximum original URL length. |
| `max_retention_days` | no | `3650` | Maximum per-URL retention value. |
| `rate_limit_requests` | no | `60` | Requests allowed per client/window. |
| `rate_limit_window` | no | `60` | Sliding-window length in seconds. |
| `production` | no | `false` | When true, disable embedded frontend routes for compatibility with deployments that serve static files separately. |
---
For example, `base_url = "https://example.com/go/links"` limits routing to
`/go/links` and produces URLs such as
`https://example.com/go/links/aB3xYz`. Requests outside that prefix return 404.
## Configuration (`config.json`)
Real API keys belong only in the deployment's ignored `config.toml`, never in
the repository or container image.
| Key | Required | Default | Description |
|----------------------|----------|----------------------|-------------------------------------------------------------------|
| `base_url` | ✅ | — | Public base URL. The path component (e.g. `/s` in `http://example.com/s`) is automatically used as the server's routing prefix. |
| `api_key` | ✅ | — | Secret key to protect write/read operations |
| `host` | | `"0.0.0.0"` | Bind address |
| `port` | | `8080` | Bind port |
| `db_path` | | `"data/urlshort.db"` | Path to the SQLite database file |
| `retention_days` | | `0` | Default retention period in days for new URLs. `0` means never expire. |
| `min_short_length` | | `6` | Minimum character length for generated short codes |
| `max_short_length` | | `32` | Maximum character length for generated short codes |
| `max_url_length` | | `2048` | Maximum allowed length for original URLs |
| `max_retention_days` | | `3650` | Maximum allowed retention_days value per URL |
| `rate_limit_requests`| | `60` | Max requests per IP per rate-limit window |
| `rate_limit_window` | | `60` | Rate-limit window in seconds |
## API
> **Backward compat:** If an old config contains `short_length`, it is automatically used as `min_short_length`.
All paths below are relative to the path in `base_url`. Trailing slashes are
accepted. Every response includes the legacy CORS and security headers.
---
### Health and frontend
## Deploying behind a sub-path (e.g. `http://example.com/s`)
- `GET /` serves the byte-identical embedded frontend when `production=false`.
- `GET /api/health` returns HTTP 200:
Just set `base_url` to include the desired path prefix — the server derives its routing prefix automatically from it:
```json
{"status":"ok","service":"url-shortener"}
```
When the configured base path is non-empty, a GET of the bare path (for
example `/s`) redirects to `/s/`.
### Create a short URL
`POST /api/shorten` and `GET /api/shorten` are both supported and do not
require an API key. Fields may be supplied as query parameters or in a JSON
body; query parameters take precedence.
| Field | Required | Description |
|---|---:|---|
| `url` | yes | Absolute `http://` or `https://` URL. |
| `retention_days` | no | Per-URL lifetime, or zero to keep forever. |
Success is HTTP 201 with only the short URL as UTF-8 plain text:
```text
https://example.com/s/aB3xYz
```
Codes begin at `min_short_length`. After ten collisions at one length the
service tries the next length, through `max_short_length`.
### List URLs
`GET /api/urls?api_key=<key>` returns all rows newest first. A missing or
invalid key returns HTTP 403 with an empty body.
```json
{
"base_url": "http://example.com/s",
...
}
```
The path component `/s` is extracted at startup. The server will only respond to requests whose path starts with `/s`; everything else returns 404.
| `base_url` | Derived routing prefix | Short URL example |
|---|---|---|
| `http://example.com` | *(none — root)* | `http://example.com/aB3xYz` |
| `http://example.com/s` | `/s` | `http://example.com/s/aB3xYz` |
| `http://example.com/go/links` | `/go/links` | `http://example.com/go/links/aB3xYz` |
### With Docker Compose
```bash
# Set base_url in config.json, then:
docker compose up --build
# Service is now available at http://localhost:8080/s/
```
### With an existing nginx vhost
`nginx.conf` contains **location blocks only** — drop them into an existing `server { }` block.
The app handles the base_path prefix internally; nginx proxies API/redirect requests and serves frontend static files.
```
Browser ──► nginx /s/ ──► static/index.html
Browser ──► nginx /s/static/… ──► static files (CSS/JS)
Browser ──► nginx /s/api/… ──► urlshort :8080 (proxy)
Browser ──► nginx /s/<code> ──► urlshort :8080 (proxy → 302)
```
To change the prefix, update `base_url` in `config.json` **and** the `location /s` blocks in `nginx.conf`.
---
## Authentication
All endpoints marked with 🔒 require the API key.
Pass it as a **query parameter** or in the **JSON request body**:
```
?api_key=<api_key>
# or in JSON body
{"api_key": "<api_key>", ...}
```
---
## Retention
URLs can have a retention period (`retention_days`). When set to a positive integer, the URL will be automatically deleted after that many days. If `0` or not set, the URL never expires.
- The **default** retention is set in `config.json` (`retention_days` key, default `0`).
- Each URL can override the default at creation time via the `retention_days` field.
- Expired URLs are cleaned up on startup and lazily on each incoming request.
---
## API Reference
### `GET /`
Serves the frontend page (if `static/index.html` exists), otherwise returns health check JSON.
### `GET /api/health`
Health check endpoint.
**Response `200`**
```json
{ "status": "ok", "service": "url-shortener" }
```
---
### `POST / GET /api/shorten`
Create a new short URL (supports both `POST` and `GET`). No API key required.
Fields can be passed as **query parameters** (URL-encoded) or in a **JSON request body**.
Query parameters take precedence over body fields.
| Field | Required | Description |
|------------------|----------|-----------------------------------------------------------|
| `url` | ✅ | The URL to shorten (must start with `http://` or `https://`) |
| `retention_days` | | Override the default retention period for this URL |
Short code length is determined automatically: the server starts at `min_short_length` and
progressively tries longer codes on collision, up to `max_short_length`.
**Response `201`** — plain text containing only the short URL:
```
http://localhost:8080/s/aB3xYz
```
---
### `GET /api/urls` 🔒
List all short URLs, newest first.
**Response `200`**
```json
{
"count": 2,
"count": 1,
"urls": [
{
"short_code": "aB3xYz",
"short_url": "http://localhost:8080/s/aB3xYz",
"original_url": "https://example.com",
"short_url": "https://example.com/s/aB3xYz",
"original_url": "https://example.org",
"created_at": 1710000000,
"visit_count": 5,
"retention_days": 0
"retention_days": 30
}
]
}
```
---
### Metadata and lookup
### `GET /api/urls/<code>`
Get metadata for a single short code. No API key required.
- `GET /api/urls/<code>` returns one metadata object without authentication.
- `GET /api/lookup?url=<encoded-url>` returns the newest matching metadata
object without authentication.
- A missing record returns HTTP 404 with an empty body.
**Response `200`**
```json
{
"short_code": "aB3xYz",
"short_url": "http://localhost:8080/s/aB3xYz",
"original_url": "https://example.com",
"created_at": 1710000000,
"visit_count": 5,
"retention_days": 0
}
### Delete
`DELETE /api/urls/<code>?api_key=<key>` returns:
- HTTP 204 with an empty body on success;
- HTTP 404 with an empty body when the code does not exist; or
- HTTP 403 with an empty body when authentication fails.
### Redirect
`GET /<code>` atomically increments `visit_count` and responds with HTTP 302 to
the stored original URL.
## Data compatibility
The existing database can be mounted directly; no export/import is required.
The schema remains:
```sql
CREATE TABLE urls (
short_code TEXT PRIMARY KEY,
original_url TEXT NOT NULL,
created_at INTEGER NOT NULL,
visit_count INTEGER NOT NULL DEFAULT 0,
retention_days INTEGER NOT NULL DEFAULT 0
);
```
**Response `404`** — code not found (empty body).
Older databases missing `retention_days` are upgraded in place with a default
of zero. Expired records are removed on startup and before each GET, POST, or
DELETE request, using the original strict expiry boundary.
---
### `GET /api/lookup`
Look up a URL by its **original URL**. No API key required.
Used by the frontend to check if a URL has already been shortened.
| Parameter | Required | Description |
|-----------|----------|-------------------------------|
| `url` | ✅ | The original URL to look up |
**Response `200`** — same metadata JSON as `GET /api/urls/<code>`.
**Response `404`** — no short URL exists for this original URL.
---
### `DELETE /api/urls/<code>` 🔒
Delete a short URL entry.
**Response `204`** — success (empty body).
**Response `404`** — code not found (empty body).
**Response `403`** — not authorized (empty body).
---
### `GET /<code>`
Redirect to the original URL (HTTP 302).
Increments `visit_count` on each hit.
---
## Example with `curl`
```bash
# Shorten a URL (POST with JSON body)
curl -X POST http://localhost:8080/s/api/shorten \
-H "Content-Type: application/json" \
-d '{"url": "https://github.com"}'
# Shorten a URL (POST with query parameters)
curl -X POST "http://localhost:8080/s/api/shorten?url=https%3A%2F%2Fgithub.com&retention_days=30"
# Shorten a URL (GET with query parameters)
curl "http://localhost:8080/s/api/shorten?url=https%3A%2F%2Fgithub.com"
# Follow the redirect
curl -L http://localhost:8080/s/aB3xYz
# Get metadata for a short URL (no API key needed)
curl http://localhost:8080/s/api/urls/aB3xYz
# List all URLs
curl "http://localhost:8080/s/api/urls?api_key=change-this-secret-key"
# Delete a URL
curl -X DELETE "http://localhost:8080/s/api/urls/aB3xYz?api_key=change-this-secret-key"
# Lookup by original URL
curl "http://localhost:8080/s/api/lookup?url=https%3A%2F%2Fexample.com"
```
---
Before a production cutover, make a filesystem-consistent backup of the
database (or use SQLite's backup command) and test the copy with the new image.
## Frontend
A clean single-page frontend is included in `static/`. It provides:
The existing `index.html`, JavaScript, CSS, locally hosted fonts, sorting,
pagination, theme selection, URL lookup, copy/delete actions, and admin flow
are unchanged. Compile-time embedding makes those files part of the standalone
artifact; no bind mount or separate web root is needed.
- **URL shortening** — paste a URL and press Enter to create a short URL
- **Existing URL lookup** — as you type a URL, the frontend checks if it already exists and shows its metadata
- **Admin table** — enter a valid API key to see all shortened URLs in a sortable, paginated table
- **Copy & Delete** — per-row copy and delete buttons (delete on hover only)
- **Theme switching** — matches system dark/light preference, with a manual toggle
For the existing `/s` deployment, place the location blocks from `nginx.conf`
in the public vhost. They proxy all traffic to the binary while retaining the
current `no-store` HTML and `no-cache` asset policies.
### Accessing the frontend
## Build and test
- **Via nginx** (production): browse to `/s/`
- **Via Python backend** (local dev): browse to `http://localhost:8080/s/`
The repository intentionally does not require a host Rust installation.
### Fonts
```bash
docker build -t ushort:local .
docker compose up -d
```
The CSS includes a Google Fonts `@import` that works out of the box. For strict local-serve deployments, replace it with locally-hosted font files (use navpage's `fetch_fonts.py` as a reference).
Run the Rust test suite in an isolated build stage:
---
```bash
docker build --target tester .
```
## Security & Hardening
The Dockerfile builds against musl and uses `scratch` for the runtime image.
Docker Buildx can publish both required architectures from the same source:
### CORS
All responses include `Access-Control-Allow-Origin: *` headers. `OPTIONS` preflight requests are handled automatically.
```bash
docker buildx build \
--platform linux/amd64,linux/arm64 \
-t sodium/ushort:latest \
--push .
```
### Rate limiting
A per-IP sliding-window rate limiter protects all endpoints. Default: 60 requests per 60-second window (configurable via `rate_limit_requests` and `rate_limit_window`). Behind a reverse proxy, the real client IP is extracted from `X-Real-IP` / `X-Forwarded-For` headers. Returns `429 Too Many Requests` when exceeded.
### Field validation
All limits are configurable via `config.json`.
| Field | Constraint (defaults) |
|------------------|------------------------------------------------|
| `url` | Max `max_url_length` (2048) chars, valid http(s) |
| `retention_days` | 0–`max_retention_days` (3650) |
| `short_code` | Alphanumeric only, max `max_short_length` (32) chars |
### Security headers
Every response includes: `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `X-XSS-Protection: 1; mode=block`.
### SQL injection prevention
All database queries use parameterized statements (`?` placeholders).
---
## Nginx configuration
`nginx.conf` contains **location blocks only** — drop them into an existing `server { }` block. Adjust the `alias` paths to match your deployment layout.
When running without nginx (Docker Compose or local dev), the Python backend serves the frontend directly at the base URL. Set `"production": true` in config to disable backend static serving (nginx handles it).
### Caching strategy
The nginx config uses a layered caching strategy designed for deployments behind Cloudflare or other CDNs:
| Resource | `Cache-Control` | Why |
|---|---|---|
| `/s/` (index.html) | `no-store` | Never cached by CDN or browser. This is the HTML entry point (~2KB) that contains `?v=` cache-buster query strings for CSS/JS. Must always be fresh so that version bumps take effect immediately. |
| `/s/static/*.css?v=…` | `no-cache` | Cached but revalidated on each request. The `?v=` query string acts as a cache key — Cloudflare and browsers treat each version as a distinct resource. Bump the `?v=` value in `index.html` whenever CSS/JS files change. |
| `/s/static/*.js?v=…` | `no-cache` | Same as CSS. |
| `/s/api/*`, `/s/<code>` | (proxied) | Not cached by nginx; backend controls caching via response headers. |
### Updating static files
When you modify `style.css` or `app.js`:
1. Deploy the updated files to the server
2. Bump the `?v=` value in `static/index.html` (e.g. `?v=20260318` → `?v=20260319`)
3. Reload nginx (`nginx -s reload`)
Since `index.html` has `no-store`, browsers and Cloudflare always fetch the latest version, which in turn references the new `?v=` URLs for CSS/JS — busting all downstream caches automatically. No manual CDN purge is needed.
## Security model
- SQL statements use bound parameters.
- Short codes are restricted to ASCII alphanumerics.
- URL, retention, code, and request-body sizes are bounded.
- API-key comparison is constant-time; protected operations return no body on
authentication failure.
- Per-IP sliding-window throttling applies to API calls and redirects; embedded
frontend files do not consume API quota. The app uses `X-Real-IP`, then the
first `X-Forwarded-For` value, then the TCP peer. Keep the container port
bound to loopback and let the trusted reverse proxy overwrite those headers.
- CORS preflight uses `OPTIONS`; it does not consume rate-limit quota.
- The runtime image is read-only, drops Linux capabilities, and runs as a
numeric non-root user through Compose.
- `SIGINT`/`SIGTERM` cleanly unblock the server for prompt container shutdown.