exodevices/deploy/README.md

# Production deployment — four EXO landings on one VPS

Four independent landings, each on its own domain root, on a single reg.ru KVM VPS.
nginx serves the built SPA off disk; a small Express process per app handles only
`/api/*` and forwards leads to amoCRM. Releases are built on the server and swapped
in atomically via a `current` symlink.

`apps.conf` is the single source of truth for domains and ports — check it before provisioning.

| App | Domain | Port | amoCRM tag |
|---|---|---|---|
| `fitnes` | `fitness.exorecovery.ru` | 3000 | `fitness-landing` |
| `hotel` | `hotel.exorecovery.ru` | 3001 | `hotel-landing` |
| `medcenterphysio` | `physio.exorecovery.ru` | 3002 | `physio-landing` |
| `medcenterstart` | `start.exorecovery.ru` | 3003 | `start-landing` |

## VPS requirements

|  | Minimum | Recommended |
|---|---|---|
| vCPU | 2 | 2–4 |
| RAM | 4 GB + 2 GB swap | 4–8 GB |
| Disk | 40 GB NVMe | 60–80 GB NVMe |
| Network | 1 IPv4, ≥100 Mbps | + IPv6 |
| Virtualization | **KVM** | KVM |

Take the **KVM** line, not OpenVZ/LXC — the systemd sandboxing in `systemd/exo@.service`
and swap control need a real kernel. Moscow or SPb DC: leads are personal data of
Russian citizens, so 152-ФЗ wants them processed on RU infrastructure.

Why these numbers: four Node processes ≈ 400 MB total; nginx ~30 MB; Ubuntu ~300 MB —
idle well under 1 GB. The only spike is a build (~0.5–1 GB, one app at a time; the
toolchain is native — TypeScript 7 in Go, Vite 8 on Rolldown, Tailwind's Rust oxide).
Disk: ~40 MB per pruned release, 3 kept × 4 apps ≈ 1.5 GB steady.

## Software

| Component | Version |
|---|---|
| Ubuntu | 24.04 LTS (Debian 12/13 fine) |
| Node.js | 24.x LTS — hard floor is `>=22.12` from transitive deps |
| npm | 11.x (bundled) |
| nginx | ≥1.24 — needs `http2`, `gzip_static`, `ssl`, all built in |
| certbot | `python3-certbot-nginx` |
| ufw, fail2ban, unattended-upgrades | distro |

No database, no Redis, no Docker, no PM2. See §14 of the plan for why not Docker.

## Provisioning

### 1. Base OS

```bash
apt update && apt full-upgrade -y
apt install -y git curl nginx ufw fail2ban unattended-upgrades gzip
timedatectl set-timezone Europe/Moscow
dpkg-reconfigure --priority=low unattended-upgrades

# Swap, if the plan doesn't provide it
fallocate -l 2G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab
echo 'vm.swappiness=10' > /etc/sysctl.d/99-swappiness.conf && sysctl --system
```

### 2. Node 24

```bash
curl -fsSL https://deb.nodesource.com/setup_24.x | bash -
apt install -y nodejs && node -v   # v24.x
```

### 3. User, directories, repos

```bash
adduser --system --group --home /srv/exo --shell /bin/bash exo
mkdir -p /srv/exo/{fitnes,hotel,medcenterphysio,medcenterstart}/releases
mkdir -p /etc/exo /var/www/certbot
chown -R exo:exo /srv/exo
chmod 755 /srv/exo          # www-data must traverse to reach dist/client
chown root:exo /etc/exo     # without the group, exo cannot traverse it at 750
chmod 750 /etc/exo

for app in fitnes hotel medcenterphysio medcenterstart; do
  sudo -u exo git clone --bare <REPO_URL> /srv/exo/$app/repo
done
```

All four clone the same repo; `exo-deploy` extracts one subdirectory each.

### 4. Env files, per app

Two files with different trust levels — see `env/app.env.example` and
`env/app.build.env.example` for annotated templates.

```bash
install -m 640 -o root -g exo env/app.env.example       /etc/exo/fitnes.env
install -m 644                 env/app.build.env.example /etc/exo/fitnes.build.env
# then edit both: PORT, AMO_LONG_LIVED_TOKEN, AMO_LEAD_TAGS, VITE_SITE_URL
```

The split matters. `VITE_SITE_URL` is a **build-time HTML substitution** — if it is
absent when `vite build` runs, the literal `%VITE_SITE_URL%` ships into the canonical
tag and OG metadata (`exo-deploy` aborts if it detects this). Keeping the amoCRM token
in the 640 runtime file means it never lands in a release directory. `dotenv` does not
override variables already in the environment, so the release `.env` (VITE vars) and
systemd's `EnvironmentFile` (secrets) coexist and the secrets win.

### 5. systemd

```bash
install -m 644 systemd/exo@.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable exo@fitnes exo@hotel exo@medcenterphysio exo@medcenterstart
```

Let the deploy user restart only its own units — `visudo -f /etc/sudoers.d/exo-deploy`:

```
exo ALL=(root) NOPASSWD: /usr/bin/systemctl restart exo@fitnes, \
  /usr/bin/systemctl restart exo@hotel, \
  /usr/bin/systemctl restart exo@medcenterphysio, \
  /usr/bin/systemctl restart exo@medcenterstart
```

### 6. Firewall and log retention

```bash
ufw default deny incoming && ufw default allow outgoing
ufw allow OpenSSH && ufw allow 80/tcp && ufw allow 443/tcp && ufw enable
```

Ports 3000–3003 are never opened, and the app binds `127.0.0.1` by default anyway
(`HOST` in `server/src/config.ts`). In `/etc/ssh/sshd_config.d/99-hardening.conf` set
`PasswordAuthentication no` and `PermitRootLogin no` — but install your key on a
sudo-capable non-root user *first*.

On an amoCRM failure the server deliberately logs the whole lead payload so a real
lead is never lost. That puts names and phones in the journal, so bound it in
`/etc/systemd/journald.conf`:

```ini
[Journal]
SystemMaxUse=500M
MaxRetentionSec=14day
```

### 7. nginx

```bash
install -m 644 nginx/snippets/*.conf /etc/nginx/snippets/
install -m 644 nginx/http-extras.conf /etc/nginx/conf.d/00-exo-http.conf

# Debian's stock nginx.conf sets some of the same http{} directives (gzip,
# server_tokens), and nginx refuses to start on a duplicate. Comment out every
# stock directive that 00-exo-http.conf now owns.
for d in $(grep -oE '^[a-z_]+' /etc/nginx/conf.d/00-exo-http.conf | sort -u); do
  sed -i -E "s@^([[:space:]]*)($d[[:space:]]+[^;]*;)@\\1# \\2@" /etc/nginx/nginx.conf
done

bin/exo-render-nginx /etc/nginx/sites-available     # after filling in apps.conf
ln -s /etc/nginx/sites-available/exo-*.conf /etc/nginx/sites-enabled/
rm -f /etc/nginx/sites-enabled/default
nginx -t && systemctl reload nginx
```

### 8. DNS and TLS

Order matters: nginx needs the port-80 vhost live before certbot can validate.

1. Per domain at reg.ru DNS: `A @ → <VPS IPv4>`, `A www → <VPS IPv4>` (+ `AAAA` if available).
2. Confirm: `dig +short <domain> @77.88.8.8`.
3. Comment out the two `443` blocks in each rendered vhost (they reference certs that
   don't exist yet), then `nginx -t && systemctl reload nginx`.
4. Issue apex + www together, per domain:
   ```bash
   certbot certonly --webroot -w /var/www/certbot \
     -d <domain> -d www.<domain> --agree-tos -m <admin-email> --no-eff-email
   ```
5. Uncomment the `443` blocks, `nginx -t && systemctl reload nginx`.
6. Make renewal reload nginx:
   ```bash
   printf '#!/bin/sh\nsystemctl reload nginx\n' > /etc/letsencrypt/renewal-hooks/deploy/reload-nginx
   chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx
   certbot renew --dry-run
   ```

### 9. First deploy

```bash
install -m 755 bin/exo-deploy /usr/local/bin/
sudo -u exo exo-deploy medcenterphysio main
```

## Runbook

| Task | Command |
|---|---|
| Deploy | `sudo -u exo exo-deploy <app> [ref]` |
| Rollback | `ln -sfn /srv/exo/<app>/releases/<older> /srv/exo/<app>/current && sudo systemctl restart exo@<app>` |
| Tail logs | `journalctl -u exo@<app> -f` |
| All leads, last hour | `journalctl -u 'exo@*' --since '1 hour ago' \| grep '\[lead\]'` |
| Restart all | `systemctl restart 'exo@*'` |
| Check amoCRM wiring | `cd /srv/exo/<app>/current && npm run amo:check` |
| Rotate amoCRM token | edit `/etc/exo/*.env` → `systemctl restart 'exo@*'` → verify `"amo":true` |
| Renew certs | `certbot renew && systemctl reload nginx` |
| Disk usage | `du -sh /srv/exo/*/releases/*` |

`exo-deploy` health-checks after the swap and **rolls back automatically** if the new
release fails to answer `/api/health`.

## Monitoring

Each app answers `GET /api/health` with `{ ok, amo, pipelineId }`. Point an off-box
uptime checker at `https://<domain>/api/health` for all four, alerting on non-200 **and
on `"amo":false`** — the latter is how an expired amoCRM token shows up, and systemd
cannot see it. `Restart=always` covers crashes.

Log markers worth alerting on: `[lead] amoCRM submission failed`, `[lead] payload was:`
(a recoverable lead sitting in the journal), `[amo] AMO_SUBDOMAIN / AMO_LONG_LIVED_TOKEN
are not set`.

**The amoCRM token is the most likely future outage.** ~1 year TTL, fails as a 401 that
visitors see as a 502 with a phone number. Calendar a rotation ~11 months out.

## Backups

No database, and the code is in git, so the surface is small. Nightly tar, off the box:

- `/etc/exo/` — the amoCRM token and per-app config
- `/etc/nginx/sites-available/`, `/etc/nginx/snippets/`, `/etc/nginx/conf.d/`
- `/etc/letsencrypt/`
- `/etc/systemd/system/exo@.service`, `/etc/sudoers.d/exo-deploy`

Also enable reg.ru VPS snapshots — a full-image restore beats rebuilding under pressure.

## Known constraints

- **One process per app.** `server/src/rate-limit.ts` is an in-memory fixed-window
  limiter, single-process by design. Horizontal scaling needs a shared store first.
  Not a concern at landing-page traffic; nginx `limit_req` is the second layer.
- **No CI.** Deployment is a manual `exo-deploy`. A GitHub Actions job that SSHes and
  runs it is a natural follow-up once the flow is proven.
- **One Metrika counter for four domains.** All four landings carry the same
  Yandex.Metrika counter (`112352796`) in `index.html`, so reports mix the domains and
  every domain has to be listed in the counter's settings, or its hits get filtered.
- **Still open before launch** (not deployment blockers): conversions are not wired to
  Metrika goals — `src/lib/lead.ts` pushes a plain custom `dataLayer` event, which
  Metrika's `ecommerce: 'dataLayer'` does not read, so a JS goal or an explicit
  `ym(..., 'reachGoal', ...)` call is still needed; and the forms show implicit 152-ФЗ
  consent text with no link to a published privacy policy.