Blueprint

Cron & news refresh

Scheduled calls to crypto-news and related APIs, secrets, and verification.

Endpoints

  • News refresh: POST /api/cron/crypto-news (GET also accepted).
  • Optional: CRON_SECRET_KEY in the app environment; callers must send
    Authorization: Bearer <same value>.

External schedulers (e.g. cron-job.org)

Use POST (or GET) with header:

Authorization: Bearer YOUR_CRON_SECRET_KEY

Keep the five-minute schedule and existing Bearer header. Do not put the token in the body. cron-job.org stops waiting after 30 seconds. Each upstream RSS request has a 6.5-second abort deadline; the handler responds within a 24-second application budget. News tables are written in parallel using batched inserts. PostgreSQL transaction advisory locks prevent overlapping refreshes across processes and deployments.

A 200 updated or degraded response means news writes have committed. degraded lists empty/unavailable sources in data.unavailable_sources; successful sources continue updating. already_running means an existing refresh owns the lock. If all sources fail, persistence fails, or the response deadline is exceeded, the handler returns 503 and preserves existing cached news. No refresh response is cached.

Reminder processing runs via Next.js after() after the HTTP response, under its own transaction lock. Check [CRON] Reminder processing / Reminder task failed logs separately: an HTTP news-refresh success does not imply reminder delivery succeeded. If the HTTP budget expires, registered work is retained for cleanup; it is not falsely reported as complete. This deployment uses a persistent Dokploy Node.js server.

Deployment migration

Back up the application database, then run the idempotent migration before deploying:

psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f src/db/migrations/20260916_news_reliability.sql

The migration adds missing event/notification columns and a news-link uniqueness index. Historical events are initially suppressed; only newly created events are pending. Existing columns, events and subscriptions are preserved. If duplicate news links exist, the migration fails without deleting them. Code rollback can leave these additive columns/indexes in place.

As observed on 2026-09-16, the public BlockBeats RSS endpoints return an empty JSON response, not RSS. It is disabled by default and listed in data.disabled_sources; its absence does not make the active feeds degraded. PANews, TechFlow, Decrypt and Cointelegraph remain independent. Set NEWS_BLOCKBEATS_ENABLED=true only after verifying the existing public RSS feeds recover. A future authenticated API requires a separate integration; this flag does not supply API credentials.

Verification

Run pnpm test, pnpm exec tsc --noEmit, pnpm lint, and pnpm build. Test upstream failure, all-source failure, failed persistence, concurrency and deadline behavior with mocks. Do not send test notifications without an explicit delivery-test request; observe the existing scheduler's natural runs after deployment. Inspect response time and cache freshness as separate checks.

Server crontab

Project scripts (see scripts/): setup-cron.sh, cron-news-update.sh, test-deployment.sh.

Example:

curl -X POST "https://your-domain/api/cron/crypto-news" \
  -H "Authorization: Bearer YOUR_SECRET"

Fidelity

If you change cron auth or routes in code, update this page and status/changelog.

Dependencies

Environment

  • CRON_SECRET_KEY — Bearer secret for /api/cron/* handlers.
  • NEWS_BLOCKBEATS_ENABLED — defaults to false; opt in only when the public RSS feeds work.
  • DATABASE_URL — Postgres for all cron writes.
  • ADMIN_SECRET_KEY — optional; some maintenance routes accept Bearer ADMIN_SECRET_KEY or CRON_SECRET_KEY.

Database (src/db/schema.ts)

  • news_items_cache, binance_alpha_news_cache, articles_cache
  • crypto_prices_cache, gas_fees_cache
  • push_subscriptions, binance_alpha_subscriptions, website_reminder_subscriptions (notification crons)
  • Additional tables per job (e.g. duplicate-event cleanup targets event-related storage — see the specific route.ts).

Direct publisher feeds from Decrypt (https://decrypt.co/feed) and Cointelegraph (https://cointelegraph.com/rss) supplement English articles. Both use the shared timeout, parser and independent source status; no third-party RSS proxy or API key is required.

POST /api/cron/cleanup-duplicate-events requires a configured CRON_SECRET_KEY and matching Bearer header. It refuses cleanup when the secret is missing. Individual Web Push sends time out after 10 seconds.

Notification delivery validation

Before testing real delivery, back up the database and apply src/db/migrations/20260916_notification_delivery.sql. Historical production tables used event_uuid, required event_type, and created_at, while the sender expected event_id, title, body, and sent_at. The additive migration supports both layouts, preserves existing records, and can run twice. No subscriptions or event flags change.

Delivery results count push-service acceptance separately from log persistence. logFailures reports missing audit writes; they do not cause a second send or prevent delivery to later devices. Logs redact endpoint credentials. A successful Web Push response is not proof of a visible device notification: a real test must target the requesting user's subscription and confirm both receipt and the notification link.

Cron & news refresh | Tracker Dog Docs