Cron & news refresh
Scheduled calls to crypto-news and related APIs, secrets, and verification.
Endpoints
- News refresh:
POST /api/cron/crypto-news(GET also accepted). - Optional:
CRON_SECRET_KEYin the app environment; callers must send
Authorization: Bearer <same value>.
External schedulers (e.g. cron-job.org)
Use POST (or GET) with header:
Authorization: Bearer YOUR_CRON_SECRET_KEYKeep the five-minute schedule and existing Bearer header. Do not put the token in the body. cron-job.org stops waiting after 30 seconds. Each upstream RSS request has a 6.5-second abort deadline; the handler responds within a 24-second application budget. News tables are written in parallel using batched inserts. PostgreSQL transaction advisory locks prevent overlapping refreshes across processes and deployments.
A 200 updated or degraded response means news writes have committed. degraded
lists empty/unavailable sources in data.unavailable_sources; successful sources
continue updating. already_running means an existing refresh owns the lock.
If all sources fail, persistence fails, or the response deadline is exceeded, the
handler returns 503 and preserves existing cached news. No refresh response is cached.
Reminder processing runs via Next.js after() after the HTTP response, under its own
transaction lock. Check [CRON] Reminder processing / Reminder task failed logs
separately: an HTTP news-refresh success does not imply reminder delivery succeeded.
If the HTTP budget expires, registered work is retained for cleanup; it is not falsely
reported as complete. This deployment uses a persistent Dokploy Node.js server.
Deployment migration
Back up the application database, then run the idempotent migration before deploying:
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f src/db/migrations/20260916_news_reliability.sqlThe migration adds missing event/notification columns and a news-link uniqueness index. Historical events are initially suppressed; only newly created events are pending. Existing columns, events and subscriptions are preserved. If duplicate news links exist, the migration fails without deleting them. Code rollback can leave these additive columns/indexes in place.
As observed on 2026-09-16, the public BlockBeats RSS endpoints return an empty JSON
response, not RSS. It is disabled by default and listed in data.disabled_sources;
its absence does not make the active feeds degraded. PANews, TechFlow, Decrypt and
Cointelegraph remain independent. Set NEWS_BLOCKBEATS_ENABLED=true only after
verifying the existing public RSS feeds recover. A future authenticated API requires
a separate integration; this flag does not supply API credentials.
Verification
Run pnpm test, pnpm exec tsc --noEmit, pnpm lint, and pnpm build. Test upstream
failure, all-source failure, failed persistence, concurrency and deadline behavior
with mocks. Do not send test notifications without an explicit delivery-test request;
observe the existing scheduler's natural
runs after deployment. Inspect response time and cache freshness as separate checks.
Server crontab
Project scripts (see scripts/): setup-cron.sh, cron-news-update.sh, test-deployment.sh.
Example:
curl -X POST "https://your-domain/api/cron/crypto-news" \
-H "Authorization: Bearer YOUR_SECRET"Fidelity
If you change cron auth or routes in code, update this page and status/changelog.
Dependencies
Environment
CRON_SECRET_KEY— Bearer secret for/api/cron/*handlers.NEWS_BLOCKBEATS_ENABLED— defaults to false; opt in only when the public RSS feeds work.DATABASE_URL— Postgres for all cron writes.ADMIN_SECRET_KEY— optional; some maintenance routes accept BearerADMIN_SECRET_KEYorCRON_SECRET_KEY.
Database (src/db/schema.ts)
news_items_cache,binance_alpha_news_cache,articles_cachecrypto_prices_cache,gas_fees_cachepush_subscriptions,binance_alpha_subscriptions,website_reminder_subscriptions(notification crons)- Additional tables per job (e.g. duplicate-event cleanup targets event-related storage — see the specific
route.ts).
Direct publisher feeds from Decrypt (https://decrypt.co/feed) and Cointelegraph
(https://cointelegraph.com/rss) supplement English articles. Both use the shared
timeout, parser and independent source status; no third-party RSS proxy or API key is required.
POST /api/cron/cleanup-duplicate-events requires a configured CRON_SECRET_KEY
and matching Bearer header. It refuses cleanup when the secret is missing.
Individual Web Push sends time out after 10 seconds.
Notification delivery validation
Before testing real delivery, back up the database and apply
src/db/migrations/20260916_notification_delivery.sql. Historical production tables
used event_uuid, required event_type, and created_at, while the sender expected
event_id, title, body, and sent_at. The additive migration supports both layouts,
preserves existing records, and can run twice. No subscriptions or event flags change.
Delivery results count push-service acceptance separately from log persistence.
logFailures reports missing audit writes; they do not cause a second send or prevent
delivery to later devices. Logs redact endpoint credentials. A successful Web Push
response is not proof of a visible device notification: a real test must target the
requesting user's subscription and confirm both receipt and the notification link.