NR
NiyatnaRoute core-v1.0

High-Throughput LLM Proxy & Gateway Router

9999 : active
Heap: 142MB / 1024MB
Gateway Throughput LIVE
2,840 req/min
+14.2% vs 1h moving avg
Median TTFT (P2C) p50 OPTIMAL
112 ms
DeepSeek V3 Fastest upstream
Token Compression RTK + CAVE
-42.6% payload
18.4M tokens saved today
Relay Health ALL GREEN
5 / 5 providers
99.98% 0 dropped streams

Live Request & Stream Inspector

SSE Active

Real-time /v1/chat/completions telemetry & routing decisions

STATUS TIME MODEL / UPSTREAM STRATEGY TOKENS (IN/OUT) COMPRESSION TTFT
200 OK 16:08:12.41
deepseek-chat
Direct Relay • HK-01
 P2C Balance 1,420 / 412 -46.2% 114ms
STREAM 16:08:11.89
claude-3-5-sonnet
Cloudflare AI Gateway
 Priority #1 3,890 / 852 -38.1% 242ms
FALLBACK 16:08:09.12
gpt-4o-mini
Vercel AI Relay
 Cost Route 820 / 164 -51.4% 186ms
200 OK 16:08:05.67
gemini-1.5-flash
Direct Google AI
 P2C Balance 2,150 / 610 -44.0% 148ms
Auto-tail active (50 items buffer)
•

Upstream Provider Matrix

P2C weighted latency & active failover health

p95 / Availability
DeepSeek V3 (Direct Relay)
Weight: 45% • P2C Primary
114 ms
99.98% up
Claude 3.5 Sonnet (Cloudflare Gateway)
Weight: 30% • Fallback #1
242 ms
100.0% up
GPT-4o (Vercel AI Relay)
Weight: 15% • Cost Route
186 ms
99.94% up
Gemini 1.5 Flash (Direct API)
Weight: 10% • High-Throughput
148 ms
99.85% up
Auto-failover threshold: >500ms or 5xx

Compression & Headroom Engines

3-tier zero-loss prompt token optimization

SAVINGS -42.6%
rtk engine AST & Whitespace
-32.4% avg
Latency overhead: 0.8ms 13.8M tokens stripped
caveman Prompt Syntax Pruning
-10.2% avg
Latency overhead: 0.4ms 4.6M tokens stripped
headroom governor V8 Heap Guard
882 MB Free
Max Heap: 1024 MB Current: 142 MB (13.8%)
Total cost saved: $5,273.40 / mo