Deploy-time analysis
Every time a deploy webhook fires, Rocketgraph:- Fetches logs from a window before and after the deploy (from Sentry, Loki, Datadog, etc.)
- Mines templates — groups raw log lines into patterns using template mining, normalizing away variable parts like IDs, timestamps, and UUIDs
- Scores novelty — uses TF-IDF cosine similarity to determine if an after-deploy template is genuinely new or just a minor variation of something seen before. This prevents false positives from things like changing thread IDs
- Detects anomalies — an Isolation Forest model trained on the before-window scores after-deploy templates for unusual frequency, error rate, or behavior shifts
- Classifies changes into: new errors, disappeared patterns, behavior changes, and stable templates
- Reports to Slack with severity, evidence, and recommended actions
Continuous monitoring
Independent of deploys, Rocketgraph runs the same ML pipeline on a recurring schedule (default: every hour):- Compares the current window against the previous window (e.g., last hour vs. the hour before)
- Detects drift — new error patterns appearing, existing patterns disappearing, frequency shifts
- Alerts on meaningful changes — filters out noise using anomaly scoring thresholds
What gets detected
| Change type | Example | Signal |
|---|---|---|
| New error | PostgresError: relation "thread_events" does not exist appeared for the first time | TF-IDF novelty score > threshold |
| Disappeared error | SMTP connection refused stopped occurring after a deploy | Template existed before, no similar match after |
| Disappeared info logs | workspace_resolved auth logs stopped appearing | Possible broken logging pipeline |
| Behavior change | POST /api/orders error rate went from 0.1% to 8% | Isolation Forest anomaly score |
| Volume shift | Total log volume dropped 85% after deploy | Before/after count comparison |
Environment filtering
When connected to Sentry, you can filter analysis by environment (e.g.production, staging, railway) to avoid mixing development noise into production analysis.