Security monitoring
Security monitoring in Orbit Pulse watches the logs or spans of a project every minute with detection rules. A rule that matches opens a signal that people triage. Rules run on the logs and spans already stored in the project.
| Role | What it can do |
|---|---|
| Project administrator | Create and change rules and suppressions |
| Member | Triage signals, comment, assign, file an Orbit bug and declare an incident |
| Viewer | Read rules and signals |
How a rule works
Section titled “How a rule works”A rule has a source (logs or spans) and up to five queries named a to e in the query syntax. Each query counts events, or counts the distinct values of a field, such as @usr.id. You can group by up to four fields.
The window runs from 1 minute to 1 day and is 5 minutes by default. For spans it is at most 15 minutes, because trace retention may sample older spans out.
| Type | How it decides |
|---|---|
| Threshold | Up to five ordered cases, each with a condition and a severity. For each group the first case that holds sets the severity. |
| New value | One query and one field. The rule learns the values each group shows during a learning period (0 to 30 days, 7 by default), then signals each value the group has not shown before. It forgets values not seen for 1 to 365 days (30 by default). |
A threshold condition compares the query values with >, >=, <, <=, == and !=. It joins them with && (and) or || (or) and allows parentheses. For example, a >= 10 && b >= 1.
Severities are info, low, medium, high and critical.
A rule can also carry:
- a message template with variables such as
{{rule.name}},{{severity}},{{group}}and{{link}}; - up to 20 tags, such as
technique:t1110-brute-force; - up to 20 project members who are notified in Orbit;
- a bug severity. Signals of at least that severity file an Orbit bug.
Write and preview a rule
Section titled “Write and preview a rule”The rule editor checks the rule and previews it on the latest window while you write. The preview shows the groups and matches that would open signals, and saves nothing. Use it to see whether a rule is too loud before you turn it on.
Triage signals
Section titled “Triage signals”A signal SIG-<n> belongs to a rule and a group. While the group keeps matching within the keep-alive time (1 hour by default), the same signal is updated. A match at a higher severity escalates it: Pulse notifies again and files a bug at the rule’s bug severity. Otherwise a new signal opens. One evaluation opens or updates at most 50 signals.
Each signal keeps its rule’s queries and window, its message, its group values and up to 20 sample events. It reads the same after the rule changes or the telemetry expires.
-
Open Security and filter signals by severity, status, assignee, rule or text.
-
Open a signal. You see what matched, the samples with their attributes and traces, the Orbit bug, the incidents and the rule.
-
Use View logs or View spans to open the signal’s events with its query and window.
-
Set the status to open, under review or archived. Archiving needs a reason: false positive, benign, true positive, testing or other. You can add a comment and assign a project member who can edit.
-
File an Orbit bug from the signal if it needs work. The work item is titled
[Pulse security] SIG-<n> · <title>, labelledpulseandsecurity, and holds the values, the group and the samples. Or use Declare incident; see Incidents and postmortems.
Suppress known noise
Section titled “Suppress known noise”A suppression leaves the events that match its query out of one rule, or out of every rule of the project. It can expire on a date or never. It applies from the next evaluation, and open signals stay until someone archives them. Suppress… on a signal pre-fills it with the signal’s group. A project can have up to 200 suppressions.
Recommended rules for Orbit’s own services
Section titled “Recommended rules for Orbit’s own services”Pulse also lists nine recommended rules. They watch Orbit’s own services: they read the spans of the Orbit API and the intake logs of Pulse itself. That telemetry is not sent to your projects, so in your project these rules would not match. Use them as examples of rule shapes, and write your own rules over your services’ logs and spans.
| Rule | Watches |
|---|---|
| Sign-in brute force | Failed sign-ins per address: 10 in 5 minutes is medium, 50 is high |
| Sign-in after many failures | A successful sign-in after repeated failures |
| Password reset bursts | Many password resets |
| Denied requests | Bursts of denied requests |
| Access changes | Member and user changes |
| Security settings changes | Password policy, agent access and grants, session revocation |
| Work item exports | Exports of work items |
| New sign-in address | A sign-in from a new address for a user, after 7 days of learning |
| Refused ingest keys | Keys that Pulse refused at intake, with the reason |
Fields your own rules need
Section titled “Fields your own rules need”Per-address and per-user rules need two fields on your server spans:
client.address: the address of the real client. If your service sees only a proxy’s address, every request looks like it came from one client.enduser.id: the user id, never an email address or a name. It lets rules group by user with@enduser.id.