# Lattice Network Performance Monitor **Training & introduction handbook** Northwind demo fabric · Internal use This is the intro you give a new NOC operator, network engineer, or executive who has never opened Lattice. Start with the 15-minute path. Then work the two playbooks at the end on the live board. --- ## First 15 minutes Do this once. You will know if you can page, and you will have touched the four screens that matter. 1. **Overview** — look at the strip. Red is Critical. Yellow is Degraded. If both are empty, the fabric is quiet. Site tiles underneath; click HQ. 2. **Map** — Internet at the top, firewall, then switches, hosts, VMs. Hover HQ-ACC-C9300. That card is the inventory. Double-click opens the dashboard. 3. **One device** — HQ-FW-PA3220. Health ring, DIA-facing interfaces, setup pills (ICMP/SNMP). If the ring is green and DIA is yellow, the circuit is the story, not the box. 4. **One report** — Reports → Element is already on DIA. Read the headline paragraph. Saved views: DIA month, Site month, Worst 10. 5. **Alerting** — Policies tab is the six rules we page on. Ack a minor. Outbox is the demo email/SMS. Three clicks if you remember nothing else: Overview strip → Map → Reports Element (DIA). PerfStack presets (DIA / Core / SQL / HV cluster) if “it is slow.” --- ## 1. What Lattice is Lattice is our Network Performance Monitor. It is a new system — not The Dude, not SolarWinds, not PRTG. It is built to the same *job* those tools do: tell you whether the network is healthy, where it is not, and why, without making you SSH into fifteen boxes. What it does today in this environment: - Watches every node, circuit, Hyper-V host/guest, wireless AP, and application we have loaded. - Rolls that into a **health score** (0–100) so you do not have to stare at raw SNMP. - Lets you **drill** from the whole fabric → a site → a neighbor → a node. - Reconstructs the path a user actually takes (**NetPath**). - Overlays CPU, latency, loss, and util on one time axis (**PerfStack**). - Stores **downtime history**, SLA, and outage *patterns* so Tuesday-at-02:00 is visible as a pattern, not a surprise. - Imports inventory (including a Dude backup) so we do not re-type the network. What it is **not** (yet): this training environment is a high-fidelity **demo fabric**. Metrics tick, incidents inject, email/SMS go to an outbox. When we put Lattice on a live poller, the screens stay the same — the numbers start coming from SNMP, ICMP, WinRM, traps, and NetFlow instead of the simulator. Voice / IP SLA is built but **turned off** for now. Do not look for it in the sidebar. --- ## 2. Who uses which part | Role | You live here | You rarely need | |---|---|---| | **NOC** | Overview, Map, NOC board, Alerting, Events | Config diffs, UnDP OIDs | | **Engineer** | NetPath, PerfStack, Insight, Configs, Hyper-V, Reports → Element | Executive rollup | | **Admin** | Admin (nodes, links, Dude import), Teams | QoE | | **Exec / manager** | Overview health, Reports → Executive, Capacity exhaustion | Drill-down topology | SSO is Entra ID (demo-connected). Roles: Admin, NOC, Engineer, Exec, Read-only. This preview is **not** locked behind login so training is not blocked. On the live server, SSO will gate the same screens. --- ## 3. The language of the board Learn this before you click anything. Lattice uses the same colors everywhere — map, alerts, reports, Hyper-V. | You see | Means | Do this | |---|---|---| | **Green / Up** | Responding. Health typically ≥ 90. | Nothing. | | **Yellow / Degraded / Warn** | Up, but not well. High CPU, loss, host pressure, crowded AP, one node of a cluster down. | Investigate *before* it is an outage. | | **Red / Down / Critical** | Not responding, circuit down, or a Critical alert. Health 0. | Incident. | | **Grey / unknown** | We are not polling it (or it is parked: VM saved/off). | Confirm it should be monitored. | **Critical alerts are red. Degraded is yellow.** That is intentional. Red is “wake someone.” Yellow is “this will be red if you ignore it.” **Suppressed** (dimmed on Alerts and Events) means we *know* and we are not paging you: either the parent is already down (DIA died, so Austin IPsec is expected to look dead) or a **maintenance window** is on. Do not ack-and-forget a suppressed child until you have checked the parent. Header (top right, every page): live health, counts of up / degraded / down. If health is not in the 90s, start at Overview or Alerts — do not wander. --- ## 4. Health score (how Lattice judges a node) A node is not “up” just because ping works. Lattice scores five slices, then rolls them: 1. **Availability** — ICMP/SNMP answering. Zero if down, lost heartbeat, or a VM that is Off/Saved. 2. **Performance** — CPU, memory, Hyper-V CPU-wait, ballooning. 3. **Hardware** — chassis temp, fans, PSUs; on Hyper-V also VHDX latency and host free RAM. 4. **Response** — ICMP compared to *that node’s own baseline* (a jump from 1 ms to 8 ms is worse than a branch that always sits at 30 ms). 5. **Interfaces** — oper-down, >80% util, error counts. **Overall health** on Overview is the fabric rollup of those node scores. Rules of thumb: - 90–100: healthy. - 70–89: degraded — yellow. Open the node. - Below 70 or 0: treat as down / incident. A SQL VM can be green on ICMP and yellow on **Apps** (page life, lock waits). Always check the layer you care about: network vs hypervisor vs application. --- ## 5. The demo fabric (what you are looking at) Unless someone imported a Dude file, you are on **Northwind**: - **HQ Chicago** — edge firewall (Palo Alto), core/access Cisco, Arista leaf, wireless, NAS, 2-node Hyper-V cluster (HQ-HV-01 / HQ-HV-02) with CSV, SQL, DC, RDS, PBX VMs. - **Austin** and **Denver** — Cisco ISR branches over IPsec. - **DIA** — the Internet circuit from ISP-PE-CHI to HQ-FW. This is the circuit we report on first. - Cloud / SaaS (M365) as a QoE target, not a box we SNMP. IPs are RFC1919 (`10.0.0.0/24` core, `10.0.20.0/24` servers, `10.0.40.0/24` wireless, `10.2` Austin, `10.3` Denver). You can **inject** incidents from Overview or Hyper-V to train: WAN loss, AP down, Hyper-V host contention, cluster failover. They expire on their own. Use them. That is what they are for. --- ## 6. First ten minutes Work this once with a trainer sitting next to you. 1. **Overview** — read overall health, the five slice bars, worst nodes, hottest links. Note open alerts. 2. Click a **site** tile → you land on the Map already drilled into that site. 3. **Map** — fabric view is all sites. Click a site, then a node. Links color with the same green/yellow/red. 4. Open **HQ-FW** from Devices. Read probes, interfaces, sensors. 5. **NetPath** — pick a path (e.g. Austin user → M365). Walk the hops. A red hop is where the user feels pain. 6. **PerfStack** — overlay DIA util + firewall CPU + Austin tunnel latency. This is how you prove “backup window” vs “ISP.” 7. **Hyper-V** — two hosts, CSV owner, quorum, guests. Note parked VMs show as dashes, not fake RAM. 8. **Reports → Element** — it opens on **DIA**. Read the paragraph at the top. That sentence is the point of the report. 9. **Admin** — search a node, do not change production IPs in training unless you intend to. Reset demo restores Northwind. 10. **Alerting** — ack one minor, preview an email. Check the outbox. Nothing leaves the building in the demo. If you only remember three clicks after this: **Overview → Map drill → Reports Element (DIA).** --- ## 7. Monitor ### Overview The operations home. Health ring, availability / performance / hardware / response / interface bars, site scores, worst five nodes, hottest links, injectors. Shortcuts under the banner: Traffic, Events, Capacity, Apps. **Run discovery** walks a simulated find (percent in the header). **Admin / import** is where real inventory work happens. ### Map Three levels: 1. **Fabric** — sites as regions, WAN/tunnels between them. 2. **Site** — devices in that building, neighbor links. 3. **Node** — the selected device and anything directly attached. Use this in an incident before you open ten device pages. If Austin is red and DIA is red, Austin is probably a victim, not the cause. That is also what **dependencies** on Events are for. Time travel (rewind) on the map plays recent snapshots so you can see the moment it broke. ### NOC board Wall / second-monitor view. Toggle widgets: overall health, open alerts, WAN/DIA, Hyper-V cluster, top talkers. Voice MOS widget is hidden while Voice is disabled. Keep this on the TV. Do not troubleshoot from it — drill out to Map / NetPath / the node. ### Devices Inventory list: name, IP, kind, site, status, health. Open a node for: - Identity (vendor, model, role, SNMP community) - Probe table (ICMP, SNMP, and extras) - Interfaces (util in/out, errors, discards, speed) - Hardware sensors - Hyper-V block if it is a host or guest This is the CMDB-ish view. If the name or IP is wrong, fix it in **Admin**, not here. ### NetPath Hop-by-hop path with latency and loss per hop — the “user said the VPN is slow” screen. Pick source and destination. A healthy ICMP to the firewall plus a red hop two hops upstream is an ISP or underlay problem, not a LAN problem. ### PerfStack You choose the metrics. Typical stacks we actually use: - DIA util + HQ-FW CPU + Austin RTT (backup vs ISP) - Host logical CPU + guest CPU-wait + VHDX latency (Hyper-V pressure) - AP clients + noise (wireless) If two lines spike together, they are related until proven otherwise. ### Wireless APs, client count, noise, stations. HQ-AP-FL1 running 27 clients against a baseline of ~16 is a real example in the seed data. Warehouse scanners live on a separate SSID/VLAN — do not “fix” them by moving them to corporate. ### Hyper-V This is **VMAN for our 2-node cluster**, not “ping the VM.” - Two hosts, one cluster, CSV owner, file-share witness, HA vs parked guests. - Host metrics: logical CPU, available RAM, VHDX latency, balloon. - Guest metrics: vCPU wait, assigned vs demand, heartbeat, state (Running / Saved / Off / Paused). - **Parked (Saved/Off) VMs must show as idle** — no fake 8 GB demand. If you see numbers on a Saved VM, that is a bug; say so. - Inject **host contention** to watch wait/balloon climb on guests. - Inject **cluster node down** to watch live migration onto the surviving host and CSV ownership move. When HV-01 is down, HV-02 going yellow is *correct* (cluster degraded, one node left). That is not a second independent failure. SQL, DC, RDS are also in **Apps**. A green host with a yellow SQL AppInsight is an application problem until the host wait is high. ### Experience (QoE) Application flows: M365, TDS/SQL, SMB/CSV, SIP, etc. Share of WAN, RTT, risk. Use with Traffic when someone says “the internet is slow” — often it is one app. --- ## 8. Analyze ### Traffic (NetFlow) Top conversations: who is talking to whom, which application, bitrate, volume. This is how you answer “what filled the DIA” and “what is pounding CSV.” Unknown UDP toward the firewall is worth a look — app-id may not have classified it. ### Configs (NCM) Running vs startup, with a simple diff. Compliance against CIS-style policies (demo). Austin NAT for shop VLAN showing **drift** is a teaching example: a change landed in running and was not saved / not in policy. When a hop dies after a change window, come here *before* you reload the box. ### Apps AppInsight-style: - **SQL** — batch requests, page life, lock waits, log flush, buffer hit. Page life falling while the VM looks fine is still an incident for the DBA, and you can prove the host was not starving it. - **AD** — LDAP bind time, replication lag, auths. - **RDS** — sessions, disconnected, input delay. ### IPAM & UDT - **Subnets** — CIDR, VLAN, used vs size, DHCP, DNS. - **IPs** — used / reserved / available / **conflict** (lobby printer vs finance laptop is in the demo on purpose). - **UDT** — switch, port, VLAN, MAC, hostname, user, IP. “Who is on Gi1/0/8?” lives here. ### Insight (Network Insight) Firewall-aware view of HQ-FW: session count, data-plane CPU, VPN tunnels (Austin/Denver) with RTT, top drop reasons (app-id deny, spoof/zone, threat). BGP table: peer, ASN, state, prefixes, uptime. ICMP to the PA is not the same as “IKE is up.” Use this screen for VPN tickets. ### Capacity Exhaustion dates (peak vs average): DIA, CSV volume, SQL RAM, host CPU, AP clients. Volumes with growth/day. **SQL RAM ~18 days** in the demo is the one that should scare you. DIA ~47 days is the circuit conversation with the ISP. Baselines: current CPU vs the last 48 samples. “High” that is normal for that box is not an incident; a jump off baseline is. ### Events Four tabs: 1. **Stream** — syslog + SNMP traps, live. Severity colors match alerts. 2. **Maintenance** — toggle windows (Tue patch on the access stack is seeded). Enabled windows suppress child noise. 3. **Dependencies** — parent → child (DIA → firewall → Austin/Denver; Hyper-V host → SQL/DC). If the parent is down, the child is suppressed. 4. **Pollers** — HQ-POLL-01/02 and custom OIDs (UnDP): PAN session util, CRC, PSU watts. --- ## 9. Operate ### Alerting Open incidents, severity, ack, notification channels, outbox. - Critical = red. Major/minor degraded = yellow. Info = quiet. - Channels: NOC email, on-call SMS, exec email (critical only), escalation pager. In the demo, **Send test** and real alerts write to the **outbox** — they do not leave the network. Read the outbox to see what *would* have been sent. - Ack removes it from the badge count; it stays in history. ### Reports This is where managers and engineers both should be fluent. **Element (default)** Left: filter DIA / Routers / Switches / Firewalls / Servers / Hyper-V / APs / Tunnels. Right: for the selected object: 1. **The story** — one paragraph. Why you ran the report. Read this first. 2. Current KPIs (health, ICMP, CPU vs baseline — or util / latency / loss / BGP for a circuit). 3. History overlay + 30-day availability. 4. Hottest interfaces *or* circuit ends. 5. **This object’s** outages only. Start every capacity or outage conversation on **DIA**. Then open the node they named. **Downtime** — fabric SLA, MTTR, MTBF, site SLA table, incident log, worst nodes. Click a worst-node name to jump into that Element report. **Patterns** — heatmap (hour × weekday), recurring windows (the Tuesday 02:00 patch is supposed to light up), flapping nodes, correlated clusters (“blast radius”), cause mix (WAN vs power vs wireless vs hardware). **Availability** — 30-day overlay per site, node vs 99.95% SLA gap. **Executive** — health, SLA, MTTR, open alerts, DIA sparkline, site scores. This is the Monday email (there is an **Email downtime report** button; demo only). **Capacity / Traffic** tabs — link headroom and interface heatmap, plus FW CPU vs Austin RTT. Windows: 24h / 7d / 30d. SLA target is **99.95%**. ### Tools Path / probe utilities and a pointer to Admin for inventory. Do not use Tools as the import home anymore. ### Teams & SSO Directory (Entra / Okta demo users), role dropdown, capability matrix. Toggle SSO connected/paused — preview stays open. On the live server this becomes the real IdP. ### Admin Three tabs: **Nodes** — add / edit / remove. Fields: name, IP, kind, vendor, model, site, role, SNMP community, notes. The table is read-only; edit in the form so live polling does not steal the cursor. **Links** — A/B ends, media (copper, fiber, tunnel, internet, virtual), bandwidth. **Import** — this is the Dude door, and the generic inventory door. - Accepts XML, JSON, CSV, `.tar.gz`, zip, SQLite harvests. - **Merge** (default) — add/update, keep Hyper-V cluster and Lattice-only objects. - **Replace** — throw away the current fabric and load only what is in the file. Dangerous. Training should use Merge. - **Load Dude sample** / **Load Lattice sample** if you do not have a file handy. - Export Lattice JSON if you need a backup of *this* fabric. After import, Map and NetPath rebuild from the new sites/devices. Reports history for brand-new nodes will be thin until they have been polled. **Reset demo** puts Northwind back. Use it after a messy import in training. --- ## 10. Playbook: working an incident Use this verbatim until it is muscle memory. 1. **Header** — health not in the 90s? How many down vs degraded. 2. **Alerting** — newest Critical (red). Read the title. Do not ack yet. 3. **Map, fabric** — is it one site or the whole WAN? If DIA/firewall is red, do not chase Austin. 4. **Events → Dependencies** — confirm parent/child. If suppressed, work the parent. 5. **NetPath** — from an affected user/site to the thing they need (DC, SQL, M365). First red hop is the fault domain. 6. **PerfStack** — overlay that hop’s util/loss with the edge CPU. Backup window? ISP loss? Host pressure? 7. **If it is a VM** — Hyper-V: host wait, CSV owner, heartbeat. Apps: SQL/AD/RDS. 8. **If it is the WAN** — Insight (IKE, BGP, drops) + Traffic (who is filling it) + Reports Element on DIA. 9. **Configs** — did someone commit in the last hour? 10. **Ack** and (in demo) preview the notice. Write the cause in language a manager can read: “DIA loss 1.8%, Austin is downstream, not a branch router fault.” Inject WAN loss from Overview and walk this list. Then inject HV contention. Then inject cluster failover. That is the training lab. --- ## 11. Playbook: running a report Someone asked “how has DIA been?” or “why does Austin keep dropping?” 1. **Reports → Element.** 2. Chip **DIA** (or Routers / the site’s ISR). 3. Read the **headline paragraph**. If it already answers the question, stop. Paste that paragraph into the ticket. 4. Check 30-day availability and the object’s own outage list — not the fabric log. 5. If you need pattern (“every Tuesday”), switch to **Patterns**. Recurring windows and the heatmap are the evidence. 6. If you need SLA language, **Downtime** or **Availability** vs 99.95%. 7. **Email downtime report** only for the exec brief (demo outbox). More data is here if you scroll. The headline is the product. Do not send a 40-row table when the paragraph said “two ISP events, 41 minutes, SLA breach.” --- ## 12. Importing from The Dude Lattice is not Dude. Dude is an **inventory source**. 1. Admin → Import. 2. Leave mode on **Merge** unless you were told to replace. 3. Drop the Dude backup (XML / export / sqlite). Or **Load Dude sample**. 4. Read the toast: device count, warnings, maps found. 5. Overview should say “Imported N nodes from …”. 6. Spot-check Map (sites), a known IP, and that the Hyper-V cluster is still two hosts if you merged onto Northwind. If names are ugly (`device-12`), rename in Admin → Nodes. Lattice will keep the IP. --- ## 13. Color and incident examples (so you calibrate) | Symptom | Color | Where | Typical cause in this fabric | |---|---|---|---| | DIA util 50s, loss 0.2% | Green/yellow | Overview, Reports DIA | Backup window, not down | | DIA loss > 1%, latency jump | Red alert | Alerting + Insight | ISP / WAN loss inject | | HQ-HV-01 CPU wait, guests balloon | Yellow | Hyper-V | Host contention inject | | HQ-HV-01 down, SQL migrating | HV-01 red, HV-02 yellow, SQL yellow briefly | Hyper-V | Failover inject — *one* incident | | AP 27+ clients | Yellow | Wireless | Overcrowd inject | | Austin ISR down, DIA also down | Austin suppressed | Events deps | Work DIA, not Austin | | SQL page life low, host fine | Apps yellow, Hyper-V green | Apps | Database, not compute | | Saved VM | Grey/down, metrics “—” | Hyper-V | Parked — do not “fix RAM” | --- ## 14. What is demo vs what we take live | Capability | In this environment | On a live poller | |---|---|---| | Screens, health, maps, reports | Real product | Same | | SNMP / ICMP / WinRM numbers | Simulated tick | Real poll | | Traps / syslog | Simulated stream | Real receivers | | NetFlow | Simulated conversations | Real IPFIX | | Email / SMS | Outbox preview | SMTP / Twilio (or equivalent) | | SSO | Directory + roles, no login wall | Entra SAML gate | | Dude import | Parser + merge | Same parser against production backup | | Inject buttons | Training | Disable or leave for staged | If a number looks too perfect, it is the simulator. The *workflow* is what you are learning. --- ## 15. Glossary | Term | Meaning | |---|---| | **DIA** | Dedicated Internet Access — HQ’s circuit to the ISP. | | **Health** | 0–100 composite, not ping. | | **NetPath** | Hop-by-hop path and quality. | | **PerfStack** | Multi-metric overlay on one time axis. | | **CSV** | Cluster Shared Volume (Hyper-V). | | **CPU wait** | Guest ready but waiting on host CPU. The Hyper-V “yellow” that ping cannot see. | | **Balloon** | Host reclaiming guest RAM. | | **UDT** | User Device Tracker — port / MAC / user. | | **UnDP** | Universal Device Poller — extra OID. | | **MTTR / MTBF** | Mean time to restore / between failures. | | **SLA** | 99.95% availability unless leadership says otherwise. | | **Merge vs Replace** | Import additively vs wipe the fabric. | | **Suppressed** | Alert muted because of parent down or maintenance. | --- ## 16. After this document You are trained enough to sit a shift when you can: 1. Explain a yellow vs a red without opening a menu. 2. Drill fabric → site → node on the Map. 3. Run the incident playbook on an injected WAN loss. 4. Open Reports → Element → DIA and read the headline out loud. 5. Merge a Dude sample without wiping Hyper-V. 6. Tell a manager whether SQL is a VM problem or a database problem. If you cannot do those six, walk the first ten minutes again with a trainer. Do not sit a shift on tribal knowledge from The Dude — the colors and the health score are the new language.