Nagios XI

The Server That Cried Before Midnight

A fictional story. The characters, organisations and events are imaginary. The products are real.

The Server That Cried Before Midnight

In 30 seconds

Setting: BayarBijak, a fictional Malaysian e-wallet, used to find out about outages from customers on social media, and its most dramatic database server, DB-07, was about to face the biggest sale night of the year.

What Nagios XI did: It warned the team more than an hour before a disk would have filled up, after a first week in which the team had to learn to tune its alerts.

The business point: Nagios XI is an infrastructure monitoring platform supported in Malaysia by Astiostech. Good monitoring is not more alerts. It is the right alert, early enough, sent to the right person.

The story

Meet Wei Jie and the server called DB-07

Wei Jie is a 29-year-old systems engineer at BayarBijak, a fictional Malaysian e-wallet with four million users. He wears hoodies in an office set to 18 degrees, eats Milo dinosaur for breakfast, and has a personal relationship with a database server called DB-07.

DB-07 is the most dramatic machine in the data centre. It slows down without warning. It sulks when traffic spikes. Once, it went offline because someone plugged a kettle into the wrong power strip.

Wei Jie loves DB-07 the way you love a cat that knocks things off tables. You never know what it will do next, so you watch it closely. The problem was that, for a long time, nobody could actually watch it at all.

Finding out from social media

Before Nagios XI, BayarBijak's monitoring system could best be described as "customers".

When something broke, the first alert usually came from a user posting a screenshot of a spinning loading icon with the caption "BayarBijak down again ah?" and three crying-laughing emojis. Customer service forwarded it to IT. IT logged in, looked around, and eventually discovered that a disk had filled up, a service had stopped, or DB-07 was having one of its moods.

The CTO, Puan Hidayah, kept a count on the whiteboard: "Days since we found out about an outage from social media." The number never stayed up for long.

After one painful Hari Raya promotion, she called a meeting. "We are not finding out from social media anymore," she said. "We are finding out before social media."

Week one: the alert storm

The team chose Nagios XI, deployed with help from Astiostech. Within three weeks it was watching every server, database, firewall and switch that mattered. DB-07 had more checks than anything else, which everyone agreed it deserved.

Then came the first lesson, and it was not a pleasant one.

In its first week, Nagios XI was enthusiastic. Every check had been set with default thresholds, and many of them were far too sensitive. A server using a little more memory than usual triggered a warning. A backup job that ran slowly at 3am triggered another. Phones buzzed all night. By Thursday, Wei Jie had muted the alerts on his phone.

On Friday morning, a real problem slipped through. A web server ran out of disk space, and the warning sat among dozens of unimportant ones. Nobody noticed until a customer did. On social media. The whiteboard went back to zero.

Puan Hidayah did not shout. She drew two columns on the whiteboard: "Alerts that need someone to wake up" and "Alerts that can wait until morning". The team spent the next week sorting every check into one of the two, setting thresholds that matched how each system really behaved, and giving every critical alert a named owner.

"An alert nobody reads," she said, "is just a very expensive diary."

The phones went quiet. When they buzzed now, it meant something.

Wei Jie unmuted his phone. The next alert that woke him, a week later at 2am, was real: the security certificate on the payment gateway would expire in three days. He renewed it before breakfast, went back to sleep, and felt that the machine was finally on his side rather than shouting at him for fun.

12.12: the big night

In December, BayarBijak ran its biggest campaign ever for the 12.12 shopping festival, with flash deals starting at midnight.

By 10pm, traffic was triple a normal night. Wei Jie was in the office with pizza, two colleagues and a second Milo dinosaur.

At 10:47pm, Nagios XI sent a warning.

WARNING: DB-07 transaction log disk above warning threshold.

Wei Jie opened the graph. The line was climbing like a durian price before a festival. At this rate, the disk would fill shortly after midnight, right as the flash deals opened. And when that disk filled, DB-07 would stop accepting transactions.

Because the team had set a warning threshold well below the critical one, they had more than an hour of notice instead of a few minutes. That was the difference between a calm fix and a public meltdown.

An hour of quiet work

Wei Jie called the database lead, Kavitha, who was at home watching a Korean drama. She logged in remotely. Together they found the cause: a reporting job meant to run once a day had been scheduled to run every ten minutes during the campaign, stuffing the log with detail nobody needed.

They stopped the job, archived the old logs and added buffer space. By 11:40pm, the graph was heading down and Nagios XI turned green.

At midnight, the flash deals opened. Transactions surged. DB-07 handled them without a single complaint, which nobody could remember happening before. On social media, there was nothing at all. Just people buying air fryers.

Kavitha went back to her drama. She had missed the confession scene. She considered this an acceptable sacrifice.

The morning after

The next morning, Puan Hidayah reviewed the night: one warning, acknowledged and fixed before any customer noticed. She looked at the two columns still on the whiteboard and decided to leave them there permanently.

Today, Nagios XI also watches BayarBijak's website from outside the data centre, its links to partner banks, its security certificate expiry dates, and the switch beside the infamous kettle power strip.

As for Wei Jie, during the 12.12 rush he had stood beside the rack and, despite himself, patted DB-07. A colleague took a photo. It appeared in the company newsletter under the caption "Engineer comforts server during difficult night". Below it, Puan Hidayah added one line to the whiteboard: "Days DB-07 has behaved: 1." Everyone agreed this was generous.

What this means for your organisation

  1. Find problems before your customers do. Nagios XI checks servers, applications, databases and network devices continuously and alerts the team when something crosses a threshold.
  2. Set warning thresholds, not only critical ones. An early warning gives your team time to fix a problem calmly.
  3. Tune alerts in the first weeks. Default thresholds are a starting point. Alerts that fire too often get ignored, so match each threshold to how the system really behaves.
  4. Give every critical alert an owner. An alert that goes to everyone often goes to no one.
  5. Watch from the outside too. Checking your website and services from outside the data centre shows what customers actually experience.

Frequently asked questions

What is Nagios XI? Nagios XI is an enterprise infrastructure monitoring platform. It watches servers, applications, databases, websites and network devices, and alerts the team when something goes wrong. Astiostech deploys and supports Nagios XI in Malaysia.

How does Nagios XI decide when to alert? Each check has warning and critical thresholds. When a value crosses a threshold, Nagios XI changes the status and notifies the people responsible.

Why do new monitoring systems sometimes create too many alerts? Default thresholds rarely match every environment. Teams usually spend the first weeks tuning thresholds and deciding which alerts need an immediate response.

Can Nagios XI monitor websites from the outside? Yes. It can check whether a website or online service is reachable and responding, as well as monitoring the servers behind it.

Who supports Nagios XI in Malaysia? Astiostech deploys and supports Nagios XI for Malaysian organisations. Learn more on the Astiostech network monitoring page.

Book a 15-minute monitoring demo

See how Nagios XI could warn your team before your customers notice. Book a 15-minute monitoring demo.


BayarBijak, Wei Jie and DB-07 are fictional. Any resemblance to a real server with a personality problem is purely coincidental.