aldnav

I acquired an old HP ProDesk 400 G5 Desktop Mini PC which I plan to use as a headless server for my first ever home lab setup.

It has the following specs:

• 6 Core Intel(R) Core(TM) i5-9500T CPU @ 2.20GHz
• 16GB SODIMM DDR4 (2 x 8GB)
• 256GB HDD
• Intel UHD Graphics 630

The plan is to use this as my main server for learning Kubernetes and other DevOps tools. I believe I scored a good deal considering the low spec requirement for my intended use. And I can alway upgrade the components later.

NixOS 🖥️

I chose NixOS because I’ve always been intrigued by the declarative approach to system configuration. Normally I would have chosen Ubuntu Server or Debian, but this time I wanted to take advantage of defining the configuration as I go and use the same configuration for other machines I plan to deploy. Having the confidence that it will “just work”.

Setting up the server 🔧

The amount of information around NixOs is overwhelming. I started customizing the packages being mindful that I would only install the necessary ones. I even opted to use NixOS without a desktop environment. Less is more.

My goal is to get the machine up and running without the need for a monitor so I can just place it in the corner of my house. SSH into it and away I go.

SSH 💻

• ✅ Install NixOS
• ✅ Establish SSH access from my laptop to the server

To do this, I have to enable OpenSSH service. Configure the firewall to allow SSH connections.

Wiki SSH: https://nixos.wiki/wiki/SSH

I also configured my router to DHCP bind the server’s IP address. This way I can access the server from anywhere in the house on the same network.

ssh [email protected] -v

I can even go further as to setup a DNS that would alias the server’s IP address to a custom machine name. Though that’s for another time.

Wake on LAN ☀️

• ✅ Confirm Wake on LAN is enabled in the BIOS

For the BIOS, Wake on LAN (WAL) is enabled by default.

I found this article that has helpful instructions on Wake on LAN.

Enabling Wake-On-LAN (In Ubuntu 20.10) https://necromuralist.github.io/posts/enabling-wake-on-lan/

• ✅ Configure Wake on LAN in NixOS

As with the article, I used ethtool to confirm WAL capabilities.

This means that the machine is capable of WAL and by default, as enabled by the BIOS, it could be triggered by a MagicPacket messages.

Which leads me to the next step.

• ✅ Setup and use wakeonlan to wake the server

I’m using the same wakeonlan tool used in the article which is https://github.com/jpoliv/wakeonlan and installed it on my laptop.

And true enough, the server has booted up.

Sleep 💤

Ironically, throughout this endeavor, it was me who is not sleeping.

Problem: Freeze/sleep/suspend when monitor is unplugged

At this point, I was able to connect to the server using SSH. And remotely wake it up.

As I was wrapping up my physical arrangement, I unplugged the Display Port cable and moved the server closer to the router.

I sat back to my table with the laptop and I observed that the SSH session is frozen. So I opened a new TMUX session and connected to the server again.

This time the connection request has timed out. The machine is still on as indicated by the steady light of its power button.

So I reconnected my monitor and rebooted the server and within 10-20 seconds, I’m back at the login prompt. I am also now able to reconnect thru SSH on my laptop. 🤯

☁️ So I can only use my supposed headless server with a monitor?

Initially I thought maybe its the power management feature of the machine itself. Reviewing the BIOS settings, there is nothing out of the ordinary. The defaults seem to be set to the most power efficient mode. Aren’t servers supposed to be power efficient?

Stuck 🤔

☁️ Maybe the machine triggers a suspend/hibernate when there are no display output active? Like on a laptop when the lid is closed?

I started to think of another approach. I can install a program that would keep the machine awake.

The problem with that is surely, servers don’t actually resort to this approach right? The session isn’t dependent on a graphical user interface which usually does have power management features.

So I have to research on configuration. Perhaps there are configs for managing behavior of sleep/suspend/hibernate and how the operating system would handle it.

Indeed, there is!

systemd as documented here Wiki Power Management will create the generated content in /etc/systemd/sleep.conf which the OS will use to read the settings for suspend/hibernate.

NixOs generated a file with the following content in /etc/systemd/sleep.conf:

[Sleep]
AllowSuspend=no
AllowHibernation=no
AllowHybridSleep=no
AllowSuspendThenHibernate=no

And to test the sleep configuration…

• ✅ Sleep config is setup and working

I quickly repeated the process to test out:

1. On my laptop, connect to the server and run ping 8.8.8.8 to simulate a workload.
2. Unplug the monitor.
3. Observe the SSH session if it will stay connected or will eventually freeze.

And ❄️ freeze ❄️ it did.

I sat back on my thinking chair. At least I know now that sleep won’t be an issue. So what could be the problem?

I plugged in my monitor again, rebooted the server and waited.

On my laptop, reconnect to the server via SSH.

💡And there it was. What’s common between the two and what’s different when I have no monitor plugged-in.

The login prompt. Actually, a few more steps back — the GRUB menu.

With the monitor connected on the server, and the laptop connected to the server via SSH, I will be prompted with the login.

But when I unplug the monitor, I wouldn’t be able to tell if the GRUB menu is ever displayed or if the user is supposed to be prompted to choose which NixOS entry to boot. Or it is just frozen.

After a few minutes searching, I found this Arch Linux BB post of a similar issue [SOLVED] Machine wont boot without a monitor.: https://bbs.archlinux.org/viewtopic.php?id=233260.

I did not follow everything that the OP did but only applied nomodset as a GRUB/loader option.

And as another quality-of-life improvement, set the user for autologin since I’m already authenticated via SSH.

• ✅ Auto-login is setup
• ✅ nomodeset applied as a boot option

Completing headless setup 🥁

Now finally, to test again:

1. On my laptop, connect to the server and run ping 8.8.8.8 to simulate a workload.
2. Unplug the monitor.
3. Observe the SSH session if it will stay connected or will eventually freeze.

And it worked! 🙌 🎉 🕺

Demo

Now I have a headless server that I can use for my home lab setup. Most importantly, I can now sleep.