AIO Ep8. A Case of Troubleshooting Lost Connection to Black Synology on ESXi

Intermittent disconnection from Black Synology — troubleshooting process log.

After initially setting up the Synology system, it would intermittently (every few hours to 3 days) become unreachable. Once disconnected, all services became inaccessible, ping failed, and shutting down the VM in ESXi would get stuck indefinitely.

At first, I suspected a hardware issue with my DIY host.

I booted into PE and ran an AIDA64 stress test overnight — no anomalies. I also ran MemTest86+ overnight, which also showed no issues.

Based on experience, system-wide unresponsiveness like this is often caused by I/O errors.

Since only the Synology system froze while ESXi remained accessible, I started suspecting that the issue might be related to Synology’s HDD hibernation or certain packages.

I disabled HDD hibernation in Synology and enabled system panic auto-reboot — but the problem persisted.

Next, I suspected the community-made network driver I used for my ESXi installation, so I began investigating ESXi itself. I logged into SSH and ran esxtop, where I noticed something abnormal: the system USB storage showed 100% utilization, yet both read and write speeds were 0.

Upon inspection, the USB drive was very hot. I replaced it with a new SanDisk Cruzer plugged into a front panel port (to avoid motherboard heat interference). The system remained stable for 10 days. (So much for the NIC driver theory…)

2021.08.31 Update:

The newly purchased USB drive developed the same issue after 17 days of continuous operation. This time, the problem was visible in ESXi’s monitoring as well.

image

ESXi got stuck trying to restore the connection due to I/O freeze, making the VM inaccessible.

I tried reseating the USB drive (as shown in the image), which finally triggered ESXi’s timeout fast-fail mechanism — otherwise, the system would have remained frozen indefinitely.

After the timeout, the Synology VM returned to normal.

Actually, only the bootloader resides on the USB drive; the main system runs from the hard drive.

On a physical “black Synology” NAS, the USB drive is theoretically inactive after boot. But in ESXi, continuous logging causes constant read/write operations on the USB drive, leading to drive dropouts and system freezes.

Solution: Purchase a 16GB Optane drive (essentially a small SSD) as the boot device. Installation is identical to using a USB drive. If issues persist in the future, I’ll update accordingly.

How to install ESXi on a small-capacity SSD with space reserved for data storage: (Note: This does not apply to USB drives. For USB, you need to resize the partition after installation — see AIO Ep.4)

1
2
3
4
5
6

When entering ESXi 7.0 installation for the first time, press Shift + O
After cdromBoot runweasel appears, append autoPartitionOSDataSize=8192 (case-sensitive)
8192 means assigning 8GB to OSDataSize. Press Enter to proceed with normal installation.

Compared to version 6.7, 8GB is more than sufficient for VMware Tools, scratch space, and coredump storage.
By specifying this size, ESXi won't automatically allocate excessive space.