5 ms·
Our BMCs are not great at keeping accurate time
- shenki0 4y agoThere are many things that BMCs are not great at. Add this to the list :)
- pabs3 4y agoSounds like you need a server that can run OpenBMC or similar. Or just run a regular Linux distro on the BMC, some of the vendor BMCs are running Linux under the hood.
- themoonisachees 4y agoAs far as i can tell the openBMC support list is quite small :/ Most vendors are indeed running Linux. For example, HPE iLO 5 is capable of starting smart storage administrator from bios, and when doing so it just launches some flavor of centos, launches SSA server then starts an antediluvian build of Firefox pointing to it. I wouldn't be surprised to learn that the chip serving iLO was Linux too.
- gpapilion 4y agoIt is. The industry has really build around 2 vendors for chips, and two vendors for software solutions for those chips, AMI and Insyde. These systems are generally pretty locked dow , but there are was to get to the Linux core(sometimes serial headers).
- rgrs 4y agoAll others except iLO is Linux.
- qhwudbebd 4y agoI'd dearly love to do this. I'd even settle for keeping the (presumably ancient) kernel shipped by the manufacturer and just replacing the user space to ditch the hardware manufacturer's pile of hacked together crap and clunky web interface. All we ever use in practice would be covered by dropbox/opensshd on a minimal rootfs with a shell where we can (presumably) toggle GPIOs through sysfs to power on/off/reset and interact with the serial console over a tty. This is almost certainly what the horrible manufacturer interfaces already build on top of. But as far as I know, the damn things only accept signed images and aren't easily stripped down and replaced like this?
- pabs3 4y agoI expect these devices have plenty of vulnerabilities over the years and there have been Linux remote exploits, so probably you could hack them to replace the software or even just brick the device to avoid the vulnerabilities. Also companies like Raptor Computing, Facebook and others are creating open BMCs people can use.
- rgrs 4y agoOpenBMC only supports one vendor's SoC.
- amboar 4y agoOpenBMC supports SoCs from Aspeed and Nuvoton, and HP were/are working to add support for their GXP (iLO) SoC.
- qhwudbebd 4y agoGiven a random board with an aspeed 2500 on it, I assume that's not sufficient to allow the manufacturer firmware to be replaced with vanilla openbmc by a user though?
- justsomehnguy 4y agoOr just enable NTP client in the BMC. A more than a decade old BMCs based on Aspeed 20xx had the NTP baked in. Because you know, it's already running Linux there. I don't remember on hw/os ancient iLOs/iDRACs were running on, but they had NTP client too. I'd say everything newer than 2010 should already have NTP client.
- gpapilion 4y agoThe oem vendors have been adding weird surcharges for the bmcs to use features. For example hp used to charge(and I assume still does) for remote console. The author mention dell charged for using ntp, which was a little surprising. But, “enterprise” features like ntp and ldap are things vendors get excited to charge you for.
- justsomehnguy 4y ago> adding weird surcharges for the bmcs to use features While this is a true statement, I don't remember NTP client being on this list. SuperMicro only wants a license for a online BIOS update, HP/HPE wants money for iKVM to a running OS[0] and some features making sense only in enterprise environments, like AD integrations and Dell's iDRAC... well I haven't used iDRAC without aN Ent. license for a decade, so I can't bet my $5 on the feature set. Overall, most BMC implementations does have NTP client free of charge. [0] You can have any time you need to install OS. You have a one or two minute limit in already running OS. Enough to for the troubleshooting.
- X-Istence 4y agoIn Dell iDrac 7/8, NTP is a licensed feature. See table here: https://www.dell.com/support/manuals/en-us/idrac8-lifecycle-controller-v2.75.75.75/idrac8_2.75.75.75_ug/licensed-features-in-idrac7-and-idrac8?guid=guid-e7b32690-602f-4f14-ab6e-ad00b4a42e4a&lang=en-us https://www.dell.com/support/manuals/en-us/idrac8-lifecycle-...
- justsomehnguy 4y ago
- johnklos 4y agoBMCs are horribly insecure and probably intentionally so. They should always be on a dedicated ethernet port and never shared (looking at you, Supermicro - no port should ever be able to accidentally become shared), and the network to which they're connected should be 100% disconnected from the Internet, which means you'd obviously need to be running your own NTP server. If you weren't previously using NTP with your BMC, then that might suggest you haven't put enough time and attention in to your configurations. If you're not already keeping your BMCs from talking to the Internet, then you really, really should, as soon as you can.
- zokier 4y ago> If you weren't previously using NTP with your BMC, then that might suggest you haven't put enough time and attention in to your configurations. > not all vendors include NTP as a free feature on all of their BMCs. Charging licensing costs for some features is a good way to insure that we don't use them, unless the licensing cost is trivial (which, spoiler, it hasn't been when we asked
- BuildTheRobots 4y agoAre there any resources that talk about the specific concerns of the bmc having outbound Internet access?
- infotogivenm 4y agoIt is general advice across most server application deployments to limit internet egress (see for example NIST NIST SP 800-190, section 4.4.2 “Unbounded network access from containers”) . A machine that can talk out to the internet usually provides an attacker more avenues to compromise it and control/exfiltrate data from it post-compromise. A classic example is a SSRF bug, like how the recent log4j/log4shell bug used an attacked-instigated outbound LDAP call to a malicious internet host to achieve RCE.
- rgrs 4y agoThe article mentions KVM over IP licensing. AFAIK only Intel and its OEMs used to do that. Who else provides licenses for features?
- jclulow 4y agoHewlett-Packard has never heard of a rent they didn't want to seek, so iLO definitely has bullshit extra fees for remote management after the firmware exits.
- technofiend 4y agoHP also unlike Dell requires a maintenance contract to get the latest ilo updates. The free ones are years behind, so if there are zero days for ilos then many price-sensitive users purchasing them on the secondary market are vulnerable.
- jwiz 4y agoDell iDrac requires the enterprise license to get KVM-over-IP.
- colechristensen 4y ago> Who else provides licenses for features? Every enterprise grade hardware provider as far as I am aware.
- deleted 4y ago[deleted]
- DannyBee 4y agoEven with shared network BMC's, NTP will work fine. I'm not sure what BMC they are using that does not have NTP - it's been around all aspeed BMC's for a decade. I'm not aware of anyone charging for it, either.
- ncmncm 4y agoIt seems to suffice to log the difference between BMC time and host time periodically. Actually correcting BMC time seems pointless.