NIST warns several of its Internet Time Service servers may be inaccurate due to a power outage — Boulder servers 'no longer have an accurate time reference'

Time waits for no man
(Image credit: Getty Images)

The National Institute of Standards and Technology has warned that several of its Internet Time Service servers could be providing inaccurate time following a failure of the primary atomic time scale, NIST-F4, at its Boulder, Colorado campus. The alert was posted to NIST’s public Internet Time Service mailing list after a prolonged utility power outage disrupted the facility on December 17, with engineers still working to fully restore normal operations several days later.

According to NIST, the Boulder campus lost utility power at approximately 22:23 UTC during a period of high winds that triggered line damage and preemptive shutdowns tied to wildfire risk in the region. While backup systems were expected to maintain continuity, NIST says a critical standby generator failure occurred downstream of the signal distribution chain that feeds its Boulder-based Internet Time Service infrastructure. As a result, the atomic ensemble time scale that underpins those services was interrupted.

NIST F-4 atomic clock in Boulder

(Image credit: National Institute of Standards and Technology)

The NIST-F4 atomic clock at Boulder uses caesium atoms to measure the exact length of a second, which is just a little bit more sophisticated than one you might be able to build on Pi. It's an important point of reference used for many applications like GPS systems, data centers, telecommunications, and power generation — all of which require extremely precise timekeeping. According to NIST, it represents the "gold standard of accuracy" in timekepeing.

The Boulder incident follows another Internet Time Service disruption on December 10 at NIST’s Gaithersburg, Maryland site, where an atomic time source failure caused a time step of approximately minus 10 milliseconds on affected hosts. At the time of writing, NIST has not provided a firm estimate for when full service will be restored at the Boulder campus.

Google Preferred Source

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

TOPICS
Luke James
Contributor

Luke James is a freelance writer and journalist.  Although his background is in legal, he has a personal interest in all things tech, especially hardware and microelectronics, and anything regulatory. 

  • bit_user
    Yikes! That sounds like they're probably working with antiquated equipment on a shoestring budget. Given how much IT infrastructure depends on it, it really ought to be bomb-proof, with multiple failovers at each point. Stuff like a power gap, when switching to a generator, sounds pretty amateur.

    They say there's geographical redundancy, but how much?
    Reply
  • kanewolf
    bit_user said:
    Yikes! That sounds like they're probably working with antiquated equipment on a shoestring budget. Given how much IT infrastructure depends on it, it really ought to be bomb-proof, with multiple failovers at each point. Stuff like a power gap, when switching to a generator, sounds pretty amateur.

    They say there's geographical redundancy, but how much?
    The current federal administration does not believe in funding basic scientific things like NIST. They probably ARE years past due for recap on something.
    Reply
  • thisisaname
    bit_user said:
    Yikes! That sounds like they're probably working with antiquated equipment on a shoestring budget. Given how much IT infrastructure depends on it, it really ought to be bomb-proof, with multiple failovers at each point. Stuff like a power gap, when switching to a generator, sounds pretty amateur.

    They say there's geographical redundancy, but how much?
    I think they dream of having a shoestring budget :o
    Reply
  • jp7189
    For some reason this phrase made me laugh: "lost utility power at approximately 22:23 UTC" . I mean wouldn't NIST of all people have a pretty specific idea about what time it was when the power went out.. or did all the clocks just start flashing 12:00?
    Reply
  • Konomi
    Well, if you're several nanoseconds late for work, now you have a valid excuse.
    Reply
  • DS426
    kanewolf said:
    The current federal administration does not believe in funding basic scientific things like NIST. They probably ARE years past due for recap on something.
    If these systems are years past due for recap, then this problem began with the prior administration.

    I don't think it would take that long for standby generators to start having issues though; even getting six months behind on proper maintenance could probably result in something like this, or staff being terminated without proper succession/re-assignment of remaining employees to perform the occasional self-tests. In other words, I don't doubt that cuts from the current admin are the main culprit here.
    Reply
  • bit_user
    DS426 said:
    If these systems are years past due for recap, then this problem began with the prior administration.
    I think US Federal budgets have been getting squeezed since the 1990's, but actually the 2013 US Federal Budget Sequestration bill did quite a lot of damage.
    https://en.wikipedia.org/wiki/2013_United_States_budget_sequestration
    In general, budgets of most federal agencies have not kept pace with inflation, while demands for new services have continued to arise.

    DS426 said:
    I don't doubt that cuts from the current admin are the main culprit here.
    I don't know and I sort of don't care, because I definitely see a situation where important government services are not understood by lawmakers or the public, which has contributed to rot. That's the bigger story, IMO. Everyone seems to understand how vital something like the FAA and air traffic control is, but that's just the tip of the iceberg in government services that support a complex and technologically advanced society & economy. DOGE is yet another symptom of this, albeit one that has itself compounded the problem.
    Reply