Downtimes of General Environment computational resources (including granite, notchpeak, kingspeak, and lonepeak) on the nights of January 14–15 and January 21–22
Update: As of the morning of January 22, all systems are back online.
Computational resources in the General Environment, including the granite, notchpeak, kingspeak, and lonepeak clusters and standalone biochemistry and cryo-EM servers, will be offline overnight on January 14–15 and January 21–22, 2026. The downtimes are a result of a utility company's infrastructure upgrades in the area, which will result in a power outage at the data center. Affected systems at the CHPC will be unavailable beginning at 8:00 p.m. on January 14 and January 21, and they will be brought back online the following mornings (January 15 and January 22, respectively).
Critical services, storage, virtual machines, and the redwood cluster (Protected Environment) are supported by generators and should not be affected by the power outages.
We have put reservations in place on the General Environment clusters (granite, notchpeak, kingspeak, and lonepeak) to prevent jobs from running during the power outages. As the outages approach, you may notice that jobs will not be started by the scheduler, Slurm, if they overlap with the reservation window. This is intentional; it serves to ensure that no jobs are running when the clusters are shut off for the power outages, which would cause running jobs to be interrupted and could potentially result in incomplete or corrupt files. With the reservations in place, any jobs that start are guaranteed to finish before the power outages occur, preventing such issues. Jobs that will finish before the outages begin (i.e., jobs that do not overlap with the reservations) will still be started by Slurm, while longer jobs will remain in the Slurm queue until outages are complete and reservations are released. If you see a message like "ReqNodeNotAvail, May be reserved for other job" or "ReqNodeNotAvail, Reserved for maintenance" when submitting or viewing the status of a job, please try shortening the runtime of the job (if possible for your job) so it does not overlap with an outage-related reservation.
The CHPC will provide updates as services are restored. Additionally, we recommend using the status page to view the status of each scheduled outage.