Jump to content
Sign in to follow this  
TCH-Dick

Tchnoc Power Outage

Recommended Posts

At approximately 2:30 PM EST we lost power to the data center due to a lightning strike and were forced to switch to backup power.

Power has now been restored and all services are back online. Thank your for your patience and please open a ticket if you have any issues.

Share this post


Link to post
Share on other sites

I will be posting a full recap as to why our onsite generator failed to take over the power loads.

 

At the moment, I am simply making sure that all services have been restored.

Share this post


Link to post
Share on other sites

Update:

 

At approximately 2:30PM EST power was lost to the TCH NOC. We were notified by local law enforcement that a bolt of lighting struck the power supply yard not far from the DC and it knocked out an entire grid.

 

The moment we lost AC power, our UPS system kicked in and we were running on backup batteries. Everything was going as it should. 3-5 minutes after the power loss, our generator started up and was running. A few moments later we moved to generator power and everything was up and running. Approximately 15 minutes under full load the generator stalled and we lost all power to the power distribution units. Several attempts to restart the generator failed. We lost all power and sat in the dark.

 

Immediately we made contact with our standby power supplier and a temporary generator was enroute to the DC.

 

At approxmiatley 3:40PM AC power was restored to the facility. We lost AC power again at around 4pm and our UPS ran us for as long as it was possible and we lost power to the servers again. Shortly therafter AC power was restored.

 

The electricians have been on site for approximately 2 hours awaiting the arrival of our temporary generator. Once the generator arrives I will post a update to this thread.

 

Total down time thus far has been 1.5 hours and all servers have been returned to full service at this point.

Share this post


Link to post
Share on other sites

The temporary generator is on site and is in the process of being hooked up. I took some pictures to share with everyone.

 

101_0410.JPG

 

101_0411.JPG

 

101_0412.JPG

 

101_0413.JPG

 

101_0414.JPG

 

101_0415.JPG

Share this post


Link to post
Share on other sites
The temporary generator is on site and is in the process of being hooked up. I took some pictures to share with everyone.

 

Excellent response time guys, in a bad situation.

 

Thanks !!

 

:group:

Share this post


Link to post
Share on other sites

We have been running on diesel power for the past two hours or so.

 

I am pleased to say things are back to normal for now.

 

Thank you for everyones support.

 

Bill

Share this post


Link to post
Share on other sites

And I though that I was having a lousy day!!!

 

My hat is off to all of you for the stand up job of dealing with the combination of problems thrown at you today.

 

Thanks TCH!!!

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.
Sign in to follow this  

×
×
  • Create New...