Service Status

Power issue at Canix datacenter

CANIX REPORT: 

On Sunday evening one of our 150KVA UPSs had a major fault in the main logic board. While transferring to bypass (Hydro Quebec) power, there was a short power drop on customers connected to this string of UPSs. 

CANIX operations staff immediately contacted our maintenance vendor, and began troubleshooting the problem UPS. They determined the main logic board within the UPS failed, and required replacement. The replacement board was put on expedite order, and planned for receipt today in the afternoon (17 May). 

At this time the board has not been replaced, however we are completing configuration and updates of firmware within the board. 

All UPSs within the string are also going to be fully checked out prior to putting the UPSs back online. This is anticipated on Thursday, May 20th. 

CANIX will provide a complete report on outage to CANIX customers upon completion of the repairs, and send a notice to customers prior to placing the UPSs back online. 

UPDATE 3:40 AM EST: At this moment all the sites have been tested and service is back to normal. We are waiting for a report of the outage incident from the datacenter and will share the details as soon as we receive it.

UPDATE 2:22 AM EST: All the affected servers are up at this moment. We are still verifying that all sites are running properly.

UPDATE 2:02 AM EST: Most of the servers are back up at this moment but we are checking every server now to make sure all the applications are running properly. Please contact us if your application is still down or having any issue.

UPDATE 12:15 AM EST: We had restored most of the affected servers and we continue working to bring the rest of the servers up.  

11:10 PM EST There have been a power outage in our Canix facility. We are working to bring the servers back up but some had taken longer than planned due extended file system checks. One of the servers also had a hardware issue and we are working to move all the affected VPS out to a new host. We will publish more updates as soon as we have the details.