Head Guru Posted December 17, 2008 Share Posted December 17, 2008 Hello, Our internal backup server #2 is currently off line for a File System Check. The raid array went into read only mode, and this is requiring a FSCK. So for the moment Backup Server #2 is not available for remote restores thru your cPanel. We will update this when we have more information. Thanks Link to comment Share on other sites More sharing options...
TCH-Ryan Posted December 17, 2008 Share Posted December 17, 2008 The file system check/repair is still in progress, we will provide further updates on this matter as soon as possible. Link to comment Share on other sites More sharing options...
Head Guru Posted December 18, 2008 Author Share Posted December 18, 2008 The FSCK is still in progress. Remember, were talking nearly 7 TB of data here. Updates soon to follow..... Link to comment Share on other sites More sharing options...
TCH-Ryan Posted December 18, 2008 Share Posted December 18, 2008 It became clear to us that that data set for backup server 2 was in such a corrupt state that the fsck was never going to complete and if it did it would have still resulted in a file system that was both inconsistent and ridden with issues. The call was made after 18h of struggling to get data restored in a viable condition, to simply reformat the partition and repopulate it with data. Although the backup servers are all raid setups, that does not protect from file system corruption as was the case on this system when we experienced the power loss at the data center. The process of repopulating the server with backups from all associated servers, will take some time and we will post updates as best possible with the progress of this task. We apologize in advance for any inconvenience this may cause and thank everyone for there patience and understanding. Link to comment Share on other sites More sharing options...
TCH-Ryan Posted December 18, 2008 Share Posted December 18, 2008 Data is still steadily repopulating onto buserver2, in total there is 48 servers that store data on this system so it may take up too 24h for this task to complete. We will provide further updates as soon as they become available, thank you again for your patience. Link to comment Share on other sites More sharing options...
TCH-Ryan Posted December 19, 2008 Share Posted December 19, 2008 We are currently at 30 of 48 systems reseeded onto the backup server, steadily getting there. Link to comment Share on other sites More sharing options...
TCH-Ryan Posted December 19, 2008 Share Posted December 19, 2008 The reseeding process of all servers that backup too buserver2 has completed, we have turned back on the regular incremental backup schedules and data is now available for restores on the server as normal. Thank you everyone for your patience while we worked to bring buserver2 back into a production state. Link to comment Share on other sites More sharing options...
TCH-Carl Posted December 19, 2008 Share Posted December 19, 2008 Link to comment Share on other sites More sharing options...
TCH-Bruce Posted December 19, 2008 Share Posted December 19, 2008 Good work Ryan! Link to comment Share on other sites More sharing options...
Dumplin Posted December 19, 2008 Share Posted December 19, 2008 The reseeding process of all servers that backup too buserver2 has completed, we have turned back on the regular incremental backup schedules and data is now available for restores on the server as normal. Thank you everyone for your patience while we worked to bring buserver2 back into a production state. Is this why my site just-for-fun-club is down? Thanks! Donna Link to comment Share on other sites More sharing options...
TCH-Thomas Posted December 19, 2008 Share Posted December 19, 2008 Hi Donna, This should not be the reason so please submit a ticket and the techs will look in to it. Link to comment Share on other sites More sharing options...
Dumplin Posted December 19, 2008 Share Posted December 19, 2008 Hi Donna, This should not be the reason so please submit a ticket and the techs will look in to it. Ok will do that when I get home. Thanks! Donna Link to comment Share on other sites More sharing options...
Recommended Posts