lincoln UPDATE

***GatorBrent*** · #1 01-07-2006, 07:35 AM

The other day the server had a hard drive failure so it was taken off the network to replace one of the drives. Upon doing so, the what was believed to be "the good Drive" completely failed after a system check.

This left us with one drive "the one we believed to be bad" which in fact, has turned out to be a bad drive and why the server is currently down.

The system is currently going through a second system check of home, at this point, it is not looking like it is going to go through. Here is our plan of attack.......

Our offsite backup is from a few days ago if we must go to this there will be a few days of data loss. So what we are going to attempt to do is mount this drive as a secondary in another box on the network. Than we will do a network restore off of this failing drive. A server will not boot up when the primary drive is needing a fsck (this means files are corrupted)
however it will boot up if it is a secondary drive allowing you to do all the files over except for the corrupted. The amount of corrupted files, could only be a few files out of millions. Basically, there should be no data loss this way.

how much downtime?

It all depends how long it takes to restore all the accounts. probably eight hours at a minimum. There is no way for us to speed this up since it is a completely automated restore.

best case scenario?
This second system check of /home works and the server boots up giving us time to move data off the failing drive. (looking unlikely at this point)

tei727 · #2 01-07-2006, 07:41 AM

8 hours? holy crapola!!

***GatorBrent*** · #3 01-07-2006, 09:35 AM

We are having to restore from the failing drive I'll keep everyone updated as it progresses. At this current moment in time, we are still waiting for new drives and have been for hours. the planet was having some major network issues, so they decided they didnot have time for our issue until they put out their own fires.

tei727 · #4 01-07-2006, 09:43 AM

Good luck!

***GatorBrent*** · #5 01-07-2006, 12:50 PM

still waiting on the reload..........

They told me everything is done but cpanel. as soon as it is back on the network once it completes we can then begin on the restore.

ahliz · #6 01-07-2006, 02:05 PM

1-5-2006
my resller is down when i chat with support he tald me just three hours !!
i wait 8 hours

1-6-2006
four domain is close
when i chat with the support he tald me please wait 2 hours
i wait 5 hours

1-7-2006
all my resller is down !!!

when i chat with the support
he tell me please wait only two hours
and he give me this link this topic !!!!

Q 1- why you don't info me you will close my resller befor you do this ?
so I can info my client !!
Q 2- why you don't move my resller to another server befor you close it like this ?!! three Dayes ... Three Dayes you play on my resller without I know anythink ?

Q 3- I know the server is hardware and it's need to fix it some time
and I know this company have more than 10 server at least
why you don't move my resller to another server from the firest day ?!!
Then you can do anythink you want on the bad server !!!
I lose My custumer !! 12 domain

really Thanks hostgator For Good service

ahliz.org

VailMedia · #7 01-07-2006, 02:24 PM

My question is why don't you inform your clients about server problems when this happens? Instead of leaving us hanging, send out an email to the clients it's going to affect or something.

I have never witnessed such horrible disregard for custom service in all my years of designing and hosting web sites.

***GatorBrent*** · #8 01-07-2006, 02:51 PM

The server is now being restored. We will keep everyone updated.

tei727 · #9 01-07-2006, 03:10 PM

I see the server is back up..just waiting for my site to be restored..says there nothing at the address..i'll continue to patiently wait..lol

bshell · #10 01-07-2006, 03:26 PM

Quote:

Originally Posted by ahliz

1-5-2006
my resller is down when i chat with support he tald me just three hours !!
i wait 8 hours

1-6-2006
four domain is close
when i chat with the support he tald me please wait 2 hours
i wait 5 hours

1-7-2006
all my resller is down !!!

when i chat with the support
he tell me please wait only two hours
and he give me this link this topic !!!!

Q 1- why you don't info me you will close my resller befor you do this ?
so I can info my client !!
Q 2- why you don't move my resller to another server befor you close it like this ?!! three Dayes ... Three Dayes you play on my resller without I know anythink ?

Q 3- I know the server is hardware and it's need to fix it some time
and I know this company have more than 10 server at least
why you don't move my resller to another server from the firest day ?!!
Then you can do anythink you want on the bad server !!!
I lose My custumer !! 12 domain

really Thanks hostgator For Good service

ahliz.org

WTF? That is absolutely great.

I do agree about emailing us when things happen, it would be helpful to know before my customers do.

Thanks for working hard to get this back up. Cheers!

tei727 · #11 01-07-2006, 03:42 PM

How would emailing help if the server is down? lol

bshell · #12 01-07-2006, 04:26 PM

that is a good reason to use a non-HG hosted email address for your HG contact.

***GatorBrent*** · #13 01-07-2006, 04:33 PM

Everyone should be back up within the hour or two with what is looking like 0 data loss so far. This would have only been a few hours downtime, had we had someone in the data center to perform what we ask with a standby server ready to go.

I'm flying to Texas next week to work this out with the planet to help minimize all downtime in the future.

tei727 · #14 01-07-2006, 04:41 PM

Quote:

Originally Posted by GatorBrent

Everyone should be back up within the hour or two with what is looking like 0 data loss so far. This would have only been a few hours downtime, had we had someone in the data center to perform what we ask with a standby server ready to go.

I'm flying to Texas next week to work this out with the planet to help minimize all downtime in the future.

The file manager says it's empty right now. Is that normal at this point?

Why do you have to GO to texas? can't you just call them? lol

***GatorBrent*** · #15 01-07-2006, 06:52 PM

It looks like all files have been restored.Iis anyone having any problems?

we are currently fixing the quotas, but please let us know if there's anything else. Missing or having problems.