Supra Final Solution. Must Read!!!

[GatorBrent]

We have been working day and night for days to get supra up and running. When the server first crashed it took a huge effort to get it back online due to the amount of corruption done to core system files. When we finally got it back online after a ton of work, we found most of the cpanel files were corrupted or completely missing altogether. We still thought we were fine because we had an offsite backup that was done on the entire server one day prior. We went to use the files from the backup that were needed, and they were all corrupted, everything was.

We still believed we could get the server fully operational again and spent every second fixing problems as we found them, which involved reconstructing data that simply did not exist. We succeeded in many ways with this. We worked our way through most of the errors and reconstruction and had one major problem left to fix (exim crashing every few minutes from hung processes). It appeared as if it needed a fsck on two partitions so we ordered the datacenter to do this. Within seconds of the box coming back online, the server had another doomsday meltdown. This is when we discovered the ram was what caused the problem to begin with, and we were completely mistaken thinking it was the raid controller. This is why our off-site backup was completely corrupted!

We should have given up at this point, but again we believed once the server was up we could get past all the problems. We had the chassis completely swapped and got it back online again and have been working up to about a hour ago to fix all issues.

We got hung up multiple times. The corruption this time around was even worse than before. Our progress has been extremely slow and there's no telling how many more issues we would have ran into. We cannot waste any more of your time or our time in dreams of recovery. We simply cannot go on with this charade.

The server is online right now and for the most part, almost all web sites are working. We have decided we will keep the current server online with all your data for close to two months. We are not a bad host, however, if you were to base your opinion about us on the events that took place with supra, we would be one of the worst hosts in the world. We have over a 120 servers and in the three years of being in business we have never had anything like this happen. This one event has completely destroyed all of us. We did everything that we could. We had a raid1 backup for instant backups and a weekly offsite backup. The corruption from the Ram destroyed the raid and the off-site backups.

What to do now...............

• A. find a new host
• B. give us another chance on a new server and get 3 months free.

We hope all of you choose B. We are a great host. What happened was a string of unlucky disasters all resulting from a ram chip that happened to fail with the worst possible results we have ever seen.
The server will be online until February 18th. We will not support any problems supra has other than people having trouble downloading their files in order to move to a new server. If you are moving and need help getting to your files, please post IN THIS THREAD. All other Supra topics have been closed at this time.

Steps to take..........

Step 1:
The first step is to get your new account up and running in less than 15 minutes time. You can do this by going to www.hostgator.com/resellers.shtml sign up and make sure to select quarterly payment and use the coupon code "evacuate" remove the quotations. This will give you three months for free. Do not select monthly payment. If you do only one month will be free. You are able to cancel at any time, and you can switch to monthly before renewal.

When ordering make sure to select existing client if this does not work you can order as a new client. If your e-mail address is taken in the system you could use another e-mail but make sure it is working.

Step 2.
Once you have your welcome e-mail for the new server, begin moving all of your sites over. You will not be able to use the backup tool cpanel offers and we will not be able to transfer the sites for you. The reason is most cpanel files are corrupted. You do not want to take these problems with you to the new server. To download your files, use something like FTP, dream weaver, or even file manager from the cpanel if it is working for you. To download your databases, go into your cpanel and click backup. "Download a MySQL Database Backup" this backup will be fine, do not use the full backup or home directory backup, or aliases/filters to move or you will be bringing many problems with you to the new server.

Step 3.
Once you have all of your files moved over to the new server you will want to update your name servers to the new IP addresses that are given in your welcome e-mail. Allow for the DNS change to take 24 hours to go through. If you bought your domain from us and you cannot make the changes on your own, please e-mail sales@hostgator.com with your domain name and that you need us to do it for you.

Step 4.
You must fill out a cancellation form for your old account or you will continue to be billed for it. This is located at hostgator.com/cancel.php In the reason for canceling field, put either "evacuating supra to a new server with you" or "supra I'm going to a new host." Do not fill this form out unless you are sure all of your files have been moved; once they are deleted, they are gone for good.

We suggest you do the above as soon as possible to avoid further problems. On February 18th the server will be unplugged. Once it is unplugged, it is to late to do anything.

How will hostgator prevent this in the future?

You can look at it from a math perspective. We have over 120 servers and have been in business close to three years without ever having a problem like this. The problem affected one server in three years time.
The above is not good enough for us. What you went through should have never happened, even if there was no way for us to prevent it.

We are getting a new weekly off server backup system for all our shared and reseller boxes. The new backup system will be more reliable/better than tape (this is going to cost a fortune, but so did the supra incident). More details to come in a few days.

We are also adding more staff since this disaster has shown how ill-prepared we were to handle phone calls and support e-mails during the event that took place. Having more staff would not have changed what happened nor improved the chances of fixing supra, but it would have given us the power to provide the support we are so known for.

Apology

On behalf of Hostgator, I cannot stress how incredibly sorry we are for these past few days. Hostgator is my life. When customers suffer, I feel I have failed with my responsibilities in providing the best hosting experience possible. We would truly love to continue in serving you, but we will completely understand any decision you make.

With sincerest apologies,

Brent Oxley
Founder and President
Hostgator, LLC

[onsite]

Brent will this allow me to access the server by ftp? I still can't get it to resolve at the ip address.

[ghumphrey]

Brent,

Thank you for the updated information.

I am now able to get into WHM.

My concern is that I need to get access to the files that are in a particular users account that no longer shows up in WHM. The site is accessible via HTTP, just not via FTP nor any other method.

I am concerned that I do not have access to these files and whatever I might do to get access to my other accounts might render it unusable.

BTW, what does this mean?

Please do intimate us in case of any queries for the same ,we would be glad to further assist you in this regard.

That was the sign off reply that just arrived to my request for the WHM password reset from earlier today.

Is is possible to get a .TAR/.ZIP of the files in the user account that I do not have access via WHM? Is there another method that we can use to resolve this issue? I appreciate your assistance.

[Brink]

I know that others don't feel this way but as for me, Apology accepted.

I did however already take your advice given here : http://forums.hostgator.com/showthre...=8240#post8240

I already sisgned for a new account & didn't use the "evacuate" coupon code because there was no code yet. You said to be patient & refunds will be given so I have been. I still have not asked for a refund because 3 weeks have not yet passed. So I'm just being patient and hoping you have a plan to deal with 3 months credits manually for users like myself when the time comes.

My new server space on infiniti works great... just like supra used to work.

[GatorBrent]

Everyone should be able to access via 67.18.2.226 for the hostname. Remember, you still need to change cpanel passwords from whm before you can FTP into them.

You can also use the cpanel file manager to download files from a site.

[Trinity31]

I'm totally pissed....

But I do accept the apology, and also understand Brent's position.
I have no hard feelings personally.

All I can do is crack a beer, prepare for a long night, and get my customer sites online before dawn.

I do thank Brent for all his hard work and late nights trying to resolve the issue.

[ghumphrey]

Originally Posted by GatorBrent

Everyone should be able to access via 67.18.2.226 for the hostname. Remember, you still need to change cpanel passwords from whm before you can FTP into them.

You can also use the cpanel file manager to download files from a site.

Brent,

One of my customer's was "merged" with another.

Customer "PV" account is showing up on "B4W" site.

Customer "PV" site is not listed at all in WHM.

Customer "PV" site was a dedicated IP site.

Site is viewable via dedicated IP.

I need to get the files from Customer "PV" account. The "B4W" account doesn't matter right now.

Help!

[onsite]

Brent I've tried to access them through the ip address and no go. How do I get my files to the other server if I can't ftp into it?

Originally Posted by GatorBrent

Everyone should be able to access via 67.18.2.226 for the hostname. Remember, you still need to change cpanel passwords from whm before you can FTP into them.

You can also use the cpanel file manager to download files from a site.

[GatorBrent]

Did the file manager work?

[Matrix0]

I paid two years in advance for this hostings. What path do I take to avoid monthly charges after 3 months?

[Matrix0]

I've put the order in. But as above I'm not sure how you wish to handle annual payers in terms of adding the 3 months on.


Also... anything you can do to get TekSalon up in the meantime? That's our biggest worry at the moment.

[nodtveidt]

Originally Posted by Matrix0

I paid two years in advance for this hostings. What path do I take to avoid monthly charges after 3 months?

The first step would have been to never pay for more than 3 months in advance. You shouldn't ever lock yourself in to one company like that, even if you plan to be with them for years to come. Hope for the best but expect the worst.

[Nutter]

Question -
With the move, I assume we'll have to get into our registrar account and change the name servers?

- Ryan

[ztechpc]

I think you shoudl do the right thing and Move the accounts for us, I personallyldo not want to go through reconfiguring oscommerce and dealing with ssl again.

If you are as good a host as your say you are then this is something you would do for your customers after over a week of downtime. I know Interland would do something like this cuz THEY are a GREAT company. Now it's time to prove you are on par with the big boys.

[geoff]

Gators, get moving and reconfigure our site homewardboundtogo.com, you have not finished your work! Our customer needs the site up and running so they can stay in business.

[Cindy]

Brent, although I was not personally affected by the Supra incident, as a new customer I have understandably been watching this situation closely. Let me just say that I could not be more impressed by how it has been handled, and I am more confident than ever that I made the right choice in hosts.

[onsite]

No. Nothing works. I can't get to anything from the ip address. with ftp I should at least get a login screen. I get nothing.

Originally Posted by GatorBrent

Did the file manager work?

[onsite]

Brent I've even ping the ip and I got 100% lost of packlets. I can't get there for some reason. HELP!

[obmedia]

Brent or anyone in HG Support: Can you please resend my welcome pkg email so I can transfer my accts from Supra. I have my new email in the secondary...the primary address is no longer accessible and I can't change it .

Thanks!
Mark

[David]

Originally Posted by ztechpc

I think you shoudl do the right thing and Move the accounts for us, I personallyldo not want to go through reconfiguring oscommerce and dealing with ssl again.

If you are as good a host as your say you are then this is something you would do for your customers after over a week of downtime. I know Interland would do something like this cuz THEY are a GREAT company. Now it's time to prove you are on par with the big boys.

I've seen you praise Interland on a couple of different threads. I am glad that you feel so good about them. I thought I would share my experience with you and others. I started to work for my current employer as a contractor in January of 2004. They were with Interland at the time and had NUMEROUS email outages and http downtime before and after I came on board. They also had bad expereinces with getting support for those issues. In addition to other things, they have me handling all of their internet/website operations. They had already made the decision to go with Webhost4life and had opened a new account (based on someones reccomendation). So, one of my first tasks was to get everything over to them, which I did. After a few months with them, their downtime was much better than Interland, but performance was really slow and the companies needs outgrew their largest package. So, I started searching WHT and other places and ended up reccomending they go with HG. They and I have been very pleased thus far with the services. Performance is an order of magnitude better than Webhost4life, and while we have experienced downtime it has been very short in nature and rare.

I am sure it is possible that Interland has cleaned up it's act since then, but I doubt I would ever host with them based on the past experience. There are just too many other good hosts out there to do that. While we have not been affected by the Supra crash, I feel for everyone who has had to deal with this and wish you all the best of luck in getting up and running again.

David

[GatorBrent]

Anyone with special needs such as yearly or biannual payments please email me your account info brent@hostgator.com You will get everything you paid for.

Because of the corruption we can't do a server to server transfer. If we did server to server it would be exactly what you have now. Missing files would still be missing, corrupted, etc.

If this was a shared server we would be able to handle moving most of the accounts via FTP for you however it is not. It is a reseller server where most of the resellers have 30 plus sites each. For those of you, that is going to be a pain, moving them. Imagine us trying to move all of them, because it is not server to server. There is no magic transfer it has got to be done manually. Even if we did decide to move them we would not know the site passwords and other desired information required when setting up the accounts.

I have no idea why you would not be able to ping the server. If you could open a support ticket on that and then e-mail me the ticket member I will try to figure out what is going on as.

There are a few sites whose FTP or cpanel username do not exist. We will be helping everyone get the files off. If you are having trouble and cannot access anything open to support ticket with what you need and we will try to deliver.


>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Big tip
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

If you are one that was to have the site switched quickly, without waiting for DNS changes log into your WHM and added the DNS zone for the site you're moving, replacing the whole IP with the new one every where you see it. You will still need to make DNS changes at your registrar but this will speed things up making your site load off the new server within minutes. I will try and finalize the details of the new backup system today and post about it.

[fuzzfree]

Transfering sites is a painful job.
It is better to try moving files with FXP from Supra to the new. Moving the sites from your PC to the new server depends on your connection (PSTN, ISDN or DSL) but if you try FXP the 2 servers will communicate with their connections (the difference is HUGE!).

Search for an FTP client with FXP support. Hope this helps.
Antonis

[TheImmortal]

Well, I have to say that given the choices I will move I stuck with you guys through all the crap; but I just don't have the confidence in you that I once had. So, I will send an Email to sales soon to discontinue my account as soon as I finish getting what data I have off your servers. It really is too bad; things were working pretty good after that first crash until you tried to fix them again and again.

Good luck to everyone sticking with HG. I wish you all and HG nothing but the best.

[airkat]

So I see the 3 months free thing, and decide, hmm I can keep lesser sites here and more important ones on the new server. I log into cpanel, initiate a site backup, logout, and now cant log back in? lol wtf, nothing works now...

*edit* after unsuccessfully trying to hange my pass for me... support points me back here... great...

thumbs up

[lordprodigy]

hello everyone,

I made backup of my site today. Now at the new server location, if I do "full restore from backup" using the zipped file I have, will that recreate all the emails, passwords, databases etc... or should I do everything manualy?

thanks for your help.