Skyline down

[David]

All of my sites appear to be down on Skyline on both reseller accounts. I tried checking them through siteuptime and alertra just to make sure it wasn't a dns issue with my ISP, but both came back failed for http on several of the sites.

Someone please look into this as soon as possible. Thanks.

David

We are aware of this issue and are working on it now. We will update you as soon as any more information is available.

[GatorBrent]

Trying to figure out what happened, the box seems to be hosed we will most likely have to restore from the disksync. Working on that now

[d-lexy]

Any idea how much time it will take if everything goes well
*Knocking on wood*

Originally Posted by GatorBrent

Trying to figure out what happened, the box seems to be hosed we will most likely have to restore from the disksync. Working on that now

No ETA yet but I will post here the minute we do, we are all working hard on this issue.

[GatorBrent]

We orginally believed this box could have been hacked but with help of planet and their logs we were able to eliminate that. We thought this because most logs were missing, and it looked intentional.... although we couldn't figure out why box wasn't just "rm'ed" What made us really suspect this was the fact that another server viper went down shortly after. Viper was up 55+ days in a row and the timing was horrible. We thought it was hacked... Got viper up after a delay from our paranoia investigating it's issue.

Upon further investigation whats left of skyline we saw no one has logged into this box recently (planets logs) and everything in home and var was garbadge. (corrupted) Thus why logs no good.

So something hardware went bad. We are having the box swapped as we speak and will be restoring everyone from disksync backup solution. I will post news as I get it.

[David]

Thanks for the updates. I have backups of all of my and company websites from last week, so that's not an issue. The boss is very concerned about lost email. What happens if mail was sent today while it has been down. Are they just lost? Also, I am guessing that any mail that was waiting to be picked up will be lost as well?

David

[GatorBrent]

Anyone still down? Looks like it is done.


While it was down emails would have bounced back, anything on server not downloaded would have been lost.

[sgcom]

I still encounter problem with some accounts. Some are restored.

[David]

All of the company's sites are back up, just old. All of mine (5) are still down with the exception of one.

Do you know what the date/time stamp on the restored sites is? I know you said roughly a week, but if it's older than 2/27 when I did my last backups I will want to restore them overtop of what is there.

Can somone answer the email questions I posed above? Am I right in thinking that all of the email is lost?

David

[GatorBrent]

I'm locking this thread everyone, please post in the official thread. (less confusion)