← back to the history
dedicatednow down?
Thread #574286 · 130 posts · started 2007-01-04 04:51 AM · archived from webhostingtalk.com
dwiguna · 2007-01-04 04:51 AM · #1
our main server at dedicatednow so slow almost down, does anyone experience the same thing?
the_fulgore · 2007-01-04 09:34 AM · #1
Least the phones are on this time.. heres hoping it's cleared up very shortly.
buymywindowshost · 2007-01-04 01:35 PM · #1
i'd like to request that people post as their servers come back up, so we can at least see that there is movement...
jxk777 · 2007-01-04 04:09 PM · #1
wish I had good news, but alas, my server is still down, my day is shot and I've lost a lot of money. I'm just dying to see the "official explanation" I just hope they have the "stones" to tell us why something that was quoted from one of their employees as "not a network issue" has kept us waiting this long. After all, if it was a power issue, they could have easily explained that hours ago and we would have understood. (although we would wonder why there was no backup generators in place) What I really don't understand is the "we will respond to what happened after all the machines are online" attitude, does that mean that everyone there is a tech and working on the problem? not one sales person, the CEO, or someone, anyone could tell us why we are getting ulcers over this mess? I'd say that it is a poor excuse to hide behind "our first concern is getting your servers online" and we will explain later, sure it sounded good for a minute, until you really think about the fact that they couldn't find one person to take 5 minutes and tell us what happened and put our minds at ease.
jxk777 · 2007-01-04 08:15 PM · #1
Gold, they must be saving the best for last, lol I'm just dying to see the reason for this nightmare. I can only hope that if anything even remotely close to this happens again that they learn from this and tell people what happened without making them wait for so long with no explanation. It certainly was the wrong way to handle this and nothing anyone can say will ever change my mind.
tcWebGuy · 2007-01-09 07:10 PM · #1
my clients have been complaining about 2-3 minute outages since the big outage last week. Seems like the network/power supply/routers/whatever is not totally stable yet.
cbwass · 2007-01-04 08:53 AM · #2
Down from Spain also
Yapluka · 2007-01-04 09:38 AM · #2
From dedicatednow's forums :
jxk777 · 2007-01-04 01:35 PM · #2
Yes, get the servers online, and then look at what caused the problem. One of our servers is working, and for many people it seems they have some working servers too. So we know they have working segments, even if they just patched us in temporaily to get us online they could always switch us back once the issue is fixed. This is insane that it is taking this long, and very expensive for many of us losing money by the minute. I think they need to pass a law that makes hosting companies liable for lost income due to extreme outages. Even if they gave us a month of free service it doesn't come close to compensating for all the lost income. After all, if any other business does something that causes financial harm to clients, they are held liable, why not hosting companies? Sure, it might raise hosting prices a bit, but the reduced downtime would certainly make up for that. PS - just checked, still dead server.
jbfung · 2007-01-04 04:30 PM · #2
Although we are all frustrated - blasting them here isn't going to be helping the situation. Keep in mind, that it would take more than a mere 5 minutes to try and prepare a suitable notice, and to be honest, I'd much rather they have everone working trying to get machines back online. To those who are posting updates of successful reboots - thanks. It's good to know some progress is being made.
Steven · 2007-01-04 08:18 PM · #2
I think it is cruddy that you will only post in your forums what happened... Maybe to prevent future customers to find out until its to late?
ForumsAddict · 2007-01-09 08:16 PM · #2
Its down again?
patchwork · 2007-01-04 08:55 AM · #3
Yes I lost connection too for a short period, I could not access anything at DN including the forums and their sites. Pete
PhMatt · 2007-01-04 09:39 AM · #3
At approximately 7:45AM EST, we experienced a partial network outage which is presently still effecting some clients. Our network and administrative staff are working as quickly as possible to identify and correct the matter and further updates will be provided as quickly as possible. An official statement will be provided later on today once all of the information is available. Thank you,
jbfung · 2007-01-04 01:39 PM · #3
Still out, ticket UGQ-508833
The3bl · 2007-01-04 04:41 PM · #3
Oh give me a break. I have been doing this a long time and it takes less than 5 minutes to make a forum post, do that once an hour with a real update on what is going and people shut up and are happy. Leave people in the dark for 8 hours and they are going to get mad and bash them, they deserve every bit of bashing they get for it too.
BrandonSCSN · 2007-01-04 08:32 PM · #3
I completly agree. At the same time, other customers (VPSColo, JVDS) can't see DNOW forums (and their own are now failing / filled with spam) so they can't find out what happened either. I only manage 2 VPSColo VPSs for another company, so if someone does get a hold of the "statement", please post it over here.
ChrisEatsAlot · 2007-01-09 08:19 PM · #3
Yep - this is BS now...
Russ Foster · 2007-01-04 08:55 AM · #4
Still down for me here
patchwork · 2007-01-04 09:52 AM · #4
I hope its not going to be too much longer, my main server has been down for 1 hour now :-( Pete
jayglate · 2007-01-04 01:41 PM · #4
Just an FYI to everyone we are working to restore service to everyones boxes. A full reason will be posted in our forums later tonight. Any questions please contact Jason Brozena.
jxk777 · 2007-01-04 04:49 PM · #4
JB, I'm not blasting, I'm stating facts. Let's face it, that can't all be working on the problem, seeing how they already fixed it hours ago. And even if they had their entire staff, sales, techs, phone support and all working on the problem, why is it taking going on 8 hours now to resolve this? Why did the admin stop anyone from registering on their own support forums? Our support ticket was posted over 5 hours ago and we still have a dead server. The simple fact is there are a lot of unanswered questions and we deserve anwsers, if we can't get them on their site then we will get them here. It seems we are getting more response here anyhow. here is my list of questions.... 1) what happened? 2) why don't they know which servers were affected without us having to tell them? 3) where is the redundancy? 4) why are we 8 hours into this with no explanation? 5) why did they stop us from being able to register on thier own forums so we wouldn't have to ask here? 6) what are they going to do to prevent this in the future? 7) why were they not prepared to handle this issue in a timely manor? These are reasonable questions, not blasting, and we deserve answers in a timely mannor, do we not?
gold777 · 2007-01-04 08:33 PM · #4
Oh believe me, it's happened before (multiple times), and we were pretty much kept in the dark then as well. This is par for the course for Dead Now which is why my web host is moving all their servers to another datacenter. They are fed up with the poor service. It's EXTREMELY rare for me to post publically when I have a complaint, but this is just getting ridiculous - at some point you just have to start calling it as you see it. After giving them plenty of 2nd chances, I haven't seen an ounce of improvement.
Gemhdar · 2007-01-09 08:38 PM · #4
At 7:15pm, our Newark links were running hot, and to prevent the slowdown, we opened up 2 more Gige Links to our Newark pop to alleviate the congestion. Also at the same time due to a Cisco STP bug causing a loop, we had to resolve this on both sides, and we were prepared, having dispatched engineers during the turn up in case something occurred, and we were able to resolve this with minimal downtime on our Newark link. If anyone is still having any issues feel free to contact me live, in real time, on IM, email or in our new IRC channel.
patchwork · 2007-01-04 08:58 AM · #5
I can access 2 of my servers now but not the third, DN's sites are still down as well. Pete
DangerMouse · 2007-01-04 09:55 AM · #5
Same here... 2 servers down over an hour now.
patchwork · 2007-01-04 01:47 PM · #5
I emailed my ticket number to Jason nearly 2 hours ago, I haven't heard anything since. RWF-542565 Pete
patchwork · 2007-01-04 04:50 PM · #5
8 Hours now and still down Ticket ID: RWF-542565 If anyone is reading this would it be possible to have a look at that ticket please and reboot the server. Pete
Ankheg · 2007-01-04 08:40 PM · #5
It's kind of sad when my (residential) Qworst DSL line has better uptime than my server there. Heck, I think that much-maligned "datacenter" in Wisconsin with the pair of almost-redundant T3's is probably doing better. Twenty quatloos says DN/FITX give the Lucy Lawless explanation - "Wizards did it." As a hedge bet, I'll take ten quatloos on a reprise of Senator Stevens' classic "tubes" explanation.
ForumsAddict · 2007-01-09 11:14 PM · #5
Cool - thanks for the update
tmwes · 2007-01-04 08:59 AM · #6
can't get on 2 of my servers, I have DN via Beachcomber, and they appear to be down as well.
the_fulgore · 2007-01-04 09:58 AM · #6
Looks like phone support is going to voicemail now.. I was in the queue then got shunned to voicemail suddenly.
BrandonSCSN · 2007-01-04 01:50 PM · #6
We had similar outages: System 1 System 2
Ankheg · 2007-01-04 04:50 PM · #6
Our machine at 65.98.19.85 is still down (unrouteable); sent in a support email a couple hours ago (hah!), but didn't even get an auto-ack. Just another regular day at FortressNOUPS...
zenx · 2007-01-04 09:09 PM · #6
One of our servers went down only for abt 10 mins last evening (i.e. in India here), but came back up with a funnily misspelt hostname - leads me to wonder if there was some issue which needed manual recovery of each server/hostname - that'd cause these long (and inconsistent across customers) outages. This incident is really scary - we went with dedNow for the terrific pricing - and am now wondering if those dollars are worth saving! We've had a good response from them most other times - but the silence on this one is a little scary.
cbwass · 2007-01-04 09:00 AM · #7
My servers are back up. One appears to have been rebooted 5 mins. ago. The others are OK. Maybe a power problem. They also had some routing problems with Telia earlier today I believe.
guipos · 2007-01-04 10:30 AM · #7
still can't open the support site of beachcomber from asia
cch · 2007-01-04 02:05 PM · #7
after 5.5hours, my server is still down. How long should we wait? my ticket [#WVZ-866894]
geekeffect · 2007-01-04 04:51 PM · #7
back up here! wooty. good luck to the rest of you.
jxk777 · 2007-01-04 09:14 PM · #7
Gold, As you can see I've been a member here for a couple years myself, and I've never posted before this. I've had servers in a few places over the last 7 years, but this is the first time one has pissed me off this much. Well there was one other time with a company that rhymes with ci-ghost - but they answered the phones and I told them what I thought directly. But for some stupid reason DN chose to not answer phones or even give any explanation and that really ticked me off. The only reason I can think of that they have done this someone in management wants to cover up some screwup. Most likely the issue wasn't really that bad, but it was mishandled and now they are scrambling to come up with an excuse that they think will do damage control - sorry, that ship sailed and sank a couple hours into this nightmare. The management needs to act like adults and stop playing these games. When something goes wrong, just tell us the truth without making us wait. It looks very suspicious when you leave so many people hanging for over 12 hours. It stinks of coverup. Yes, my server is back online, but I'm not going anywhere until I get an explanation. As clients, we deserve much better treatment and if you are not willing to give it...
tmwes · 2007-01-04 09:01 AM · #8
one is back
patchwork · 2007-01-04 10:44 AM · #8
Any news anyone, things have gone very quiet and its comming up to the 2 hour mark. Pete
buymywindowshost · 2007-01-04 02:06 PM · #8
our server has been down now for 5.25 hours according to our monitoring system. Please post if your server comes back up and tell us when it was restored...
R1CH · 2007-01-04 04:53 PM · #8
It's not like this is the first time DN have had problems and gone completely silent too. Each time everyone complains about the lack of updates, and each time DN says they'll try and "improve" things, but it seems to have little effect. For those wishing to gain access to the DN forums, you create your account through the DMS ( https://customer.dedicatednow.com/ ) in the Preferences / Forum Accounts.
geekeffect · 2007-01-04 10:43 PM · #8
seems as though all servers are working, but still no explanation as promised.
patchwork · 2007-01-04 09:05 AM · #9
The 2 servers I can access have not been rebooted. Pete
dinsolglobal · 2007-01-04 11:06 AM · #9
No news yet. My servers are also down. They should atleast provide some more information about this outage in their forum.
jxk777 · 2007-01-04 02:07 PM · #9
someone should get fired for this, after all where is the network redundancy? Isn't that one of the first things that should be in place in a server enviroment? Who designed this data center and forgot to add in at least some level of redundancy? I was a CNE back when Novell 3.11 was the thing to use and even back then we built redundancy into every network. Even back then we had backup equipment already pre-configured to drop into place in case of a problem. As we moved into the WAN age, we even had extra routers pre-configured, just in case we needed to replace a dead unit. It made it very easy to make a few quick changes and drop in a working router in minutes, not hours. I can only hope that they learn from this major design blunder and take steps to stop this from ever happening again. Don't get me wrong, I do understand that sometimes things break, but 5 hours of downtime is from poor planning on someones part. It is inexcusable and this lack of at least some hint as to what has happened makes me very curious. After all, by now they should have some clue as to whether it is a router, a switch, a cable, or? And why they didn't have a plan in place to handle this kind of failure in a timely manor is beyond me. Yes, I am upset, I've lost a lot of money already and I do have a right to be upset that it has taken this long to fix it.
gold777 · 2007-01-04 05:16 PM · #9
My host is moving me to a new server and much better data center after this. They've had enough with incidents like this, as have I. If there's an issue like this, the absolute WORST thing you can do is leave people hanging with no consistent updates on progress.
dwiguna · 2007-01-04 10:46 PM · #9
server back online after down almost 12 hours, but everything still slow, ftp, web, mail........:-(
Jon69 · 2007-01-04 09:10 AM · #10
I have a box through Beachcomber there as well. Had some trouble earlier today, seemed to be back to normal but now down
Gemhdar · 2007-01-04 11:22 AM · #10
Anyone still having issues or not fully back online, please feel free to contact me directly, and the sooner, the better. But PLEASE have a ticket in the system to reference. If you cannot submit a ticket for whatever reason, send me your details at jbrozena@fortressitx.com
24Hostingnow · 2007-01-04 02:44 PM · #10
2 of our servers are still down for more than 5.5 hours and yet none of DN team has given any explanation to contact our clients! I am also upset too. Our ticket id to DN is UBU-123791.
geekeffect · 2007-01-04 05:16 PM · #10
ankheg, from past experience, if u didnt get the auto email response, they didnt get the email. I would continue trying to send an email to support@dedicatednow.com until you get a reply, even if only automated to be sure you are in the queue.
jbfung · 2007-01-04 11:34 PM · #10
dwiguna - thanks for the post. Still keeping my fingers crossed.
jcjc · 2007-01-04 09:12 AM · #11
3 Server unreachable :-( Oh boy. Any news from DN? Their site is down too and support tickets don't go through.
cch · 2007-01-04 11:36 AM · #11
My server is not back online. please check my ticket [#WVZ-866894] thanks
patchwork · 2007-01-04 02:52 PM · #11
6 Hours Now Ticket ID: RWF-542565 Pete
The3bl · 2007-01-04 05:24 PM · #11
Not always my first ticket submitted by email 8 hours ago just got a auto response 30 minutes ago. My ticket I submitted direct to the help desk got a response right away. So we have two tickets now over 8 hours old.
jayglate · 2007-01-04 11:50 PM · #11
If your server is slow, please open a ticket and the techs can look at it. but all boxes should be up.
DangerMouse · 2007-01-04 09:12 AM · #12
looks like a power outage 3 servers came back up, 2 havent yet. cant create support tickets as DN server down also. now in queue on telephone.
dinsolglobal · 2007-01-04 11:36 AM · #12
I dont have any ticket number as my email did not get any reply nor any automated response. I sent you a email with my servers name and ip. Please check and put them up. thanks.
jbfung · 2007-01-04 02:55 PM · #12
I agree - our servers are still down - we are unable to email and no one is answering phones...support ticket UGQ-508833 if anyone is monitoring what is going on?
jxk777 · 2007-01-04 05:30 PM · #12
8 hours, 15 minutes, still no server...
jbfung · 2007-01-05 12:12 AM · #12
What I fail to understand is why additional tickets should be necessary, when the first was never addressed, and is still listed as open? Regardles, things appear to be running now.
Yapluka · 2007-01-04 09:14 AM · #13
Our server on 65.98.xxx is unreachable but the others on 69.72.xxx and 208.116.xxx are online. Wait & see
Yapluka · 2007-01-04 11:39 AM · #13
You can submit a ticket at http://support.dedicatednow.com
tcWebGuy · 2007-01-04 03:10 PM · #13
I was able to get in contact with them. They said the primary issue has been resolved, and that they are going around and rebooting machines right now. If you haven't submitted a support ticket, do so right away, so they know that your box is down.
host4profit · 2007-01-04 06:06 PM · #13
We have 3 servers down :-( What I find weird is this issue appears to be fairly limited. Really, for 9 hours of downtime, if it was a "larger" issue, this thread would be 30 pages long already based on the past. If that is the case, what is the holdup. Was there some type of power surge and many power supplies were lost? Seems simple enough to at least give a status. Oh well, have 3 servers prepped and ready to be moved to once they are back.
PhMatt · 2007-01-05 12:42 AM · #13
The following are the accounts which transpired today causing network / facility outages to occur: Early this morning at approximately 7:45 AM EST, we experienced what initially was believed to be a network failure shortly after a blip of a power outage seen in the main offices lasting roughly ½ a second or less. Network admins began working to diagnose the outage and services were restored by approximately 8:30 AM EST after rebooting the core routers and awaiting our BGP sessions to re-converge. The UPSs sounded their normal alarms of running in normal mode as generator power had kicked on supplied power through our ATS (automatic transfer switch) and we awaited to restore power back to normal at that time. During the time same between generator kick (roughly 15 seconds) the ups battery failure became apparent on UPS1 and servers lost power. Additional alarms sounded on our UPS indicating that the batteries drained in 11seconds compared to the rated 15minutes it should have had and our other UPSs have . Technicians were called onsite as well as emergency personnel as several office servers including our mail server and ticket system were down and additional hands were needed. By approximately 9AM EST, corporate servers were back online and functioning and we began to restore service to any servers reported as downed. It was later determined that some of the batteries inside UPS1 had died since our last routine maintenance which was in the middle of November thereby causing power loss to certain areas of the datacenter, while others were unaffected. Many of the downed servers required manual FSCKs or Power Supply swaps to bring them back online and team members responded as quickly as possible to each customers needs. During this time many multiple tickets were opened and hung in a mail server backlog only to dump all at once when the issue was determined with the mail server as well, which created additional support headaches to wean the false or duplicate tickets from the legitimate server failures. We continued to run fully on generator until approximately 6PM EST this evening to allow replacement batteries to be installed in the UPS, and will be replacing all of the UPS batteries early next week to ensure better protection for our clients. Though we have experienced power issues through normal city failures over the past 3-4 months, our failover systems have acted accordingly, in one instance running for approximately 16 hours while our main power feed into the building was replaced due to normal maintenance, and routine maintenance contracts on our UPSs showed no service failures that we would need to resolve prior to todays issue, it is unfortunate that in but a brief instance so many problems could arise. In light of this, we will be increasing our maintenance schedules on equipment and replacing components such as batteries over shorter scheduled periods of time to avoid any such future failures from occurring. We will also be deploying new methods next week in which to combat issues such as multiple tickets clogging up the support queues and taking time away from the task at hand. We will have an open discussion period planned for mid next week to talk further about these matters, and will announce these once they are set in place. As always, we thank you for your time and consideration in this matter, and resolve to continue to do all that we can to ensure your continued happiness with our services.
cch · 2007-01-04 09:16 AM · #14
My server is unreachable ... dedicatednow.com is down too
patchwork · 2007-01-04 11:41 AM · #14
Your email address doesn't work, hotmail bounced the message back. Pete
jxk777 · 2007-01-04 03:11 PM · #14
I can't think of any possible excuse for this considering they have working network segments. I bet they are negotiating with Stephen King to write this horror story about what happened. It would take one heck of a fiction writer to come up with an excuse that we "might" believe after this length of time.
ChrisEatsAlot · 2007-01-04 06:16 PM · #14
My server went offline too at around 7:45am, but, almost immediately came back online... Sorry to hear about everybody else's downtime...
The3bl · 2007-01-05 02:15 AM · #14
That is a great after the fact post, but it still fails to address the fact that you provided ZERO communication with us for over 10 hours other than to come here and tell us to put a ticket in which we had already done. Your major problem is not your lack of UPS, back up equipment, or anything else, your major problem is your lack of customer support and your obvious lack of concern for your customers need to know what is going on during an outage like this. Even taking 5 minutes once an hour to write a post in the forums stating what is going on and giving an on going status update would have solved and saved some very negative feelings. Instead as you have done in the past you chose to ignore your customers plea's for information. I had two servers down, I had tickets open from those two customers and the only thing I could tell them was I have put in a ticket and I have and contnue to pace calls and I have no idea what is going on or when your server will be up again. So I hope you are happy cause these two customers have now demanded that we move them off your servers and into our racks at Gnax. That is if they do not cancel their accounts with me altogether thanks to your lack of information. Matt I used to consider you one of the best in the business at customer service, but after this I have to wonder what happened to that.
jcjc · 2007-01-04 09:18 AM · #15
I have two boxes on 69.72.XX. None of them are available.....
patchwork · 2007-01-04 11:46 AM · #15
Thanks I now have a ticket number RWF-542565 Pete
geekeffect · 2007-01-04 03:14 PM · #15
WHT seems to be the official news and outage place. I assume this is the official place for reboot requests too (jsut found out i had to put a request in because i was watching DNow forums). Please reboot machine with primary IP of 216.67.237.17
Ankheg · 2007-01-04 06:29 PM · #15
We're back up. Between this and Apache problems on another server, today's been the worst day of the year for us, so far.
Cirtex · 2007-01-05 05:04 AM · #15
Seems like our boxes at DedicatedNow are also having issues now, and the ETA we got is less than a hour to fix... :\
patchwork · 2007-01-04 09:21 AM · #16
The DN sites are back online, although I still have 1 server down :-( Pete
dinsolglobal · 2007-01-04 11:46 AM · #16
Thanks. I just submitted a ticket to them. My ticket id is Ticket ID: CRY-432867 Yes, this issue is really making me CRY
John D · 2007-01-04 03:20 PM · #16
Any way to check if a domain is on the network? Neteller.com is down - First time I have seen it down, thought they would have had good backup servers, not good for something like that to go down. EDIT: There site is up with an error message now, think it was about 40 mins ago I first saw it was down.
jxk777 · 2007-01-04 06:37 PM · #16
9 hours, 15 minutes, no server, ticket submitted over 7 hours ago. I keep hoping that any minute my server will be back like a few (very few) others. but no. Worse yet, not even a peep from them about what happened to cause such a nightmare for so many people. This tells me that whoever is making the decision on when to inform us doesn't give a rats arse about us or they wouldn't leave us hanging here with nothing to tell our clients. I'm sure the support people are probably working hard to fix this, but why doesn't the management care about us enough to keep us posted? Frankly, I find this very disturbing that they would let such a simple thing as not keeping their clients informed put such a stain on their reputation. I'm sure they must know by now that this thread exists and that there are many people very upset, but yet they won't take 2 freaking minutes to put our minds at ease. My clients deserve better, and so do we.
CDH-Hosting.com · 2007-01-05 12:29 PM · #16
My Dedicated server has been down for over 28 Hrs. now. Ready to switch!!! Any Ideas?
jbfung · 2007-01-04 09:21 AM · #17
DangerMouse, please let us know what you learn when you manage to geth through on the phone. Thanks, ~ Joseph
Jon69 · 2007-01-04 11:52 AM · #17
Arrrggggh - 3 hours now. YIA-461191 Thanks in advance.
jxk777 · 2007-01-04 03:32 PM · #17
This is a quote I got from my local server admin that they got from someone at DN "there were no network issues what so ever, and the issue that did occur, was resolved hours ago" They went on to say that they are now rebooting servers in the order of the tickets submitted. Ok, this makes me wonder, if it was not a network or power issue, what the heck would cause this and the need for them to reboot all the servers?
jxk777 · 2007-01-04 06:44 PM · #17
Host4profit, That just backs up what I said earlier, why couldn't they just switch us to a working segment? Even if it was just temporary, why not put us back online the fastest way possible and fix the issue after we were online? They could have easily moved us back one at a time once it was fixed and we wouldn't be down for over 9 hours with no answers.
jayglate · 2007-01-05 02:02 PM · #17
CDH-hosting pm your ticket # as i don't see any boxen down that long or anywhere near that. ALso a good idea is a phone call
cbwass · 2007-01-04 09:21 AM · #18
Dednow forum is working now.
24Hostingnow · 2007-01-04 11:57 AM · #18
I am not getting any autoreply from your support ticket system. I have sent email to jbrozena@fortressitx.com Awaiting our servers to come back online again!
geekeffect · 2007-01-04 03:35 PM · #18
must be nice to get some news.... if all servers need to be rebooted, do it, whats this ticket nonsense?
Marty · 2007-01-04 06:49 PM · #18
I have one server still down as well. I have exhausted all methods of contact with them. I have emailed the addresses in this thread, submitted a ticket, and am now waiting on hold. This is utterly rediculous.
jxk777 · 2007-01-05 06:57 PM · #18
I have to agree with what was said here, it is nice to finally know what happened, but leaving us all in the dark for so long was like throwing us to the wolves. Even if someone had simply posted that there was a glitch in the power and that there were other problems that have yet to be diagnosed it would have at least gave us something to tell our clients. I'll also mention that the support queue wouldn't have gotten so many repeat tickets if someone would have at least acknowledged that we existed. Think about it, if someone would have gave us any clue as to what was going on there wouldn't have been so many support tickets opened. But instead we were told to OPEN tickets. Then there is the issue that you have nothing in place to tell you if a server is down. If you had some kind of monitoring in place that told you which servers were down you could have said "we know which servers are down, please don't open any tickets" and once again, that wouldn't have been an issue. The fact is when you don't say anything you are leaving us all here getting madder and madder. Of course we are concerned when our servers are down and our clients want answers - and we get NOTHING. In my mind, your lack of keeping us even slightly updated caused a bad situation to get much worse. In other words, you brought it on yourself. You already knew there was a power issue, who was the dumbass that decided it was best not to tell anyone until hours later? If you would have just said that in the first place we would have known something to tell our clients and this thread would have never gotten to this point. Please tell the person that made the call to not inform us of anything that it was without doubt the stupidist thing possible to do. How many clients did it cost you because of this "lets not tell our clients anything" attitude? How much harm has it done to DN's reputation? Please tell that person to take some classes in customer service - friken idiot.
tmwes · 2007-01-04 09:21 AM · #19
mine is 69.72.xxx and is up. 69.98.xxx appears not to be.
jxk777 · 2007-01-04 12:12 PM · #19
My server with DN is down, has been for a few hours now. Incredibly bad timing since I have programmers that are being paid hourly to work on our main site that are sitting here drinking coffee and not able to work. Over the past month it seems to have had one issue after another, most fixed very quickly, but still it has been a roller coaster. I hope they get everything resolved soon. Between the issues and the scheduled maintenence it seems that we are down at least once a week for anywhere from 30 minutes to an hour+ and now this time it has been hours with no access and I'm losing revenue by the minute.
tcWebGuy · 2007-01-04 03:37 PM · #19
i think that some servers came back up on their own, and others didn't. So not everything needed to be rebooted? It does point to the fact that they don't actually monitor any servers if they need us to tell them when they are down....
jayglate · 2007-01-04 07:22 PM · #19
No emails have been ignored all duplicate tickets were closed, which delayed getting to the actual tickets that needed getting to. A full post will be posted in our forums later tonight. Just to update some people, forums login are available to all customers but you must login through DMS to register your forum account, that was explained to all users clearly via email once or twice and is clearly laid out in our open sales forum on how to get access to our forums. We answered and resolved tickets in the order recieved, most boxes should be up except for a few which we are working on. Yes marty including yours. If anyone should have any questions we will be having an "Executive Chat" session next week via an IRC channel or another method. So we can answer any your questions in real time we are looking to do this on tuesday night.
insiderhosting · 2007-01-05 08:46 PM · #19
Hi Matt and Jay, Generally, I don't participate in these threads, but I have to agree with Monte and the others. When you guys have a major issue, there is very little communication exchanged, so clients are left in the dark. There is no worse feeling. When we had some leftover servers there, we experienced it first hand. This also isn't the first time your mail servers have failed during a power outage. What I recommend is this, and I say this only to help you and your customers. Get an offsite server, so in the event of a disaster/catastrophe, your clients can get an update on what is happening. Update that page every say 5-10 minutes (it only takes a few seconds to post a status update), and your clients can then be informed as to where things stand. Also a small investment for a backup mail server might come in handy. Just my $.02. -Steven
cbwass · 2007-01-04 09:25 AM · #20
After a power problem it will take some time to restart all the servers that went down.
jcjc · 2007-01-04 12:18 PM · #20
Jay, 4 hours and counting.. Ticket JYQ-869813 Thanks
jxk777 · 2007-01-04 03:41 PM · #20
better yet, if it was resolved hours ago, why are we still offline? You mean to tell me they don't know what servers were affected? Also, if it wasn't a major issue, why don't we know what happened yet considering it was resolved hours ago? How long does it take to tell the truth? This just keeps getting better....
gold777 · 2007-01-04 07:27 PM · #20
jayglate... Duplicate tickets is not an excuse for no updates or replies after 8 hours. And my server is still down, no change here at all. I'll be moving servers. This is getting really old.
layer0 · 2007-01-05 09:08 PM · #20
I agree. Personally I think all providers should have offsite status pages [at the least].
adminsami · 2007-01-04 09:27 AM · #21
Still down... anyone has got any updates?
cch · 2007-01-04 12:33 PM · #21
my server is down. yes, 4 hours and counting my ticket [#WVZ-866894]
jbfung · 2007-01-04 03:44 PM · #21
If they are manually rebooting machines, it may simply be the amount of time it takes to manually boot up each machine?
jxk777 · 2007-01-04 07:41 PM · #21
10 and half hours, still dead server. Our server admin only put in one ticket and it was over 7 hours ago. I don't understand why you can't just tell us what happened, by now you have to know. Who had the bright idea of keeping us all in the dark for over ten hours? All someone had to do was tell us what happened, it is clear that you already know because some boxes are coming back online. Why are we still left with nothing to tell our angry clients after this many hours? We have been left to look like we don't know what we are doing because some genious there decided it was best to not tell us until later tonight. You have the time to post in here, why not just tell us what the heck happened? Don't we deserve an answer after over 10 hours of downtime? Or should we wait a little longer and be left to look even worse to our clients than we already do? This is just freaking mind blowing that a company would take this approach to customer service! I understand that you were probably told by someone not to disclose what happened yet, but whoever made that call is an idiot and has no idea what the term "customer service" means.
jxk777 · 2007-01-06 11:51 AM · #21
Hey, We would have been happy with even once an hour updates, anything is better than nothing. But if you notice people from DN had posted in this thread, they had to have some idea what was going on, but I'm guessing someone in management told them not to say anything until they prepared "the official statement" - that person that made that decision should be tar and feathered. Customer service is extremly important in any business. If they don't understand that then they should not be making decisions that effect the companies reputation. I really hope that person learns something from this, even though it appears they haven't learned from it in the past.
patchwork · 2007-01-04 09:28 AM · #22
I think its more likely to be a network problem or failure, the 2 servers I can access at the moment never lost power. Pete
jxk777 · 2007-01-04 12:58 PM · #22
I'd really like to know what the problem is, not sure why they can't just patch us into a network segment that works. I would have thought that they would have other segments in place for growth and adding new servers. You would think that all they would need to do is patch us into a working segment.
geekeffect · 2007-01-04 03:46 PM · #22
5 hours since ANY official news from DNOW and servers still down. 7 hours total downtime. that I know of. 98% uptime this month at the most now. 81% "usable" uptime last month.
Marty · 2007-01-04 07:43 PM · #22
All of my servers are back. Thank you, Jay.
tcWebGuy · 2007-01-09 07:06 PM · #22
here we go again??
DangerMouse · 2007-01-04 09:28 AM · #23
nope still on hold here...
jcjc · 2007-01-04 01:04 PM · #23
You would expect that wouldn't you... seems not This is becoming very ugly for us.
Jon69 · 2007-01-04 03:49 PM · #23
Praise the lord - 7 hours later, my box is back. 65.98.xx.xx for those interested. Hope the rest of you guys get your boxes up soon.
gold777 · 2007-01-04 08:00 PM · #23
I agree, my host has been left in the dark completely as well. I ask them for an ETA or updates as to what's happening and they can't tell me anything - to no fault of their own, they aren't getting any replies or responses from DeadNow.
ForumsAddict · 2007-01-09 07:08 PM · #23
Its back strangely after a small hiccup
the_fulgore · 2007-01-04 09:30 AM · #24
Pretty funny I was just thinking the other day. "Gee it's been a while since DN had any screw ups lately...must be due for one major catastrophic failure soon!" I hope this isn't another 16 hour downtime for my business.
patchwork · 2007-01-04 01:18 PM · #24
Down for nearly four and a half hours now. They should not have any single points of failure, its called redundancy, but I agree how long does it take to unplug a network cable and plug it in to something that is working, its not rocket science. Pete
geekeffect · 2007-01-04 03:59 PM · #24
congrats! means somethings happening at least.
jxk777 · 2007-01-04 08:00 PM · #24
Ok, Mine just came back too. Now, if we could only get an answer to what happened so we can tell our clients. All I can say is it better be one heck of an answer after all of this, my clients are so pissed off that anything short of an act of God is not going to be good enough for them. If we have been kept in the dark all this time over something as simple as a power failure or a network outage they are not going to buy it.
patchwork · 2007-01-09 07:08 PM · #24
Yep I was down too for a little while, seems ok now though. Pete
DangerMouse · 2007-01-04 09:32 AM · #25
so it was you that jinxed it...?
dinsolglobal · 2007-01-04 01:35 PM · #25
Server are up but did not heard from them regarding the outage. Do anyone know the reason for this outage? Thanks.
The3bl · 2007-01-04 04:09 PM · #25
DN now has the worst customer service I have ever seen from any company, close second is servermatrix. But at least there they answer the phone. These guys are a joke, ghetto network and data center, lousy customer service and bad attitude. Wonders how they stay in business.
gold777 · 2007-01-04 08:03 PM · #25
Congrats! Mine still down, no change.
ChrisEatsAlot · 2007-01-09 07:10 PM · #25
7 minutes for me