URL: http://www.flightadventures.com/cgi-bin/dcforum/dcboard.cgi
Forum: DCForumID4
Thread Number: 223
[ Go back to previous page ]

Original Message
"VPC unreachable 12-09-02"

Posted by Ben_Chiu on 12-10-02 at 06:57z
Due to problems at the company that controls our Name Server, we were down for most of today. The short synopsis is about 3am Pacific, a Denial Of Service attack/ HUGE influx if DNS queries bombarded their main facility in Seattle. In the following hours everything on that network became extremely slow making most of the services provided on that network (DNS,URL forwarding, Email Forwarding, websites, DB) appear unavailable or really slow to anyone outside the network. This kind of activity has happened before but never of this magnitude.

The unfortunate side effect of this activity is that it overloaded both the primary and secondary firewalls causing them to reset connections about every 2 minutes. Their senior network engineer was woken up and after having no luck with a remote fix headed to our co-lo facility. He arrived to find the firewalls rebooting under a large deluge of traffic. He couldn't even get information off of the firewalls about what was actually happening.

In the meantime, the downtime at our co-lo in Seattle caused all DNS to be directed to our east coast facility. The facility also was brought down by the volume in traffic. As they tried to diagnose what the problem was so that they could know what to cut off, the traffic just kept coming. After some conflict with the co-lo provider, finally at 5pm PST, they filtered out all traffic destined for the nameserver in the Seattle co-lo. This immediately enabled all services on that network to the outside world.

Isn't this stuff fun?

Ben


-= VPC OffLine Reader 2.1 =-
Registered to: Ben Chiu
-OLR.PL v1.80-


Table of contents

Messages in this discussion
"RE: VPC unreachable 12-09-02"
Posted by James on 12-10-02 at 16:35z
Hi Ben et al,

>> Isn't this stuff fun? >>

Isn't it just <LOL>

Regards,

James (CONman) Anderson
*** The plane in front is a Boeing ***

-= VPC OffLine Reader 2.1 =-
Registered to: James Anderson
-OLR.PL v1.80-


"RE: VPC unreachable 12-09-02"
Posted by vgbaron on 12-10-02 at 16:54z
Ben -

Were they able to trace back the DOS attack? Altho any really good hacker capable of a DOS attack has probably taken over other peoples systems and using them to carry it out. Any reason why *they* were targeted?

Fun stuff.

Vic

Of course, I could be wrong. I thought I was wrong once before, but I was mistaken.


-= VPC OffLine Reader 2.1 =-
Registered to: Vic Baron
-OLR.PL v1.80-


"RE: VPC unreachable 12-09-02"
Posted by Ben_Chiu on 12-10-02 at 23:22z
Greeting Vic:

> Were they able to trace back the DOS attack?

It seems to have originated from an attack to one of the 600,000+ sites on the name server. This caused the chain reaction and shut everyone down. However, they're still looking into it and no names have been named yet.


> Altho any really good
> hacker capable of a DOS attack has probably taken over other peoples
> systems and using them to carry it out. Any reason why *they* were
> targeted?

Don't know, but as I explained above, our provider was just a casualty of the main/originating assault. Regardless, measures are in place so it won't happen again. <fingers crossed>

We're also discussing internally fall back methods to keep us up if the NS shuts down again.

Ben


-= VPC OffLine Reader 2.1 =-
Registered to: Ben Chiu
-OLR.PL v1.80-