[BMA] What happened to my server?

Issues, news, and discussion specific to any other servers.
Post Reply
Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

[BMA] What happened to my server?

Post by Berendol » Sun 19.12.2004, 15:05

Something's very wrong with the server. The /home directories and a bunch of critical system libraries are missing, and I can't get to the logfiles to check what's up.<br><br>I don't know what to say... my server is offline until I can fix it.
By appreciation, we make excellence in others our own property. (Voltaire)

Big_Juan_Teh_Furby
Iridescent Beetle
Posts: 244
Joined: Sun 27.10.2002, 21:16
Location: Eugene, OR
Contact:

Re: What happened to my server?

Post by Big_Juan_Teh_Furby » Sun 19.12.2004, 16:47

My guess is that you got hax0red by someone who doesn't like you very much.

3 guesses?  ;)
When the winds of change blow hard enough, the most trivial of things can turn into deadly projectiles.

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Sun 19.12.2004, 17:32

I do have three guesses... but any info you have would be appreciated. I can't believe anyone would really want to do this to a game server.

I don't know if I should continue to run the server. Maybe I should cease my efforts to make MAngband live on and prosper.

Why would anyone want to hack it? It's such an incapable machine as it is. Mounting an attack from that box would be worthless, because although it has fat pipes, it has low computational horsepower.

Things I am missing on the server:
/home
/sbin (can't reboot remotely now)
/var/www (my web server.. the front page only comes up because it's cached in RAM by the server)
All filesystem checking and mounting programs
Any functionality in the commands cat, tee, tar, ftp, ls, dir.
Nano, vi, less, more, wget, emerge, ps, passwd, ... ... ...

Write access is gone. The whole freaking thing is mounted read-only and mount, umount, and remount are gone; and I can't check the tables.

Any info on how to recover this remotely would be sincerely appreciated. Geeze... I'll reward you handsomely. I'll give you actual money (some US Dollars) if it works and I can recover the server. However I must stipulate that you must contact me by email at my publicly-noted address instead of here on the forums, since my hacker may be reading these.
By appreciation, we make excellence in others our own property. (Voltaire)

Big_Juan_Teh_Furby
Iridescent Beetle
Posts: 244
Joined: Sun 27.10.2002, 21:16
Location: Eugene, OR
Contact:

Re: What happened to my server?

Post by Big_Juan_Teh_Furby » Sun 19.12.2004, 18:22

Given what's missing and has been changed, it really DOES sound like a hack.  Sounds dumb, but did you have the server behind any sort of firewall?

I hope you bring the server back.  It was fun to play on...it was a nice change from Crimson's server.  Not to say that his server isn't fun, but it was great fun to play on your server.
When the winds of change blow hard enough, the most trivial of things can turn into deadly projectiles.

Narchan
Giant Spotted Rat
Posts: 51
Joined: Sat 02.11.2002, 11:33

Re: What happened to my server?

Post by Narchan » Mon 20.12.2004, 20:37

Strange.  The ToME/oook server got hacked recently.  Don't know if this is related to what happened to your server.  If anything is does show that there are hostile people out there....
#############
ZZZZZZZZ@ZZZZZ
ZZZZZZZZZZZZZZZ
#############

uh oh

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Mon 03.01.2005, 15:18

Well I am settled in back at my apartment in NY and the server didn't burst into flames or anything, so I will be attempting to mount the drive in my other computer shortly.

Expect an update on the problem later today/tonight...
By appreciation, we make excellence in others our own property. (Voltaire)

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Tue 04.01.2005, 02:13

As promised, here is your update!

1) I noticed a LOT of lag and server reboots the day-of, as was astutely noted by so many people.

2) The last error message was an error in my boa log: [tt][18/Dec/2004:03:23:04 +0000] malformed request: "CONNECT 1.3.3.7:1337 HTTP/1.0"[/tt]

3) There were a myriad of failed root logins from several IP addresses around the globe. The root password on the box was not "easily crackable" by any standard... letters, numbers, 8 characters, no relevance to me/the game, and no dictionary words in any language. I'd be willing to say there's no pronunciation for it in any language.

4) Received this several times, before they switched to a numeric IP address: [tt]Dec 18 02:52:09 [sshd] reverse mapping checking getaddrinfo for masarykova.mlynska.cz failed - POSSIBLE BREAKIN ATTEMPT![/tt]

5) The hard drive is working just fine, and passed a full filesystem and bad sector scan with fsck. (Forced of course.)

6) All the MAngband stuff is still there in the right location. In fact, everything is as I left it - which has me utterly confused.

Maybe it was a successful breakin and rootkit. I was a few weeks out of date on my software patches, after all.

Before I put DServ back online, I will be comparing the source code with the most recent good copy just to make sure there are no special additions. I will also reformat the hard drive, and install a fresh copy of the latest stable Gentoo everything on it. This time I may go the selinux route. I will also change all of the passwords (including RNG and the console) to 10+ characters.

All of the savefiles will be left intact!

So... expect to see DServ online by Saturday at the latest.
By appreciation, we make excellence in others our own property. (Voltaire)

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Tue 04.01.2005, 02:29

Update #2.

The latest chkrootkit and rkhunter did not detect any evidence of tampering.

What??
By appreciation, we make excellence in others our own property. (Voltaire)

Big_Juan_Teh_Furby
Iridescent Beetle
Posts: 244
Joined: Sun 27.10.2002, 21:16
Location: Eugene, OR
Contact:

Re: What happened to my server?

Post by Big_Juan_Teh_Furby » Tue 04.01.2005, 15:39

Weird, but I'm glad it's intact!
When the winds of change blow hard enough, the most trivial of things can turn into deadly projectiles.

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Wed 05.01.2005, 00:16

I'm working on installing the new OS. If all goes well, I may be able to finish bringing it online early.

This time, I'm using a bleeding-edge (stable) hardened Gentoo system, using the complete hardened toolchain to compile even the base system software. I'm also installing fewer packages, although there weren't that many on the original. I may install tripwire as well, and there will be scheduled rootkit checks.

This is going to be at least an overnight compile on my well-optimized 3 GHz Pentium 4. I just yanked the hard drive out of the 200 MHz box and plugged it in so it wouldn't take two weeks.

Trying to get into this system will be a colossal waste of time... passwords are cryptologically sound, also >10 characters long, and even buffer overflow attacks will be prevented by the new software.

Check out the features... grsecurity, PaX, PIE, SSP ... your server may benefit from this software as well.
By appreciation, we make excellence in others our own property. (Voltaire)

Big_Juan_Teh_Furby
Iridescent Beetle
Posts: 244
Joined: Sun 27.10.2002, 21:16
Location: Eugene, OR
Contact:

Re: What happened to my server?

Post by Big_Juan_Teh_Furby » Wed 05.01.2005, 01:17

How 'bout hiding the box behind a firewall, with only the appropriate ports open for MAngband?  I worked in online security enough to know it's retarded to NOT have something that mattered behind a firewall.  Trust me.
When the winds of change blow hard enough, the most trivial of things can turn into deadly projectiles.

Berendol
Evil Iggy
Posts: 868
Joined: Mon 11.11.2002, 19:13
Location: Loot Pile
Contact:

Re: What happened to my server?

Post by Berendol » Thu 06.01.2005, 09:43

Got any recommendations for firewall software that runs on kernel 2.6.7, preferably with a Gentoo ebuild available? I don't own a special hardware firewall box.

By the way, the only software with any open ports other than MAngband will be SSH and a different web server, which will operate on a non-standard port. I've taken care to not install anything like a mail server, ftp, telnet, etc.
By appreciation, we make excellence in others our own property. (Voltaire)

Post Reply