| Author |
Message |
|
Guilhem Achikbach #1 / 14
|
 Server crash
Hello! Our linux box crashed this week-end (Sat. around 4PM), but i haven't been able to find why. I checked /var/log/* (including various programs log such as mysql, apache, and so), but none of the files I saw contained an error at that time. Am I missing something ? Where should I look for more ? The guy who rebooted the server this morning said it just froze (after a month of correct operations). Is this common under 2.2.16-22SMP ? Should I use another Kernel ? I've been using my linux box for over 200 days without a reboot (with a 5.2 redhat), so I can't just say "Well it's been a long time, it had to hang/reboot, la M$oft"... Our config: 2xP3 Intel, RedHat 6.2, kernel 2.2.16-22SMP, SCSI/RAID5, the box is in a rackable case. Thanks for your help, Guilhem Achikbache
|
| Fri, 02 Jan 2004 16:05:56 GMT |
|
 |
|
dbian.. #2 / 14
|
 Server crash
Well, not knowing anything about what it was doing it's quite difficult to say anything. I would stick with the current kernel if you don't have any other problem or real necessity to update. Davide
|
| Fri, 02 Jan 2004 16:19:03 GMT |
|
 |
|
Guilhem Achikbach #3 / 14
|
 Server crash
Actually, it is a web server with apache/mod_ssl, doing mostly CGIs (perl+mysql), but it's also our mailserver. Other running processes: proftpd, mysql, postgresql, RealServer, plus the basic ones (sendmail, xinetd, etc.) load average: 0.05, 0.06, 0.01 (not heavily used ;-) Thanks -- Guilhem Achikbache ---------------------------------------------------------------- Downgrade your system for only 89 dollars! Install Windows!
|
| Fri, 02 Jan 2004 16:52:36 GMT |
|
 |
|
dbian.. #4 / 14
|
 Server crash
A lot of opened doors... I'd investigate the possibility that someone cracked it (or tried to crack). Ok, it's not a pleasant possibility, but I will not discard it first-hand. Davide
|
| Fri, 02 Jan 2004 17:26:19 GMT |
|
 |
|
Guilhem Achikbach #5 / 14
|
 Server crash
First, thank you, i really appreciate the time you are taking for me! What do you mean '' ? are you speaking in terms of crash (A lot of opened possibilities) or attacks (A lot of opened backdoors) ? in the latter case, which programs could have 'opened doors' ? PS: Actually, nobody got in connected since Friday (and that was me, looking at the logging time) Thanks again! -- Guilhem Achikbache @nsweb - http://www.answeb.net ---------------------------------------------------------------- On the other hand, you also have 5 fingers.
|
| Fri, 02 Jan 2004 21:38:19 GMT |
|
 |
|
dbian.. #6 / 14
|
 Server crash
Any application that "listen" on a port is an opened door, you have: ftp, sendmail, http (apache), mysql maybe telnet and maybe something else. All these have well-known vulnerability (check if they are the last versions or older versions, in this case, update them). Someone could have tried to use one or more of these doors to attack your system, resulting in a crash. I tend to discard the possibility of a crash due to any of these services, unless you have a very low memory condition or a very full hard disk condition, in this case one of the service could have triggered a situation unhandled by the kernel resulting in a crash, but it's quite difficult to tell (leave alone solve).
If you are sure of this (the server is not connected to the Internet) you should be safe. Davide
|
| Fri, 02 Jan 2004 22:00:48 GMT |
|
 |
|
Guilhem Achikbach #7 / 14
|
 Server crash
I understand what you mean. I will check for these updates if necessary. Actually, mysql is only localhost-based, and no telnet is available (using ssh).
The server has 512Megs of RAM, so i guess it wasn't the pb, the hard disks neither (around 20% full). here is what free gives me: total used free shared buffers cached Mem: 517004 509928 7076 104840 53648 397812 -/+ buffers/cache: 58468 458536 Swap: 530064 3160 526904 I guess I have 7076+458536 Megs of free RAM (without swap)... do I ? or is it 397812Mb ? ;-)
It is connected, but the eventual cracker would have had erased his log entries. Would that be possible (which means getting root, which means decrypting SSHv3, cause I always su, never log in root) ?
Thanks again ! -- Guilhem Achikbache ---------------------------------------------------------------- I love BBSing: All the social dynamics of kindergarten!
|
| Sat, 03 Jan 2004 01:15:18 GMT |
|
 |
|
dbian.. #8 / 14
|
 Server crash
It is possible, and the first thing for a "good" cracker (Ok, it's a contradiction in terms but you got the picture) is disable logging and/or remove all the traces of his presence/activity. Never overestimate your security, being paranoid is better than being overconfident. I suggest you to read "the cookoos eeg", actually is quite old, but the attack strategies described are still good... and it's also a good reading for a rainy afternoon. Davide
|
| Sat, 03 Jan 2004 15:03:09 GMT |
|
 |
|
Rasmus B?g Hanse #9 / 14
|
 Server crash
Actually RedHat has released a 2.2.19 update. I haven't checked out what fixes it contains as I compile my own kernels by hand... Rasmus -- -- [ Rasmus 'M?ffe' B?g Hansen ] --------------------------------------- I don't suffer from insanity, i enjoy every minute of it! --------------------------------- [ moffe at amagerkollegiet dot dk ] --
|
| Sat, 03 Jan 2004 16:47:35 GMT |
|
 |
|
Guilhem Achikbach #10 / 14
|
 Server crash
First, thanks for the news. Then I jump on your remark 'as I compile my own kernels by hand' : what are the benefits of (re)compiling a kernel by myself (other than size) ? performance ? stability ? security ? Will our twin-processors' performances be significantly increased ? Should I do it ? Thanks for your answers -- Guilhem Achikbache @nsweb - http://www.answeb.net ---------------------------------------------------------------- Bad Command:(A)bort (R)etry (T)ake RAM hostage
|
| Sat, 03 Jan 2004 20:37:53 GMT |
|
 |
|
Rasmus B?g Hanse #11 / 14
|
 Server crash
I like to have it in control - knowing what drivers I need as modules and which I don't. Also I run reiserfs, which is not supported in the redhat kernels.
Performance... Well, you can optimize exactly for you CPU type - I do not - however - know how much is gained here. Stability, probably no - and the same with security.
Not when you are already running an SMP kernel. Only do it, if either 1) you want to, 2) you have really tight memory, 3) You want to run a newer kernel than the ones provided by RedHat. And if you have never done it before - don't(tm) do it on a production machine. Most people (including myself) fail to do it 100% correct the first time. Rasmus -- -- [ Rasmus 'M?ffe' B?g Hansen ] --------------------------------------- Programming is a race between programmers, who try and make more and more idiot-proof software, and universe, which produces more and more remarkable idiots. Until now, universe leads the race. - R. Cooka --------------------------------- [ moffe at amagerkollegiet dot dk ] --
|
| Sat, 03 Jan 2004 21:13:06 GMT |
|
 |
|
Guilhem Achikbach #12 / 14
|
 Server crash
Actually, I used to recompile my home box's kernel, but on this prod server, I won't ;-). You just confirmed the fact that when it's working, don't fix it ;-). I might however try the new 2.2.19 you were talking about (maybe later). Again, thank you for your time and help -- Guilhem Achikbache ---------------------------------------------------------------- The hangman let us down.
|
| Sat, 03 Jan 2004 22:29:48 GMT |
|
 |
|
Guilhem Achikbach #13 / 14
|
 Server crash
I will follow your advices. I hope it won't crash again... Many thanxx -- Guilhem Achikbache @nsweb - http://www.answeb.net ---------------------------------------------------------------- The hangman let us down.
|
| Sat, 03 Jan 2004 22:31:01 GMT |
|
 |
|
Robert Davie #14 / 14
|
 Server crash
My view is that is wise, my experience was that a custom kernel, on a production server, didn't offer significant performance advantages, and it cost considerable time responding to security advisories. It's probably not really worth the bother, unless you have special security requirements, hardware requirements or want to work with patched kernel. The distro's _do_ test their kernels, and too often I see advice to download latest Linus source, ignoring the known bugs, with patches available, and/or hardware support lacking in standard kernel, which has been added. My view at present is it's a feel good factor, and also a lack of understanding of how to work with modules which leads to the 'compile your own advice'. I would love to seem some hard figures on the 'inefficiency' of modules, my tests couldn't see any significant difference in practice, possibly because the kernel, tends not to use much CPU time anyway. Rob
|
| Sun, 04 Jan 2004 19:01:40 GMT |
|
 |
| |