5 ms·

She found out about it pretty quickly due to having syslog be a constant presence in one of her gnu screen windows I'm amazed that this is possible. How would

She found out about it pretty quickly due to having syslog be a constant presence in one of her gnu screen windows I'm amazed that this is possible. How would I set something like that up? A realtime log of only the most significant events of a remote system? In fact, I'd like to take this opportunity of ignorance-admitting to ask the community for general linux/bsd sysadmin resources. What books should I read, or what topics should I study? I want to become an expert at modern sysops. Modern deployment, hardening, backup, managing dozens of boxen, etc. I've been thinking of going through any MIT OCW on the subject, but it seems like hard-earned experience might not necessarily translate well to an academic setting. What would you recommend I do?

ixmatus 13y ago
Papertrail is great for this...you can of course setup syslog to route to a central server and just be logged in tmux / screen on that particular machine to read off the stream of logs (I prefer papertrail though + saved searches and hipchat "pings" when saved searches are matched on incoming events). General devops / sysops/ sysadmin knowledge can be had through a variety of means - I got most of my knowledge from simply reading the FreeBSD manual and making a lot of mistakes with my own servers.
sentenza 13y ago
What position are you starting from? My old workplace was a university group where we (admins) were recruted from the available pool of PhD students. So I'm used to guiding people from "no knowledge" to "enough knowledge to be dangerous". The first step was to force the prospective admins to run a specific system on their "productive machine" and keep it in such conditions that _everything_ works. This way, a complete admin newbie would learn about digging through the systems by working out the kinks of practical everyday problems. Remember, this is only the most basic instruction, nowhere near enterprise-grade. If there was a "prospective admin" who had never before run Linux, I'd tell them to install and use Ubuntu/Mint. (Those guys whould usually only be trained to be a helping hand for a "senior" admin.) If he'd already used Ubuntu at home, I'd tell them to start using Debian and work out how to set up an SSH server and set up their home machine so they could access it remotely. If they had dabbled with Debian, Fedora, SuSe or something similar, I'd tell them to install Arch and set up some "interesting things", like a mail server or a nis server. If they were using Arch or Gentoo at home, I'd just personally show them the important things about our system and have them wingman with me for a few days. If you are already an advanced Linux or BSD user, my approach is of course not appropriate. Instead I'd recommend to pick skills that you want to learn (iptables? Exim?) and set that up. Read Manuals! Read RFCs! Best of luck.
- sillysaurus2 13y ago
  What position are you starting from? Accurately assessing one's own competence is difficult and makes for boring reading, but since it's probably necessary here, I'll give some background. If he'd already used Ubuntu at home, I'd tell them to start using Debian and work out how to set up an SSH server and set up their home machine so they could access it remotely. If they had dabbled with Debian, Fedora, SuSe or something similar, I'd tell them to install Arch and set up some "interesting things", like a mail server or a nis server. If they were using Arch or Gentoo at home, I'd just personally show them the important things about our system and have them wingman with me for a few days. I'd say my current skill level is a mixture of those three. For example, I don't know how to deploy a web service which can send out email for users to e.g. reset passwords. So I don't know anything about email. On the other hand, I've been trying to hone my skills by hardening a Debian server using iptables. On my third hand, while I could set up a box at home that can be SSH'd remotely, I'm not yet confident I know all the best practices. I think the best SSH practices are: change the default SSH port, disable root login, and disable password-based login (use a password-protected keyfile instead). Beyond that, what is interesting to me is being able to set up dozens or hundreds of systems. Doing this by hand is fraught with error, so it seems like I should learn about virtualization + deployment systems. I've heard good things about Ansible and Salt, but I've also heard Salt considered security an afterthought, which didn't sound good. It's sounding like my best bet is just to try things, but I want to set things up correctly from a security perspective. I should also enhance my knowledge of networking... perhaps by spending a few weeks on OCW material regarding the networking stack. How packets are routed, the details of TCP, that sort of thing. Thanks so much for your insight!
  - sentenza 13y ago
    You're welcome. If you want to deploy and maintain many machines, then maybe FAI[1] might be worth a look. It allows you to maintain a consistent state over an arbitrary number of machines running a Debian-based distribution, with _and without_ virtualization. We used it to run about 40 user-facing desktop machines and about the same number of cluster nodes. You basically have a central server that contains configuration, configuration-modifying scripts and package configurations. It is possible to define classes of machines, and one machine can belong to multiple classes, so you can have a part of the configuration identical on all machines and then other parts only on some of them. [1] http://fai-project.org http://fai-project.org
kordless 13y ago
Splunk: http://splunk.com/ http://splunk.com/. Free download, free to use up to X many GB.
- michaelmior 13y ago
  Also Splunk Storm https://www.splunkstorm.com/ https://www.splunkstorm.com/
dredmorbius 13y ago
Remote syslogging, generally. There are any number of log tailers. One I've recently discovered and love is multitail by Folkert vanHeusden: http://www.vanheusden.com/multitail/ http://www.vanheusden.com/multitail/ Incidentally, a related utility (and one which works really well with multitail) is rsstail: http://www.vanheusden.com/rsstail/ http://www.vanheusden.com/rsstail/ One option is to get an RSS/Atom feed of critical system events.
azth 13y ago
> I've been thinking of going through any MIT OCW on the subject Are there any MIT OCW courses on this? I haven't come across any.
atsaloli 13y ago
Check out www.sabok.org Sys Admin Body of Knowledge

ixmatus 13y ago

Papertrail is great for this...you can of course setup syslog to route to a central server and just be logged in tmux / screen on that particular machine to read off the stream of logs (I prefer papertrail though + saved searches and hipchat "pings" when saved searches are matched on incoming events). General devops / sysops/ sysadmin knowledge can be had through a variety of means - I got most of my knowledge from simply reading the FreeBSD manual and making a lot of mistakes with my own servers.

sentenza 13y ago

What position are you starting from? My old workplace was a university group where we (admins) were recruted from the available pool of PhD students. So I'm used to guiding people from "no knowledge" to "enough knowledge to be dangerous". The first step was to force the prospective admins to run a specific system on their "productive machine" and keep it in such conditions that _everything_ works. This way, a complete admin newbie would learn about digging through the systems by working out the kinks of practical everyday problems. Remember, this is only the most basic instruction, nowhere near enterprise-grade. If there was a "prospective admin" who had never before run Linux, I'd tell them to install and use Ubuntu/Mint. (Those guys whould usually only be trained to be a helping hand for a "senior" admin.) If he'd already used Ubuntu at home, I'd tell them to start using Debian and work out how to set up an SSH server and set up their home machine so they could access it remotely. If they had dabbled with Debian, Fedora, SuSe or something similar, I'd tell them to install Arch and set up some "interesting things", like a mail server or a nis server. If they were using Arch or Gentoo at home, I'd just personally show them the important things about our system and have them wingman with me for a few days. If you are already an advanced Linux or BSD user, my approach is of course not appropriate. Instead I'd recommend to pick skills that you want to learn (iptables? Exim?) and set that up. Read Manuals! Read RFCs! Best of luck.

sillysaurus2 13y ago
What position are you starting from? Accurately assessing one's own competence is difficult and makes for boring reading, but since it's probably necessary here, I'll give some background. If he'd already used Ubuntu at home, I'd tell them to start using Debian and work out how to set up an SSH server and set up their home machine so they could access it remotely. If they had dabbled with Debian, Fedora, SuSe or something similar, I'd tell them to install Arch and set up some "interesting things", like a mail server or a nis server. If they were using Arch or Gentoo at home, I'd just personally show them the important things about our system and have them wingman with me for a few days. I'd say my current skill level is a mixture of those three. For example, I don't know how to deploy a web service which can send out email for users to e.g. reset passwords. So I don't know anything about email. On the other hand, I've been trying to hone my skills by hardening a Debian server using iptables. On my third hand, while I could set up a box at home that can be SSH'd remotely, I'm not yet confident I know all the best practices. I think the best SSH practices are: change the default SSH port, disable root login, and disable password-based login (use a password-protected keyfile instead). Beyond that, what is interesting to me is being able to set up dozens or hundreds of systems. Doing this by hand is fraught with error, so it seems like I should learn about virtualization + deployment systems. I've heard good things about Ansible and Salt, but I've also heard Salt considered security an afterthought, which didn't sound good. It's sounding like my best bet is just to try things, but I want to set things up correctly from a security perspective. I should also enhance my knowledge of networking... perhaps by spending a few weeks on OCW material regarding the networking stack. How packets are routed, the details of TCP, that sort of thing. Thanks so much for your insight!
- sentenza 13y ago
  You're welcome. If you want to deploy and maintain many machines, then maybe FAI[1] might be worth a look. It allows you to maintain a consistent state over an arbitrary number of machines running a Debian-based distribution, with _and without_ virtualization. We used it to run about 40 user-facing desktop machines and about the same number of cluster nodes. You basically have a central server that contains configuration, configuration-modifying scripts and package configurations. It is possible to define classes of machines, and one machine can belong to multiple classes, so you can have a part of the configuration identical on all machines and then other parts only on some of them. [1] http://fai-project.org http://fai-project.org

kordless 13y ago

Splunk: http://splunk.com/ http://splunk.com/. Free download, free to use up to X many GB.

michaelmior 13y ago
Also Splunk Storm https://www.splunkstorm.com/ https://www.splunkstorm.com/

dredmorbius 13y ago

Remote syslogging, generally. There are any number of log tailers. One I've recently discovered and love is multitail by Folkert vanHeusden: http://www.vanheusden.com/multitail/ http://www.vanheusden.com/multitail/ Incidentally, a related utility (and one which works really well with multitail) is rsstail: http://www.vanheusden.com/rsstail/ http://www.vanheusden.com/rsstail/ One option is to get an RSS/Atom feed of critical system events.

azth 13y ago

> I've been thinking of going through any MIT OCW on the subject Are there any MIT OCW courses on this? I haven't come across any.

atsaloli 13y ago

Check out www.sabok.org Sys Admin Body of Knowledge