teach me to w00t for r00t.

July 10, 2007 10:57 AM

*nixfilter: Help me get to the "root" of understanding the vastness that is a unix/linux environment.

I've searched out a few other posts on here, and followed a few of the links, but I think my quest differs slightly.

I have casually used linux on and off over the years, starting with Gentoo 1.4 through 2005.0, and more recently using Ubuntu/Kubuntu's latest offerings, but for the most part my exposure was limited to just using X and staring at the neat stuff you can do with Beryl.

I have been talking with members of a higher end engineering group in my company about possibly transferring there, and the general opinion is that I need to beef up my knowledge of what's going on under the hood in a linux/unix/bsd environment.

So basically what I'm looking for is recommendations on books, wiki's, recommended distro's, etc that will teach me the nuts and bolts of using the command line, building kernels, etc. I want to stay away from the easy stuff like portage, aptitude; basically anything that tries to automate things that I should know.

I have a spare P4 3.2GHz PC that I can put something onto, and I have a Powerbook G4 running OSX Tiger if that matters.

Discuss!

posted by Industrial PhD to Computers & Internet (22 answers total)

Stick with gentoo. Don't install X. Upgrade your kernel. Don't use genkernel. Read man pages religiously. Print them out for bathroom reading.
Break it. Figure out how to fix it. Wash, rinse, repeat.

The Linux Documentation Project is your friend.
posted by Cat Pie Hurts at 11:07 AM on July 10, 2007

Why not just pick any common distro and do not install/load a GUI. Have it boot straight into the cli (init 3). Start using unix like you would on a shared system. Make this your primary computer for a week or three. Try to do everything you normally do. If it comes down to it, load up x, and use whatever web or office apps you need, but only if there is no cli alternative.

Give yourself some tasks. Write a small dynamic website. Only use a web browser to test your work. Write cron jobs to do something helpful like empty out a temp directory or send you a reminder email. FInd random projects on sf or freshmeat, try to compile them and get them running. Run a dns server. Learn to use ipchains to firewall everything off. More fun things to do circa 1997.
posted by damn dirty ape at 11:09 AM on July 10, 2007

Ubuntu is, by far, the easiest distro I've tried, especially for new linux users. The real strength of Ubuntu is that the community is great and supportive.

You'll find info on damn near any problem you'll have on the Ubuntu Guide and the Ubuntu Forums
posted by chrisamiller at 11:11 AM on July 10, 2007

My two indispensable books, both from O'Reilly: Unix in a Nutshell and Unix Power Tools. For free stuff, you can't go wrong with the How-Tos from the Linux Documentation Project.

Any of the Linux distros can give you command-line exposure, and nearly all of them have some sort of package-management system available (apt, rpm, etc). Even FreeBSD has the ports system which (can) automate package compilation and installation.

You might try a couple of simple system projects like building/installing apache, correctly configuring the local MTA (postfix, sendmail, etc) or setting up a local caching nameserver. Any of those should have you mucking around at the prompt for a little while at least.

If you're building stuff from source, you should know how to install the basic build-tools for your platform (GCC, libraries, autoconf, automake, and the rest).
posted by jquinby at 11:13 AM on July 10, 2007

Another thing:
Install webmin.
Make system modifications using webmin, then explore your /etc dir and figure out how the changes made using the gui reflect on the actual config files.
posted by Cat Pie Hurts at 11:15 AM on July 10, 2007

Get yourself just about any of the Linux books in the Bible series. They're inexpensive and pretty comprehensive. Any by Christopher Negus are good, or Naba Barkakati. Essentially they rewrite the same book for each distro, so it becomes pretty polished.

Forget about Gentoo. Saying you run Gentoo will have people laugh at you in certain Linux circles.

I'd probably look at Ubuntu, because Ubuntu is popular on the desktop, and it's likely an academic institution will run Debian on its servers (Ubuntu is based on Debian). Red Hat is used in business. SUSE might not be around for much longer, so I wouldn't risk putting your eggs in that basket.

Search for the Linux System Administrator's Guide on Google. That's how I started. It's pretty down to earth and distro agnostic.

Try and setup a Samba server on the Linux machine you create. That will teach you how to play around with network settings, and it's a great way to get inside how Linux functions (aka the Unix logic). You can connect to it using your Powerbook, which will prove when/if you get it setup right.

There's no benefit to be had in installing Linux on your Powerbook. It's almost identical to Linux on an X86 computer, although you might have the intriguing possibility of some devices not working, in which case you'll have to hack kernel modules. This in itself can be educational.
posted by humblepigeon at 11:23 AM on July 10, 2007

Give yourself some tasks.

This is the key. You can install different flavors til you're blue in the face and not learn a thing.

You need to give yourself a few things to do. Turn your extra computer into a webserver, an email server, or something, and give that service away to your friends for free. Beg them to use it, because they will find problems or make requests and ask you to fix it. If not web or email, perhaps chat, or a fileserver for your friends to leave stuff on, or a gameserver.
posted by poppo at 11:29 AM on July 10, 2007

The dominant linux distributions in the enterprise server space are Redhat and SUSE. For Redhat you can run Fedora or Centos, I am unsure what the non commerical SUSE equivalent is.

For most day to day stuff the difference between the flavors out there is the tools, nearly every distribution uses apt or rpm now.

The standard for linux application offers generally is built around LAMP. Read up on LAMP (Linux Apache, Mysql, PHP) and it will give you a decent foundation in one of the more commonly deployed top down configurations.
posted by iamabot at 11:31 AM on July 10, 2007

Installing "Linux From Scratch" will help understand how a system is put together. There's also a spectacular series of articles that walks through the NetBSD source tree, to really understand the components that go into an OS.

Slackware is traditionally a minimalist linux distro. Debian used to be, except for apt, but they've been moving away from that. If you grab a Debian woody distro (not sarge, sarge added a half-broken installer, and not etch, because etch is completely different), that's not a bad way to go (warning: it will be old and out of date).

Installing all of the BSDs is also excellent experience. You'll run into more problems than you could ever imagine, especially if you have slightly unusual hardware, and a lot of those things you take from granted on linux will just be gone (net/openbsd's tar didn't have bzip2 support, so while you could do 'tar xvzf' for .tar.gz, you couldn't do 'tar xvjf' for .tar.bz2 - little things like that add up). Also, learning how different systems configure themselves givse you a much better understanding of how your own does it (and with the BSDs, you actually have a sane init system, and a kernel that integrates well into the userland distribution).

Otherwise, the advice above is dead on - for each system, do somethign useful. Break it and then fix it - resist the temptation to wipe clean, and hack away at a single installation until it not only works but is clean.
posted by devilsbrigade at 11:58 AM on July 10, 2007

Oh, and if your company works with Java, it's a MUST that you learn how to install multiple Java versions, set up J2EE, Tomcat/Apache if you're using that, JBoss if you're using that, etc. Those things are very tricky (especially when you have multiple versions) and don't necessarily come easily.
posted by devilsbrigade at 12:00 PM on July 10, 2007

Here is the NetBSD article. Fast but good reading, and a good place to start.
posted by devilsbrigade at 12:02 PM on July 10, 2007

I agree with some others that the real way to learn *NIX is to avoid the GUIs. By design, they hide a lot of complexity, and that complexity is what you want to get into.

You can pick your favorite Linux distro, or even a non-Linux distro (if you think you're going to be using "real UNIX" at some point in the future, playing with one of the BSDs would probably be helpful).

What I've done and found to be educational, is to set up a machine somewhere in the house and use it 100% remotely. It doesn't have to be 1000 miles away, but pretend that it is.

Then I'd just do whatever seems interesting at the moment. You could set it up as an rsync server, and have your other computers back their data up to it in the middle of the night. (For extra points, put a bunch of hard drives into it, so that each user's data is on a separate drive. Good way to learn about fstab and mountpoints.)

You could set up Apache and play around with some blogging/CMS platforms, and other web stuff, if you find that interesting. Running your own mailserver can also be enlightening. And a DNS server even more so.

But honestly there are so many things you could do that it's really a question of what you're interested in. I find networking pretty interesting, so that's what I'm always playing with on mine. But if you're not, don't force it -- install software that you think is interesting, solve problems that you've always wanted solutions to -- the learning will come from using the system without a GUI to mediate everything.
posted by Kadin2048 at 12:15 PM on July 10, 2007

Linux from Scratch is a great idea. Gentoo can be educational if you pay attention to the whys and wherefores of choosing your USE flags. But any command-line based system would be fine -- I'd avoid the slick, glossy, KDE and Gnome based distros (this isn't a slur on those projects or distros; it's just that a big part of the point of a desktop environment is to attempt to hide the inner workings from the user, which isn't what's wanted here.) I start with an Ubuntu command-line system (installed from the alternate installation CD.)

Don't think that portage and aptitude are the enemies of understanding. Yes, you'll want to be able to build things from source, but they're your friends for getting the dependencies installed. Writing your own package in one or the other would be a useful learning experience.

Learn the shell and the crucial commands -- the file & directory manipulation commands, find, grep, xargs. Know how to redirect STDERR and STDOUT and build command pipelines. Learn enough Awk and Sed, or Perl, or Ruby for text file munging.

Know what happens in what order when you boot the machine.

Lock down your box with a firewall. Open up an SSH server. Watch your logs. Figure out how to stop the script kiddies doing dictionry attacks.

And like everyone said, pick specific projects and do them.
posted by Zed_Lopez at 12:19 PM on July 10, 2007

My 2cents.
Won't matter what distro you use - just don't use any of the package manager apps, and install from source.
I'd recommend Apache, PHP, OpenLDAP, Build/upgrade the kernel, write a small program that does something (how about a script that backs up to an NFS share somewhere else on a regular basis, only files that have changed since last run).

I have found that if you can script in some language (shell, perl, python, whatever), and you can build from source, and you understand the configure/make/make install process, and can run some performance commands (like top or ps) you can do 90% of what you'd need to do that is not application specific.
posted by niteHawk at 12:23 PM on July 10, 2007

Seconding ORA's "Unix Power Tools".

Seconding, "install, but not a GUI".

Learning to compile a kernel and whatnot is only good for knowing how to compile a kernel. It won't teach you anything that you can use outside of that.

How about some topics instead? Most of being good at it is knowing where to look for info. At a shell,
$ man man
Google, man pages, mailing list archives, source code -- all are your friends.

Learn:

- What defines Unix. open() close() fork() exec() read() write() .
- What a "user" is in Unix. How does the boot process work, and how is it that your shell becomes you? Boot with kernel parameter "init=/bin/bash" .
- Filesystems. Creating, mounting, unmounting. Links and Symbolic links. The name of the file is not the file.
- Try a BSD also. Learn the cultural differences, so you know what is Unix and what is a local idiom.
- Program! Unix is great for programming against, and in doing so you'll learn some things. Try Python as a first language. Perl and PHP are good once you have experience with the shell, Awk, and Sed -- start it too early and it will eat youR BraNE!!$^
posted by cmiller at 12:31 PM on July 10, 2007

Oh yeah, it's a rich culture. See also the jargon file!
posted by cmiller at 12:33 PM on July 10, 2007

The UNIX Programming Environment. The book is timeless, I've never seen a better explanation of the Unix philosophy. Understanding it makes the difference between a novice who learned how to install Linux from scratch by rote and a master.
posted by reynaert at 12:54 PM on July 10, 2007

Install and learn different shells. The differences between sh, bash, Korn, c-shell, zsh and others shouldn't throw you for a loop, when you move between systems. Not every system will present you bash. Learn to work with shells, including minor shell scripting.
posted by paulsc at 2:15 PM on July 10, 2007

I cut my teeth on Slackware 7, doing what many of my Unixy friends did; building a NAT firewall and general network hub -- you start off with the basics of making it do something, getting network and SSH set up, text editing, hosts.allow, filesystem layout, high level overview of how the rc and config system works and some basic understanding of shell scripts, and moving on from there.

Pretty soon you (hopefully) have a fairly secure system which can sit in front of your other computers, at which point you probably want to start making it do more than a $40 consumer level NAT router -- so you set up bind to give your networked computers names and cache lookups which go outside the network; then maybe a caching [ad filtering?] web proxy to speed up browsing; an email server and spam/virus filters; a web server; some scripting languages to play with; MySQL and any web application which takes your fancy; RAID, and backup strategies for all your work; and so on.

I found I fairly rapidly went from having a Linux box which was just a toy, to having it as an integral part of my home network, and the things I learned by starting from the basics with Slackware helped me when I migrated to Debian, and later FreeBSD.
posted by Freaky at 2:20 PM on July 10, 2007

I needed to understand more about *nix too. I tried lots of different distros etc. but the thing really made it click was ORA's Essential System Administration. It talks about all the stuff you need to do to keep your system working and secure, and how to solve problems. The normal distro books just talk about installation by comparison.
This advice is years old, so if you want a GUI based environment it doesn't make sense, but using the CLI is the only way the real experts get stuff done.
Definitely second the "do real stuff" suggestions.
posted by bystander at 7:39 PM on July 10, 2007 [1 favorite]

I don't know how much I buy the "don't install X" school of thought. At times the terminal is going to be one's only lifeline into the system and being able to multi-window through the console beats the pants of trying to run the thing on a single-instance terminal window. My first NAT/web/FTP server setup was X-free and man, what a mistake that was.

Configure by hand, sure, and build as many of the services as you can 'cause the process will teach you a lot, but being able to see your README while you build is often invaluable.

When you inevitably hit google for help, having that X-based Firefox is going to make all the difference...
posted by Ogre Lawless at 3:16 PM on July 12, 2007

Ogre: Um.. single instance? Unless you're in single user mode, most systems have at least half a dozen virtual terminals.

X is much nicer, yes, but.. well, you need to be able to deal when that breaks too, and with many systems you really don't want the extra instability X can introduce, or the bloat. VT's are fine for fixing networks and SSH, and from there you can just use a proper workstation or laptop to talk to it.
posted by Freaky at 10:02 AM on July 13, 2007