What is Linux? More than just on operating system, GNU/Linux is becoming a universal and industry standard computing platform. Through the work of the Free Software Foundation and thousands of developers worldwide, Linux and the software which runs on it are the world's largest software technical reference library. For any idea you could wish to develop on a computer, there are is a wealth of free information available under the GPL or similar licenses. There's no longer a reason to reinvent the wheel.
If you just want to give Linux a spin, you should try KNOPPIX - Live Linux Filesystem On CD.It runs off your hard drive, without you having to install anything. Just set your BIOS to boot from the CDROM before the hard drive, and restart your computer with the CD in.
Planet CCRMA at Stanford is a wonderful site geared toward multimedia production on Linux (and there's also a bunch of stuff that runs fine under MicroSoft Windows™, too). Their CD images (and apt-get servers) build upon the RedHat and Fedora Core distributions with custom kernels for Intel and AMD processors (both SMP and single CPU) and alsa sound drivers. They have a huge repository of audio applications, from recording, MIDI, effects processing, live streaming, analysis and synthesis. Source code is also available for all these goodies, so you can see how they work and modify or build your own applications.
Ever wanted to count the number of violins in Handel's
Baudline™ is a wonderful FFT spectrum analyzer written by SigBlips, a custom software provider. While it is free to download, it is not Open Source, so there is no peeking at the code, unfortunately. Also, it cannot be redistributed by a mirror or distribution, only from SigBlips' http server. Even so, it is a real hoot to play with. It supports mp3, Ogg Vorbis, aiff, au, snd, wav, and a whole slew of formats I've never heard of. Windowing from Blackman to Kaiser, drift integration, heterodyne conversion, phase tracking, EQ, and a bunch of stuff you'll need a DSP textbook to understand.
Jack is digital patchbay for audio applications, allowing for all kinds of bizarre realtime interactions and conversions. For more info, check out the CCRMA site for instructions and links.
Sfront is an audio Mpeg-4 compiler. With it you can create portable programs which can play back compositions, be controlled my a MIDI keyboard, or which can compose music automatically. Building blocks include allpass and comb filters, oscillators, wave tables, and fourier transforms. The syntax is similar to C. While similar to CSound, it makes much more efficient use of the CPU.