The "Systems" part of a typical CS curriculum glosses over many exciting engineering designs involved in execution of almost any program on almost any computer. Not surprisingly, these same designs must be covered in "hacker"-style books as a pre-requisite to the rest of their material. Here we collect some of the resources that cover these topics.
The Executable and Linkable Format (ELF) — The ELF format is key to many exploits. In particular, infecting executables requires an intimate understanding of it. The ELF specification is somewhat dense, and does not describe many sections that are present in any executable or shared object file on a modern GNU/Linux system. John Levine's book is a gentler introduction to the subject, especially to relocation, but referring to the ELF spec and other sources is still inevitable.
Hardware and operating system support for debugging — Understanding the underpinnings of debugger implementation goes a long way towards being able to analyze resistant malware and spyware, and also makes clear several important considerations of modern OS design.
Linux Rootkits — Patching the kernel of a running system is the most efficient method of rootkit-ing it, that is, hiding the attacker's presence on the system and any use made of systems resources for illicit purposes, which includes files, processes and network connections.