news aggregator
Keeping an Eye Out When Sites Go Down
miller60 writes "Are major web sites going down more often? Or are outages simply more noticeable? The New York Times looks at the recent focus on downtime at services like Twitter, and the services that have sprung up to monitor outages. When a site goes down, word spreads rapidly, fueled by blogs and forums. But there have also been a series of outages with real-world impact, affecting commodities exchanges, thousands of web sites and online stores."
Read more of this story at Slashdot.
Gentoo 2008.0 Released
An anonymous reader notes that the Gentoo 2008.0 final release is available. From the announcement: "Code-named 'It's got what plants crave,' this release contains numerous new features including an updated installer, improved hardware support, a complete rework of profiles, and a move to Xfce instead of GNOME on the LiveCD. LiveDVDs are not available for x86 or amd64, although they may become available in the future. The 2008.0 release also includes updated versions of many packages already available in your ebuild tree."
Read more of this story at Slashdot.
The Internationalization of Malware
Ant brings us a write-up from a former malware analyst about the difficulties in fighting malware as it expands beyond English-language targets and into societies with different standards for privacy and security. Quoting: "One of the most fascinating facets of the increasing internationalization of malware is the cultural assumptions around such software. What is considered malware in the US may be commonly accepted in China or Japan, and this is largely due to the society that it exists in. Anti-cheating rootkits are very common in games released in these countries. What is considered to be invasive in the North American or European world is acceptable there. These anti-cheating rootkits would hook into the kernel space in a very invasive way, and have the behavioral characteristics of malware such as hooking into the keyboard driver. This made it very difficult from a purely technical standpoint to distinguish them."
Read more of this story at Slashdot.
Scaling Large Projects With Erlang
Delchanat points out a blog entry which notes, "The two biggest computing-providers of today, Amazon as well as Google, are building their concurrent offerings on top of really concurrent programming languages and systems. Not only because they want to, but because they need to. If you want to build computing into a utility, you need large real-time systems running as sufficiently as possible. You need your technology to be able to scale in a similar way as other, comparable utilities or large real-time systems are scaling — utilities like telephony and electricity. Erlang is a language that has all the right properties and mechanisms in place to do what utility computing requires. Amazon SimpleDB is built upon Erlang. IMDB (owned by Amazon) is switching from Perl to Erlang. Google Gears is using Erlang-style concurrency, and the list goes on."
Read more of this story at Slashdot.
US Justice Dept. Sued For Cellular Tracking Information
tpaudio writes "The ACLU and the EFF are suing the Department of Justice over how the government might be using GPS and location data from cell phones. With over 200 million Americans carrying cell phones, this could be pretty important for setting guidelines. We have already seen other frightening powers related to cell phones, such as 'cell mic tapping.'" The ACLU press release is also available, and it contains links to the complaint and the Freedom of Information Act request. We've previously discussed instances of cell phone tracking in the US and elsewhere.
Read more of this story at Slashdot.
Bletchley Park Faces Financial Rescue
biscuitfever11 writes "Just two months ago it seemed that Bletchley Park, the home of Station X, Britain's secret code-breaking base during the War, was doomed as the codebreakers' huts rotted and the site fell into disrepair. But today Britain's Lottery Fund is set to step in with a grant to rescue the ailing heritage site. (There was an earlier story on ZDNet.)"
Read more of this story at Slashdot.
AVG Backs Down From Flooding the Internet
Simon Wright writes "As a website that is featured heavily in many Google Australia search results, Whirlpool (Australia's largest technology forum) has been particularly affected by AVG's LinkScanner. We've seen a traffic increase as much as 12 hits per second from these bots. So we've actively and loudly campaigned against this move by AVG, encouraging all users of AVG 8.0 to uninstall the product. The discussion starts here. And AVG's backing down is posted here." From that URL:"'As promised, I am letting you know that the latest update for AVG Free edition has addressed and rectified the issue that [Whirlpool] have brought to our attention. This update has now been released to users and has also been built into the latest installation package for AVG Free.' — Peter Cameron, Managing Director, AVG Australia."
Read more of this story at Slashdot.
home
LinuxMan's technical notes
About this wiki
About this wiki
- Cheat sheets
- Clustering Shorewall
- Creating a CentOS thin client server
- Creating a Debian server for a small nonprofit
- Dual head setup on Debian Sarge
- Email practices
- Essential spam-fighting techniques
- General Debian installation notes Generic Linux installation procedure The Great Linux Groupware Dilemma
- Hardware notes
- Linux installation procedure , including:
- Linux web content management systems
- Major Linux distributions
- Non-Linux notes
- Password practices
- Programming notes
- PPPPPPS (or, Paul's Principles for Practical Provision of Packet Processing with Shorewall )
Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Change sysstat config.
- (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname
Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Change sysstat config.
- (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname
Linux installation procedure
System planning Disk design
Services Host services As far as possible, all hosts should have some basic accounting and performance collection configured, and store it for at least one month, preferably more. Packages that should be configured are:
Network services
Other packages you might want:
Notes on specific packages that are usually installed by default:
- RAID
In general, Linux software RAID is preferable for OS disks. - file system layouts
In general, the fewer file systems present, the easier a system is to manage. For most servers, separate volumes for the following file systems should be configured:- / (root)
- /boot
- /tmp
- /var
- Additional file systems should be added when applications need isolation. For small server disks (anything under 72 GB or so), this may not be feasible.
- location of data
In general, unless there is a good reason to do otherwise, application data should be stored under /srv/APPNAME for system-provided applications, or /var/opt/APPNAME for 3rd-party applications which comply with the FHS standard for optional packages. - Commonly used disk-related commands
- Determine trust models for network services.
- Determine location in network topology.
Services Host services As far as possible, all hosts should have some basic accounting and performance collection configured, and store it for at least one month, preferably more. Packages that should be configured are:
Network services
- Host naming
- Local
/etc/hosts needs to be edited post-install on most distributions. The exception to this is dynamic IP servers on modern distributions such as openSUSE 10.2 or Ubuntu 6.10. - DNS
- DNS Stuff is a useful site for testing your DNS server & configuration.
- client
- Local
- Remote access
- Firewall
Shoreline Firewall is a policy-based firewall that provides great flexibility and does not require the manual construction of iptables commands. - Time synchronisation: NTP
- Intrusion detection/prevention
- Virus scanning
- Spam filtering
- Email
- server
- client
- acct (a.k.a. psacct) - Process accounting (logs CPU/disk/memory usage of particular processes).
- chkrootkit - intrusion detection
- clamav - virus scanner
- findutils-locate - locate files on the system
- ethereal - protocol analyser
- ltrace - library call trace (ltrace-32bit also for 64-bit systems)
- net-snmp - allows monitoring of various aspects of the system
- nmap - network diagnostics tool
- ntp (a.k.a. xntp) - Network Time Protocol. This package ensures that the system's clock is kept in sync.
- rcs - revision control system
- rsync
- sensors (a.k.a. lm-sensors) - motherboard health monitoring
- snort - Intrusion detection system
- strace - system call trace
- sysstat - Collects system performance statistics that can be viewed later with sar or isag.
Other packages you might want:
- aide - intrusion detection
- amanda - backup
- bind - DNS server
- iptraf - IP traffic monitor
- linux-iscsi/open-iscsi - for network storage via iSCSI
- multipath d - disk load balancing & failover
- rsnapshot - rsync-based backup tool that saves time and disk space
- webalizer - gather web server statistics from logs
- xpdf - For reading system documentation in PDF.
Notes on specific packages that are usually installed by default:
Linux installation procedure
System planning Disk design
Services Host services As far as possible, all hosts should have some basic accounting and performance collection configured, and store it for at least one month, preferably more. Packages that should be configured are:
Network services
Other packages you might want:
Notes on specific packages that are usually installed by default:
- RAID
In general, Linux software RAID is preferable for OS disks. - file system layouts
In general, the fewer file systems present, the easier a system is to manage. For most servers, separate volumes for the following file systems should be configured:- / (root)
- /boot
- /tmp
- /var
- Additional file systems should be added when applications need isolation. For small server disks (anything under 72 GB or so), this may not be feasible.
- location of data
In general, unless there is a good reason to do otherwise, application data should be stored under /srv/APPNAME for system-provided applications, or /var/opt/APPNAME for 3rd-party applications which comply with the FHS standard for optional packages. - Commonly used disk-related commands
- Determine trust models for network services.
- Determine location in network topology.
Services Host services As far as possible, all hosts should have some basic accounting and performance collection configured, and store it for at least one month, preferably more. Packages that should be configured are:
Network services
- Host naming
- Local
/etc/hosts needs to be edited post-install on most distributions. The exception to this is dynamic IP servers on modern distributions such as openSUSE 10.2 or Ubuntu 6.10. - DNS
- DNS Stuff is a useful site for testing your DNS server & configuration.
- client
- Local
- Remote access
- Firewall
Shoreline Firewall is a policy-based firewall that provides great flexibility and does not require the manual construction of iptables commands. - Time synchronisation: NTP
- Intrusion detection/prevention
- Virus scanning
- Spam filtering
- Email
- server
- client
- acct (a.k.a. psacct) - Process accounting (logs CPU/disk/memory usage of particular processes).
- chkrootkit - intrusion detection
- clamav - virus scanner
- findutils-locate - locate files on the system
- ethereal - protocol analyser
- ltrace - library call trace (ltrace-32bit also for 64-bit systems)
- net-snmp - allows monitoring of various aspects of the system
- nmap - network diagnostics tool
- ntp (a.k.a. xntp) - Network Time Protocol. This package ensures that the system's clock is kept in sync.
- rcs - revision control system
- rsync
- sensors (a.k.a. lm-sensors) - motherboard health monitoring
- snort - Intrusion detection system
- strace - system call trace
- sysstat - Collects system performance statistics that can be viewed later with sar or isag.
Other packages you might want:
- aide - intrusion detection
- amanda - backup
- bind - DNS server
- iptraf - IP traffic monitor
- linux-iscsi/open-iscsi - for network storage via iSCSI
- multipath d - disk load balancing & failover
- rsnapshot - rsync-based backup tool that saves time and disk space
- webalizer - gather web server statistics from logs
- xpdf - For reading system documentation in PDF.
Notes on specific packages that are usually installed by default:
Mozilla Launches Security Metrics Project
Earthweb passes along a ZDNet article which notes, "In partnership with indie security consultant Rich Mogull, Mozilla has launched a valuable Security Metrics Project that could help to — we can only hope — put an end to the silly notion that patch-counting helps to determine a product's security posture. The idea is to develop a metrics model that goes beyond simple bug counts to accurately reflect the effectiveness of secure development efforts and the relative risk to users over time. Mogull has released a spreadsheet (.xls) with a preliminary version of the model and Mozilla's Window Snyder is actively seeking feedback to make the project open and meaningful."
Read more of this story at Slashdot.
Installing Ubuntu On an OLPC XO
Matt Lincoln Russell writes "Installing Ubuntu Netbook Remix on the OLPC XO is not for the faint of heart, but Drew Beckett has got the process down. This setup is pretty slow on the XO, but the good news is that Netbook Remix is a work in progress, and can be expected to get better."
Read more of this story at Slashdot.
Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Enable startup in /etc/default/sysstat (if applicable). Change sysstat config.
- (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname
General Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Enable startup in /etc/default/sysstat (if applicable). Change sysstat config.
- (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname
Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Enable startup in /etc/default/sysstat (if applicable).
- On vservers, it (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname
General Debian installation notes
Start See Generic Linux installation procedure first!
Caveats
Caveats
- These reflect my preferences
- They're mostly based on etch (Debian 4.0), but should work for other derivatives and earlier versions.
- They're mostly here for my benefit, particularly when installing VServer s, VMware VMs, or Xen domUs.
- Change /etc/apt/sources.list to comment unneeded repositories (i don't use deb-src on most machines) and add contrib & non-free (if necessary).
- Add the following to /etc/apt/apt.conf :
Aptitude::Recommends-Important "false"; This ensures that aptitude doesn't enforce "soft" (recommends) dependencies, only "hard" (requires) dependencies. It doesn't seem to be necessary on new vservers, since this file is copied from the host. - (sarge only) Reduce the noise of package installs:
dpkg-reconfigure debconf (Choose high when prompted for the priority.) This step isn't necessary any more on etch, since high is the default. - Install latest OS updates that didn't get added during the install process:
aptitude update
aptitude upgrade - Set ftp_proxy , http_proxy , and https_proxy environment variables.
- Remove non-preferred apps in base install:
aptitude purge vim-tiny nvi nano - Install preferred software.
- Core set for all machines:
aptitude install acct apt-show-versions at bc bind9-host bzip2 deborphan \
debsums file ftp isag less logwatch lsof lsscsi ltrace make openssl patch \
perl postfix psmisc rsync ssh strace sysstat subversion sysvconfig telnet \
time vim - Additional packages for physical hosts (i.e. not VMs):
aptitude install bind9 pciutils
- Core set for all machines:
- (I usually share one bind9 instance across all VMs on one physical host.)
- Purge unneeded library dependencies:
while [ `deborphan | wc -l` -gt 0 ]; do
aptitude purge `deborphan`
done
- Set up /etc/resolv.conf with appropriate name servers & domains. (The installer only allows one domain and one name server.)
- (VMs only) Edit the mess that debootstrap/xen-create-image makes of /etc/hosts .
- Add your ssh public key to /root/.ssh/authorized_keys2 .
- Edit /etc/syslog.conf to put syslog messages on tty12. Add this line:
*.debug /dev/tty12 - Verify contents of /etc/network/interfaces
- Enable startup in /etc/default/sysstat (if applicable).
- On vservers, it (VServers only) It seems necessary to change the following line in /etc/postfix/main.cf :
inet_interfaces = all to
inet_interfaces = $myhostname






