Categories
General

RAID-1 sans disks

In preparation for a somewhat more dramatic future experiment, I’ve been trying out RAID-1 failure modes using linux’s loopback capabilities to avoid having to actually buy any more real hard drives. You can simulate drive failures, failover and easily see what the current disk contents are. It should go without saying that, if you have a real RAID system currently running, you probably don’t want to execute these commands without thinking a bit first:

# Creating and destroying disks from the safety of your own console
mkdir ~/raid; cd ~/raid

# Create two 10Mb files called disk0 and disk1
for d in 0 1; do dd if=/dev/zero of=disk${d} bs=1024 count=10240; done

# Make them available as block devices using the loopback device
for d in 0 1; do sudo losetup /dev/loop$d disk$d; done

# Combine the two 'disks' into a RAID-1 mirrored block device
# Using '--build' rather than '--create' means there is no device
# specific metadata, and so the contents of the disks will be identical
sudo mdadm --build --verbose /dev/md0 --level=1 --raid-devices=2 /dev/loop[01]

# Create a filesystem on our raid device and mount it
sudo mkfs.ext3 /dev/md0 
mkdir /tmp/raidmnt
sudo mount /dev/md0 /tmp/raidmnt
sudo chown $USER /tmp/raidmnt

# The contents of both disks change in unison
md5sum disk[01]
date > /tmp/raidmnt/datefile
sync
md5sum disk[01]

# If we mark one disk as failed, disk contents diverge
sudo mdadm --fail /dev/md0 /dev/loop0
date > /tmp/raidmnt/datefile
sync
md5sum disk[01]

# Remove the failed disk and readd it, and RAID1 will sync
sudo mdadm --remove /dev/md0 /dev/loop0
sudo mdadm --add /dev/md0 /dev/loop0
sleep 1
md5sum disk[01]

# Add a third (unused) disk into the system to test failover
dd if=/dev/zero of=disk2 bs=1024 count=10240
sudo losetup /dev/loop2 disk2
sudo mdadm --add /dev/md0 /dev/loop2
sudo mdadm --detail /dev/md0

# When one of original two disks fail, the new disk gets used
md5sum disk[012]
sudo mdadm --fail /dev/md0 /dev/loop0
date > /tmp/raidmnt/datefile
sync
md5sum disk[012]

# Tidy up the world
sudo umount /dev/md0
sudo mdadm -S /dev/md0
for x in /dev/loop[012]; do sudo losetup -d $x; done
rm -rf /tmp/raidmnt ~/raid
Categories
General

Reading mode for emacs

Reading lots of text on a computer isn’t the most fun thing in the world. I still like paper (or e-ink screens!). But computers still have the advantage of flexibility. I’ve tried some “alternative” reading methods like dictator before but didn’t like them much. I think a less radical approach is better. So, I’ve been hacking up a basic reading mode in emacs which just highlights a sentence at a time. I’ve found it pleasantly useful, especially for technical documents. Here’s the screenshot (code still in flux):

Sentence highlighting

Dead simple, but it keeps my mind focused.

Categories
General

Yi, the haskell editor

My latest shinything fascination is with Yi, an emacs-like editor written in Haskell. State! GUIs! Monads! What fun. It is entirely made of awesome.

It’s not the easiest program to start playing with, so I’ve written up some installation instructions and a beginner guide here. If you want to see what a real-world Haskell application looks like, yi is a great example.

Categories
General

The best programming book I’ve ever read: Beautiful Code

Beautiful Code is probably the best programming book I’ve read (and I’ve read a lot of them!).

Bookshops are filled with books about the mechanics of programming; “Learn X in N days”. But there are precious few books which manage to capture the thought processes and the wisdom of good programmers. Reading “Beautiful Code’ is like spending several hours in the pub in deep conversation with a bunch of really sharp programmers. If you care about programming, it’s a must-read.

When you write code, you make lots of decisions. Some are small scale, like whether to split a long conditional across two lines. Others are larger scale, like how to model concurrency in a large system. Ideally, each decision you make should be an informed one where you are aware of the trade-offs you are making. However, that thought process is rarely captured in source code. When another programmer reads your source in six months, they are probably going to wonder why you did it that particular way instead of different way.

So that’s the great thing about “Beautiful Code”. There are 33 articles, all written by programmers about systems they’ve built and care deeply about. The articles talk about the trade-offs and choices they made. Some articles focus on the design of small bits of software, such as Python’s dictionary implementation. Others focus on cool new techniques, such as Simon Peyton Jones’ article on transactional memory. There’s an amazing article about designing software for Stephen Hawking’s computer system (he can only operate a single input button). These are not trivial toy examples; they are real world systems with history and design.

There’s both a breadth and a depth to this book which really impresses me.

See Beautiful code at amazon.com or amazon.co.uk.


Categories
General

Unix tools, and how I use them

I love the unix philosophy: carry around a small set of tools and use them to build bigger custom tools to solve problems. Over the last few years, I’ve the following programs to my ‘must have’ list:

strace – trace system calls

Strace tells you which system calls a process is making. It gives loads of information about errant processes – is it blocked on network or file i/o? Is it stuck in a loop? I used this recently to find out why ssh logins were slow on one of my machines. I used ‘ps ax | grep sshd’ to find the pid of sshd, then ran “sudo strace -f -t -p PID”. The ‘-f’ means to also trace any child processes, and ‘-t’ gives timestamps. This showed that sshd was doing a reverse DNS lookup when I logged in (wrongly set up dns) and also that the default ubuntu .bashrc takes a good while to run.

lsof – list open files

lsof is useful in conjunction with strace; strace will show you that a process is reading on file descriptor 7, but what is that used for? Running “lsof -p PID” will tell you what each file descriptor is connected to.

cstream – filter, monitor and bandwidth-limit stream

cstream is great for monitoring long running jobs. I use this often to monitor the progress of mysql imports from mysqldump files. Eg. “cat dump.sql | cstream -l -T1 | mysql DATABASE” lets me know how much of the file has been processed so far.

You can also use cstream to bandwidth-limit a stream, but I tend to do my bandwidth limiting via rsync (–bwlimit) or scp (-l).

socat – like netcat, but better

Socat is netcat for network, processes, files, sockets, etc, etc. It also doesn’t buffer output in the same annoying way that netcat does, which makes it more useful for creating mock servers. For example, I recently used it to create a dummy HTTP server for testing erlang’s inets library:

Create a script called “reply-204” containing

#!/bin/bash
sleep 1
echo -ne 'HTTP/1.1 204 No Content\r\nSomeHeader: foornrn'
sleep 100

.. then run “socat tcp-listen:9999,reuseaddr exec:./reply-204”.

watch – run a command repeatedly

I used to write this loop lots: “while true; do ls -l somefile; sleep 1; done”. Now I just use watch, for example “watch -n1 ‘ls -l somefile'”. The “-d” flag is also useful – it highlights difference between each run.

Commands which run other commands are the happiest commands in the world.

iftop – what traffic is going where?

iftop is like top, but for network traffic. Great for getting a quick overview of why your network connection has suddenly slowed to a crawl. Also good for noticing weird connections (aka, why is my machine sending traffic there?).

tcpflow

Like tcpdump, tcpflow captures network packets. Additionally, it stores each “conversation” in a seperate file which makes it easy to futher analyze.

Whilst running tcpflow a minute ago, my browser happened to request this page and tcpflow let me see that it returns this header: “Server: Modified Atari-ST”. Do you think it’s true?

iperf – how fast can my network go?

iperf is a simple end-to-end network performance tool. Answers the question: What’s the maximum transfer rate between two machines? I recently moved all my photos and videos onto a separate media server box, and loading up big jpegs was taking a few seconds. I used iperf to check my actual network speed, but sadly the performance was pretty close to the theoretical maximum .. sadly, moving lots of bits still takes a while!

Not forgetting

  • My favourite grep flags: “-o” to only show matching text, and “-P” to get perl regexps (eg. non-greedy quantifiers)
  • My favourite cat flags: “-T” to show tabs as “^I” … useful for eyeballing tab-separated files
  • My favourite less flags: “-S” makes long lines get truncated, rather than wrapping.