3e8.org

This is a short guide to starting up Symbolics Concordia in a Virtual Lisp Machine (VLM) running OpenGenera. It was written because the existing tutorials on the Symbolics VLM teach you the basics of building one and starting it up, but stop after the login prompt.

Concordia is a software documentation tool that produces output that can be read in Genera's Document Examiner, an early hypertext browser. The documentation system was extremely advanced for its time and can still be admired today for its usefulness, thoroughness, and excellent technical writing; it is worth exploring.

We assume you've already started up a VLM and configured it for your site, that the VLM machine is called GENERA, and the VM host system is called GENERA-HOST, as in this tutorial. We'll also pretend you used ARIANA for your site name when you defined your site at first boot. You should have run through the Genera Workbook in Document Examiner to become familiar with Genera as well. At the very least, you will need to know where a few important keys are, such as <Select> on F1, <Help> on F12, <Abort> on F6, and <Super> on F7.

Concordia won't be available as an Activity (selectable application) nor will its documentation appear in Document Examiner until you load the Concordia system:

Continuing from my previous post, another way to redirect output to a serial console is to use Google's sgabios option ROM, which mirrors BIOS output from the screen to a serial port. Just place this ROM alongside qemu's other roms (e.g. in /usr/share/qemu) and start qemu with -device sga.

Since this works for LILO, we can get into the LILO boot prompt on the serial console, and start Linux with linux console=ttyS0. Once Linux boots up we can modify lilo.conf and run lilo. This avoids some of the rigamarole of the previous method, although guestfish is still useful to edit lilo.conf, due to the lack of an editor in the test image.

However, there's no prebuilt sgabios package for Ubuntu 12.04, and qemu 1.0 doesn't come with sgabios included. So I built sgabios along with a .deb that drops it into qemu's ROMs directory.

I now present to you:

• a script which downloads, builds and packages sgabios,
• the qemu-sgabios .deb, and
• the ROM file itself.

As provided, QEMU's test linux image expects to be on either a graphical or curses-based terminal. Let's say you'd like to patch the image to use the serial console instead. To make this more interesting, let's not resort to booting in graphical or curses mode to set up the serial console in the first place.

First, grab the test image.

$ mkdir linux-test && cd linux-test
$ curl http://wiki.qemu.org/download/linux-0.2.img.bz2 | bunzip2 > linux-0.2.img

The image uses LILO; to get LILO to interact with the serial port, we have to add serial=0,9600n8 to lilo.conf. We also need to get the kernel to use the serial line by appending console=ttyS0 to the kernel arguments in lilo.conf.

One problem: the image doesn't come with an editor (not even ed, man!). Also, we have no interactive console yet either way. So, in order to manipulate the disk image directly, we use the amazing guestfish.

$ guestfish -i -a linux-0.2.img
><fs> edit /etc/lilo.conf

Now we can edit lilo.conf to add the serial and console arguments. Updated, it looks something like:

boot = /dev/hda
serial = 0,9600n8
root = /dev/hda
# [... 5 lines omitted ...]
append = "console=ttyS0 sb=0x220,5,1,5 ide2=noprobe ide3=noprobe ide4=noprobe ide5=noprobe"
image = /boot/vmlinuz-2.6.20
  label = linux

It might have been nice to automate this editing with guestfish's support for augeas, but unfortunately augeas lacks a lens for lilo.conf.

Anyway, we still have to execute lilo to update the boot sector. We can try to do this via guestfish as well; the guestfish instance is running a mini-VM attached to your disk, and you can actually run code located on your disk within the mini-VM.

><fs> command lilo
libguestfs: error: command: Fatal: open /dev/hda: No such file or directory

Ugh. Our disk in the guestfish VM is located at /dev/vda, not hda. Well, we can fake it with a symlink:

><fs> command "ln -s /dev/vda /dev/hda"
><fs> command lilo
libguestfs: error: command: Fatal: Sorry, don't know how to handle device 0xfd00

By device 0xfd00 it means /dev/vda (major 253, minor 0); apparently it doesn't like virtblk devices. It might work by explicitly specifying the geometry, or upgrading LILO, but let's try another method.

What we'll do is use QEMU's support for directly loading the kernel and bypassing the boot loader. We can copy the kernel out of the image, and launch our VM using that external kernel, pointing it to the serial console. We can then run LILO to update the boot sector.

(Naturally, using this method, we could skip messing around with LILO at all, and just start up the VM this way all the time. But we've come this far, so let's make the change permanent.)

><fs> copy-out /boot/vmlinuz-2.6.20 .
><fs> quit

$ ls -l vmlinuz-2.6.20
-rw-r----- 1 jim jim 2040204 Nov 27 22:16 vmlinuz-2.6.20

$ kvm -nographic -hda linux-0.2.img -kernel vmlinuz-2.6.20 \
      -append "console=ttyS0 root=/dev/hda"

The machine should boot up and print its output to the serial console (your terminal). Run lilo when you get the login shell:

sh-2.05b# lilo
Added linux *
sh-2.05b# exit

Now this image has been permanently converted to communicate with the serial console. Boot it like so and you'll see both the LILO and kernel output on your terminal:

$ kvm -nographic -hda linux-0.2.img

Via Shocco, a neat little technique for templating in shell. The idea is to encapsulate a here document in a function, and use the function arguments ($1 $2 ...) as the template variables. The cool part is the use of $(cat) in the template body, allowing you to pipe in data to the function and have it wrapped in the output. This is extremely useful in the middle of a pipeline; much of shocco is, in fact, a gigantic pipeline.

Here's a simple example. We assume the input is already properly escaped for HTML.

layout() {
  cat <<HTML
<html><head>
<title>$1</title>
<body><p>$(cat)</p></body>
</head></html>
HTML
}

echo 'My body is ready' | layout 'Status'

Output:

<html><head>
<title>Status</title>
<body><p>My body is ready</p></body>
</head></html>

Shocco is shoc-full (sorry) of interesting shell techniques and I'd recommend checking it out if you're at all into shell programming.

In jQuery 1.4.3 and later, you can use .data() to access the value of custom data attributes on your HTML5 elements. jQuery tries to figure out what type of data you're passing and do the appropriate result conversion, falling back to string if it can't figure it out. This is convenient, but a problem can occur when you want to pass a string but your value looks like (for example) a boolean:

<input id='abc' data-foo='true'>

$('#abc').data('foo')  /* true (boolean) */

It's not possible to quote the string value either; you'll just get a string with embedded quotes. The official workaround is to use .attr('data-foo'), which will return the raw string. Of course, this means that to safely pass a string without interpretation, you can't accept any other object. Some people are unhappy with this behavior, but... wontfix.

However, there's another trick you can use. Pass a quoted string inside a 1-element array, then reference that element. This will ensure the JSON parser is engaged because the value starts with [. It also lets you pass arbitrary objects you do want converted (in which case you couldn't use .attr()). Example:

<input id="abc" data-foo='["true"]' data-bar='[true]'>

$('#abc').data('foo')[0] /* "true" (string) */
$('#abc').data('bar')[0] /* true   (boolean) */

It's a little clunky, but at least you'll get consistent results. If you're passing several values, it's probably easier just to use a JSON hash.

ccze is a nice way to colorize your logs. The defaults are tuned more for following logs in a dedicated window, than for viewing them inline at a prompt. I use an alias like this:

alias cz='(ccze -m ansi | less -MnFRX)'

allowing usage like

$ cz < /var/log/messages
$ tail -n 20 /var/log/syslog | cz
$ dmesg | tail | cz

While working on the zguide for the zmq egg in Chicken, I encountered a problem with terminating a process in a timely manner when it has a SIGTERM handler installed.

(use posix)
(on-exit (lambda () (print "exiting")))
(set-signal-handler! signal/term
                     (lambda (s) (print "terminating") (exit)))
(read)
(print "finished")

If you compile and run this program, and kill -TERM it from another window, nothing happens right away. You have to hit Enter afterwards, and then it will print terminating and exiting.

The reason for this is that the posix unit installs signals with signal(3), which sets the sigaction flag SA_RESTART under the hood. "Slow" system calls (read, zmq_recv) which have received no input yet will automatically restart when the signal handler returns. Chicken has a single global signal handler that just sets a flag indicating which signal was received, schedules the interrupt handler to run in a moment, and returns immediately. The syscall is then restarted before the interrupt handler gets a chance to run.

To hack around this behavior you can add

(foreign-code "siginterrupt(SIGTERM, 1);")

to the top of your file, so slow syscalls will immediately exit with EINTR even when no input is available, and your handler will be invoked. To wit:

(foreign-code "siginterrupt(SIGTERM, 1);")
(use posix)
(on-exit (lambda () (print "exiting...")))
(set-signal-handler! signal/term
                     (lambda (s) (print "terminating...") (exit)))
(read)
(print "finished")

If you do not explicitly hook SIGTERM, a TERM signal will interrupt read immediately and terminate. This is true on Linux and Mac OS X at least, but it's not clear to me that this can be relied upon.

You can see this behavior as well with SIGINT. By default Ctrl-C terminates a read immediately. However, when the posix egg is loaded, it hooks Ctrl-C with signal(3) and so Ctrl-C will not immediately terminate a read.

 (use posix)
 (read)
 ^C^C^C^C^C^C

Instead, you have to press Enter afterward (as line buffering is active by default). To terminate immediately you can do:

 (use posix)
 (foreign-code "siginterrupt(SIGINT, 1);")
 (read)

This situation might need to be addressed with an egg that provides more nuanced signal handling than the posix unit.

Final note: It's not possible to restore the default signal handler (SIG_DFL) with the posix unit, either. Passing #f to set-signal-handler! will ignore the signal (SIG_IGN).

The stock SXML->XML serializer from sxml-tools has a couple aesthetic issues related to namespace output.

1. It doesn't support default namespaces at all, which makes the already-verbose XML positively prolix, and could cause some non-conformant XML processors to fail.
2. It does not allow redeclarations of XML prefixes, so you may sometimes get an autogenerated prefix name even when you provided a mapping. (This was a "design goal," though.)
3. It does not support declaring all prefixes in the root element, which in certain cases can elevate the natural redundancy of XML to dizzying heights.

I added support for 1. and 2. in version 0.2 of the sxml-serializer egg, released yesterday. No.3 unfortunately will take some time to think about, as the code is geared to declare prefixes as locally as possible.

So I was going to write a blog post with a lot of examples and in-depth explanation, but instead, I just documented the egg! See The default namespace and Redeclaring XML prefixes for more details.

Here's a preview, though. The change introduces a new *default* pseudo-namespace which we can use to map any number of URIs to the default namespace. This works with nested elements and also handles the empty namespace correctly. Below is a pretend Atom document that is rendered without any prefixes:

> (serialize-sxml
    '(*TOP* (@ (*NAMESPACES*
                (atom "http://www.w3.org/2005/Atom")
                (xhtml "http://www.w3.org/1999/xhtml")))
       (atom:feed (atom:entry
                   (atom:content (@ (type "xhtml"))
                    (xhtml:div (xhtml:p "I'm invincible!"))))))
    ns-prefixes: '((*default* . "http://www.w3.org/2005/Atom")
                   (*default* . "http://www.w3.org/1999/xhtml")))

<feed xmlns="http://www.w3.org/2005/Atom">
  <entry>
    <content type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p>I'm invincible!</p>
      </div>
    </content>
  </entry>
</feed>

Instead, if we omit the *default* mappings from ns-prefixes -- or just use the stock serializer -- every element is prefixed:

<atom:feed xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:entry>
    <atom:content type="xhtml">
      <xhtml:div xmlns:xhtml="http://www.w3.org/1999/xhtml">
        <xhtml:p>I'm invincible!</xhtml:p>
      </xhtml:div>
    </atom:content>
  </atom:entry>
</atom:feed>

I like to call this "terse mode," because irony is the spice of life.

Last time, we talked about the distinction between SXML shortcut names and XML prefixes, and particularly about the *NAMESPACES* node. This time let's talk about the prefixes you pass to the parser and the serializer.

The default SSAX parser in Chicken is ssax:xml->sxml. It accepts an alist that maps user namespace shortcuts (symbols) to namespaces (URI strings). Let's see what happens if we pass it a null list:

> (ssax:xml->sxml
   (open-input-string
    "<cars:part xmlns:cars=\"http://www.cars.com/xml\" />")
   '())

(*TOP* (http://www.cars.com/xml:part))

Because we didn't give it an association between shortcuts and URIs, the parser returns fully-qualified names. It completely discards the XML prefix in the original document, because the prefix is only meaningful for languages that cannot qualify element names with URIs (i.e. XML 1.0). An important consequence is that you cannot recreate the original XML document prefixes without additional information provided by the user.

Now let's associate the shortcut prefix cs to the namespace URI for the XML prefix cars, and parse the same document:

> (ssax:xml->sxml
   (open-input-string
    "<cars:part xmlns:cars=\"http://www.cars.com/xml\" />")
   '((cs . "http://www.cars.com/xml")))

(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
 (cs:part))

I used cs here so that you could see that the prefix cars is completely irrelevant to the output, except on the XML side in determining the universal name. It is not even retained in the SXML document. The fully qualified SXML name http://www.cars.com/xml:part is now, via the user's association, changed to the short name cs:part. This association is also recorded in the *NAMESPACES* node so that the namespace URI can be reconstructed when transformed back to XML. Notice that during parsing, the alist actually maps from URI strings to shortcut symbols, which is the opposite of what you'd expect.

Let's now consider transforming the two documents above back to XML. In the previous post, we saw the erroneous case where we tried to use a shortcut namespace without defining that association, so the shortcut was treated as a full namespace URI:

> (->xml '(*TOP* (cars:part)))
<prfx1:part xmlns:prfx1="cars" />

However, the first SXML document we produced above is valid, and will produce valid output:

> (->xml '(*TOP* (http://www.cars.com/xml:part)))
<prfx1:part xmlns:prfx1="http://www.cars.com/xml" />

There's no assocation to an XML prefix, though, so prfx1 is generated for you. This is a perfectly legal XML document, and is readable by any conformant parser, even though it looks strange.

We can pass another association list, this time to the serializer, to map URI strings to XML prefixes. Below, the namespace URI http://www.cars.com/xml is mapped to the XML prefix cars, instead of the auto-generated prfx1.

> (->xml '(*TOP* (http://www.cars.com/xml:part))
         ns-prefixes: '((cars . "http://www.cars.com/xml")))
<cars:part xmlns:cars="http://www.cars.com/xml" />

It's important to remember that the alist you pass to the serializer in ns-prefixes does not map shortcut prefixes to URIs. That's the job of the alist passed to the parser--or more accurately, the job of the *NAMESPACES* node. ns-prefixes only maps URIs to XML prefixes. Returning to the erroneous example above,

> (->xml '(*TOP* (cars:part))
         ns-prefixes: '((cars . "http://www.cars.com/xml")))
<prfx1:part xmlns:prfx1="cars" />

we can see ns-prefixes has no effect. There is no such namespace URI as http://www.cars.com/xml in that document, nor does ns-prefixes create a shortcut mapping to that URI.

Turning to the second example which was generated by the SSAX parser,

> (->xml '(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
           (cs:part)))
<cs:part xmlns:cs="http://www.cars.com/xml" />

we can see there is a namespace association in *NAMESPACES* from the shortcut cs to the URI http://www.cars.com/xml. Recall this association was created by the user when calling the parser, not by the XML document. During serialization, this assocation causes the shortcut namespace cs to be translated to the URI http://www.cars.com/xml. However, we still need to determine the resulting XML prefix. By default, the serializer conveniently uses the shortcut name cs as the XML prefix!

That was, however, just the default. If we actually provide a mapping from URI to XML prefix in ns-prefixes, that will override the default:

> (->xml '(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
           (cs:part))
         ns-prefixes: '((cars . "http://www.cars.com/xml")))
<cars:part xmlns:cars="http://www.cars.com/xml" />

So the shortcut cs is translated to the namespace URI http://www.cars.com/xml via *NAMESPACES*, and the URI is then mapped to the XML prefix cars via ns-prefixes. Again, notice how ns-prefixes contains no reference to the shortcut cs, because it never sees the shortcut names; it only sees the namespace after expansion into a URI.

Confused yet? Let me sum up.

• When parsing XML to SXML, XML prefixed names are mapped to universal names (local names qualified with URIs), based on the xmlns attributes in the document. Optionally, these URIs are mapped to shortcut names based on an alist passed by the user, and this mapping is also stored in the *NAMESPACES* node.
• When serializing SXML to XML, shortcut names are mapped back to URI namespace strings using the associations stored in the *NAMESPACES* node, and all URIs are then mapped to XML prefixes based on a user-provided alist. Since universal names are illegal in XML 1.0, all URIs must be mapped to prefixes; therefore, automatic prefixes are created if you do not provide a mapping.

In the next installment, we'll take a look at a more complex example.

Namespaces in SXML are tricky. The SSAX SXML parser takes a list of namespace prefix to URI assocations; the SXML document itself contains *NAMESPACE* nodes mapping prefixes to URIs; and the SXML serializer takes a list mapping prefixes to URIs as well! How does it all fit together?

In the discussion below I will be using the sxml-serializer egg for Chicken. I also define the following helper function which transforms SXML to XML and prints it to stdout:

(define (->xml doc . opts) (print (apply sxml-serializer doc opts)))

In SXML, element names usually consist of a qualifying URI and a local name, separated by a colon. This is similar to the universal names described in XML namespaces:

<{http://www.cars.com/xml}part />      <!-- XML universal name -->
(http://www.cars.com/xml:part)         ;; SXML name

XML 1.0 cannot handle such identifiers, because they contain illegal characters. So it does prefix mapping instead:

<cars:part xmlns:cars="http://www.cars.com/xml" />

However, SXML can handle these identifiers directly, as shown above, without any need for prefix mapping. To find the local name, you take everything right of the rightmost colon, in this case part.

Now, the application developer might not like dealing with these long URIs when querying or typing in a document, so SXML provides a way to define a shortcut name for the URI. These shortcuts are defined in the document inside *NAMESPACE* administrative nodes, usually at the top level.

> (->xml
   '(*TOP* (@ (*NAMESPACES* (cars "http://www.cars.com/xml")))
     (cars:part)))
<cars:part xmlns:cars="http://www.cars.com/xml" />

Now you can use cars to mean http://www.cars.com/xml anywhere in the document. Also, invididual elements may have their own local associations:

> (->xml
   '(*TOP*
     (cars:part (@ (@ (*NAMESPACES* (cars "http://www.cars.com/xml")))))
     (cars:part)))
<cars:part xmlns:cars="http://www.cars.com/xml" />
<prfx1:part xmlns:prfx1="cars" />

There, in the first cars:part element and any children, cars stands for http://www.cars.com/xml. Outside of it, the association does not exist.

What's with the second cars:part element, though? It's been rendered as the XML element prfx1:part with the namespace URI cars! Well, that's because we gave no association for the shortcut cars, so the serializer treats cars itself as the qualifying URI. Remember, when SXML elements include a colon, the left side is the full qualifying URI unless you explicitly specify a shortcut association. And when URIs don't have an associated XML prefix, one is generated for you, such as prfx1.

This brings us to our first insight, which is that XML prefixes are not SXML shortcut names. They may look similar, and even overlap sometimes in naming, but the distinction is critical.

Next time, more fun with namespaces.

The atom egg v0.1 for Chicken has been released. It can read and write Atom 1.0 feeds and I am using it to generate the feed for this site.

Recently I have optimized this site and chickadee for display on iPhone and other small displays, using CSS3 @media queries to alter the layout as described at hicksdesign and elsewhere. This site has always sported a fluid layout so the conversion was not difficult. However, I'd like to note a couple things that I didn't see explored sufficiently elsewhere.

1. Use only initial-scale=1 in specifying viewport.

   Most sites seem to recommend the use of

   <meta name="viewport" content="width=device-width" />

   which will make iPhone treat the viewport as 320px across. This is wrong for landscape mode, which is 480px, and will render larger instead of adding to available horizontal space. Instead in a properly fluid layout you should omit an explicit width and use

   <meta name="viewport" content="initial-scale=1" />

   Webkit browsers will then set the width to device-width in portrait mode, and device-height in landscape.

2. Prefer width instead of device-width in media queries not explicitly targeted at iPhone.

   Since width may now be the device width or height, you need not use the orientation property to distinguish between portrait and landscape, but rather just the width of the viewport. For example with the two queries below you can handle both orientations on iPhone, and also handle desktop browsers with small windows:

   @media screen and (max-width: 600px) {}
   @media screen and (max-width: 400px) {}

   Of course you can still target iPhone and iPad directly with max-device-width, for example to increase link size for the touchscreen.

3. Turn off Webkit's automatic font size adjustment.

   Mobile Safari will sometimes increase your font size without your permission, usually in landscape mode. This can be useful for the average website not optimized for mobile use, but is counterproductive in our case. You should turn this off for iPhone:

   @media only screen and (max-device-width:480px) {
     html { -webkit-text-size-adjust: none; }      }

   iPad already has this property set to "none". Also, this directive will cause desktop Safari to ignore the user's text zoom setting, so make sure it is wrapped in the iPhone-only media query as above.

   Edit: Setting the value to "100%" instead of "none" supposedly allows text zoom, avoiding the need for the iPhone media query, but I haven't tested it yet.

My chickadee Chicken documentation server is now operational. It features fast access to Chicken docs and incremental search, in a package that's Lisp-machine-chic. It uses my chicken-doc backend, which also provides docs at the command-line.

I like to change Emacs' selective display indicator—the hidden-line ellipsis in outline and org mode—so it stands out more. But in emacs 23, my original recipe (derived from here) failed because it relied on hardcoded bit-shifting. Below is an improved version that uses glyphs and works in both emacs 22 and 23. It changes the indicator from "..." to " >>>" in yellow text:

(make-face 'invisible-text-ellipsis-face)  ;; arbitrary name
(set-face-foreground 'invisible-text-ellipsis-face "yellow")
(set-face-background 'invisible-text-ellipsis-face nil)
(let ((dot (make-glyph-code ?> 'invisible-text-ellipsis-face)))
    (set-display-table-slot standard-display-table 'selective-display
                            (vector ?\  dot dot dot)))

Until recently I used rsync+hfsmode to back up my Mac to a Linux box. I stopped only because I needed the features of rsync 3, but that lost me the ability to back up extended attributes in AppleDouble format, which can reside on a plain, non-HFS+ filesystem. Anyway, it turns out rsync+hfsmode had been creating corrupt AppleDouble files for three years—ever since I moved from PPC to Intel. So much for testing your backups.

Briefly, it's an endian issue. Use htonl(), guys! So, I wrote a simple script, adouble-fix.pl to detect and repair broken AppleDouble files created by rsync+hfsmode. It's limited to files which contain only Finder Info and resource forks, but this is exactly what the --hfs-mode=appledouble option produces.

$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -n -q
Dry-run finished.  1878 files seen, 1852 fixed, 26 skipped, 0 errors.
Dry-run finished.  1895 files seen, 1641 fixed, 254 skipped, 0 errors.
Dry-run finished.  1511 files seen, 1301 fixed, 210 skipped, 0 errors.
real 0m23.586s  user 0m1.460s  sys 0m1.450s

$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -q
Finished.  1878 files seen, 1852 fixed, 26 skipped, 0 errors.
Finished.  1895 files seen, 1641 fixed, 254 skipped, 0 errors.
Finished.  1511 files seen, 1301 fixed, 210 skipped, 0 errors.
real 0m3.555s  user 0m2.240s  sys 0m0.980s

$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -q
Finished.  1878 files seen, 0 fixed, 1878 skipped, 0 errors.
Finished.  1895 files seen, 0 fixed, 1895 skipped, 0 errors.
Finished.  1511 files seen, 0 fixed, 1511 skipped, 0 errors.
real 0m1.069s  user 0m0.820s  sys 0m0.400s

Computers are fast these days—real fast. One might venture to say too fast, two times fast. Back in my day, you could `type a:autoexec.bat`, drink a sip of coffee, change your mind and Ctrl-C it before you cluttered up your screen with unwanted output. I got nostalgic for those halcyon days of clean screens, and that's why I bought the 640GB Western Digital Caviar Green.

The WD6400AACS is an adequate performer, and runs nice and cool; exactly what I want in a RAID 1. But it was this stellar feature which really caught my eye: after 8 seconds of inactivity, the drive parks its heads. The first subsequent access incurs a half-second pause, which I like to call my "me time".

Eight. Unconfigurable. Seconds. Configuration? No thank you! You'd lose the primary nostalgic benefit: frequent, jarring pauses during interactive use, which is why I bought the thing in the first place!

Let's have a look at an example session with the Caviar Green.

$ ls /usr/share/doc       # what the heck was I looking for?
  [dramatic half-second pause before results appear]
  [you peruse the directory list for 10 seconds; clunk, heads park]
$ cd mutt-1.5; ls         # found it!
  [dramatic half-second pause; exeunt results]
  [you get distracted by an errant cat for a moment; parks heads, clunk]
$ cat README.Debian       # there's my bedtime story
  [dramatic-half-second pause music]
  [you skim the readme for 10 seconds; clunks park, head]
$ date                    # it feels like aeons have passed
  [ironically dramatic half-second pause]
  2300 AD                 # gato, is that you?

Meanwhile, while I pause to gather my thoughts, the drive is continuously falling asleep and being woken up again every 15 seconds anyway as Linux periodically squirts a bit of data at it, causing an absolutely adorable constant clacking sound.

Now, in an alternate universe where I don't actually enjoy using my command-line like it was the gas pedal on an old man's Cadillac, I might scour the internet for hours and come up empty except for an unsupported, DOS-based, placebo utility which only pretended to fix the problem, and eventually resort to a backgrounded while loop touching a file every 7 seconds.

But in this universe, I and Ferris Bueller rate the WD6400AACS a Strong Buy.

Welcome back to 1993.

Thousands of days ago, during my OS/2 phase, I used the mail client PMMail. Recently, I discovered a long-forgotten cache of 1996-era PMMail-format messages, hidden amongst a surplus of hyphens. I thought there perhaps might be something worth reading in there—a poignant reminder of the human mind's infinite capacity for self-delusion—and in an effort to recover these messages, I wrote a trivial converter from PMMail to Maildir format.

Now, one may explore the dark days of college to one's heart's content. Chiefly, one finds that one was once able to write real good English, which that I can't do anymore.

I redid the site in Scheme. You won't notice any difference.

JEdict (4.5.3) does not let you change the font used in its WebKit web browser, nor does it let you specify a custom user stylesheet. Kill two birds with ... well, two stones:

$ defaults write -app JEDict WebKitUserStyleSheetLocationPreferenceKey -string "~/css/jedict.css"
$ defaults write -app JEDict WebKitUserStyleSheetEnabledPreferenceKey -boolean true

A minimal stylesheet for viewing something like 2ch ascii art on OS X might be:

body { font: 14pt "MS-PGothic" !important; }  /* 16.5pt works too -- but not 16 */
table { font-size: 100%; }

I made the following changes to Jonesforth, which are available here:

Tail-call optimization; DOVAR/DOCON -> simplified CONSTANT, VARIABLE, and VALUE; FIG-FORTH-style CREATE ... DOES> support; case-insensitive FIND; Pentium optimizations; SEE aborts on word not found; constant, variable and DOES> decompilation (and indicate primitives). Also added optional space-saving DOES> support, which cannot be decompiled and is probably slower.

I use SQLite3 for my searches database and I was looking for a way to create a frequency table of search terms--either updating the term count or inserting new termsas needed. Evidently, according to this blog post, MySQL has an extension INSERT ... ON DUPLICATE KEY UPDATE which lets you insert or update as needed. The solution in that post for SQLite3 is programmatic: check if an INSERT fails, and do an UPDATE if so.

However, I thought this was not particularly elegant and it also would not work from a trigger. Here is the autovivification solution I came up with:

CREATE TABLE terms(count INTEGER, term TEXT PRIMARY KEY);
...
INSERT OR IGNORE INTO terms(count, term) VALUES (0, 'my term');
UPDATE terms SET count = count + 1 WHERE term = 'my term';

Execute these two statements whenever you want to bump a term's frequency. The ON CONFLICT IGNORE (aka OR IGNORE) causes the INSERT to silently fail on constraint violation; specifically, the uniqueness of the primary key. The base value of 0 ensures the count starts at 1.

Persistent connections exhibited the same issue on spiffy, so I wrote a socket egg which allows you to disable Nagle's algorithm (among other socket options) on Chicken TCP connections:

(http:listen-procedure 
 (lambda (port backlog host)
  (let ((L (tcp-listen port backlog host)))
    (set! (tcp-no-delay (tcp-listener-fileno L)) #t) L)))

Benchmarking lighttpd on OS X with httperf, I found that persistent connections that were fetching files of certain sizes (some big, some small) had their first request satisfied in under 1ms, but experienced a 40ms delay on subsequent requests. Apache did not have this problem. Only one guy on the entire Internet reported seeing something like this, and he got (surprise) no response.

Studying ktrace/strace and tcpdumps from both sides indicated it might be a Nagle problem, and since httperf does disable Nagling, that left lighttpd. Indeed, I found that lighttpd does not (as of 1.4.18) set the TCP_NODELAY flag on its sockets. I guess it assumes the OS disables the Nagle optimization globally, which is generally true on Linux, but not OS X. Or evidently Solaris, since Sun said they patched lighttpd themselves.

I rebuilt the lighttpd from MacPorts and added a small patch:

port uninstall lighttpd
port -d extract lighttpd    # -d to see extract path
(patch network.c with TCP_NODELAY option)
port install lighttpd 

and it solved the issue.

I've started coding for the Dreamcast again recently, and posted a few things in the Dreamcast section. First, serpent, a version of the KOS bubbles demo that runs about five times faster, due to an optimized assembly loop that transforms and sends vertices to the PVR via the store queues. This sparked a discussion on DMA, and a DMA-based variant coded by Dan Potter was added to the examples tree. Second, oceano, which uses the specular highlight feature of the PVR to simulate sparkles on water. It doesn't try to be mathematically correct--rather it's meant as a jumping off point for others. Third, punch, a test harness for benchmarking large polygon performance, originally written to test punch-through polygons. There are also a few patches I wrote along the way, some of which are in the main KOS tree now.

Over the past year I've added several items: brkout, a breakout clone for the Dreamcast; u6edit, my first world editor for Ultima 6; and several musical pieces. Today I added pu6e, which supersedes u6edit and offers many more features.