jimhttp://3e8.orgatom egg for Chickentag:3e8.org,2010-07-27:/blog/atom10In EBCDIC we trust.3e8.org2016-04-05T03:45:25Z
<p>This is a short guide to starting up Symbolics Concordia in a Virtual Lisp Machine (VLM) running OpenGenera. It was written because the existing tutorials on the Symbolics VLM teach you the basics of building one and starting it up, but stop after the login prompt.</p>
<p>Concordia is a software documentation tool that produces output that can be read in Genera's Document Examiner, an early hypertext browser. The documentation system was extremely advanced for its time and can still be admired today for its usefulness, thoroughness, and excellent technical writing; it is worth exploring.</p>
<p>We assume you've already started up a VLM and configured it for your site, that the VLM machine is called GENERA, and the VM host system is called GENERA-HOST, as in <a href="https://github.com/ynniv/opengenera/blob/master/README.md">this tutorial</a>. We'll also pretend you used ARIANA for your site name when you defined your site at first boot. You should have run through the Genera Workbook in Document Examiner to become familiar with Genera as well. At the very least, you will need to know where a few important keys are, such as <Select> on F1, <Help> on F12, <Abort> on F6, and <Super> on F7.</p>
<p>Concordia won't be available as an Activity (selectable application) nor will its documentation appear in Document Examiner until you load the Concordia system:</p>
<pre>Command: Load System Concordia</pre><img src="/blog/assets/load-system-concordia-short.png" alt="Depiction of loading the Concordia system" />
<p>The necessary pieces are loaded into memory, including the documentation, and the Concordia activity is made available on <tt><Select> W</tt>. You can see this with <tt><Select> ?</tt>.</p>
<p>Now we need to enable file locking. Without this, Concordia will refuse to save files, and you can't generate any docs. For example, if you try to save (<tt>c-X c-S</tt>) a new buffer to <tt>GENERA‑HOST:/var/lib/symbolics/test.sab</tt>, an error will occur.</p><img src="/blog/assets/concordia-lock-failure.png" alt="An example lock failure error" />
<p>(Press <tt><Super> A</tt> or <tt><Abort></tt> to continue after this error — hopefully, you figured out where your keys are.)</p>
<p>We will enable locking (LOCK-SIMPLE protocol over TCP) on host GENERA (our local VLM) and specify it as a surrogate locking server. This surrogate server will handle locking for servers that do not support the LOCK-SIMPLE protocol, such as our GENERA-HOST file server where the Concordia system and our files reside.</p>
<p>Update the VLM's HOST namespace object to advertise the LOCK-SIMPLE service:</p>
<pre>add services to hosts lock-simple tcp lock-simple GENERA</pre><img src="/blog/assets/add-lock-simple-service.png" alt="Depiction of adding the lock-simple service" />
<p>Enable the LOCK-SIMPLE service:</p>
<pre>enable services lock-simple</pre>
<p>Now set GENERA up as a surrogate locking server by editing the namespace object for the entire site ARIANA, and adding a property which instructs everyone in the site to fall back to GENERA for locking:</p>
<pre>edit namespace object site ARIANA
-> User Property: LOCK-SIMPLE-SERVER GENERA
-> Save Object
-> Quit</pre><img src="/blog/assets/namespace-editor-lock-simple.png" alt="Namespace editor screen showing the lock-simple property" />
<p>Now Concordia can lock and save files.</p><img src="/blog/assets/concordia-lock-success.png" alt="A successful lock message" />
<p>At this point you can read the Symbolics Concordia Workbook through Document Examiner, which walks you through creating a simple document. Below, <Select> should be on the F1 key.</p>
<pre><Select> D
Show Documentation Symbolics Concordia Workbook
<Select> W
; start creating documentation!</pre>
<p>Here is an example of the record you'll create in the first section of the workbook.</p><img src="/blog/assets/concordia-workbook-record.png" alt="Example Concordia workbook record" />
<p>The lock-simple service runs on TCP port :666. For further details, try:</p>
<pre>Load System version-control-doc
show doc The Lock-simple Network Service
show doc Symbolics Concordia Installation Guide</pre>tag:3e8.org,2016-04-04:/blog/symbolics-concordia-in-a-virtual-lisp-machine2016-04-04T22:29:00-05:00Symbolics Concordia in a Virtual Lisp Machine2016-04-04T22:29:00-05:00
<p>Continuing from my <a href="/blog/2012/11/28/qemu-linux-test-image-with-serial-console/">previous post</a>, another way to redirect output to a serial console is to use Google's <a href="http://code.google.com/p/sgabios">sgabios</a> option ROM, which mirrors BIOS output from the screen to a serial port. Just place this ROM alongside qemu's other roms (e.g. in <tt>/usr/share/qemu</tt>) and start qemu with <tt>-device sga</tt>.</p>
<p>Since this works for LILO, we can get into the LILO boot prompt on the serial console, and start Linux with <tt>linux console=ttyS0</tt>. Once Linux boots up we can modify lilo.conf and run <tt>lilo</tt>. This avoids some of the rigamarole of the previous method, although guestfish is still useful to edit lilo.conf, due to the lack of an editor in the test image.</p>
<p>However, there's no prebuilt sgabios package for Ubuntu 12.04, and qemu 1.0 doesn't come with sgabios included. So I built sgabios along with a .deb that drops it into qemu's ROMs directory.</p>
<p>I now present to you: </p>
<ul>
<li><a href="https://gist.github.com/4163790">a script which downloads, builds and packages sgabios</a>, </li>
<li><a href="/pub/sgabios/qemu-sgabios_0+svn8_all.deb">the qemu-sgabios .deb</a>, and </li>
<li><a href="/pub/sgabios/sgabios.bin">the ROM file itself</a>.</li></ul>tag:3e8.org,2012-11-28:/blog/qemu-and-sgabios2012-11-28T14:27:00-06:00QEMU and sgabios2012-11-28T14:27:00-06:00
<p>As provided, <a href="http://wiki.qemu.org/download/linux-0.2.img.bz2">QEMU's test linux image</a> expects to be on either a graphical or curses-based terminal. Let's say you'd like to patch the image to use the serial console instead. To make this more interesting, let's not resort to booting in graphical or curses mode to set up the serial console in the first place.</p>
<p>First, grab the test image.</p>
<pre class="prettyprint lang-sh">
$ mkdir linux-test && cd linux-test
$ curl http://wiki.qemu.org/download/linux-0.2.img.bz2 | bunzip2 > linux-0.2.img
</pre>
<p>The image uses LILO; to get LILO to interact with the serial port, we have to add <tt>serial=0,9600n8</tt> to lilo.conf. We also need to get the kernel to use the serial line by appending <tt>console=ttyS0</tt> to the kernel arguments in lilo.conf.</p>
<p>One problem: the image doesn't come with an editor (not even <i>ed</i>, man!). Also, we have no interactive console yet either way. So, in order to manipulate the disk image directly, we use the amazing <a href="http://libguestfs.org">guestfish</a>. </p>
<pre>
$ guestfish -i -a linux-0.2.img
><fs> <b>edit /etc/lilo.conf</b></pre>
<p>Now we can edit lilo.conf to add the serial and console arguments. Updated, it looks something like:</p>
<pre>
boot = /dev/hda
<b>serial = 0,9600n8</b>
root = /dev/hda
# [... 5 lines omitted ...]
append = "<b>console=ttyS0</b> sb=0x220,5,1,5 ide2=noprobe ide3=noprobe ide4=noprobe ide5=noprobe"
image = /boot/vmlinuz-2.6.20
label = linux
</pre>
<p>It might have been nice to automate this editing with guestfish's support for <a href="http://augeas.net">augeas</a>, but unfortunately augeas lacks a lens for lilo.conf.</p>
<p>Anyway, we still have to execute <tt>lilo</tt> to update the boot sector. We can try to do this via guestfish as well; the guestfish instance is running a mini-VM attached to your disk, and you can actually run code located on your disk within the mini-VM.</p>
<pre>><fs> command lilo
libguestfs: error: command: Fatal: open /dev/hda: No such file or directory</pre>
<p>Ugh. Our disk in the guestfish VM is located at <tt>/dev/vda</tt>, not <tt>hda</tt>. Well, we can fake it with a symlink:</p>
<pre>><fs> command "ln -s /dev/vda /dev/hda"
><fs> command lilo
libguestfs: error: command: Fatal: Sorry, don't know how to handle device 0xfd00</pre>
<p>By device 0xfd00 it means <tt>/dev/vda</tt> (major 253, minor 0); apparently it doesn't like virtblk devices. It might work by explicitly specifying the geometry, or upgrading LILO, but let's try another method.</p>
<p>What we'll do is use QEMU's support for directly loading the kernel and bypassing the boot loader. We can copy the kernel out of the image, and launch our VM using that external kernel, pointing it to the serial console. We can then run LILO to update the boot sector.</p>
<p>(Naturally, using this method, we could skip messing around with LILO at all, and just start up the VM this way all the time. But we've come this far, so let's make the change permanent.)</p>
<pre>><fs> copy-out /boot/vmlinuz-2.6.20 .
><fs> quit
$ ls -l vmlinuz-2.6.20
-rw-r----- 1 jim jim 2040204 Nov 27 22:16 vmlinuz-2.6.20
$ kvm -nographic -hda linux-0.2.img -kernel vmlinuz-2.6.20 \
-append "console=ttyS0 root=/dev/hda"</pre>
<p>The machine should boot up and print its output to the serial console (your terminal). Run lilo when you get the login shell:</p>
<pre>sh-2.05b# lilo
Added linux *
sh-2.05b# exit</pre>
<p>Now this image has been permanently converted to communicate with the serial console. Boot it like so and you'll see both the LILO and kernel output on your terminal:</p>
<pre>$ kvm -nographic -hda linux-0.2.img</pre>tag:3e8.org,2012-11-28:/blog/qemu-linux-test-image-with-serial-console2012-11-28T00:45:00-06:00QEMU linux test image with serial console2012-11-28T00:45:00-06:00
<p>Via <a href="https://github.com/rtomayko/shocco">Shocco</a>, a neat little technique for templating in shell. The idea is to encapsulate a here document in a function, and use the function arguments (<tt>$1 $2 ...</tt>) as the template variables. The cool part is the use of <tt>$(cat)</tt> in the template body, allowing you to pipe in data to the function and have it wrapped in the output. This is extremely useful in the middle of a pipeline; much of shocco is, in fact, a gigantic pipeline.</p>
<p>Here's a simple example. We assume the input is already properly escaped for HTML.</p>
<pre class="prettyprint lang-sh">
layout() {
cat <<HTML
<html><head>
<title>$1</title>
<body><p>$(cat)</p></body>
</head></html>
HTML
}
echo 'My body is ready' | layout 'Status'
</pre>
<p>Output:</p>
<pre class="prettyprint lang-html">
<html><head>
<title>Status</title>
<body><p>My body is ready</p></body>
</head></html></pre>
<p>Shocco is shoc-full (sorry) of interesting shell techniques and I'd recommend checking it out if you're at all into shell programming.</p>tag:3e8.org,2012-11-20:/blog/shell-templates2012-11-20T21:33:00-06:00shell templates2012-11-20T21:33:00-06:00
<p>In jQuery 1.4.3 and later, you can use <tt>.data()</tt> to access the value of <a href="http://www.w3.org/TR/html5/elements.html#embedding-custom-non-visible-data-with-the-data-attributes">custom data attributes</a> on your HTML5 elements. jQuery tries to figure out what type of data you're passing and do the appropriate result conversion, falling back to string if it can't figure it out. This is convenient, but a problem can occur when you want to pass a string but your value looks like (for example) a boolean:
<pre><input id='abc' data-foo='true'>
$('#abc').data('foo') /* true (boolean) */</pre>
<p>It's not possible to quote the string value either; you'll just get a string with embedded quotes. The official workaround is to use <tt>.attr('data-foo')</tt>, which will return the raw string. Of course, this means that to safely pass a string without interpretation, you can't accept any other object. <a href="http://bugs.jquery.com/ticket/7579">Some</a> <a href="http://bugs.jquery.com/ticket/7231">people</a> are unhappy with this behavior, but... <i>wontfix</i>.</p>
<p>However, there's another trick you can use. Pass a quoted string inside a 1-element array, then reference that element. This will ensure the JSON parser is engaged because the value starts with <tt>[</tt>. It also lets you pass arbitrary objects you do want converted (in which case you couldn't use <tt>.attr()</tt>). Example:</p>
<pre>
<input id="abc" data-foo='["true"]' data-bar='[true]'>
$('#abc').data('foo')[0] /* "true" (string) */
$('#abc').data('bar')[0] /* true (boolean) */</pre>
<p>It's a little clunky, but at least you'll get consistent results. If you're passing several values, it's probably easier just to use a JSON hash.</p></p>tag:3e8.org,2011-12-13:/blog/jquery-and-html5-data-attribute-conversion2011-12-13T19:00:00-06:00jQuery and HTML5 data attribute conversion2011-12-13T19:00:00-06:00
<p><a href="http://freshmeat.net/projects/ccze/">ccze</a> is a nice way to colorize your logs. The defaults are tuned more for following logs in a dedicated window, than for viewing them inline at a prompt. I use an alias like this:</p>
<pre class="prettyprint lang-sh">alias cz='(ccze -m ansi | less -MnFRX)'</pre>
<p>allowing usage like</p>
<pre class="prettyprint lang-sh">$ cz < /var/log/messages
$ tail -n 20 /var/log/syslog | cz
$ dmesg | tail | cz</pre>tag:3e8.org,2011-08-07:/blog/ccze2011-08-07T16:10:00-05:00ccze2011-08-07T16:10:00-05:00
<p>While working on the <a href="https://github.com/ursetto/zbguide">zguide</a> for the <a href="http://api.call-cc.org/doc/zmq">zmq</a> egg in <a href="http://call-cc.org">Chicken</a>, I encountered a problem with
terminating a process in a timely manner when it has a SIGTERM handler installed.</p>
<pre class="prettyprint lang-scm">(use posix)
(on-exit (lambda () (print "exiting")))
(set-signal-handler! signal/term
(lambda (s) (print "terminating") (exit)))
(read)
(print "finished")</pre>
<p>If you compile and run this program, and <tt>kill -TERM</tt> it from
another window, nothing happens right away. You have to hit <tt>Enter</tt>
afterwards, and then it will print <tt>terminating</tt> and <tt>exiting</tt>.</p>
<p>The reason for this is that the posix unit installs signals with
<tt>signal(3)</tt>, which sets the <tt>sigaction</tt> flag <tt>SA_RESTART</tt> under the hood.
"Slow" system calls (<tt>read</tt>, <tt>zmq_recv</tt>) which have received no input
yet will automatically restart when the signal handler returns.
Chicken has a single global signal handler that just sets a flag
indicating which signal was received, schedules the interrupt handler
to run in a moment, and returns immediately. The syscall is then
restarted before the interrupt handler gets a chance to run.</p>
<p>To hack around this behavior you can add</p>
<pre class="prettyprint lang-scm">(foreign-code "siginterrupt(SIGTERM, 1);")</pre>
<p>to the top of your file, so slow syscalls will immediately exit with
<tt>EINTR</tt> even when no input is available, and your handler will be
invoked. To wit:</p>
<pre class="prettyprint lang-scm">(foreign-code "siginterrupt(SIGTERM, 1);")
(use posix)
(on-exit (lambda () (print "exiting...")))
(set-signal-handler! signal/term
(lambda (s) (print "terminating...") (exit)))
(read)
(print "finished")</pre>
<p>If you do not explicitly hook <tt>SIGTERM</tt>, a <tt>TERM</tt> signal will interrupt <tt>read</tt> immediately and terminate. This is true on Linux and Mac OS X
at least, but it's not clear to me that this can be relied upon.</p>
<p>You can see this behavior as well with <tt>SIGINT</tt>. By default Ctrl-C
terminates a <tt>read</tt> immediately. However, when the posix egg is
loaded, it hooks Ctrl-C with <tt>signal(3)</tt> and so Ctrl-C will not
immediately terminate a <tt>read</tt>.</p>
<pre class="prettyprint lang-scm"> (use posix)
(read)
^C^C^C^C^C^C</pre>
<p>Instead, you have to press <tt>Enter</tt> afterward (as line buffering is
active by default). To terminate immediately you can do:</p>
<pre class="prettyprint lang-scm"> (use posix)
(foreign-code "siginterrupt(SIGINT, 1);")
(read)</pre>
<p>This situation might need to be addressed with an egg that provides more
nuanced signal handling than the posix unit.</p>
<p>Final note: It's not possible to restore the default signal handler
(<tt>SIG_DFL</tt>) with the posix unit, either. Passing #f to <tt>set-signal-handler!</tt> will ignore the signal (<tt>SIG_IGN</tt>).</p>tag:3e8.org,2011-08-02:/blog/take-off-every-sigterm2011-08-02T12:34:56-05:00Take off every SIGTERM2011-08-02T12:34:56-05:00
<p>The stock SXML->XML serializer from <a href="http://www.modis.ispras.ru/Lizorkin/sxml-tutorial.html#htoc32">sxml-tools</a> has a couple aesthetic issues related to namespace output.</p>
<ol>
<li>It doesn't support default namespaces at all, which makes the already-verbose XML positively prolix, and could cause some non-conformant XML processors to fail.</li>
<li>It does not allow redeclarations of XML prefixes, so you may sometimes get an autogenerated prefix name even when you provided a mapping. (This was a "design goal," though.)</li>
<li>It does not support declaring all prefixes in the root element, which in certain cases can elevate the natural redundancy of XML to dizzying heights.</li></ol>
<p>I added support for 1. and 2. in version 0.2 of the <a href="http://3e8.org/chickadee/sxml-serializer">sxml-serializer</a> egg, released yesterday. No.3 unfortunately will take some time to think about, as the code is geared to declare prefixes as locally as possible.</p>
<p>So I was going to write a blog post with a lot of examples and in-depth explanation, but instead, I just documented the egg! See <a href="http://3e8.org/chickadee/sxml-serializer#sec:The_default_namespace">The default namespace</a> and <a href="http://3e8.org/chickadee/sxml-serializer#sec:Redeclaring_XML_prefixes">Redeclaring XML prefixes</a> for more details.</p>
<p>Here's a preview, though. The change introduces a new <tt>*default*</tt> pseudo-namespace which we can use to map any number of URIs to the default namespace. This works with nested elements and also handles the empty namespace correctly. Below is a pretend Atom document that is rendered without any prefixes:</p>
<pre class="prettyprint lang-scm">> (serialize-sxml
'(*TOP* (@ (*NAMESPACES*
(atom "http://www.w3.org/2005/Atom")
(xhtml "http://www.w3.org/1999/xhtml")))
(atom:feed (atom:entry
(atom:content (@ (type "xhtml"))
(xhtml:div (xhtml:p "I'm invincible!"))))))
ns-prefixes: '((*default* . "http://www.w3.org/2005/Atom")
(*default* . "http://www.w3.org/1999/xhtml")))
</pre>
<pre class="prettyprint lang-xml"><feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<p>I'm invincible!</p>
</div>
</content>
</entry>
</feed></pre>
<p>Instead, if we omit the <tt>*default*</tt> mappings from <tt>ns-prefixes</tt> -- or just use the stock serializer -- every element is prefixed:</p>
<pre class="prettyprint lang-xml"><atom:feed xmlns:atom="http://www.w3.org/2005/Atom">
<atom:entry>
<atom:content type="xhtml">
<xhtml:div xmlns:xhtml="http://www.w3.org/1999/xhtml">
<xhtml:p>I'm invincible!</xhtml:p>
</xhtml:div>
</atom:content>
</atom:entry>
</atom:feed></pre>
<p>I like to call this "terse mode," because irony is the spice of life.</p>tag:3e8.org,2010-08-01:/blog/default-namespaces-in-sxml2010-08-01T23:36:20-05:00Default namespaces in SXML2010-08-01T23:36:20-05:00
<p><a href="http://3e8.org/blog/2010/07/30/namespaces-in-sxml-part-1/">Last time</a>, we talked about the distinction between SXML shortcut names and XML prefixes, and particularly about the *NAMESPACES* node. This time let's talk about the prefixes you pass to the parser and the serializer.</p>
<p>The default SSAX parser in Chicken is <a href="http://3e8.org/chickadee/ssax#def:ssax:xml-.3esxml">ssax:xml->sxml</a>. It accepts an alist that maps user namespace shortcuts (symbols) to namespaces (URI strings). Let's see what happens if we pass it a null list:</p>
<pre>> (ssax:xml->sxml
(open-input-string
"<cars:part xmlns:cars=\"http://www.cars.com/xml\" />")
'())</pre>
<pre>(*TOP* (http://www.cars.com/xml:part))</pre>
<p>Because we didn't give it an association between shortcuts and URIs, the parser returns fully-qualified names. It completely discards the XML prefix in the original document, because the prefix is only meaningful for languages that cannot qualify element names with URIs (i.e. XML 1.0). An important consequence is that you cannot recreate the original XML document prefixes without additional information provided by the user.</p>
<p>Now let's associate the shortcut prefix <tt>cs</tt> to the namespace URI for the XML prefix <tt>cars</tt>, and parse the same document:</p>
<pre>> (ssax:xml->sxml
(open-input-string
"<cars:part xmlns:cars=\"http://www.cars.com/xml\" />")
'((cs . "http://www.cars.com/xml")))</pre>
<pre>(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
(cs:part))</pre>
<p>I used <tt>cs</tt> here so that you could see that the prefix <tt>cars</tt> is completely irrelevant to the output, except on the XML side in determining the universal name. It is not even retained in the SXML document. The fully qualified SXML name <tt>http://www.cars.com/xml:part</tt> is now, via the user's association, changed to the short name <tt>cs:part</tt>. This association is also recorded in the *NAMESPACES* node so that the namespace URI can be reconstructed when transformed back to XML. Notice that during parsing, the alist actually maps <i>from</i> URI strings <i>to</i> shortcut symbols, which is the opposite of what you'd expect.</p>
<p>Let's now consider transforming the two documents above back to XML. In the previous post, we saw the erroneous case where we tried to use a shortcut namespace without defining that association, so the shortcut was treated as a full namespace URI:</p>
<pre>> (->xml '(*TOP* (cars:part)))
<prfx1:part xmlns:prfx1="cars" /></pre>
<p>However, the first SXML document we produced above is valid, and will produce valid output:</p>
<pre>> (->xml '(*TOP* (http://www.cars.com/xml:part)))
<prfx1:part xmlns:prfx1="http://www.cars.com/xml" /></pre>
<p>There's no assocation to an XML prefix, though, so <tt>prfx1</tt> is generated for you. This is a perfectly legal XML document, and is readable by any conformant parser, even though it looks strange.</p>
<p>We can pass another association list, this time to the serializer, to map URI strings to XML prefixes. Below, the namespace URI <tt>http://www.cars.com/xml</tt> is mapped to the XML prefix <tt>cars</tt>, instead of the auto-generated <tt>prfx1</tt>.</p>
<pre>> (->xml '(*TOP* (http://www.cars.com/xml:part))
ns-prefixes: '((cars . "http://www.cars.com/xml")))
<cars:part xmlns:cars="http://www.cars.com/xml" /></pre>
<p>It's important to remember that the alist you pass to the serializer in <tt>ns-prefixes</tt> <i>does not map shortcut prefixes to URIs</i>. That's the job of the alist passed to the parser--or more accurately, the job of the *NAMESPACES* node. <tt>ns-prefixes</tt> only maps URIs to XML prefixes. Returning to the erroneous example above,</p>
<pre>> (->xml '(*TOP* (cars:part))
ns-prefixes: '((cars . "http://www.cars.com/xml")))
<prfx1:part xmlns:prfx1="cars" /></pre>
<p>we can see <tt>ns-prefixes</tt> has no effect. There is no such namespace URI as <tt>http://www.cars.com/xml</tt> in that document, nor does <tt>ns-prefixes</tt> create a shortcut mapping to that URI.</p>
<p>Turning to the second example which was generated by the SSAX parser,</p>
<pre>> (->xml '(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
(cs:part)))
<cs:part xmlns:cs="http://www.cars.com/xml" /></pre>
<p>we can see there is a namespace association in *NAMESPACES* from the shortcut <tt>cs</tt> to the URI <tt>http://www.cars.com/xml</tt>. Recall this association was created by the user when calling the parser, not by the XML document. During serialization, this assocation causes the shortcut namespace <tt>cs</tt> to be translated to the URI <tt>http://www.cars.com/xml</tt>. However, we still need to determine the resulting XML prefix. By default, the serializer conveniently uses the shortcut name <tt>cs</tt> as the XML prefix!</p>
<p>That was, however, just the default. If we actually provide a mapping from URI to XML prefix in <tt>ns-prefixes</tt>, that will override the default:</p>
<pre>> (->xml '(*TOP* (@ (*NAMESPACES* (cs "http://www.cars.com/xml")))
(cs:part))
ns-prefixes: '((cars . "http://www.cars.com/xml")))
<cars:part xmlns:cars="http://www.cars.com/xml" /></pre>
<p>So the shortcut <tt>cs</tt> is translated to the namespace URI <tt>http://www.cars.com/xml</tt> via *NAMESPACES*, and the URI is then mapped to the XML prefix <tt>cars</tt> via <tt>ns-prefixes</tt>. Again, notice how <tt>ns-prefixes</tt> contains no reference to the shortcut <tt>cs</tt>, because it never sees the shortcut names; it only sees the namespace after expansion into a URI.</p>
<p>Confused yet? Let me sum up.</p>
<ul>
<li>When parsing XML to SXML, XML prefixed names are mapped to universal names (local names qualified with URIs), based on the xmlns attributes in the document. Optionally, these URIs are mapped to shortcut names based on an alist passed by the user, and this mapping is also stored in the *NAMESPACES* node.</li>
<li>When serializing SXML to XML, shortcut names are mapped back to URI namespace strings using the associations stored in the *NAMESPACES* node, and all URIs are then mapped to XML prefixes based on a user-provided alist. Since universal names are illegal in XML 1.0, all URIs <i>must</i> be mapped to prefixes; therefore, automatic prefixes are created if you do not provide a mapping.</li></ul>
<p>In the next installment, we'll take a look at a more complex example.</p>tag:3e8.org,2010-07-31:/blog/namespaces-in-sxml-part-22010-07-31T13:40:00-05:00Namespaces in SXML, part 22010-07-31T13:40:00-05:00
<p>Namespaces in SXML are tricky. The SSAX SXML parser takes a list of namespace prefix to URI assocations; the SXML document itself contains *NAMESPACE* nodes mapping prefixes to URIs; and the <a href="http://www.modis.ispras.ru/Lizorkin/sxml-tutorial.html#hevea:serializ">SXML serializer</a> takes a list mapping prefixes to URIs as well! How does it all fit together?</p>
<p>In the discussion below I will be using the <a href="http://3e8.org/chickadee/sxml-serializer">sxml-serializer</a> egg for <a href="http://call-cc.org">Chicken</a>. I also define the following helper function which transforms SXML to XML and prints it to stdout:</p>
<pre>(define (->xml doc . opts) (print (apply sxml-serializer doc opts)))</pre>
<p>In SXML, element names usually consist of a qualifying URI and a local name, separated by a colon. This is similar to the universal names described in <a href="http://www.jclark.com/xml/xmlns.htm">XML namespaces</a>:</p>
<pre><{http://www.cars.com/xml}part /> <!-- XML universal name -->
(http://www.cars.com/xml:part) ;; SXML name</pre>
<p>XML 1.0 cannot handle such identifiers, because they contain illegal characters. So it does prefix mapping instead:</p>
<pre><cars:part xmlns:cars="http://www.cars.com/xml" /></pre>
<p>However, SXML can handle these identifiers directly, as shown above, without any need for prefix mapping. To find the local name, you take everything right of the rightmost colon, in this case <tt>part</tt>.</p>
<p>Now, the application developer might not like dealing with these long URIs when querying or typing in a document, so SXML provides a way to define a shortcut name for the URI. These shortcuts are defined in the document inside *NAMESPACE* administrative nodes, usually at the top level.</p>
<pre>> (->xml
'(*TOP* (@ (*NAMESPACES* (cars "http://www.cars.com/xml")))
(cars:part)))
<cars:part xmlns:cars="http://www.cars.com/xml" /></pre>
<p>Now you can use <tt>cars</tt> to mean <tt>http://www.cars.com/xml</tt> anywhere in the document. Also, invididual elements may have their own local associations:</p>
<pre>> (->xml
'(*TOP*
(cars:part (@ (@ (*NAMESPACES* (cars "http://www.cars.com/xml")))))
(cars:part)))
<cars:part xmlns:cars="http://www.cars.com/xml" />
<prfx1:part xmlns:prfx1="cars" /></pre>
<p>There, in the first <tt>cars:part</tt> element and any children, <tt>cars</tt> stands for <tt>http://www.cars.com/xml</tt>. Outside of it, the association does not exist.</p>
<p>What's with the second <tt>cars:part</tt> element, though? It's been rendered as the XML element <tt>prfx1:part</tt> with the namespace URI <tt>cars</tt>! Well, that's because we gave no association for the shortcut <tt>cars</tt>, so the serializer treats <tt>cars</tt> itself as the qualifying URI. Remember, when SXML elements include a colon, the left side is the full qualifying URI unless you explicitly specify a shortcut association. And when URIs don't have an associated XML prefix, one is generated for you, such as <tt>prfx1</tt>.</p>
<p>This brings us to our first insight, which is that <i>XML prefixes are not SXML shortcut names</i>. They may look similar, and even overlap sometimes in naming, but the distinction is critical.</p>
<p>Next time, more fun with namespaces.</p>tag:3e8.org,2010-07-30:/blog/namespaces-in-sxml-part-12010-07-30T17:35:00-05:00Namespaces in SXML, part 12010-07-30T17:35:00-05:00
<p>The <a href="http://3e8.org/chickadee/atom">atom egg</a> v0.1 for Chicken has been released. It can read and write Atom 1.0 feeds and I am using it to generate the <a href="http://3e8.org/blog/atom10.xml">feed</a> for this site.</p>tag:3e8.org,2010-07-27:/blog/atomized2010-07-27T22:55:04-05:00Atomized2010-07-27T22:55:04-05:00
<p>Recently I have optimized this site and <a href="/chickadee">chickadee</a> for display on iPhone and other small displays, using CSS3 @media queries to alter the layout as described at <a href="http://hicksdesign.co.uk/journal/finally-a-fluid-hicksdesign">hicksdesign</a> and elsewhere. This site has always sported a fluid layout so the conversion was not difficult. However, I'd like to note a couple things that I didn't see explored sufficiently elsewhere.</p>
<ol>
<li>Use only <tt>initial-scale=1</tt> in specifying viewport.
<p>Most sites seem to recommend the use of </p>
<pre class="prettyprint lang-html"><meta name="viewport" content="width=device-width" /></pre>
<p>which will make iPhone treat the viewport as 320px across. This is wrong for landscape mode, which is 480px, and will render larger instead of adding to available horizontal space. Instead in a properly fluid layout you should omit an explicit width and use</p>
<pre class="prettyprint lang-html"><meta name="viewport" content="initial-scale=1" /></pre>
<p>Webkit browsers will then set the width to <tt>device-width</tt> in portrait mode, and <tt>device-height</tt> in landscape.</p></li>
<li>Prefer <tt>width</tt> instead of <tt>device-width</tt> in media queries not explicitly targeted at iPhone.
<p>Since width may now be the device width <em>or</em> height, you need not use the <tt>orientation</tt> property to distinguish between portrait and landscape, but rather just the width of the viewport. For example with the two queries below you can handle both orientations on iPhone, and also handle desktop browsers with small windows:</p>
<pre class="prettyprint lang-css">@media screen and (max-width: 600px) {}
@media screen and (max-width: 400px) {}</pre>
<p>Of course you can still target iPhone and iPad directly with <tt>max-device-width</tt>, for example to increase link size for the touchscreen.</p></li>
<li>Turn off Webkit's automatic font size adjustment.
<p>Mobile Safari will sometimes increase your font size without your permission, usually in landscape mode. This can be useful for the average website not optimized for mobile use, but is counterproductive in our case. You should turn this off for iPhone:</p>
<pre class="prettyprint lang-css">@media only screen and (max-device-width:480px) {
html { -webkit-text-size-adjust: none; } }</pre>
<p>iPad already has this property set to "none". Also, this directive will cause desktop Safari to <em>ignore the user's text zoom setting</em>, so make sure it is wrapped in the iPhone-only media query as above.</p>
<p>Edit: Setting the value to "100%" instead of "none" supposedly allows text zoom, avoiding the need for the iPhone media query, but I haven't tested it yet.</p></li></ol>tag:3e8.org,2010-07-07:/blog/phoning-it-in2010-07-07T00:00:00-06:00Phoning it in2010-07-07T00:00:00-06:00
<p>My <a href="/chickadee">chickadee</a> Chicken documentation server is now operational. It features fast access to Chicken docs and incremental search, in a package that's Lisp-machine-chic. It uses my <a href="/chickadee/chicken-doc">chicken-doc</a> backend, which also provides docs at the command-line.</p>tag:3e8.org,2010-05-10:/blog/hello-bird2010-05-10T00:00:00-06:00Hello, bird2010-05-10T00:00:00-06:00
<p>I like to change Emacs' selective display indicator—the hidden-line ellipsis in outline and org mode—so it stands out more. But in emacs 23, my original recipe (<a href="http://groups.google.com/group/gnu.emacs.help/browse_thread/thread/e284ff4f77286ba4/095c51bf22c3368e?pli=1">derived from here</a>) failed because it relied on hardcoded bit-shifting. Below is an improved version that uses glyphs and works in both emacs 22 and 23. It changes the indicator from "..." to " >>>" in yellow text:</p>
<pre class="prettyprint lang-el">
(make-face 'invisible-text-ellipsis-face) ;; arbitrary name
(set-face-foreground 'invisible-text-ellipsis-face "yellow")
(set-face-background 'invisible-text-ellipsis-face nil)
(let ((dot (make-glyph-code ?> 'invisible-text-ellipsis-face)))
(set-display-table-slot standard-display-table 'selective-display
(vector ?\ dot dot dot)))
</pre>tag:3e8.org,2010-02-28:/blog/selective-service2010-02-28T00:00:00-06:00Selective service2010-02-28T00:00:00-06:00
<p>Until recently I used <a href="http://www.quesera.com/reynhout/misc/rsync+hfsmode/">rsync+hfsmode</a> to back up my Mac to a Linux box. I stopped only because I needed the features of rsync 3, but that lost me the ability to back up extended attributes in AppleDouble format, which can reside on a plain, non-HFS+ filesystem. Anyway, it turns out rsync+hfsmode had been creating corrupt AppleDouble files for three years—ever since I moved from PPC to Intel. So much for testing your backups.</p>
<p>Briefly, it's an endian issue. <i>Use htonl(), guys!</i> So, I wrote <a href="/pub/adouble-fix.pl">a simple script, adouble-fix.pl</a> to detect and repair broken AppleDouble files created by rsync+hfsmode. It's limited to files which contain only Finder Info and resource forks, but this is exactly what the --hfs-mode=appledouble option produces.</p>
<pre>$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -n -q
Dry-run finished. 1878 files seen, 1852 fixed, 26 skipped, 0 errors.
Dry-run finished. 1895 files seen, 1641 fixed, 254 skipped, 0 errors.
Dry-run finished. 1511 files seen, 1301 fixed, 210 skipped, 0 errors.
real 0m23.586s user 0m1.460s sys 0m1.450s
$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -q
Finished. 1878 files seen, 1852 fixed, 26 skipped, 0 errors.
Finished. 1895 files seen, 1641 fixed, 254 skipped, 0 errors.
Finished. 1511 files seen, 1301 fixed, 210 skipped, 0 errors.
real 0m3.555s user 0m2.240s sys 0m0.980s
$ time find 4 -name "._*" -type f -print0 | xargs -0 adouble-fix.pl -q
Finished. 1878 files seen, 0 fixed, 1878 skipped, 0 errors.
Finished. 1895 files seen, 0 fixed, 1895 skipped, 0 errors.
Finished. 1511 files seen, 0 fixed, 1511 skipped, 0 errors.
real 0m1.069s user 0m0.820s sys 0m0.400s</pre>tag:3e8.org,2009-07-24:/blog/lilliput2009-07-24T00:00:00-06:00Lilliput2009-07-24T00:00:00-06:00
<p>Computers are fast these days—real fast. One might venture to say <i>too fast</i>, two times fast. Back in my day, you could <tt>`type a:autoexec.bat`</tt>, drink a sip of coffee, change your mind and Ctrl-C it before you cluttered up your screen with unwanted output. I got nostalgic for those halcyon days of clean screens, and that's why I bought the 640GB Western Digital Caviar Green.</p>
<p>The WD6400AACS is an adequate performer, and runs nice and cool; exactly what I want in a RAID 1. But it was <i>this</i> stellar feature which really caught my eye: after 8 seconds of inactivity, the drive parks its heads. The first subsequent access incurs a half-second pause, which I like to call my "me time".</p>
<p>Eight. Unconfigurable. Seconds. Configuration? No thank you! You'd lose the primary nostalgic benefit: frequent, jarring pauses during interactive use, which is why I bought the thing in the first place!</p>
<p>Let's have a look at an example session with the Caviar Green.</p>
<pre>$ ls /usr/share/doc # what the heck was I looking for?
[dramatic half-second pause before results appear]
[you peruse the directory list for 10 seconds; clunk, heads park]
$ cd mutt-1.5; ls # found it!
[dramatic half-second pause; exeunt results]
[you get distracted by an errant cat for a moment; parks heads, clunk]
$ cat README.Debian # there's my bedtime story
[dramatic-half-second pause music]
[you skim the readme for 10 seconds; clunks park, head]
$ date # it feels like aeons have passed
[ironically dramatic half-second pause]
2300 AD # gato, is that you?
</pre>
<p>Meanwhile, while I pause to gather my thoughts, the drive is continuously falling asleep and being woken up again every 15 seconds anyway as Linux periodically squirts a bit of data at it, causing an absolutely adorable constant clacking sound.</p>
<p>Now, in an alternate universe where I don't actually enjoy using my command-line like it was the gas pedal on an old man's Cadillac, I might scour the internet for hours and come up empty except for an unsupported, DOS-based, placebo utility which only pretended to fix the problem, and eventually resort to a backgrounded while loop touching a file every 7 seconds.</p>
<p>But in this universe, I and Ferris Bueller rate the WD6400AACS a Strong Buy.</p>tag:3e8.org,2009-07-17:/blog/zeos-rip2009-07-17T00:00:00-06:00Zeos, RIP2009-07-17T00:00:00-06:00
<p><a href="gopher://3e8.org">Welcome back</a> to <a href="http://chicken.wiki.br/eggref/4/phricken">1993</a>.</p>tag:3e8.org,2009-02-14:/blog/walentyn2009-02-14T00:00:00-06:00Walentyn2009-02-14T00:00:00-06:00
<p>Thousands of days ago, during my OS/2 phase, I used the mail client <a href="http://pmmail.os2voice.org/index.php?title=PMMail_for_OS/2">PMMail</a>. Recently, I discovered a long-forgotten cache of 1996-era PMMail-format messages, hidden amongst a surplus of hyphens. I thought there perhaps might be something worth reading in there—a poignant reminder of the human mind's infinite capacity for self-delusion—and in an effort to recover these messages, I wrote a trivial <a href="pub/pmmail-to-maildir.pl">converter from PMMail to Maildir format</a>.</p>
<p>Now, one may explore the dark days of college to one's heart's content. Chiefly, one finds that one was once able to write real good English, which that I can't do anymore.</p>tag:3e8.org,2009-01-22:/blog/heiraten2009-01-22T00:00:00-06:00Heiraten2009-01-22T00:00:00-06:00
<p>I redid the site <a href="src/">in Scheme</a>. You won't notice any difference.</p>tag:3e8.org,2008-06-04:/blog/shift-command-z2008-06-04T00:00:00-06:00Shift-Command-Z2008-06-04T00:00:00-06:00
<p>JEdict (4.5.3) does not let you change the font used in its WebKit web browser, nor does it let you specify a custom user stylesheet. Kill two birds with ... well, two stones:
</p>
<pre>$ defaults write -app JEDict WebKitUserStyleSheetLocationPreferenceKey -string "~/css/jedict.css"
$ defaults write -app JEDict WebKitUserStyleSheetEnabledPreferenceKey -boolean true</pre>
<p>A minimal stylesheet for viewing something like 2ch ascii art on OS X might be:</p>
<pre class="prettyprint lang-css">body { font: 14pt "MS-PGothic" !important; } /* 16.5pt works too -- but not 16 */
table { font-size: 100%; }
</pre>tag:3e8.org,2008-05-24:/blog/the-aa-vote2008-05-24T00:00:00-06:00The AA vote2008-05-24T00:00:00-06:00
<p>I made the following changes to <a href="http://annexia.org/forth">Jonesforth</a>, which are <a href="/pub/jonesforth-modified.tar.gz">available here</a>:</p>
<p>Tail-call optimization; <tt>DOVAR/DOCON</tt> -> simplified <tt>CONSTANT</tt>, <tt>VARIABLE</tt>, and <tt>VALUE</tt>; FIG-FORTH-style <tt>CREATE ... DOES></tt> support; case-insensitive <tt>FIND</tt>; Pentium optimizations; <tt>SEE</tt> aborts on word not found; constant, variable and <tt>DOES></tt> decompilation (and indicate primitives). Also added optional space-saving <tt>DOES></tt> support, which cannot be decompiled and is probably slower.</p>tag:3e8.org,2008-03-28:/blog/jonesing2008-03-28T00:00:00-06:00Jonesing2008-03-28T00:00:00-06:00
<p>I use SQLite3 for my searches database and I was looking for a way to create a frequency table of search terms--either updating the term count or inserting new termsas needed. Evidently, according to <a href="http://blog.modp.com/2007/11/sqlite3-and-on-duplicate-key-update.html">this blog post</a>, MySQL has an extension <tt>INSERT ... ON DUPLICATE KEY UPDATE</tt> which lets you insert or update as needed. The solution in that post for SQLite3 is programmatic: check if an INSERT fails, and do an UPDATE if so.</p>
<p>However, I thought this was not particularly elegant and it also would not work from a trigger. Here is the autovivification solution I came up with:</p>
<pre class="prettyprint lang-sql">CREATE TABLE terms(count INTEGER, term TEXT PRIMARY KEY);
...
INSERT OR IGNORE INTO terms(count, term) VALUES (0, 'my term');
UPDATE terms SET count = count + 1 WHERE term = 'my term';
</pre>
<p>Execute these two statements whenever you want to bump a term's frequency. The <tt>ON CONFLICT IGNORE</tt> (aka <tt>OR IGNORE</tt>) causes the INSERT to silently fail on constraint violation; specifically, the uniqueness of the primary key. The base value of 0 ensures the count starts at 1.</p>tag:3e8.org,2008-02-29:/blog/schaltjahr2008-02-29T00:00:00-06:00Schaltjahr2008-02-29T00:00:00-06:00
<p>Persistent connections exhibited the same issue on <a href="http://chicken.wiki.br/spiffy">spiffy</a>, so I wrote a <a href="http://chicken.wiki.br/socket">socket egg</a> which allows you to disable Nagle's algorithm (among other socket options) on Chicken TCP connections:</p>
<pre class="prettyprint lang-scm">(http:listen-procedure
(lambda (port backlog host)
(let ((L (tcp-listen port backlog host)))
<i>(set! (tcp-no-delay (tcp-listener-fileno L)) #t)</i> L)))</pre>tag:3e8.org,2008-02-25:/blog/and-your-little-spiffy-too2008-02-25T00:00:00-06:00And your little spiffy, too2008-02-25T00:00:00-06:00
<p>Benchmarking lighttpd on OS X with httperf, I found that persistent connections that were fetching files of certain sizes (some big, some small) had their first request satisfied in under 1ms, but experienced a 40ms delay on subsequent requests. Apache did not have this problem. Only <a href="http://www.webmasterworld.com/apache/3320954.htm"> one guy on the entire Internet</a> reported seeing something like this, and he got (surprise) no response.</p>
<p>Studying ktrace/strace and tcpdumps from both sides indicated it might be a Nagle problem, and since httperf does disable Nagling, that left lighttpd. Indeed, I found that lighttpd does not (as of 1.4.18) set the TCP_NODELAY flag on its sockets. I guess it assumes the OS disables the Nagle optimization globally, which is generally true on Linux, but not OS X. Or evidently Solaris, since <a href="http://www.sun.com/servers/coolthreads/tnb/lighttpd.jsp">Sun said</a> they patched lighttpd themselves.
</p>
<p>I rebuilt the lighttpd from MacPorts and added a small patch:
</p>
<pre>port uninstall lighttpd
port -d extract lighttpd # -d to see extract path
<a href="/pub/lighttpd-1.4.18-tcpnodelay.diff">(patch network.c with TCP_NODELAY option)</a>
port install lighttpd
</pre>
<p>and it solved the issue.</p>tag:3e8.org,2008-02-18:/blog/lighttpds-naggling-issue2008-02-18T00:00:00-06:00lighttpd's naggling issue2008-02-18T00:00:00-06:00
<p>I've started coding for the Dreamcast again recently, and posted a few things in the Dreamcast section. First, <a href="/hacks/dc/#demos">serpent</a>, a version of the KOS bubbles demo that runs about five times faster, due to an optimized assembly loop that transforms and sends vertices to the PVR via the store queues. This sparked a discussion on DMA, and a DMA-based variant coded by Dan Potter was added to the examples tree. Second, <a href="/hacks/dc/#demos">oceano</a>, which uses the specular highlight feature of the PVR to simulate sparkles on water. It doesn't try to be mathematically correct--rather it's meant as a jumping off point for others. Third, <a href="/hacks/dc/#demos">punch</a>, a test harness for benchmarking large polygon performance, originally written to test punch-through polygons. There are also a few <a href="/hacks/dc/#patches">patches</a> I wrote along the way, some of which are in the main KOS tree now.</p>tag:3e8.org,2004-02-18:/blog/dreamcast-hacking2004-02-18T00:00:00-06:00Dreamcast hacking.2004-02-18T00:00:00-06:00
<p>Over the past year I've added several items: <a href="/hacks/brkout">brkout</a>, a breakout clone for the Dreamcast; <a href="/hacks/ultima6/">u6edit</a>, my first world editor for Ultima 6; and several <a href="/music/">musical pieces</a>. Today I added <a href="/hacks/ultima6">pu6e</a>, which supersedes u6edit and offers many more features.</p>tag:3e8.org,2003-04-30:/blog/summary-of-additions2003-04-30T00:00:00-06:00Summary of additions.2003-04-30T00:00:00-06:00