As I recently mentioned, I’m a big fan of the “Unified Logging Format” for instant messaging logs. Unfortunately, of the two IM clients I use most — Adium on Mac OS X and CenterIM on Linux — only Adium uses it. CenterIM uses a simple flat-file format, delimited with linefeeds and formfeeds.

Since I’d really like to get all my logs in one place, in the same format, I wrote a little Python script to convert CenterIM’s flat-file format into something approximating ULF as implemented by Adium. It’s not perfect, and I’d suggest that persons with weak constitutions and functional programmers not look at the code, but it does seem to work fairly well on my logs. To get an idea of what it does, this is a snippet of a CenterIM log, showing an incoming message followed by an outgoing reply:

^L
IN
MSG
1190126325
1190126325
hey
^L
OUT
MSG
1190126383
1190126383
how's your day going?

With “^L” representing the ASCII form-feed character. In Adium format / ULF, this might appear as:

<chat account="joeblow" service="AIM" version="0.4">
  <message sender="janedoe" time="2007-12-04T14:47:35-0000">hey</message>
  <message sender="joeblow" time="2007-12-04T15:34:38-0000">how's your day going?</message>
</chat>

The major limitations the converter suffers from are a consequence of the differing structure of CenterIM’s logs and Adium’s. CenterIM stores chats in a single file for each contact, with one record for each message sent or received. ULF/Adium use one file per ‘conversation,’ which is apparently all the messages sent or received in a single window (i.e. when you close the window, a new conversation begins on the next message). CenterIM has no concept of conversations, only messages. This means that when you convert a CenterIM log to Adium’s format, Adium sees it as one long conversation, and it appears this way in Adium’s log viewer.

Also, while Adium and ULF store the account names of the conversation participants in the logs, CenterIM simply marks messages as ‘IN’ or ‘OUT’, requiring you to look at the log’s enclosing directory to get the name of the participant. Currently, my script doesn’t do this: it just expects the sender’s and receiver’s account names as command-line arguments.

The syntax is:
$ python cimconverter.py filename yoursn theirsn service
Where filename is the name of the log file you want to convert (usually “history”), yoursn is your screen or account name, theirsn is the account name of the person you had the conversation with, and service is the name of the IM service (AIM, MSN, etc.).

At some point, I will try to fix it so it can grab more of the parameters (at least theirname and service) from the history file’s path. But for now it’s just the bare minimum. I can’t guarantee that the output actually conforms to the ULF specification, since to my knowledge nothing formal exists; however, it does produce output that Adium’s log viewer processes and displays, and that’s basically the de facto standard at the moment.

The “Unified Logging Format” is one of those ideas that you can’t believe hasn’t been done already. Its goal is simple: define a standard format for instant messaging logs that can be used across applications and across platforms, instead of the mishmash of poorly defined, application-specific microformats that exist today. Although currently only one IM program supports it (Adium for Mac OS X), and the informal standardization process seems to have stalled, it’s such a good idea and would benefit so many users that I really hope other developers will see the light.

If you’ve ever switched from one IM client to another you’ve probably had to abandon all your old logs, or keep a copy of the old client around in case you ever wanted to look at them again. With ULF, this wouldn’t happen: you could take the logs from one IM client and move them over happily to a different one. In fact, rather than having each IM application manage and store its logs separately, you could define one location on your system for logs, and let various applications all dump their stuff there. Rather than having to read your logs through whatever minimal reader the client program provides, you could use a purpose-built log viewer (which there would probably be more of, because writing a log viewer is a lot easier when you only have to worry about one format of log files).

Plus, it opens the door to log-file synchronization across multiple systems with ease, even if your computers are running different OSes (and thus use different IM clients). And since ULF is at its core a text-based XML format, it’s far more ‘future proof’ than an application-specific format that’s going to fade into obscurity and become unreadable once the application ceases to be developed.

There’s really no bad here at all.

I made an unwitting discovery earlier this week regarding Apple Mail and its built-in S/MIME functionality, when used in combination with Sen:te Software’s free (and excellent!) GPGMail: for reasons that I can’t quite figure out so far, if you send an encrypted S/MIME message and also sign it using GPG (OpenPGP style, not ASCII-armored), the resulting message will be corrupted and unreadable by the recipient.

I verified this using Apple Mail 2.1 and GPGMail 1.1.

Note that it’s perfectly okay to send a message that’s signed both ways, and I do this frequently. Both signatures will verify on the other end (assuming nothing gets mangled in the mail system). The problem just seems to occur if you try to encapsulate an OpenPGP signed message inside an S/MIME encrypted one.

So far I haven’t tested the reverse, an S/MIME signed message encrypted as OpenPGP and sent that way, because many more of my correspondants use S/MIME than GPG.

As a result of this Slashdot FP, I spent a fair bit of time this afternoon reading up on “Permissive Action Links” or PALs. PALs are the systems which prevent the unauthorized use of nuclear weapons in the U.S. and allied arsenals; they’re the real versions of the ‘arming code’ devices that Hollywood loves so much.

Steven M. Bellovin, a professor at Columbia University in the Computer Science department, has a fascinating page on the topic, excellent not only for its analysis but for the depth of the material it references.

PALs are interesting because they (hopefully) represent the most extreme, highest-stakes use of both physical and electronic security measures. However, in reading about them, it’s easy to see parallels to more mundane scenarios.

Bellovin quotes from Assuring Control of Nuclear Weapons:

There are two basic means of foiling any lock, from an automobile ignition switch to a PAL: the first is to pick it, and the second is to bypass it. From the very beginning of the development of PAL technology, it was recognized that the real challenge was to build a system that afforded protection against the latter threat. … The protective system is designed to foil the probes of the most sophisticated unauthorized user. It is currently believed that even someone who gained possession of such a weapon, had a set of drawings, and enjoyed the technical capability of one of the national laboratories would be unable to successfully cause a detonation without knowing the code.

Does this sound familiar? It should: you could just as easily be describing the hardware design goals of a DRM system like AACS. And why shouldn’t it? A PAL really is just a high-stakes DRM system. The point is to allow access by authorized users who possess a code, while denying others, even if the people you want to deny have access to the whole assembly.

Based on the declassified information available, the PAL consists of a tamper-resistant ‘secure envelope’ or ‘protective skin,’ into which certain arming components are placed. This envelope can be thought of both as a physical and a logical region. It protects the components inside against both physical tampering and remote sensing (X-rays, etc.), as well as informational attacks (brute forcing of the key code, replay attacks); a breach of the envelope results in irreversible disabling of the device. The inputs and outputs from the secure envelope are carefully designed according to the “strong/weak link” principle:

Critical elements of the detonator system are deliberately “weak”, in that they will irreversibly fail if exposed to certain kinds of abnormal environments. A commonly-used example is a capacitor whose components will melt at reasonably low temperatures. The “strong” link provides electrical isolation of the detonation system; it only responds to very particular inputs.

Strong and weak links need not be electromechanical; one could envision similar constructs in modularized software, for instance. In fact, most of the basic concepts of tamper-resistance can be envisioned both literally (as hardware systems; sealed boxes full of pressure and X-ray sensors) and abstractly (modules and their handling of exceptions).

PALs are interesting because they represent the logical conclusion of tamper-resistance systems. It’s my view that if you look at the direction that commercial content-protection systems are going in, and the subsequent cat-and-mouse games with the hacker/cracker community, consumer electronics will begin to increasingly include PAL-like tamperproof elements in the future. Thus, a conceptual understanding of PALs might be exactly the sort of knowledge you’d want to acquire, if your desire was to end-run the inevitable (in my view, given the current climate) next generation of DRM hardware.

Of course, the obvious downside of this is that the same research that a relatively innocent hacker might conduct into the avoidance or circumvention of annoying DRM systems, might also be the same sort of knowledge that you’d need to circumvent the PAL on a nuclear weapon. Regardless of whether this is a valid national security threat, it’s exactly the sort of justification you’d want if your goal was to quash research that was threatening your business (or political-contribution) model. Given the not-infrequent collusion between the industries that benefit from DRM and the government (also c.f. The Attraction of Strong IP by yours truly), I’m not sure this is as farfetched as it might sound.

For the last few days I’ve been fiddling around, trying to solve an annoying problem with an OpenBSD machine (running OBSD 4.1) and my Mac. Basically, what was happening is that when I connected to the OBSD box via a terminal on the Mac, using either Apple’s Terminal.app or the freeware iTerm, after quitting Emacs, all the text subsequently written to the terminal would have the same background color as was set in Emacs. It was as though Emacs just refused to unset the terminal’s background color on exit.

I sought help from the comp.unix.bsd.openbsd.misc newsgroup, and they didn’t let me down. I got two very helpful responses, from Thomas Dickey and Christian Weisgerber: both suggested that it was my choice of ‘xterm-color’ as a terminal type that was to blame.

Changing the TERM setting on my OBSD machine (in “.profile”) to ‘xterm-xfree86’ seemed to do the trick, and now I get a nice colorized Emacs, and it drops back cleanly into the Terminal’s defaults on exit.

Thomas Dickey also gave a link to a terminfo database that’s significantly more up to date than the default one included with OpenBSD; it’s available via FTP at his site here. However, even in his version there’s no specific termcap entry for Apple’s Terminal.app; the best fit still seems to be xterm-xfree86.

The one caveat of all this is that the cursor color seems to be unaffected by whatever’s specified in the “.emacs” configuration file when you connect via a remote (SSH) terminal, so it makes sense to choose an Emacs color scheme that’s similar to your terminal default (or else you may end up with a black cursor on a black background).

If you use a Mac, you may have at some point saved a pointer to an interesting page by dragging its ‘favicon’ (the little icon that sits to the left of a page’s URL in the URL bar in most browsers) to the Finder, which creates a neat little file.

I’d also been doing this, and blindly assumed that the files created in the Finder were standard “.url” files — basically nothing but an ASCII text file containing the page’s address. However, they’re not.

As a quick peek in the Get Info window will show you, they’re actually .webloc files. They’re actually somewhat more complex than basic .url’s.

For starters, the data is XML, formatted as a “PLIST” (one of Apple’s favorite XML schemas). The .webloc file for “http://kadin.sdf-us.org” is shown below.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>URL</key>
        <string>http://kadin.sdf-us.org/</string>
</dict>
</plist>

This is the case in Mac OS 10.4, at least. With previous versions of the OS, it seems that the URL data might have been contained in the file’s resource fork instead, or in addition to, the XML PLIST.

Although normally I’d berate Apple here for ignoring an established de facto standard (the .url file) that works well, the .webloc format is interesting, because it’s easily extended. You could, for instance, encapsulate not just the URL of a page, but its entire HTML contents, or an MD5 hash, into the .webloc, if you wanted to. And, of course, it’s UTF-8 rather than ASCII (and it makes it clear that it’s UTF-8, rather than leaving the determination up to the user’s application), so it has obvious localization advantages.

Email encryption is a topic that comes up frequently in both technical and privacy circles. Pretty much everyone with any sense agrees that it would be a good thing — or at least a better situation than we have right now — if encryption was more widespread and not limited to geeks and the occasional criminal, but exactly how to get encryption into the hands of the masses in a usable form remains a challenge.

One of the problems is that most email-encryption products that offer end-to-end privacy (as opposed to simple transport-layer privacy, like SSL) are designed as part of a traditional desktop MUA, and many people are moving away from POP-based email and desktop MUAs in favor of server-stored messages and webmail.

This presents a problem, since without a desktop MUA, it’s not clear where the encryption/decryption logic will live. Some schemes in the past (e.g. HushMail, at least based on my understanding of how it works) that offer ‘encrypted webmail’ do the message encryption on the server, relying on transport-layer security to get the message to and from the user’s web browser.

This approach is seriously flawed: it requires that the user trust the webmail provider, something I think they probably should be wary of doing. (After all, the webmail provider may not be ‘evil,’ but almost certainly has priorities that are different from those of any randomly-chosen individual user.) Once you send your unencrypted message off to the server, even if it’s via SSL, you really have no idea what becomes of it or who can read it.

For real security, you need to encrypt the message before you let it out of your sight. What’s needed is something that combines the security of end-to-end encryption and client-side logic, with the convenience of webmail.

Naturally, I’m not the first person to have gone down this path. Herbert Hanewinkel, of Hanewin.net, even has a nice example implementation of GPG encryption in Javascript, under a freely-modifiable and re-distributable license. With it, you can plug in a public key, type some text, and have it encrypted for that key, all right in your browser. As he points out, this has several advantages:

• All code is implememented in readable Javascript.
• You can save the page and verify the source code.
• No binaries are loaded from a server or used embedded.
• No hidden transfer of plain text.

As-is, this is a nice way to submit forms (he has a contact form on his site that encrypts the message with his public key and sends it); combined with a matching decryptor, it could be the basis for a secure webmail system that doesn’t require the user to trust their ISP or the mailserver operator. (Sort of, anyway: the user would have to be constantly vigilant that the JS applet that they were being sent was the real thing…)

John Walker at Fourmilab.ch has a more generalized version called Javascrypt that does both encryption and decryption. (Hanewinkel’s encryptor seems to be based on Walker’s, but includes some performance enhancements.) His page also has a nice summary of the benefits of browser-based cryptography and some of its weaknesses and vulnerabilities.

While it would be nice if Google built a JavaScript implementation of GPG into its next version of Gmail, I’m not going to hold my breath (for starters, it would make their business model — basically data-mining all your stored messages — impractical). But I don’t think it would be too difficult to take the examples that are around right now, and work them into some of the more common OSS webmail packages.

A few years ago I typed up a fairly substantial document, in response to what I perceived as a lot of general ignorance concerning the origin of the “right to privacy” in the United States. Although jurisprudence is not my profession, it’s something of an interest of mine, and I tried to sum up a few of the major cases and issues involved. At the very least, my hope is that it will give the lay reader an appreciation for any upcoming Supreme Court cases, or at least allow them to hold their own in a conversation.

My original version was written in late 2002, and never read by anyone but myself. I recently updated it to cover the biggest development between then and now (Lawrence v. Texas, in 2003), and now I’m tossing it online. Please be aware: this is for basic education/entertainment only — it’s not a scholarly work and you certainly shouldn’t cite it anywhere.

It’s available as a Markdown-formatted UTF-8 text document, and minimally-formated XHTML. It’s licensed CC-BY-SA.

Since I think the probability that anyone out there actually reads this thing is fairly low, at least right now, I haven’t bothered to do much in the way of making it easy for readers to contact me. I realize this is semi-obnoxious behaviour, and I’m working to fix it.

I so far have hesitated from putting an email address up because I know it would just become flooded with spam anyway, and because people coming to this site from Slashdot.org can already get an email address for me fairly easily there, and SDF members can simply send me an email address within the system.

But just in case there’s anyone out there who isn’t a member of those two groups, and would like to drop me a message, here’s a ROT-13 encoded address you can feel free to use: “oybt1.xnqva@fcnztbhezrg.pbz”. (Yes, it’s a Spamgourmet address.)

In the very near future, I may set up comments here on the blog. If all goes well and I don’t get too inundated with spam, that will probably be the best way for random passers-by to comment or respond, should they want to.

I have a love/hate relationship with Wikipedia. On one hand, it’s a great project, and I use it — mostly as a ‘casual reference’ for settling friendly arguments or digging up little gems of information — all the time. But on the other hand, sometimes I’m pained by Wikipedia, because I can’t help but look at it and see how much potential it has, for going above and beyond what it is right now. And that’s frustrating.

There have been lots of criticisms of Wikipedia since its inception, and I’m not going to go over the well-trod ground of reliability or factual correctness. Wikipedia is “good enough,” apparently, for lots of people to use, and that’s what matters.

No, what gets me about Wikipedia is its desire to be ‘encyclopediac,’ manifested in its ‘notability’ requirements for articles. I think this is a huge misstep.

Our notions of what an “encyclopedia” is — for all but the very youngest among us — is driven by memories of the old, dead-tree variety. Paper encyclopedias, by their very nature, couldn’t contain everything about everything; it would just be impractical. There isn’t enough paper to print such a beast, and even if there was, certainly you couldn’t economically mass-produce it. So when we think about an encyclopedia, we think about a series of relatively short, introductory articles, on key topics. The best encyclopedias had the most and longest articles — the greatest breadth and depth of content — but they were still limited.

But that’s not what it has to be. That’s a limitation borne of physical restrictions, which don’t necessarily exist in the digital electronic realm, particularly with the ever-falling price of bandwidth and mass storage.

The Wikipedia Gods seem to get this, to a certain extent. One of WP’s tenets is that it’s ‘not paper.’ But despite this, it still sticks to certain key assumptions about what is fit for an encyclopedia, about what an encyclopedia is, that are based on analog premises and ideas.

Put simply, there’s no reason to reject any content that’s well-written and well-researched, on ‘notability’ grounds. There’s just no reason to do it. There is no such thing as bad information, as long as it’s correct.

There are better ways to keep the main namespace clear, and the signal-to-noise ratio high, than by constantly destroying information. Articles that get crufty can (and should!) be rewritten and pared down; cruft can be left in the historical versions for those that want to find it. Articles that get top-heavy with trivia or ‘popular culture’ sections can move the extra content to sub-pages within the main article’s namespace, to preserve the cleanliness of the main page, without deleting anything. The result would be a resource with much more depth in its articles, and potentially much more breadth as well.

Wikipedia as it currently exists strikes me as a terrible waste of potential. Within a generation, Wikipedia and other online resources like it are going to own the concept of ‘encyclopedia’ within the public consciousness. Young people growing up today will probably never think of a stack of large books when they hear that word — yet the online resources are being designed with constraints thoughtlessly inherited from their dead-tree ancestors.

I love Wikipedia for what it is, but sometimes I can’t help but hate it for what it is, too, because of the gap between what it is and what it could and can be.

While reading Ars Technica’s excellent new review of Mac OS 10.5, I ran across a link to IndieHIG, a rather fascinating effort to produce a consistent set of Human Interface Guidelines for the Mac, to replace the official ones that Apple has neglected for so long.

I think it’s a really interesting effort, and I sure hope it’s successful — if only to shame Apple a little into producing a useful HIG. But more than that, if they do produce something that is widely used (and I suspect getting developers to adopt it may be more difficult than coming up with the guidelines themselves), it could be an interesting model for other environments where there is no central authority to promulgate an interface guideline.

After an afternoon of fiddling, I’ve managed to get GNU nano working on my Mac with full UTF-8 support. Although I’m monolingual, this allows the use of nice things like “smartquotes” and emdashes, right in the actual text file.

The version of nano that ships with Mac OS 10.4 doesn’t support Unicode. In order to get it, you’ll need to install Darwinports and then install nano through it, explicitly enabling UTF-8 as you do so. I recommend using the command

sudo port install nano +utf8 +wrap +mouse +color

This enables not only UTF-8, but also support for automatic line-wrapping (which can be disabled on command and is disabled by default for root, for ease in editing config files that don’t like hardwrapped lines), mouse, and color.

Once Darwinports does its thing, you’ll probably want to rename the old version of nano, located in /usr/bin, to something like ‘nano-1.2.4’, and then make a symlink from /usr/bin/nano to the newly installed version in /opt/local/bin/nano. This isn’t required if you have /opt/local/bin in your PATH, but it’s just nice to do in case other users don’t. (Just type sudo ln -s /opt/local/bin/nano /usr/bin/nano)

To make sure it worked, type nano --version and make sure it shows version 2.0.6 or newer, and that --enable-utf8 is shown as one of the compiled options. On my system, the following is shown:

$ nano --version
GNU nano version 2.0.6 (compiled 17:26:26, Oct 27 2007)
Email: nano@nano-editor.org    Web: http://www.nano-editor.org/
Compiled options: --disable-nls --disable-speller 
--disable-wrapping-as-root --enable-color --enable-nanorc --enable-utf8

Now, there’s one final step: you need to set your LANG environment variable to a locale that specifies UTF-8, or nano won’t use it. This is as easy as adding export LANG=en_US.UTF-8 to the bottom of your .bash_profile file. You’ll need to close and reopen your terminal window or SSH session in order for this to take effect.

To test, open nano and try using the curly-open-quote character ( Option-[ ). If you see a quote, it worked. If you see a bunch of question marks, something’s wrong.

A few days after they started rolling it out, the Google Gods finally saw fit to smile on me and grant me the gift of IMAP. And, oh, what a gift it is. Gmail and IMAP are two systems that just seem made for each other, and despite some oddities, I’m shuddering just thinking of POP.

However, I have noticed a few interesting caveats with Gmail. These apply to Gmail as of 10/27/07 used with Apple Mail 2.1.

• You will get many copies of the same outgoing message in your “All mail” folder/store on Gmail, if you have the “Store draft messages on server” option (in Preferences:Accounts:Mailbox Behaviors) set. Basically, every time Mail sends a new draft version to the server (which seems to happen about once a minute or whenever you pause typing), Gmail interprets this and stores it as a new message, which will appear in your “All mail” view. For some people, this might be a feature — looking back, you can actually see the process of composing all your messages. But for most of us, I think it’ll just be clutter and take up space. Better to leave it off.

• You will get duplicate sent messages if you have “Store sent messages on server” checked. This option isn’t necessary as long as you’re using Gmail’s outgoing SMTP server to send messages — you can feel free to uncheck it, and you’ll still get sent messages in your “Sent Mail” folder/view. If you have it checked, when you send a message, Mail will send it twice: once via SMTP, and then it will upload it via IMAP to a ‘Sent Items’ folder that it creates itself (which is not the usual Gmail ‘Sent Items’ view). Since Google is unusually smart and intercepts the outgoing SMTP message, you don’t need the second IMAP copy. Therefore, you probably want this off.

• If you don’t want to mess with Google’s folder structure, turn “Store Junk messages on server” and “Move deleted messages to the Trash mailbox” off also.

• There is no way in Apple Mail to switch an account from POP to IMAP. You just have to delete or mark the POP one as inactive, and then create a new IMAP one. (You may want to back up the POP one first, since deleting it will erase all mail stored under that account on the local system.) At the very least, make sure the IMAP one is working and fully replicated locally before you delete the POP account.

• On all machines except for your main one (and possibly even that one, if you trust Google a lot), you probably want to change the “Keep copies of messages for offline viewing” option on the “Advanced” tab. By default it’s set to download and mirror all mail in the IMAP account, which could be several GB if you’ve had Gmail for a while. For a laptop or other mobile system, you may want to consider “Only messages I’ve read” as an alternative.

• Google seems to do something funny to message attachments, but I’m not clear exactly what. I send all digitally-signed email (generally S/MIME), and I’ve noticed that outgoing messages that I send, which have attachments and are digitally signed, come up with invalid-signature errors when I view them later in my Sent Mail box. (However, they go out fine, and the recipient gets them without errors.) Google is changing something in the message — perhaps the names of the attachments or something — that is causing the signature verification to fail. Be aware of this if you use digital signatures with your Gmail account.

Anyway, that’s all I’ve noticed so far. Mostly just little things; it’s a great improvement to the service overall.

I discovered something interesting about the SDF’s web server today: it doesn’t seem to have a rule set up for serving .xhtml content by default.

I didn’t notice this initially, and my XHTML pages were coming through okay, because they use the “http-equiv content-type” meta element. But a quick check with the Web Sniffer revealed that they were actually being sent with a content-type of text/html, rather than the correct application/xhtml+xml. I would have thought that this would make them fail to validate cleanly, but apparently since the media type is a “SHOULD” rather than a “MUST,” it doesn’t fail. Or maybe it uses the META http-equiv and ignores the one actually coming from the server, for purposes of validation. I’m not sure.

Inserting AddType application/xhtml+xml .xhtml into my .htaccess file in the root ~/html/ directory solved the problem, and now my XHTML pages (mostly static articles produced using MultMarkdown from plaintext) are served correctly.

Anyway, just something for other people to note, I guess. I don’t know what accepted best practice is in this area, but I think I’m going to go and explicitly specify the content-type that I want transmitted for all the various file types that I use on the site, just to make sure; the way things are looking, I’m going to end up with a mix of plain text (including some UTF-8), XHTML, and plain old HTML.

I recently ran across AnimatedKnots.com, and due to always having a soft spot for knot-tying (but absolutely no talent for it), thought it was fairly interesting. They get a gold star for being — as the name suggests — animated, which is a lot more useful than the majority of reference materials around. Most of the ‘animations’ are short (<10 frame) photo series, showing the construction of each. There’s also the ability to step slowly through the procedure, which is nice if you’re slow, like I am.

The site requires JavaScript and is probably best if you actually have a piece of rope to follow along with.

It surprises me a little, given how superior animations are to static images at teaching a skill like this, that such a site didn’t get created a long time ago, but it looks fairly new.

Ever wondered how those guys at the IETF get the plaintext versions of the RFCs looking so darn good? Sure you have. (If you haven’t, well, you might as well skip this, because it’s all downhill from here.) As it turns out, it’s not really that hard. With a little work, you too can create great looking fixed-width plaintext — or at least fake your own RFC.

The backbone of ‘RFC Standard Format’ is ASCII text, 58 lines per page with each page terminated by a form feed, and 72 characters per line with each line terminated by a CR/LF. On top of that basic format, there are headers, footers, and a consistent title page, all of which can be seen on any RFC.

The traditional method of producing RFCs was with the Unix ‘nroff’ program, combined with a few handy addons. This process is documented in RFC 1543, “Instructions to RFC Authors.” Although still usable, this method, which requires a fair bit of manual formatting and doesn’t really separate content from presentation, has since been superceded by XML.

The current best practice is taken from RFC 2629 and its unofficial update. The RFC 2629 DTD is available here.

Basically, RFCs are written in a text editor using some simple, semantically-relevant (i.e. not style-based) elements. The entire document sits inside the <rfc> element, analogous to the <html> element in (X)HTML. There are other elements for the title, author’s name, section and subsection headers — even ASCII art. A template is available to get you started.

When complete, the XML document is fed into a processor and transformed into any number of output formats, including plaintext, HTML, or PDF. This is the beauty of XML: you write what you mean, the processor does the formatting. If you’re interested in looking at how the processor works, it’s all open source.

Since most everything in the RFC process is both well-documented and publicly available, it’s an excellent way to start learning about how to incorporate XML into a flexible workflow. And if you’re feeling up to it, you can pretty easily tweak the RFC DTDs and postprocessing utilities for your own needs.

The Open Content Alliance, better known as “those people who run Archive.org” is finally getting some good press.

Although I’m a bit disappointed that the NYT decided to focus on the negative side of the equation — libraries shunning Google and Microsoft — rather than the positive, it’s nice to see a project like this being mentioned either way. And it brings up a lot of important issues concerning restrictions on digital materials, particularly public-domain materials, being compiled by corporations.

See more ...

As reported by a large segment of the Technorati crowd, Walter Mossberg recently wrote a nice piece on the current state of the U.S. cellphone market, and why it, for lack of a better word, sucks so much. The key is all in the subsidized phone racket.

[The] whole cellphone subsidy game is an archaic remnant of the days when mobile phones were costly novelties. Today, subsidies are a trap for consumers. If subsidies were removed, along with the restrictions that flow from them, the market would quickly produce cheap phones, just as it has produced cheap, unsubsidized versions of every other digital product, from $399 computers to $79 iPods.

I think he’s the first mainstream journalist that I’ve read who has really gotten this. Phone locking, enforced mutual incompatibilities, application restrictions — the entire culture of control — all springs from subsidies. If people just bought their phones outright, they’d probably be significantly cheaper (not to mention more full-featured), there would be a greater secondary market (meaning less waste), and they’d be more prone to shop for networks based on price, service, and quality.

Perhaps as it becomes more obvious that the iPhone is, despite being (in Mossberg’s words) “the best-designed handheld computer ever made,” a costly white elephant because of carrier-mandated restrictions, there will be greater national conversation about the state of cellular telephony.

As Mossberg points out, we’ve been through this with landline phones before, prior to the disassembly of AT&T as the national monopoly carrier. It took almost a century for consumers to get first comfortable with the technology, and then impatient with the restrictions placed upon it: I don’t think cellular will take that long.

Warren Meyer, over at Coyoteblog.com, has a nice little article on what I find to be the most intellectually bothersome part of the modern Democratic/big-government-liberal plank: the constant tension (shall we call it hypocrisy?) between state interest in virtually all aspects of life, and a small number of seemingly arbitrary ‘sacred cows’ where no interference is allowable, e.g. abortion. He sums it up nicely:

The left pushes constantly for expansion of government regulation into every corner of our lives. They are trying to walk a line, a line so narrow I don’t think it even exists, between there being no state interest in 16 year old girls getting abortions without their parents’ knowledge or consent and there being a strong state interest in breast implants, painkillers, seat belt use, bike helmets, tobacco use, fatty foods, etc. They somehow have to make the case that that a woman is fully able to make decisions about an abortion but is not able to make decisions, without significant government regulation and intervention, about her retirement savings, the wages she accepts for her work, her use of a tanning booth, and her choice of painkillers.

Lest anyone think I’m making an anti-choice argument, here, I’m not; I’m as vehement a proponent of a woman’s right to choose as anyone, but I find the intellecual gyrations required to be both pro-choice on abortion but ‘anti-choice’ on drug use or seatbelts a bit much.

A little while ago I finally succeeded in getting mobile internet working through my phone, my Mac, and a USB cable. Although not exactly something that deserves to be in the software configuration hall of fame, the end result — internet on my laptop, via my cellphone’s GPRS connection — has a fairly good ‘wow’ factor. However, it’s not the sort of thing most non-technical people are going to be able to set up easily.

And so, with that audience in mind, I wrote up my experiences in the form of a “mini-HOWTO”. It’s a quick explanation of how to get one particular hardware configuration working, step by step. Since the hardware I’m using isn’t that uncommon (a bog-standard Motorola V3 “Razr” phone, with service though T-Mobile, a Mac laptop running the currently-latest version of the OS, and a mini-USB cable), I hope maybe it’ll be of use to somebody else, somewhere on the internets.

Permanent link here. I tossed it up there as GFDL, so if anyone would like to redistribute it, or use it as the basis for a more general HOWTO, feel free.

(This was originally posted to my Slashdot Journal, which allows comments and is available here.)

So I recently ran across a new site, courtesy of the fine folks at MetaFilter: Rememble. In a nutshell, it’s a sort of ‘digital scrapbooking’ site. It describes itself as “a ‘washing line’ for your digital bits and pieces. Thread together texts, photos, videos, sounds, scribbles, scans, notes, tweets… so they’re not drifting in a digital wasteland.”

See more ...