[Originally posted Friday 29 Aug 2008; corrected to fix formatting and typos.]

While flipping through the latest issue of “Cabling and Installation Maintenance” magazine (setting aside all questions of taste in reading material), I noticed a fairly neat product: CATV coax to Category 7A patch cables.

Apparently, the new (draft) ISO/IEC Cat 7A cabling standard has so much available bandwidth — supposedly useful to more than 1GHz — that you can run analog cable TV over it without anything more than a simple balun to convert the 75-ohm unbalanced coax connection to the 100-ohm balanced one used by Cat 7. This isn’t IPTV or digital compression, it’s just running the analog RF signal right over the balanced network wiring.

That’s pretty impressive — in comparison, Cat 5e UTP wiring is only useful up to around 100MHz, and Cat 6 up to 250. And it opens up some neat possibilities for home wiring. Rather than having to decide which rooms you want to run coax to for cable TV, and which rooms to run Cat 5/6 to for data and phone, you could just run one type of cable everywhere. If you want cable TV, just hook it up (in the wiring closet / basement) to the incoming cable line; if you want data, plug it into a switch; if you want POTS, into a punchblock.

Having just spent far too much time screwing around with home wiring, that sounds like a pretty nice proposition.

I decided to do a little playing around earlier this weekend with Python and CGI scripts. Just for something to do, I kludged together a little comment form for this site. It’s not yet operational — I still haven’t figured out how to get ReCaptcha working via a CGI here on the SDF — but it hopefully will show up some day.

Anyway, I ran into a weird issue when trying to write to an “mbox”-format mail spool file using Python. Basically, rather than actually sending email from within my CGI script, I instead just wanted to take the user’s form input and write it to an mbox-style spool file somewhere on the filesystem, for later perusal using an MUA.

In theory, this should be fairly simple. Python comes with a standard library called “mailbox” that’s purpose-built for working with a variety of spool/mailbox file types, and can add messages to them with ease. Unfortunately, I can’t seem to get it to work right; specifically, the message envelope delimiters don’t seem to be getting written correctly.

In an mbox-format spool file, each message is delimited by a string consisting of a newline, the word “From”, and a space. What comes after the word “From” isn’t really that important, but typically it’s the actual ‘From’ address followed by a timestamp. The crucial part in all this is that, with the exception of the very first message in an mbox file, the delimiter line that begins each message must be preceded by a blank line.

In other words, when writing new messages to an mbox file, you need to always start by writing a newline, or else you need to be religious (and check for the presence of) about ending the text of each message with no less than two newline characters, in order to guarantee a blank line at the end. (According to the Qmail docs, the blank line is considered part of the end of the preceding message, rather than part of the ‘From_’ delimiter.)

Supposedly, when you use Python’s mailbox.mboxMessage class in conjunction with mailbox.mbox to create message objects and write them to a file, this should all be handled. However, it doesn’t seem to be working for me.

The code looks something like this (similar lines removed for clarity):

mailmsg = mailbox.mboxMessage()
mailmsg['To'] = 'Kadin'
mailmsg['From'] = formdata['from'].value
# Other headers removed...
mailmsg.set_payload( formdata['message'].value )

mboxfile = mailbox.mbox('/tmp/'+str( datetime.date.today() )+'.mbox',factory=None,create=True)
mboxfile.lock()
mboxfile.add(mailmsg)
mboxfile.unlock

From my reading of the documentation and some similar code samples, this should produce a correctly-formatted mbox file — but it doesn’t. Instead, it produces this:

From MAILER-DAEMON Sun Aug 31 06:48:30 2008
To: Kadin
From: Testuser
Subject: FORMMAIL:Test Subject
Date: Sun Aug 31 02:48:30 2008
Reply-To: test@test.example

Test message would go here.
From MAILER-DAEMON Sun Aug 31 06:48:46 2008
To: Kadin
From: Testuser2
Subject: FORMMAIL:Test Subject 2
Date: Sun Aug 31 02:48:46 2008
Reply-To: test2@test.example

Another message would go here.

Notice that there’s no empty line between the two messages? That means that when the mbox file is parsed by most applications, they don’t see all the messages in the box. Instead, they simply assume that (since there’s no valid delimiters) there’s just one really long message, and display it as such.

While I think I might be able to fix this by just adding a couple of newlines onto the entered text before it gets incorporated into the message object’s payload, that doesn’t seem like how things should have to work. Unless I’m just misunderstanding the mbox format (there are enough varieties of it, so it’s possible), it doesn’t seem like that ought to be required.

Most likely, I’m doing something wrong, but I can’t seem to figure out what … time to throw in the towel and come back to it tomorrow.

For the past several weeks, ever since moving into a new apartment, I’ve been racking my brain (and amassing a vast array of new drill bits) trying to figure out how to wire it for data. After some bad experiences trying to get MythTV to stream MPEG-2 video acceptably over 802.11g, I was convinced that the only network worth having was one built on good old UTP.

In the old apartment, I’d managed to successfully run Ethernet cabling from floor to floor and room to room without a lot of destruction or (almost equally importantly) visibility, by running it through the air ducts, along the plumbing, and through carefully-bored holes in closets and crawl-spaces. Unfortunately, none of the tricks that worked in previous places got me anywhere in the new one. All the ductwork has long horizontal runs and mysterious corners; the plumbing is sealed behind walls; there’s no attic or unfinished basement to run through … it’s just generally not friendly to guerilla networking projects.

In desperation — more than a week of MythTV-less existence was not winning me any friends — I started researching power-line and phone-line networking as alternatives to actually running new cable. Quite a lot has developed since the last time I ran a home network over phone wires (when Farallon’s PhoneNet was high tech), and there are quite a few options available.

The first decision to make is which medium you want to run data over: power lines, phone lines, or coax. Each has advantages and disadvantages.

Cable TV coax provides a high-quality medium for data transmission, but in many homes and apartment buildings that were constructed before cable TV became the norm, coax may only run to one or two locations. Also, the dominant standard for home networking over coax, HomePNA 3.0, supposedly doesn’t coexist with DOCSIS cable modems. That was enough to scare me away, since the last thing I want to do with my home LAN is interfere with my only Internet connection option.

The next-best option for a wired home LAN would seem to be phone wiring. HomePNA is also the dominant standard there, although you could probably cobble something together with VDSL equipment if you could get the gear. Unfortunately, I didn’t find many models available after filtering out the older HomePNA 1.0 and 2.0 devices, which are too slow to really compete with 100BT on Cat5. Apparently the dominant distributor of HomePNA chipsets, CopperGate, is focusing their attention mostly on integrating the technology into IPTV STBs and FiOS gateways. This was one of the few standalone HomePNA-to-Ethernet bridges that I found for sale, and at $83 per unit they’re not cheap.

The other option, and the least elegant in my opinion, is running data over the AC power lines throughout the house. Although they can be prone to creating RF interference, and can have widely varying performance even between different rooms in the same house (or even separate outlets in the same room), they do offer data communication over a basically ubiquitous medium. They’re also some of the easiest devices to find — I found them for sale both in the local computer warehouse (MicroCenter) and Best Buy.

Unfortunately, not all power-line networking devices are created equal. Over the years there have been several (mutually incompatible, naturally) attempts at producing a dominant data-over-mains standard, several of which are available:

• HomePlug 1.0

The HomePlug 1.0 standard operates at 14Mb/s and was an attempt to reduce the number of incompatible vendor-specific protocols that were proliferating a few years ago, before WiFi took off. HomePlug 1.0 devices are available from quite a few vendors, although not all of them mark them as such. They have the benefit today of being relatively cheap and easy to find, but 14Mb (under optimal conditions) is unacceptably slow for what I needed them to do. The Netgear XE102 was among the least-expensive and easiest-to-find devices using HomePlug 1.0. Linksys apparently still sells one, the PLEBR10, but I didn’t see it for sale anywhere.

• HomePlug 1.0 with Turbo

“Turbo” HomePlug 1.0 devices aren’t part of the official HomePlug standard, but exist as a sort of de facto standard because of a feature in a particular Intellon chipset (the INT5500) that was used in many devices. “Turbo” mode provides up to 85Mb/s under optimal conditions, with reports putting real-world performance down around 20-30 megabits. Theoretically, HomePlug 1.0 Turbo devices from various vendors ought to be compatible. As with HomePlug 1.0, not all vendors seem to be forthcoming about labeling their products with the standard they actually use, but as far as I know, 1.0 Turbo devices are the only ones likely to be labeled as “85 Mbps”. Netgear labels their Turbo devices as “85Mbps Powerline”, eschewing the HomePlug branding completely.

• Netgear “Powerline HD”

As far as I can tell, Netgear’s “Powerline HD” is a proprietary protocol used only by a handful of their power-line networking devices. It allegedly provides up to 200Mb/s, but isn’t compatible with 200 megabit devices from other vendors. The HDX101 seemed to be the most common device using this scheme, although there’s also the HDX111 which (despite being called “Powerline HD Plus”) is apparently identical except for providing a pass-thru outlet.

• HomePlug AV

The newest version of the HomePlug multi-vendor standard is the ‘AV’ variant, which provides for speeds up to 200Mb/s (150Mb/s usable, after overhead) under optimal conditions, with QoS and AES encryption. HomePlug AV devices are available from several vendors, and seem to becoming the dominant power-line networking standard, displacing the 85Mb ‘Turbo’ and 200Mb proprietary devices at the top of Linksys’ and Netgear’s lineups. Netgear offers HomePlug AV — calling it “Powerline AV” — in the XAV101, priced at an MSRP of $80 ea. or two for $140 as the XAVB101. Linksys matches this with the PLE200 (PLK200 for the bundle of two), priced similarly.

At this point, I think a person would be foolish to buy anything except the newest HomePlug AV devices, since any of the earlier revisions are likely to become obsolete and hard to find soon. In particular, the proprietary 200Mb devices seem like they should be avoided like the plague.

A few months ago, without giving it a whole lot of thought, I set up my home computers to retrieve my work email, by adding my work account to Apple Mail. This is, as any Apple Mail user can attest, pretty trivial to do. Even the relatively old version of Mail that I use, version 2.1.3 (753.1/753), has built-in support for Exchange servers, and is known to work reasonably well with the beast from Redmond. All you need to do is enter your Exchange server (“Incoming mail server” in Mail), Outlook Web Access server, and SMTP server information, and Mail does the rest.

How exactly Mail deals with Exchange servers is still a bit of a mystery to me; I think it stops short of emulating the Exchange protocol completely, but instead uses a combination of IMAP and HTTP connections; IMAP for some functions (mailbox and personal folders) and WebDAV into the OWA server for others (public folders).

I didn’t give it a whole lot of thought because it ‘just worked’ as soon as I put in the correct server information. My work email started showing up in my Mail inbox, and I considered the project finished. No trouble at all.

At least, no trouble on my end. Several weeks passed, and all the while my Macs were happily connecting to the hosted Exchange service that runs my work’s email (which shall, for the moment anyway, remain nameless), downloading messages and attachments. I even got S/MIME to work without issues. But then, out of the blue, I got an email from the operations department of the email-hosting service, asking me to give them a call right away.

When I called, I learned that something I was doing was causing thousands of “rendering errors” to pop up in their server logs. Initially they thought this was due to a corrupt message, but after checking all my messages (a tedious process), I mentioned that I was using Mail to connect. When I disabled Apple Mail, the errors stopped flowing. When I turned it back on, they restarted.

This, of course, went over pretty much like a fart in church. Since they don’t officially support anything but Outlook on Windows, if the problem couldn’t be resolved, I’d just have to stop using Mail. (Sadly the alternative — get another hosting provider — isn’t really an option.)

So far, I’ve yet to find a solution. It’s made difficult by the fact that the errors don’t seem to affect me on this end — as far as I can tell, Apple Mail works perfectly. But whatever it’s doing on the far end seems to really displease the hosting service’s sysops.

Googling hasn’t turned up anyone else having the same problem, either. This strikes me as odd — given that Apple Mail has a distinct option for connecting to an Exchange server, I doubt I’m the only person to try and use it. Furthermore, the problem seems to be specific to the desktop version of Mail; other people I know who get their email via the iPhone haven’t gotten any nastygrams, so it’s not an all-IMAP or even all-Apple issue. Yet it seems to happen with both of my Macs (both running 10.4 and Mail.app 2.1.3), regardless of whether they’re configured to connect to the hosting service via Exchange or IMAP, and whether they connect from home or some other location.

If anyone has ever experienced this problem, I’d be eager to hear any reports of possible solutions, or even just descriptions of what was happening. (Details on the errors from the server side in particular would be welcome, since the hosting company hasn’t been all that forthcoming with what’s going on.) Given the MS-centricity of the hosting provider, I think that finding my own solution to the issue is going to be the only way to continue using Mail.

The only potential solution I’ve come up with so far is to run a MUA/MTA on my home server (Ubuntu Linux), and have it fetch messages from the hosting provider via IMAP every few minutes, spool them, and then make them available to my Macs using UW-IMAPD or Courier IMAP. This strikes me as a nasty kludge and a possible source of significant delay in receiving messages, but it would at least create a Linux “buffer” between Apple Mail and Exchange. If they can’t be made to play nice with each other, this may be the only way to keep everyone happy.

This is just a quick breakdown, as far as I’ve been able to determine, of which models will allow you to access the ‘net through them, and which are too braindead/broken/Windows-centric. (This is not necessarily an exhaustive list, and is not guaranteed to be correct! Be sure to do your own research before purchasing or signing a contract.)

Options to consider:

• BlackBerry 8800 (GSM)

Reported to work; source is Tom Yager of Enterprise Mac. The discussion forum on Fibble.org also has some instructions. This seems to be the starting point for most of the 88xx variants.

• BlackBerry 8820 (GSM and WiFi)

Reported to work by BlackBerryForums member “CatherineLW”, via Bluetooth only.

• BlackBerry 8830 (CDMA and Euro-GSM)

Reported to work via Bluetooth, not via USB. Follow-up article here. It’s also described as working (in “exactly 2 minutes”) in this article.

• BlackBerry 8300 “Curve” (GSM)

Reported to work. Page includes links to required modem script and information on init strings. In the comments there are intermittent problems reported, so it’s apparently not a foolproof solution.

All information seems to relate to Bluetooth tethering; there’s no mention of success (and lots of failures) trying to tether via USB. Apparently USB tethering is, for some reason, only possible from Windows.

• BlackBerry 8320 (GSM and WiFi)

The 8320 is a special version of the 8300 made for T-Mobile; it includes some additional features including 802.11a/b/g UMA calling.

There are some scattered reports indicating that it works, and some others saying that it doesn’t. It seems like it ought to work; problems may relate to bad software revisions.

• BlackBerry 8100 “Pearl”

Reported as working by Dave Taylor of AskDaveTaylor.com, and Grant Goodale of Fibble.org.

However there are serious issues with particular software revisions. Software version 4.2.1.107 in particular, which was pushed out to T-Mobile phones, is known to have issues.

Options to avoid:

• BlackBerry 8700

For some reason, normal Bluetooth DUN methods don’t work with the 8700 series, which is unfortunate because it’s relatively inexpensive and in all other respects a nice phone (particularly the ‘G’ revision).

There’s a whole saga of efforts, including a substantial ‘bounty’, put towards getting this thing working as a USB or Bluetooth-tethered connection, but there doesn’t seem to be a very satisfactory solution. The closest anyone seems to have gotten is a $50 software package called “Pulse” which allows tethering via a proxy server that you must run (or pay for the use of), through which all traffic flows. Although I appreciate the effort involved, this doesn’t strike me as particularly elegant — frankly it’s unacceptable that it’s even necessary. Anything that requires that much of a workaround to use is broken.

The bottom line:

It’s not really much of a surprise that RIM doesn’t seem to focus very heavily on anything except Windows, given their established userbase in the corporate market, but it’s still a bit disappointing. The best BB device for Mac users at the current time seems to be one of the 8800 series, either the 8800 itself (currently retailing for $280) or the 8820/30 variants depending on whether you want GSM or CDMA service within the US. Either the Curve or the Pearl would seem to be a close second; I only give the 8800 an edge because it’s newer, and will probably be getting more attention for longer than either of the older models.

While this isn’t breaking news or anything, Jeff Starr has a nice tutorial posted over at PerishablePress.com, explaining how to set up a BlackBerry Curve as a Bluetooth DUN device with a Mac. This allows you to connect to the Internet from the Mac via the BlackBerry, provided you have a data plan that supports ‘tethering’. (This includes — to my knowledge anyway — all T-Mobile unlimited data plans including ‘BlackBerry Unlimited’, but only some AT&T plans.)

The solution is of the same form as for most other modern phones: a custom modem script that gets dropped in /Library/Modem Scripts/ and a CID string that tells the phone to open a data connection to the network.

The instructions are specifically for the Curve, aka the 8300, but a very similar procedure works for the 8800, with a slightly different modem script. (Also see this EnterpriseMac article.)