I spent a little time earlier this evening looking at hard drive prices. Since I’m a spreadsheet junkie, ‘comparison shopping’ tends to involve a lot of copying and pasting into Excel or Google Spreadsheet. This was no exception, and the results clearly showed a price-per-GB “sweet spot” in the 750GB drives.

Although we’d expect drives to get cheaper, in terms of capacity per dollar, over time (that’s what all those engineers at Seagate and Hitachi are paid to do, after all), it’s almost always been the case that the cheapest storage trails the technological bleeding edge by a certain amount. Principally I think this is due to the drive manufacturers overpricing the newest drives compared to older ones, in order to squeeze the early adopters for all they’re worth.

Right now, 1TB drives are selling at a slight per-GB premium compared to 750 and 500 GB models; it’s not until you get down into 320 and 250 GB drives that the per-GB price starts to creep back up above them. Hence, if you’re not desperate for the full terabyte today, it’s better to buy a slightly smaller drive and wait a few months for prices to drop some more.

Anyone interested in the actual data can view it either via Google Docs, or as CSV. (The Google Docs is preferable unless you have a burning desire to load it into Excel and do an X/Y plot.)

Google Docs Link
August 2008 Hard Drive Prices - 1kB Comma-Separated ASCII text (CSV)

Several months ago I wrote about the legal problems facing electronic ‘alternative currencies’ and the shuttering of one particularly sketchy operation — e-gold-based ‘meta-currency’ 1MDC.

Now it seems that the owners of E-Gold are facing stiff fines and possible prison time after pleading guilty to conspiracy to engage in money laundering and operating an unlicensed money-transmitting business, an indictment E-Gold’s founder once called “a farce.”

Basically, the Feds really didn’t like the core strength of E-Gold, which was that it provided a way to anonymously transfer funds without any sort of user verification. E-Gold didn’t make you prove who you were, and thus there wasn’t any prohibition on how many accounts you could have, which meant that there wasn’t a way to really bar someone from using the service — close down one account, and they could just open up a new one.

Unsurprisingly, the plea agreement includes a “comprehensive money-laundering-detection program that will require verified customer identification” — in short, an end to anonymous transfers.

Although E-Gold never amounted to much in the world of legitimate commerce, and it probably would be little missed by most people if it disappeared completely as a result of the changes, it’s unfortunate and sad to see yet another early-Internet dream — that of anonymous, untraceable electronic currency, immune to the whims of national law or taxation — go (dare I say it) down the tubes.

Although I suspect that I’m probably among the last to read it, I ran across Richard W. Fisher’s excellent speech to the Commonwealth Club of California, earlier today. Called “Storms on the Horizon”, it was delivered May 28 in San Francisco.

I think it’s worth a read by anyone; despite being a few months old at this point, it’s still quite topical. His main focus is on fiscal (as opposed to monetary) policy, which hasn’t been getting very much attention lately. In particular, he concentrates on the issue of unfunded Social Security and Medicare liabilities, and the effect they will have on the overall government budget deficit.

His general premise — that both Social Security and Medicare, but especially the latter, cost tremendous amounts of money — is not very controversial. Where he splits from the current administration’s party line is over whether we’ll have the ability to pay for them in the not-too-distant future without going into the red.

In keeping with the tradition of rosy scenarios, official budget projections suggest [the current] deficit will be relatively short-lived. They almost always do. […] If you do the math, however, you might be forgiven for sensing that these felicitous projections look a tad dodgy. To reach the projected 2012 surplus, outlays are assumed to rise at a 2.4 percent nominal annual rate over the next four years — almost double the rate of the past seven years. Using spending and revenue growth rates that have actually prevailed in recent years, the 2012 surplus quickly evaporates and becomes a deficit, potentially of several hundred billion dollars.

That deficit is driven in large part by the costs of Social Security and Medicare, which — especially when viewed long-term — are staggering to behold. Fisher gives the net present value of only the unfunded portion of both programs as $99.2 trillion USD; if paid yearly (‘pay-as-you-go’) instead of up front, as they would in a balanced budget, they represent 68% of current income tax revenue.

If that doesn’t give you immediate pause, it should. Particularly as we seem to be headed for an economic downturn, that 68% will only increase if income tax receipts decline. The bottom line is brutal:

No combination of tax hikes and spending cuts, though, will change the total burden borne by current and future generations. For the existing unfunded liabilities to be covered in the end, someone must pay $99.2 trillion more or receive $99.2 trillion less than they have been currently promised. This is a cold, hard fact. The decision we must make is whether to shoulder a substantial portion of that burden today or compel future generations to bear its full weight.

Or, of course, the third path, the one no politician wants to mention: cut back drastically on benefits. In reality I think it’s inevitable that this will be a major part of any solution. Nothing else will work, particularly if there’s a serious recession or depression. Fat chance selling the American public on that, though, especially those who have spent decades paying into a system that was supposedly for their retirement, but was actually being looted by Congress for other purposes.

Fisher warns against the temptation presented by the Mint:

We know from centuries of evidence in countless economies, from ancient Rome to today’s Zimbabwe, that running the printing press to pay off today’s bills leads to much worse problems later on. The inflation that results from the flood of money into the economy turns out to be far worse than the fiscal pain those countries hoped to avoid. […] Even the perception that the Fed is pursuing a cheap-money strategy to accommodate fiscal burdens, should it take root, is a paramount risk to the long-term welfare of the U.S. economy. The Federal Reserve will never let this happen. It is not an option. Ever. Period.

This at least is reassuring — or, rather, it should be. But as many have noted, the Fed has essentially been playing the cheap-money game for a while, and continues to play it today, by stoking the bubble economy with bargain-basement interest rates. While this admittedly isn’t Zimbabwe or Weimar Republic-style money printing, it certainly undermines the Fed’s credibility when it claims to have long-term health rather than short-term painlessness in mind.

Towards the end of the speech, Fisher points the finger at the place where the buck really stops: voters.

When you berate your representatives or senators or presidents for the mess we are in, you are really berating yourself. You elect them. You are the ones who let them get away with burdening your children and grandchildren rather than yourselves with the bill for your entitlement programs.

However, I take a little issue with his conclusion:

Yet no one, Democrat or Republican, enjoys placing our children and grandchildren and their children and grandchildren in harm’s way. […] You have it in your power as the electors of our fiscal authorities to prevent this destruction.

While I appreciate the sentiment (and his need to end on something other than a doom-and-gloom note), I see no evidence to support his assertion that either Democrats, Republicans, or the American public at large have any problem burdening their children and grandchildren in order to get a check cut today. Over and over again, we have seen just that happen. Voters are only too happy to pay Tuesday for their hamburgers today.

The voters have it in their power to prevent a disastrous fiscal policy crisis from taking shape, but they haven’t done so thus far, and I see little reason why that will change at the 11th hour.

As the markets have sunk further and further over the past few weeks, predictions of where we’re headed, either as a nation or the entire world, whether rosy or Mad-Maxian, have flourished. I admit to having a slightly more-than-academic interest in all this — after all, I live here, too — but rather than take my own shot in the dark, I thought it would be more useful to try and round up a few of the most interesting predictions or forward-looking statements made by others.

The diversity of opinion on where we’re headed really can’t be overstated. Although I think the overall outlook is getting bearish, there’s still room for disagreement (or at least there appears, to my eye, to be room for disagreement) as to how bad things are going to get, how quickly they’re going to go, and how long they’re going to take to recover — if they ever will.

Time will just have to tell which were in retrospect prescient and which were following in the great tradition of being somewhat less so.

(Macroeconomic predictions are by their nature almost always incorrect in one respect or another, so this is purely for entertainment, and shouldn’t be used to judge any of the people involved, now or in the future.)

Russ Winter: Super-Bear

Russ Winter, of the Wall Street Examiner’s “Winter Watch” blog, paints a grim picture: corrupt Washington insiders looting the U.S. economy for everything it’s worth, fleecing consumers and taxpayers in order to ensure they’ll live like kings in the coming economic downturn.

This isn’t just a recent bandwagon he’s hopped on, either; he’s been saying it pretty consistently for several years now. Back in 2006 he correctly called BS when Alan Greenspan predicted that the worst of the housing bubble “may well be over.” (The foreclosure rate has only continued to climb since then.) I haven’t looked back much further in his archives, but he’s been writing online since late 2005.

Select Predictions and Statements:

• The current Fannie and Freddie Mac ‘crisis’ is being used as a “Reichstag Fire” to provide an excuse for vast amounts of assets to be transferred to foreign interests, while a small number of cronies reap the profits.

• “I believe an incredibly large amount of American assets and economic capacity will pass fairly quickly into the hands of Pig Men [“the financial sphere, typically brokers, banks, Fed dealers”] interests before Bush leaves office. There is going to be a massive unprecedented rearrangement of the money tree.” Source. (Def. of “Pig Men” from here.)

• “[I]t really does look like the next crisis [after Fannie and Freddie] is Lehman Brothers.” Source. (With a link in the original to this page.)

• “Oil demand numbers in the US and globally are clearly falling off a cliff. Don’t be too surprised to see a panic drop in oil soon enough, maybe on the order of $15-20 in one day to catch up with what has already happened.” Source.

Metafilter’s Own: Malor

Malor, a well-known contributor on MetaFilter, also has a less-than-rosy outlook.

Rather than putting words into his mouth, I’ll quote him directly. The following is excerpted from this post:

[T]he financial system [has become disconnected] from any kind of on-the-ground reality. Stock and house prices have gone far into ridiculous territory, driven there by a combination of stupidly low interest rates and a massive oversupply of what looked like available capital. It also has caused an enormous, gigantic, unbelievable trade imbalance and debt position on the part of the consumer. […]

There is no good outcome here. None. We’ve backed ourselves into a corner from a series of incredibly bad decisions. If the Fed screws up, or if it miraculously realizes that we’re doomed if it doesn’t, we will have a massive deflationary crash, the Second Great Depression. As the debt we’ve taken on goes bad, it will cause deflation, the deflation that has been hidden from us by the monetary games. This is the best possible outcome.

A deflationary crash is one possibility; hyperinflation another (from this post:

If the Fed can keep us on the tracks, [hyperinflation is] inevitably where we’re going to end up. We have too much debt, and to try to hide that fact, the Fed is causing more and more debt to be issued.

This is, he continues, fundamentally flawed:

The US government is doing the same thing Zimbabwe is doing; trying to extract more value out of the economy than the economy can support. We’re already over fifty trillion dollars in debt, in today’s dollars. […] We can’t pay those debts. We can’t pay off our personal debts. And we can’t service the enormous position that other countries have in our currency, which is another, hidden form of debt.

The bottom line:

It’s a house of cards. It has to collapse. Which way it will collapse, I don’t know, but it has to go into either deflation or hyperinflation.

From here, which has photos of each. Unfortunately there’s not an easily downloadable list so you can compare your alcoholism to that of your friends’, so I typed one up. Enjoy.

Best Hotel Bars List: 1.17kB ASCII.

One of the things that’s frustrated me for a while in Emacs is working with diacritics (accented characters) and other international text. Although as a basically monolingual English-speaker I do most of my writing well within the low-ASCII range, every once in a while I find it necessary to reproduce an accented word or string of international text.

Although typing accented characters (and other Latin-1 symbols) is very easy on a Mac in a native editor like TextMate, I’d never spent the time to figure out how to do it in Emacs. However, since Emacs is sort of the least-common-denominator editor, I decided it would be worth figuring out; unlike OS-specific dead-key methods, the Emacs way should work anyplace Emacs is installed. (And I use Emacs regularly on Mac OS X, Windows, Linux, and NetBSD — although the latter two are usually only through SSH sessions.)

Anyway, actually entering accented characters and other basic non-ASCII characters is the easy part. The easiest way is to turn on ‘iso-accents-mode’ within Emacs, and then let it convert character sequences (like “-a for ä) to their Latin-1 equivalent.

The trickier part was getting them to display correctly. The first time I tried using iso-accents-mode, the non-ASCII characters were just displayed as question-mark (?) characters. I quickly traced this to a problem in Emacs, rather than in my terminal (by saving the file and then displaying it with cat, which showed the characters properly), and then with a little more research, to an issue with the “terminal-encoding” parameter.

Basically, Emacs’s “terminal encoding” controls what character set Emacs uses when displaying text (sending it to the terminal device that you’re using to interact with it). It’s distinct from the character set that the file is actually being interpreted using, and also possibly separate from the character set that’s used to interpret keyboard input.

Since I have a UTF-8 terminal (set using the “Window Settings” window, under the Terminal menu, in OS X’s Terminal.app), I set Emacs to use UTF-8 as its terminal encoding by adding the following to my .emacs file:

(set-terminal-coding-system 'utf-8)

With this done (both locally and on the remote systems I SSH into), I was able to see all the non-ASCII characters properly. In fact, not only were Latin-1 characters correctly displayed, but Unicode smartquotes and symbols were also correctly displayed for the first time.

The only issue I anticipate with this is that, when I do connect from a non-UTF-8 terminal (like Cygwin’s Win32 version of rxvt), I’m probably going to get garbage instead of Unicode. However, that’s not really the fault of Emacs, and it’s always possible to temporarily change the terminal encoding back to ASCII if necessary. I just want UTF-8 to be the default.

References:

• Information on permanently setting the terminal-coding-system came from this osdir thread.
• General information on Emacs terminal encoding came from the Emacs documentation, section 27.14, accessible here.
• Also handy is section 27.17 on “Undisplayable Characters”

I ran across a nice blog posting by Steven Frank’s while trolling through Reddit earlier today, and I thought he was right on: “Don’t Use FTP” is pretty good advice for just about anyone.

It’s not that FTP wasn’t a good idea when it was designed; it was nice, it worked, and it served us all well for many years. But it just hasn’t aged well. As Frank points out (see “Note 2” down towards the bottom), although there are many other protocols still in use that were created around the same time, most of them have been extensively updated since then. FTP hasn’t; the defining document for the protocol — insofar as one actually exists — is still RFC 959, written in 1985.

It’s a bit unfortunate that it’s been allowed to languish, because it does serve a need (which is why it’s still around, despite its insecurity and firewall-traversal issues and everything else): it’s a lingua franca for bulk file transfers between systems. It’s certainly better, in theory if not in practice, than abusing port 80 and HTTP for the same purpose. However, given that alternatives (SFTP in particular) exist, there’s really no excuse for using it in new installations or for interacting with a modern hosting environment. Any commercial provider that only offers FTP as a bulk-transfer option should be called publicly onto the carpet; that’s simply not acceptable practice in 2008.

It’s a new week, and for me, that means a new plane trip. And a new plane trip means new reading material.

Having finished Jared Diamond’s (excellent) Collapse — post forthcoming, eventually — I’ve moved on to GMU Professor Rick Shenkman’s book Just How Stupid Are We?

I saw Shenkman on “The Daily Show” a few weeks ago and ordered the book based pretty purely on that; he seemed like an intelligent guy making an interesting point. (Also, I needed something to round out an Amazon order. Yay for free shipping.)

It’s a short book, written in fairly large type. Perhaps this is appropriate given Shenkman’s overall thesis: over the past 50 or 60 years, we as a society have given the ‘American Voter’, otherwise known as ‘The People’ (as in “we the People…”) far too much credit and far too little blame for our policy failures as a nation. In other words, we’re all a lot more stupid than we like to think (and have our leaders tell us) we are.

In our search for places to lay blame, few stones have been left unturned. Bankers, investors, lobbyists, corporate executives, trial lawyers, members of the media, and of course politicians generally have all faced criticism. But only very rarely does anyone take the American people, collectively and as a group, to task for their complicity for the outcomes of government.

It’s a controversial question to ask because most of us have been taught, and probably believe quite sincerely, that “more democracy = better”, and it’s hard to blame the people for much of anything without considering whether that’s necessarily always true. Put bluntly: ‘Is more democracy really better democracy, if the people, by and large, show little-to-no inclination to do anything besides blindly accept whatever they’re told?’ Even raising the question endangers some very sacred American cows, and opens the questioner to accusations of being “undemocratic” or “elitist”.

One thing that I haven’t encountered in the book so far — and I’m about 60% of the way through, and will hopefully finish it later this week — are any proposed solutions to fix the system that we’ve created. It’s all well and good to criticize how we got to where we are, but that doesn’t provide much help in moving forward. So I’m hopeful that he’ll make some suggestions as to how the level of discourse or the system in general can be improved.

I’m holding off overall judgment on the book until I’ve finished it, but in general I thought the premise was pretty good. We’ll see if my feelings change once I make it through the conclusion.

I do most of my Usenet reading through an SSH session using the slrn newsreader, which in my opinion is one of the best around (better than gnus even, although I still use Emacs as an editor). One of the better things about it is its very flexible killfile system. In reality slrn doesn’t have a “killfile” per se, instead it has a “scorefile”, which allows you to apply numeric scores to articles based on regular expressions, killing them when they drop below a threshold.

Anyway, since it allows the use of regular expressions, it’s useful for filtering out “sporgery” and spam designed to defeat less-flexible filtering, like the MI5 Persecution nonsense.

Here’s a set of rules I set up for killing the latest batch of crap:

[*]
% Kill "MI-5 Persecution" crap
Score:: =-9999
   Subject: [A-Z][',-`. ]I[',-`. ]5[',-`. ]P
   From: MI5Victim

The first rule (the Subject: one) is designed for the latest batch, which have varied subject lines and randomly-generated From-addresses. The second rule (the From: one) is for catching the older batch of messages, which all used the same From-address and didn’t vary their headers as much. I keep the old rule around because I sometimes like to read groups where there isn’t much activity, and thus end up seeing them almost as often as I do the new ones.

It’s almost a certainty that the rules will have to be tweaked, or a new rule added, the next time a bunch of messages come out, if the spammer continues to ‘enhance’ the headers to defeat filtering. That is, of course, unless MI5 gets him first. But somehow I doubt we’re all that lucky.

The regexp used to catch the newer messages was taken from Wikipedia, and it seems to work fine, although I’ve been thinking of tweaking it a little more. Ideally I’d like to broaden it until there’s no possible permutations of the subject that wouldn’t get caught, regardless of letters placed in-between the message characters, or any similar-character replacements (e.g. replacing the letter “I” with “|”, or other similar L33T-type stuff).

I’ve only begun playing more seriously with slrn and its scoring features, so as I get a decent scorefile worked out, I’ll probably post some occasional updates, just in case somebody wants to use it as a starting point.

I was pleased to read today that Netflix has come to its collective senses and decided to save the “Profiles” feature. For those of you living under a rock, Profiles was a neat feature that Netflix offered, allowing you to essentially split your account into ‘sub-accounts’ each with their own queue and number of simultaneous movies. This was pretty nice if you had multiple people (say, family members, or you and a S.O.) sharing the same account.

Their elimination of the feature was ostensibly to simplify the website by removing a feature that few users actually took advantage of, but many felt it was done more to encourage the purchase of multiple accounts (which cost more than one account, even one with many movies at a time).

This is by any measurement a good thing. Netflix avoided doing something very stupid, and alienating its userbase (probably driving more than a few of them right into the arms of the competition, Blockbuster) by announcing its intentions, listening to the response, and then changing their tune when it became obvious they were about to shoot themselves in the foot. All good. This should be a lesson to others on how to craft policy that affects your users.

Unfortunately, they had already disabled access to the feature for most users, apparently in preparation for killing it outright. (Which is a bit of a drag for folks like me, who were holding out because they’d only heard of it as a result of the hubbub and didn’t want to try something that was on its way out.) But according to the official blog, the option to create new profiles will return in a couple of weeks. Here’s hoping.