At some point, Yahoo started sticking a really annoying popup on basically every single Flickr page, if you aren’t logged in with a Yahoo ID. Blocking these popups is reasonably straightforward with uBlock or ABP, but it took me slightly longer than it should have to figure it out.

As usual, here’s the tl;dr version. Add this to your uBlock “My filters”:

! Block annoying Flickr login popups
www.flickr.com##.show.mini-footer.signup-footer

That’s it. Note that this doesn’t really “block” anything, it’s a CSS hiding rule. For this to work you have to ensure that ‘Cosmetic Filters’ in uBlock / uBlock Origin is enabled.

The slightly-longer story as to why this took more than 10 seconds of my time, is because the default uBlock rule that’s created when you right-click on one of the popups and select ‘Block Element’ doesn’t work well. That’s because Yahoo is embedding a bunch of random characters in the CSS for each one, which changes on each page load. (It’s not clear to me whether this is designed expressly to defeat adblockers / popup blockers or not, but it certainly looks a bit like a blackhat tactic.)

Using the uBlock Origin GUI, you have to Ctrl-click (Cmd-click on a Mac) on the top element hiding rule in order to get a ‘genericized’ version of it that removes the full CSS path, and works across page reloads. I’d never dug into any of the advanced features of uBlock Origin before — it’s always just basically worked out of the box, insofar as I needed it to — so this feature was a nice discovery.

Why, exactly, Yahoo is shoving this annoying popup in front of content on virtually every Flickr page, to every non-logged-in viewer, isn’t clear, although we can certainly speculate: Yahoo is probably desperate at this point to get users to log in. Part of their value as a company hinges on the number of active users they can claim. So each person they hard-sell into logging in is some amount more they’ll probably get whenever somebody steps in and buys them.

As a longtime Flickr user, that end can’t come soon enough. It was always disappointing that Flickr sold out to Yahoo at all; somewhere out there, I believe there’s a slightly-less-shitty parallel universe where Google bought Flickr, and Yahoo bought YouTube, and Flickr’s bright and beautiful site culture was saved just as YouTube’s morass of vitrol and intolerance became Yahoo’s problem to moderate. Sadly, we do not live in that universe. (And, let’s be honest, Google would probably have killed off Flickr years ago, along with everything else in their Graveyard of Good Ideas. See also: Google Reader.)

Perhaps once Yahoo is finally sold and broken up for spare parts, someone will realize that Flickr still has some value and put some effort into it, aside from strip-mining it for logins as Yahoo appears to be doing. A man can dream, anyway.

For a while now I’ve been thinking about getting a Kindle. When they first came out, I was pretty negative on the whole concept: expensive, too much vendor lock-in, not enough of a step up over existing product (paper books). Anyway, three years has changed my assessment somewhat.

First, I’ve started to do a lot of traveling again. I say ‘again,’ because I was doing a fair bit of travel back in 2007/08, but then stopped for a while. But it’s now picking up again, and for me, travel means lots of time sitting around in train stations, airports, and on various modes of transport, often without Internet access or AC power. So I do a lot of reading.

Second, the price of the Kindle has come way down. At its original $400 pricetag, I wasn’t interested. But at the current sub-$200 price (and that’s for the 3G model; the base model is only a relatively small step up from a C-note), it goes from ‘expensive toy’ to something I could put in my briefcase and not be obsessively concerned about.

Third, the ebook marketplace has matured. Amazon now has competition, and although there is still a lot of vendor lock-in with the Kindle, you can get ebooks from sources besides AMZ. And that’s really important; I wouldn’t have bought an iPod if the only source for music had been the ITMS (and, I strongly suspect, very few others would have either). Anyone who buys a hardware device that can only be used in conjunction with content purchased from one vendor is a fool.

Now, I’m not saying that the Kindle marketplace is bad or wrong, or that you shouldn’t buy stuff from it. I suspect that if I do get a Kindle, I’ll probably be spending some significant money there. But I wouldn’t even consider the Kindle if that was the only way to load content onto it.

Which brings me around to The Pirate Bay. TPB now has an “E-Books” section — it’s a top-level category, right next to “Music” and “Movies”. If you ever needed a sign that electronic books are here to stay, that’s it. A quick glance seems to suggest that many of them are PDFs, meaning that they’re probably meant for consumption on a computer with an RGB display, rather than an e-ink device like the Kindle or Nook, but e-reader-friendly formats like MOBI and E-PUB (and venerable old ASCII, with a smattering of HTML) seem to be reasonably popular.

The really fascinating is what seems to be getting read. After all, this is a marketplace where the only “cost” to a user is a few minutes of their Internet connection. By sorting the list by either seeders or leechers, we can find out what users are either choosing to redistribute to others, or merely downloading for themselves. The top picks are interesting:

Sorted by leechers, as of 22 November 2010, 0331 UTC:

• Cooking Ingredients

Unless this is some sort of joke or codeword for something else, it’s a 300+ MB scan of an out-of-print chef’s reference from 2002. Apparently there’s a huge unmet demand in the bowels of the Internet for ingredient identification. Who knew?

• Largest fiction library (english), ebooks - 80000 authors -9000

This is pretty interesting. The #2 entry on the list isn’t a book per se, it’s actually a huge collection of books — so big that the contents of the archive is actually provided as a separate torrent. This is presumably so that you can pick out various titles that you want, and tell your BT client to only get the ones you care about. However, I suspect that most people are just grabbing the whole thing, judging by the number of users involved.

There’s a whole separate post for another day, just going through what gets included in archives like this. Real-world librarians have to balance demands like shelf space when deciding what books to keep and which to cull … but someone putting together a torrent doesn’t have any limitation except disk space, and that’s not much of an issue today. This particular collection weighs in at 23.41 GiB, compressed, and claims to unpack to 32 GB. (Depending on the format, the books inside would still be compressed, though.)

I’m pretty sure this has to be bigger than the fiction collections of many local libraries, so the idea that someone can just download something like this in one go is a big mind-blowing.

• The Ultimate Encyclopedia of Wine Beer Spirits & Liqueurs

Another scanned cookbook as a PDF. I think I may actually own this one, on paper, picked up in the bargain section of B&N. What strikes me as odd about this one is that it doesn’t seem like it would translate well to electronic reading on a computer. I leave it sitting out on the bar and flip through it occasionally when I’m looking for a new drink to try out — advances in e-readers aside, that’s still easier on paper than on bits.

If you have a computer in front of you, there are much better options for learning about drinks: sites like WebTender and iDrink can automatically provide you a list of drinks you can make given ingredients at hand, etc. If you were going to go to the work of scanning in a book and pirating it, this wouldn’t be at the top of my list… and yet, hundreds of people are downloading it.

• For Dummies Ebooks A-Z

This makes a bit more sense. It’s another big collection, this time of various “xyz For Dummies” books. Although I still question whether these aren’t squarely in that minority of books that are still better on paper than on a screen, the piracy appeal at least seems more clear. They tend to be fairly expensive and have a short lifespan: I tend to purchase them when I want to learn about a specific product or skill, and then move on to more specialized resources once I’ve gotten the basics down.

• Oxford University Press Ebook Pack 652 Books

Now this is interesting. Not only is Anonymous terribly interested in cooking and mixology, the Internet would also like to know about such varied topics as “The Destruction of Sodom, Gomorrah, and Jericho: Geological, Climatological, and Archaeological Background” and “Lyndon Johnson and the Escalation of the Vietnam War”.

More seriously, I suspect the cachet here is not so much the subject matter — although it does appear to contain so much stuff that you’d have to be a real vegetable to not find something of interest in there — but its perceived value: OUP books tend to be fairly expensive, and if we use $40 as an average figure, works out to a ‘street value’ of over twenty five thousand dollars for the archive. (Naturally, like the RIAA’s figures for the ‘value’ of downloaded MP3 files, that’s ridiculous. First because you could get the paper books used for a fraction of the new price, second because I doubt that more than a rare handful of users getting the PDFs would ever have purchased any of the paper books.)

• Fine Woodworking 2010 - All Issues

Another surprising choice. But the entire back catalog of Fine Woodworking does have a lot more value than most technology-oriented magazines, so perhaps it’s not so strange. A 1996 issue if MacWorld is pretty much only good for laughs and/or nostalgia value, but FW from the same year could still be handy, if it contained a project that you were interested in. As a result, the collection taken as a whole of Fine Woodworking is pretty valuable, in a way that many other magazines wouldn’t be.

It’s not hard to think of other magazines that might be similarly interesting if put together in a big collection; hobbyist mags of the ‘project per month’ format are probably the best candidates, long-form journalism are probably up there too, while purely news magazines would seem to have less value.

• Decision Points - George W. Bush

I’m not sure what it means that this book is the first in-print title to be simultaneously on the Pirate Bay top ebooks list and also the New York Times Best-Seller list (where it is currently #1). On one hand, maybe it cuts across demographics. Or, maybe the demographic represented by TPB’s users are interested in the book’s content, but don’t want to pay for it.

My personal guess is that it’s not politically motivated, and you’d see any book with the amount of press coverage that ‘Decision Points’ is getting represented on the Pirate Bay’s first page. (Tracking TPB versus the NYT would be interesting, though.)

Joking aside, the Pirate Bay’s listings provide a window into a corner of the market that’s under-represented in other measures, like the NYT’s lists. It’s a justified under-representation if all you care about are financial transactions, since this sort of piracy occurs without any money changing hands. But that doesn’t mean it doesn’t both reflect and perhaps affect other aspects of the market.

Given the shift we’re about to see towards ebooks and electronic publishing, you’d have to be foolish to ignore what’s going on outside the squeaky-clean, vendor-approved channels.

According to the powers that be at Flickr, you have until June 1, 2010, 12:00 PM PDT to get your historical referrer-log data, if you are a Pro member and are interested. You can download it as CSVs from the bottom of your stats page.

Apparently it is “not sustainable” to keep the data available forever. I suspect this translates to ‘it’s really expensive and only 0.001% of our users actually care or are even aware of it.’ In the future, they will be providing access to 28 days worth of data via the API, but probably nothing beyond that.

I wasn’t even aware that this feature existed until I saw the notice that it was going away, and although their reasons for terminating it are understandable, it is surprisingly interesting data. My strong recommendation is that, if you’re a Pro member and you have a few megabytes of disk and transfer to spare, you might as well take thirty seconds and download them.

Note to anyone thinking of being clever and using curl or wget to batch-download all the files at once: don’t bother, it’s really not worth your time. (Trust me on this, I looked into it.) You’d have to authenticate using Flickr’s API and it’s guaranteed to take longer than just pointing your browser’s download-destination folder to some appropriate place and Alt-clicking on them.

If you download your logs as CSV files — and really why wouldn’t you? Excel gives you no advantage here — you can use this small Python script I wrote to dump them into a SQLite database. The script requires Python 2.5 or later (or possibly an older version with the appropriate PySQLite add-on package, but I haven’t tested that and probably won’t). Bug reports and enhancements are welcomed although it’s not meant to be pretty, since it won’t be of much use to anyone after June 1. The schema should be obvious just from looking at the script; it’s two tables — one for daily and the other for weekly data — and the columns in the CSVs are all carried over into the DB. There’s no date conversion or other fancy stuff.

What you do with it once you get it there is your business; I haven’t really decided what, if anything, to do with it, but it seemed like having everything in a couple of DB tables was a lot more convenient, whatever I might decide to do with it, than having it in dozens of CSVs. (Do keep the CSVs though. They’re small and there’s no good reason not to.) If you have any suggestions of interesting things to do with or ways to analyze the data, let me know by SDF email or in the comments.

Maybe once they get the API access set up, I’ll write something to grab new stats and shove them into the same SQLite DB on top of the existing records. It would only have to run once a month or so to stay on top of the feed, which isn’t that bad.

I really like Yelp, which is probably why I’ve bothered to spend time typing up reviews for it, despite it being a commercial service that could theoretically pull a CDDB at any time. I’ve found a lot of neat little restaurants that I wouldn’t have otherwise found, particularly while traveling, via Yelp, and in general have found the ratings and reviews there to be of very high quality.

However, I’ve noticed that as Yelp’s userbase has grown and expanded beyond the computer-savvy foodie demographic that seemed to have been some of its first users, the average ratings for a particular business are no longer as useful as they once were. It used to be, if a restaurant had five stars and more than a handful of ratings, it was almost certainly phenomenal. Similarly, if a place was languishing at one or two stars, it was probably best avoided — after all, if a place is bad enough to actually get someone (who isn’t being paid) to spend the time to write a negative review, something must be pretty wrong. And if something was in the middle, chances are it was pretty much just average for whatever cuisine it was trying to represent.

Lately, though, I’ve noticed that many places — and this is especially true of eclectic or “acquired taste” restaurants — are getting pushed towards middling reviews not because anyone is actually rating them that way, but because very good and very bad reviews are being averaged out into two or three stars. This isn’t really surprising: reviewing restaurants is a “matter of taste” practically by definition. But that doesn’t make the result very useful. When I’m looking down the search results in Yelp, I want to know what I am likely to enjoy, not what some hypothetical “average user” is going to like. (I’m not the first to notice this problem, either.)

As more and more users join Yelp and start writing reviews, the average review will naturally start to approach what you’d get from reading the AAA guide, or any other travel or dining guide aimed at a general audience. That’s not necessarily bad, and when you’re writing a travel book or dining guide it’s pretty much exactly what you want: try to give an idea of what most people will think of a particular restaurant.

But that’s certainly not the best that an interactive system can do, not by a long shot. The benefit of a website, as opposed to a book, is that the website doesn’t necessarily have to show the same exact thing to everyone. This is why the front page of Netflix is more useful than the top-ten display down at your local Blockbuster, or why Amazon’s recommendations are typically more interesting than whatever happens to be on the aisle-end display at Borders. It’s not that Blockbuster or Borders aren’t trying — they’re doing the best they can to please everyone. The beauty of a dynamic website is that you don’t have to try to please everyone with the same content; you can produce the content in a way that’s maximally useful to each user.

If Yelp took this approach, ratings from users who tend to like the same things that I do would be weighted more heavily when computing an establishment’s overall score; if you brough up the same restaurant (or if it came up in your search results, more importantly), it might have a different score, if your preferences — as expressed via your reviews — are significantly different than mine. This makes perfect sense, and provided that there’s still some way to see what the overall, unweighted average review was (Netflix shows it in small print below the weighted-average), it’s a no-lose situation from the user’s perspective.

I’m sure that Yelp’s engineers are aware of the Netflix model and how it could be applied to Yelp, so this isn’t a suggestion so much as a hope that it’ll get implemented someday.

Earlier today I read a blog entry by Ben Metcalfe that really hit home. The entry is called “My GMail password scares me with its power,” and I’d like to say that he’s not the only one. Particularly in light of the widespread (and apparently quite successful) phishing attacks going around, it’s a good idea to think about how much of your life and personal information are stored behind that one password, and whether that password is really up to snuff.

Metcalf puts forward what I think is a very modest proposal, which I think boils down to two main points. Neither are trivial, but neither are either one a real stretch on technical grounds:

1. Google ought to allow you to enforce some sort of privilege separation: rather than just having one password for everything, more sensitive services (GMail, Google Checkout, Search History) should be able to be configured to use a separate password. This would ensure that the cached password saved in the chat program you use at work couldn’t be used to log into your mail, or make purchases to the credit card associated with your Google Checkout account.

2. Users who are security-conscious could buy a two-factor authentication token, like an RSA SecureID, to use with some or all Google services. This wouldn’t be mandatory and it wouldn’t be free — so it wouldn’t help the clueless or the broke — but it would let those people who are honestly concerned about security but who lack the ability to replicate Google’s services themselves (and, lets face it, just about nobody can replicate Google’s services at this point) to get that security on top of Google’s offerings.

Perhaps neither are economically feasible right now; too few users may care about security—and be willing to pay for it—to cover the cost that either would mean to Google to implement. But as users put more and more of their data in the hands of managed services like Google’s, and security breaches start having more serious consequences, the demand will come.

In the meantime, what’s a concerned user to do? The best thing you can do is to choose a more secure password. If you don’t mind potentially creating something that you can’t memorize, use a random-password generator and either write the results down, or store it in a ‘password keeper’ program that encrypts its data file with one (good!) master password. I take this latter approach, and use the open-source Password Safe on Windows and Linux, and Password Gorilla (which opens Password Safe database files) on Mac OS X. And, of course, take all the usual precautions against potential phishing attacks.

Until Google sees fit to improve on the one-username/one-password architecture for all its services, that’s about the best you can do.

Earlier today I got an instant message from a friend I haven’t talked to since 2003. Although normally I’d be pleased to hear from an old friend, the fact that the message contained nothing but a link to a web site in the .ru TLD made me suspicious.

Out of curiosity, I grabbed a copy of the page using curl, and then examined it using a text editor. This is the safest way I know of to investigate potentially-hostile web pages; even if the page exploits a flaw in your browser, chances are it’s not designed to exploit a bug in emacs or vi when it’s just being read locally. To no surprise at all, the page was nothing but a bit of JavaScript. (Which is a good reason to browse with something like NoScript enabled.)

Since I’ve just recently started to play around with JS, I thought it would be interesting to take the program apart and see what it does. For safety reasons and because I don’t want to give the malware authors any additional traffic, I’m not going to link to the original Russian site or actually host their index page, but in the interest of science, I’ve put it up on Pastebin for anyone who wishes to poke at. Just be careful and don’t run the thing outside of a sandbox.

Pastebin link to the page’s raw HTML.

They’ve done some (fairly trivial) obfuscation to hide the actual code by way of the two script elements on the page. The first <SCRIPT> defines a Decode function and includes the actual payload in a long string; the second <SCRIPT> calls the decoding function.

Their decoder:

function Decode()
{
    var temp = "", i, c = 0, out = "";
    var str = "60!33!68!79!67!84!89!...blahblahblah";
    l = str.length; 
    while (c <= str.length - 1) {
       while (str.charAt(c) != '!') {
          temp = temp + str.charAt(c++);
       }
       c++;
       out = out + String.fromCharCode(temp);
       temp = "";
   }
   document.write(out);
}
Decode();

Obviously I’ve truncated the value of str here for brevity; it’s several thousand bytes long. What we’re looking for — the actual, presumably-malicious code — is inside that string. There are a number of ways we could get at the contents, but since the malware authors have so helpfully supplied us with a decoder, why not use it? Of course, we don’t want to run it from within a browser, or using any of the online JS shells (which might — stupidly — run the code that’s being obfuscated), but the js CLI shell is a pretty safe option.

If we weren’t absolutely sure what the code was going to do when we ran it, we might want to take additional precautions, like running it inside a walled-off VM, but in this case the code to be executed is trivial.

In order to make Decode() run inside the js CLI shell instead of inside a browser’s JS environment, one small change is necessary: where the code above has document.write(out), we need to change this to a simple print(out). This writes the results to standard output when we run the decoder via js -f badscript.js > badscript.out or something similar.

What we’re left with after running this is the page that the hapless victim actually arrives at, but which the malware author attempted to hide inside the script.

I haven’t had a chance to step through the resulting page completely yet, but it seems like a mess of advertisements combined with scripts designed to make it impossible to close the page. I assume there’s probably more nastiness buried in it besides the obvious, however: since the link was sent to me automatically, it’s a good bet it has a way of propagating itself.

One of the strengths of the Internet and general and the Web in particular is the ease with which it lets an individual set up a site or page on a subject of interest to them, and share it with the world. Although that low barrier to entry opens the flood-gates to thousands of blogs (like this one), it also allows for vast amounts of information to be published on niche subjects by people who are truly passionate about them.

Leadholder.com is a perfect example of this in action. It’s a wonderful site — well-designed, easy to navigate, brimming with information — on a topic that I suspect most people would never cross paths with: a utilitarian drafting and drawing implement called the ‘lead holder.’

Now, I admit I find this sort of thing fascinating — I have an admitted weakness for precision tools in general, and drafting tools most of all — but even if you don’t share quite the level of interest in the subject matter that I do, it’s still a cool example of one of the greatest strengths of the web.

[Found via MeFi’s YoBananaBoy.]

Several months ago I wrote about the legal problems facing electronic ‘alternative currencies’ and the shuttering of one particularly sketchy operation — e-gold-based ‘meta-currency’ 1MDC.

Now it seems that the owners of E-Gold are facing stiff fines and possible prison time after pleading guilty to conspiracy to engage in money laundering and operating an unlicensed money-transmitting business, an indictment E-Gold’s founder once called “a farce.”

Basically, the Feds really didn’t like the core strength of E-Gold, which was that it provided a way to anonymously transfer funds without any sort of user verification. E-Gold didn’t make you prove who you were, and thus there wasn’t any prohibition on how many accounts you could have, which meant that there wasn’t a way to really bar someone from using the service — close down one account, and they could just open up a new one.

Unsurprisingly, the plea agreement includes a “comprehensive money-laundering-detection program that will require verified customer identification” — in short, an end to anonymous transfers.

Although E-Gold never amounted to much in the world of legitimate commerce, and it probably would be little missed by most people if it disappeared completely as a result of the changes, it’s unfortunate and sad to see yet another early-Internet dream — that of anonymous, untraceable electronic currency, immune to the whims of national law or taxation — go (dare I say it) down the tubes.

I was pleased to read today that Netflix has come to its collective senses and decided to save the “Profiles” feature. For those of you living under a rock, Profiles was a neat feature that Netflix offered, allowing you to essentially split your account into ‘sub-accounts’ each with their own queue and number of simultaneous movies. This was pretty nice if you had multiple people (say, family members, or you and a S.O.) sharing the same account.

Their elimination of the feature was ostensibly to simplify the website by removing a feature that few users actually took advantage of, but many felt it was done more to encourage the purchase of multiple accounts (which cost more than one account, even one with many movies at a time).

This is by any measurement a good thing. Netflix avoided doing something very stupid, and alienating its userbase (probably driving more than a few of them right into the arms of the competition, Blockbuster) by announcing its intentions, listening to the response, and then changing their tune when it became obvious they were about to shoot themselves in the foot. All good. This should be a lesson to others on how to craft policy that affects your users.

Unfortunately, they had already disabled access to the feature for most users, apparently in preparation for killing it outright. (Which is a bit of a drag for folks like me, who were holding out because they’d only heard of it as a result of the hubbub and didn’t want to try something that was on its way out.) But according to the official blog, the option to create new profiles will return in a couple of weeks. Here’s hoping.

One of my favorite Google products is Google Notebook, and one of my more frequent uses of it is to keep track of particularly insightful or pithy posts that I read online. Sure, most sites have their own methods for doing this, but Notebook keeps them all in one place. Unfortunately, I never really end up doing much with all the stuff I save.

Earlier today I found myself reading through some of my notes, and thought I’d share a few. Any one of them could be an entry in itself, but honestly I think there’s little I can add to most of them, so I’ll just point you back to the originals and leave it at that.

On Hillary Clinton’s ‘Prayer Breakfasts’, by MetaFilter’s dw:

[…] Hillary attending the prayer meetings is all about triangulation for her. She knows where the business of the GOP elite gets done, so she’s just going to walk right in there. If they were into watching pre-op trans burlesque while drinking paint thinner, Hillary would show up at the door with a copy of The Crying Game and a gallon of turpentine. […]

boubelium had an insightful quip about the difference between politicians and economists:

[…] if a charismatic politician tells you that he has seen the economic future, he hasn’t. He isn’t smart enough or boring enough to undertake the effort.

“Tom Collins” of Tom Collins’ World Wide Web Log — sort of a ‘Fake Steve Jobs’ of the Beltway, with the best understanding of that milieu on the Internet — sums up everything you need to know:

“Veronica, this is the United States of America. With the exception of short period of reform that lasted about forty years during the last century, the entire history of this country has been nothing more or less than the work of lying, thieving, cheating, amoral, greedy, inhuman scum bags.”
“Which means?”
“That, given the chance, you should always go with the lying, thieving, cheating, amoral, greedy, inhuman scum bags. Do that, and you can’t lose - it’s the American Way.”

On a slightly less cynical note, Vorfeed has one of the better comments I’ve read about the gun control ‘debate’ in a while:

[…] A little less than half of US households (and about 25% of all US adults) own at least one gun, and yet only about 30,000 people are killed by them per year, and more than half of those are suicides. … Criminalizing 25% of the country in order to save 30,000 lives is a terrible trade-off — if saving lives is really the issue, we’d do much better if we built a huge public transportation network and then banned cars. … As far as I can tell, the “gun control debate” in this country serves merely to distract from the actual issue — that is to say, the problem is violence, not guns! Rather than myopically concentrating on the instrument used, both sides of the gun debate could probably benefit from some realistic, holistic thinking about ways to mitigate the root causes of violence.