Bjarni says!

Last Friday I launched the web-site of my fledgling company, the Beanstalks Project. I expect most of my work-related blogging is probably going to happen over there, not on this page.

Incidentally, for my geekier friends, I aggregate all my different RSS feeds into one at this URL: bre.klaki.net/u/?Bjarni.rss. That feed includes pretty much everything I write on the web (except for my Tweets) as well as links to Stuff I Like.

...

In other news, this weekend was a major weekend of mid-summer cleaning: Ewelina and I took everything out of the downstairs storage room and into the back-yard. There we sat in the sun and sorted it all into four different piles: trash, charity, keep and friends. The keep pile was the smallest of the three, so now there is lots of room in the store-room for more "stuff".

But more importantly, now we actually know what we have.

And, I got ever so slightly sunburned!

Today I made massive progress on a 6-year-old carpentry project: closets in my bedroom.

The sliding doors are assembled and in place, and the room already looks so much nicer. Later this week I may try to dye the wood a darker color, paint the internal shelves and put them all in place, and if all goes well, next weekend I'll finish the drawers under the window.

It's weird to suddenly make rapid progress on something I've been putting off or ignoring for so many years. Of course, I was distracted by things like a divorce, single life and living abroad for four of those six years... but still. It's strange, but very gratifying, to finally see this stuff getting done.

When Ewelina and I finally get around to having a house warming party, this place is going to look completely different. And awesome! :-)

I spent today evaluating what kind of back-end data storage system to use for my little startup.

(Oh yeah, I am starting a company, did I forget to mention that? It's gonna be awesome. Details later.)

As many of you may know, I detest RDBMS databases, and have since University. Every time I have tried to use one, it has sucked, possibly with the exception of sqlite. Luckily the data I am working with in this project is a terrible fit for an RDBMS, so I was free to go exploring the open-source NoSQL universe with a clear conscience. Whee!

There are some pretty cool projects out there.

I found some nice Bigtable work-alikes (Cassandra, Hypertable), which tempted me a bit after my experience with Bigtable at Google. But I don't need all their features, the complexity and requirements would probably be more of a hindrance than a help to my project. I was tempted a bit more by Voldemort, which is a fair bit simpler, as it is a plain key/value store, not an ordered table. But still, it looked too complicated in some areas, and not mature enough in others.

Finally, I discovered redis: an in-memory, disk-backed hash table which natively supports the most important data types (ints, floats, lists, sets) and allows high-level atomic operations. It also knows how to do replication, a requirement for my project, and the implementation is simple enough that I should be able to promote a slave to master status in an emergency with little difficulty. The 2.0 release (already in beta) adds support for pub/sub and larger-than-RAM data-sets (swapping out unpopular entries).

In short, it looks totally awesome!

It was really weird to read about the features of redis, though - I had this ridiculous feeling of deja-vu.

See, redis is exactly the program I would (try to) write if I rewrote BottleNeck (the data-store behind Partalistinn and Walkabot) for serious use.

It does the same things, in the same way: it is a single threaded, in-memory hash server. It has a human-readable client/server protocol, running over TCP. Every now and then it forks out a new process to write a checkpoint of the current DB state to disk. It can track incremental changes using an operation log. The up-coming 2.0 version even has pub/sub features (BottleNeck has "watches").

Freaky. But of course, redis is in almost every way superior to my naive Perl implementation. It is faster. It is properly tested. It has way more high-level operations, natively supports complex data-structures like lists and sets, and it supports replication. Version 2.0 even supports larger-than-RAM data-sets.

It's like I got to design the thing and set requirements, and then a much better programmer went and implemented it all (and more) while I wasn't looking.

I can't wait to start using it!

For the past few months I've been working on a system for building blogs and web-sites, named Walkabot.

(I was on 'walkabout' with my Android... Get it? Get it?)

Today I decided to document and release what I have written so far, which although far from perfect, is also decidedly non-trivial.

I started working on Walkabot when I hit a wall in my WhereBlogger development - I needed a site online to handle some of the heavy lifting for embedding maps and videos in WhereBlogger authored blog-posts.

Things quickly got out of hand (turns out I have lots of opinions on what systems like this should do) and now I'm just writing Walkabot because I want to. It's lots of fun.

Once I've got it working well enough, I hope to migrate both the Sacrifiction travel stuff and my own personal web-site to Walkabot - but it isn't quite ready for that yet. Soon!

However, Walkabot is ready to run a web-site dedicated to describing Walkabot itself. So in the spirit of release early and release often, I present: code.walkabot.net

Don't ruin my fun if you think I am wasting my time, but feel free to tell me if you think it's awesome. :-)

I just sent an e-mail to the perl5-porters mailing list, describing a bug in and a potential fix for the IO::Select module.

Googling it, this bug has been reported at least once before (in 2005), but no fix was proposed and it seems no-one could be bothered to track it down. The bug is probably 12 years old by now. Crazy. I will be pretty happy if my solution is accepted.

Update: Resubmitted via perlbug, issue number 75156.

...

For the nerds in the audience, here is the story:

IO::Select is Perl's standard OO wrapper around the POSIX select() call. It is in my opinion an example of a good OO abstraction, hiding away the arcana of constructing select() bitmaps and interpreting their results behind a nice simple API.

However, every time I have used it, my IO::Select based servers have been prone to going into infinite CPU-sucking loops - every now and then, IO::Select would simply stop working.

This is trivially triggered by closing a file-handle which IO::Select is still watching.

I found I could avoid the problem (for the most part) by taking care to always remove file-handles from my IO::Select objects before closing them, but this was only a partial solution as either the OS or lower-level APIs could (and would) still close file-handles without 'asking me' first, on certain conditions.

I could trigger exactly that in an IO::Select based HTTP proxy I was working on this weekend, simply by hitting ESC at the right moment in my web-browser. Not exactly an edge case!

This problem can be detected with careful error handling around the IO::Select call, but it turns out that even when the problem has been detected, there is no way to remove the trouble-making file-handle from IO::Select. The only option left is to discard the IO::Select object entirely and build a new one, which I consider a relatively ugly workaround.

This is in my opinion a bug, and last Friday, on a rainy night in a guest-house in Kazimierz Dolny, I decided to see if I could fix it. I think I succeeded, but we'll see what the perl5-porters think.

The problem I found in the IO::Select module was that all the code to do with adding or removing file-handles depended on the result of a fileno() call - but fileno() returns undef once a file-handle has been closed. So, in the case of fileno() being undef, I simply added a linear search for the object in question. It seems to work.

Of course, my code still has the ugly work-around I described above (and probably always will), because it will take a while for my change to get reviewed, approved, integrated and then distributed to every single Linux machine on the planet... oh wait, that will probably never happen.

The workaround, for posterity, looks something like this:

my $newSel = IO::Select->new();
foreach my $handle ($oldSel->handles()) {
    $newSel->add($handle) if (defined $handle->fileno());
}
$oldSel = $newSel;

I hope my little bug-fix will make some future Perl programmers' life a little bit easier. And in the meantime, I hope people Googling for help will find this blog-post.

I just wrote the following e-mail to Facebook recruiting, in response to a job offer they sent me:

Hello [recruiter],

Thank you for your e-mail and your offer. However, I must respectfully decline, as I do not want to support a company which has demonstrated such a profound lack of respect for their users and users' privacy, as Facebook has.

Sorry, Bjarni

I just realized that I never blogged about receiving the FSFÍ Freedom Award last December. Oops!

My (possibly weak) excuses for not even saying thank-you at the time, were a lack of decent connectivity in the Peruvian jungle, worries about my health (Typhoid is scary) and generally being preoccupied with my trip.

Thinking about it now though, I feel like quite the ingrate. What I should have said, and will say now is: Thank you very much!

As I understand it, I was granted this award for being very involved in (and sometimes instigating) much of the early Icelandic grassroots activity to do with Free Software and digital rights.

It felt, and still feels, a bit odd to be awarded for my efforts in a field I have been neglecting of late, and odd to be singled out when I know I was rarely alone in those efforts, and odd to be awarded for work that I still don't really know whether made a difference... but I guess that is the nature of awards like this. And I am very grateful for the award, and more importantly, inspired by it.

I aim to do a better job living up to the award, now that I am no longer employed by a giant, secretive company and no longer backpacking around the Americas: if I don't release some interesting free software and take active part in the community in Iceland this summer, please spank me.

But anyway, I'll say it again: Thank you, FSFÍ!