Economic reform of government housing distortions

Expensive housing is entirely counter-productive. Nobody celebrates when petrol goes up, or when electricity prices rise. So why celebrate when house prices rise so much, that your children have to pay relatively 3 times as much of their income for their first house, as you did for yours?

Unaffordable housing creates a social divide between classes, and between the old and the young. House prices have risen massively since 1985, and right now we have the mother of all housing affordability problems in Australia, and quite possibly one of the world’s few remaining housing bubbles (along with Hong Kong and possibly Canada).

Much of this problem originates with government distortions of free markets, which have made housing a particularly attractive form of investment for borrowed money, which in turn has driven up prices.

Tonight is Australia’s federal budget night, so today seems like an appropriate day on which to talk about those government taxes & incentives & rules.

I would like to see 5 changes made by government in relation to housing, and the rules and taxes governing it, to remove these distortions and get Australian housing closer towards being a free and efficient market, with affordable and sustainable prices:

  1. Eliminate the First Home Owners Grant. This would save the taxpayer in the region of $700 million per year, whilst also removing a market distortion that has achieved the exact opposite its stated aim (by making homes demonstrably less affordable to new buyers).
  2. End the negative gearing tax subsidy. Currently this subsidy costs the Australian taxpayer $4.8 billion per year. “Very few other ”advanced” economies are as generous in their tax treatment of geared investments as Australia is. In the United States, investors can only deduct interest incurred on borrowings undertaken to purchase property or shares up to the amount of income (dividends or rent) earned in any given financial year; any excess of interest expense over income (as in a ”negatively geared” investment) must be ”carried forward” as a deduction against the capital gains tax payable when the asset is eventually sold. In Australia, by contrast, that excess can be deducted against a taxpayer’s other income (such as wages and salaries) thereby reducing the amount of tax otherwise payable on that other income.”
  3. A dollar’s income should be taxed the same, whether it is earned from wages, from interest, from dividends, or from asset price rises. The current tax system penalizes productive work far more than passive speculation, as the highest rates are imposed on wages and income from savings; lower rates are imposed on income from investments. In a fair tax system, the 50 % CGT reduction should be eliminated, and CGT would apply to all assets equally, including the family home. All forms of income should be taxed at the individual’s marginal rate. Removing the 50% CGT reduction alone would save the Australian taxpayer close to $10 billion per year.
  4. The “price guides” given by real estate agents for auctions in almost all cases are significantly under the final sale price, and almost always under the reserve, wasting buyer’s time and money. Also known as “under-quoting”, and it is “the difference between the guide given to buyers, and an agent’s sale estimate as written on the sale authority contract. It’s hard for buyers to prove under-quoting because they don’t have access to the contract.” A far more transparent process for sale by auction would be for the reserve price to have to be disclosed 1 or 2 weeks prior to auction. This would still leave the market to determine prices, but would let buyers know what the seller’s lower bound was, prior to having to spend money on building inspections and solicitors, only to then find out that the property was never in their price range. Agents who do not publish the reserve in time or who do not call the property on the market at the reserve price would have their real estate licenses revoked.
  5. The money saved from the above (over $15 billion per year) spent on improving public infrastructure and transport, and reducing or eliminating stamp duty.
  • Stamp duty on housing is a significant additional transfer cost, and as such it mildly discourages people from changing housing, such as when their current housing is no longer appropriate. It encourages people to buy housing they think they will need in a 5 or 10 or 15 years time, rather than the housing they actually need now. This is a large part of why couples without kids want to buy 3 or 4 bedroom houses, in anticipation of the 2.3 children they plan to have, rather than buying the housing they actually need now, such as a 2 bedroom apartment close to work.
  • The terrible state of public transport and infrastructure, especially in outer suburbs in cities like Sydney both discourages people from living further out from the city, and encourages transport by car rather than public transport. The result is higher prices for homes close to the city and congestion on the roads as people shun public transport for a more reliable and convenient form of private transport. To make our cities better places to live, and to reduce pollution, we need more public transport routes, higher frequency, more interconnection points, better timing of connections, and a single city-wide electronic ticketing system that covers every type of public transport.

In a perfect world, some or all of these reforms would be addressed in tonight’s budget. However our current politicians, from both major parties, are weak-willed poll-driven career-politicians, who are afraid of offending current home-owners. So instead of doing what’s in the economy’s and future home-owner’s best long-term interests, I fully expect more of the same. The longer the current incentives are left in place, the worse the problem will be for Australia when it inevitably corrects.

Update: Just after finishing writing this, I saw Steve Keen make some very similar suggestions today, with good graphs showing private and government debt levels.

Recovering from a corrupted MySQL install due to a dying hard disk

Might as well write these steps down in case I ever need them again:

Background: A test box started making a faint high-pitched squealing sound, and then powering off. Happened semi-randomly, but most commonly during periods of hard disk access (such as boot-up). Fixed the hardware problem by replacing the Power Supply Unit ($25 from MSY).


Then the above PSU semi-random-poweroff problem in turn caused the box’s 5-year-old hard disk to start playing up (age, plus having the power repeatedly die mid-write probably doesn’t help any).

Steps for moving to a new hard disk:

  • Buy a new hard disk of equal or larger size.
  • Install it in one of the USB external single hard disk enclosures that supports both an SATA or an IDE hard disk ($23 at MSY).
  • Download, burn, and boot from System Rescue CD (v1.5.6 is the current latest). Take the default boot menu option, and the default keymap.
  • After it boots, turn on and plug in the USB drive.
  • At the shell, see which disk has which device name: fdisk -l
  • Recover from the old to the new disk: ddrescue -b 2M /dev/hda /dev/sda ./ddres.txt
  • The above copied about 17 Mb per second, and claimed zero disk errors were found.
  • Poweroff, and swapped the old disk out of the machine and the new disk in.
  • Boot the new disk, was dropped into a shell during boot due to finding file system errors carried over from the old disk. Run fsck on the affected partion: fsck /dev/hda8 -y
  • Reviewing the SMART warnings in the syslog from the old disk seemed to indicate that it probably was dying, confirming that swapping the disks was the correct course of action.

Then found the previous HDD corruption had in turn corrupted a MySQL database (aren’t cascading failures great?). This manifested itself as at least 10 different MySQL errors/warnings/problems:

Error in /var/log/syslog when starting mysqld: Failed to open log (file ‘/var/log/mysql/mysql-bin.000348’, errno 2) The cheat was to delete the last line, the one referencing the /var/log/mysql/mysql-bin.000348 file, from the /var/log/mysql/mysql-bin.index file. Note that in this case I knew that the last log file contained no updates that mattered, so it really was no loss.
vim /var/log/mysql/mysql-bin.index

MySQL server would no longer start, instead giving a “Fatal error: Can’t open and lock privilege tables: Can’t find file: ‘host’ (errno: 2)” message in the logs. For me, the problem was that the /var/lib/mysql/mysql/host.MYI file was missing. What fixed it: Repair the host table.
mysqld_safe --skip-grant-tables &
mysql
mysql> use mysql
mysql> REPAIR TABLE host USE_FRM;
mysql> exit

Try to reset the host table with useful starting data, to fix this error when starting mysql client: error: ‘Access denied for user ‘debian-sys-maint’@’localhost’ (using password: YES)’
mysql_fix_privilege_tables
mysqladmin shutdown

To fix these errors in /var/log/syslog :
[ERROR] /usr/sbin/mysqld:Fatal error: Can’t open and lock privilege tables: Table ‘./mysql/db’ is marked as crashed and should be repaired
[ERROR] /usr/sbin/mysqld: Table ‘./mysql/db’ is marked as crashed and should be repaired

cd /var/lib/mysql/mysql
myisamchk db
myisamchk *.MYI

To fix these warnings in the syslog:
myisamchk: warning: Table is marked as crashed
MyISAM-table ‘db.MYI’ is usable but should be fixed

myisamchk -r db

To fix this error in /var/log/syslog :
[ERROR] /usr/sbin/mysqld: Incorrect information in file: ‘./mysql/tables_priv.frm’
This did not work: repair table tables_priv USE_FRM;
Cheated: just copied /var/lib/mysql/mysql/tables_priv.* from another working machine.
chmod -x,o-r,g+w tables_priv.*
chown mysql.mysql tables_priv.*

Recurrence of this error:
ERROR 1045 (28000): Access denied for user ‘debian-sys-maint’@’localhost’ (using password: YES)
And a new one:
Access denied for user ‘root’@’localhost’ (using password: NO)
… and at this point a “desc user;” showed that the user table file must have had a doubly-claimed inode with another table during the fsck, as it was a completely different user table schema from that found on another machine.
Cheated again: just copied /var/lib/mysql/mysql/user.* over from another working machine.
chown mysql.mysql user.*
/etc/init.d/mysql start

Then to fix: ERROR 1045 (28000): Access denied for user ‘debian-sys-maint’@’localhost’ (using password: YES)
cat /etc/mysql/debian.cnf
Copy the “password” field’s value for the “debian-sys-maint” user to the clipboard.
mysql
mysql> use mysql
mysql> GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'insert_password_copied_above_from_clipboard' WITH GRANT OPTION;
mysql> exit

To fix this warning in /var/log/syslog: WARNING: mysqlcheck has found corrupt tables
Force a check of all tables:
mysqlcheck -A

Fix for this error when granting privileges: ERROR 126 (HY000): Incorrect key file for table ‘./mysql/db.MYI’; try to repair it
mysql
mysql> GRANT ALL PRIVILEGES ON dbname.* to 'dbuser'@'localhost' IDENTIFIED BY "fakefake";
ERROR 126 (HY000): Incorrect key file for table './mysql/db.MYI'; try to repair it
mysql> use mysql
mysql> REPAIR TABLE db USE_FRM;
+----------+--------+----------+----------------------------------------------------+
| Table    | Op     | Msg_type | Msg_text                                           |
+----------+--------+----------+----------------------------------------------------+
| mysql.db | repair | info     | Wrong bytesec: 255- 37- 32 at 0; Skipped           |
| mysql.db | repair | info     | Found block that points outside data file at 424   |
....
| mysql.db | repair | info     | Found block that points outside data file at 24960 |
| mysql.db | repair | status   | OK                                                 |
+----------+--------+----------+----------------------------------------------------+
151 rows in set (0.01 sec)
mysql> GRANT ALL PRIVILEGES ON dbname.* to 'dbuser'@'localhost' IDENTIFIED BY "fakefake";
Query OK, 0 rows affected (0.00 sec)
mysql> exit
mysqladmin shutdown
/etc/init.d/mysql start

To fix these errors in the syslog on mysqld startup:
/etc/mysql/debian-start[4592]: ERROR 1017 (HY000) at line 116: Can’t find file: ‘columns_priv’ (errno: 2)
/etc/mysql/debian-start[4592]: ERROR 1017 (HY000) at line 516: Can’t find file: ‘proc’ (errno: 2)

mysql
mysql> use mysql
mysql> REPAIR TABLE proc USE_FRM;
mysql> REPAIR TABLE columns_priv USE_FRM;
mysql> exit

… and after all the above, the box powers on, and stays on, the disk errors are gone, the mysqld service starts cleanly, and from a quick cursory glance, the data still looks okay.

On Showbags

So my partner went to the Royal Easter Show last Saturday, and returned after dark, totally exhausted, and with a sore back and sore feet from hauling lots of heavy showbags over a kilometer from the showbag hall to the carpark. On Sunday she proceeded to lay the contents of each bag out in the middle of the living-room floor, in what can only be described as “an orgy of consumer crap”. See photo (this by the way is not everything) :

showbag-stuff
Of particular amusement to me was the 1 kilo of flour buried at the bottom of one of the bags. I have to salute the sheer evil genius of whoever came up with that idea.

To explain, I view showbags as kind of like carry-on lugguage for a flight, except it’s packed by someone else, and you only get to properly unpack it at its destination. You also have to buy this luggage, and your destination is home.

So, the ideal showbag is inexpensive, and contains a variety of interesting things that you will use, that you would not normally buy, and is light because of the distance you will have to carry it.

Examples of good things to get in a showbag: Sachets of sauces, samplers for new varieties of tea and coffee, small packets of unusual sweets, light magazines, useful vouchers. Examples of bad things to get in a showbag: Heavy bottles of liquid, heavy useless glossy magazines.

Flour, of course is diabolically brilliantly bad, because it’s readily available at any supermarket or corner store, it’s a commodity (so it can never be unusual or interesting), it’s heavy, it’s bulky, and it’s dirt cheap (e.g. around 85 cents per kilo for no-frills brands). It is however arguably useful.

It made me wonder: What’s the worst thing you could put in a showbag, and still get away with? I’m thinking maybe a gardening showbag, that contains a 1.5 kilo bag of sand at the bottom of the bag. Sand is very heavy, it’s readily available, it’s bulky, it’s even cheaper than flour, and it’s even less useful. However sand does have plausible deniability for gardening, which I think is important. (This for example is why putting a large rock in the bottom of a showbag would probably never work).

Has anyone got any ideas for something worse to get in a showbag than flour or sand?

Oh and my favourite product sample was this:

underarm-sweat-liners

That’s right: underarm sweat liners. And if you’re wondering: yes, they do make a male version. I had assumed at first that you stuck these into your armpits, and had to rip them off like a bandaid (together with any hairs) at the end of the day. Disappointingly, you stick them to your clothes – but I think the manufacturers aren’t thinking outside-the-box enough with this. Having sat in small rooms that reeked of body odour with other sweating geeks at previous LCAs, I personally think that a version that you stuck to your underarm would be a socially useful addition to the swag bag, if only we could get everyone to wear them :-)

Email bankruptcy: the cop-out wuss option

Declaring email bankruptcy has always seemed like a total cop-out to me. It says “I am helpless and incapable of handling the flows of information that I have selected for myself. Won’t someone save me from myself? Failing that, I will give up, and push the problem on to you – because if really want me to read something you sent me, you now need to send it to me again”.

Take Rich’s 16000 unread email example: don’t you just know that most of that mail must be complete crap? Lose it, dude!

Add a rule to move all the mailing list stuff into separate folders, run that rule on your inbox, and mark all those as read, and then unsubscribe from those mailing lists. Clearly, you’re not keeping up with them.

For everything else automated, if you get something opt-in that you are not keeping up with, then unsubscribe (unless it’s non-opt-in / spam, in which case you want better spam filtering). Anything (especially items that are work-related or software-related) that can be moved into trac / bugzilla / a wiki / any form of tracking system or knowledge repository should be, so that it doesn’t get lost forever, and so that you and your managers and coworkers can see what tasks are pending, and can allocate extra resources if needed.

After removing all the above (mailing lists, spam, work-related, software-related), from 16000 mails, probably only 100 max were from humans / businesses you know and actually require you to do anything, and have real-world consequences if you don’t. Move those into a separate folder (I call mine “action”), and just work through them. If there is no actionable item from a friend’s or workmate’s or client’s email, but it feels rude to not reply, then send a one line thank-you reply to be polite, mark as read, and move on!

Honestly, 16000 unread mails is probably the world’s way of telling you to consume less information, and learn to ignore more stuff.

Migrating email from Outlook to Evolution: Linux’s final frontier

At various times in Linux’s history, various things about Linux have really sucked:

  • Getting hardware to work used to really suck, and you used to have to patch the kernel and recompile your own kernel… and then the kernel got a lot better, and the hardware support got a lot better, and I haven’t had to recompiled a kernel in years, and I’m happier because it largely “just works”.
  • Setting up printers used to really suck, with stuffing around with printcap files, and printer configurations, and desperately trying to get it to work … and then cups and printer detection improved, and now it’s generally all painless and point-and-click to install a printer, and I’m happier because it largely “just works”.

  • Setting up X-windows used to really suck, with editing X config files and mucking around with modelines… and then monitor detection got better, and I haven’t had to do anything with a X config file in years, and I’m happier because it largely “just works”.

  • Getting your Microsoft Office documents migrated from Windows to Linux used to really suck … and then Open Office came along, and it does a fairly good job of importing Office’s documents, and I’m happier because it largely “just works”.

This week I came to realize that there’s one last frontier remaining, where Linux still really sucks. And that frontier is migrating from a Windows graphical email client (Outlook in this case) to a Linux graphical email client (Evolution in this case). It does not “just work” … not at all. This blog entry will now explain why I say this.

Earlier this week, my main hard disk on my main desktop machine (a Windows machine) died, with a horrible repetitive clicking and grinding sound. By sheer random good luck, I had backed up all my data onto an external hard disk about one hour before this happened, plus I had a brand new machine ready to go on which I was considering trying Linux anyway. It looked like the stars were in alignment: Linux on the desktop of my main machine, here we come!

Installing and configuring Ubuntu was totally painless. (If you care, the exact steps followed are here: http://nickj.org/Ubuntu_8.04.1_desktop_setup_steps ). And there are a lot of things that I’m really liking with this new operating system, including the following:

  • Complete hardware support: My old Windows install could not detect & use all the cores of my CPU (Windows 2000 Pro does not detect / use a Quad core CPU, whereas Ubuntu 8.04.1 does). In Windows, I needed special drivers for my sound card and my mouse and my keyboard and video card, all of which had to be manually installed or downloaded. In Ubuntu 8.04.1, all my hardware “just works”, or it offers to install restricted drivers (for the video card), and then it “just works”. Nice.
  • The ease of installing and uninstalling software (synaptic + aptitude). Having a well-integrated package manager for installing and uninstalling everything is most pleasant.
  • The way when I play a video, and don’t have the right codec installed, it will offer to download and install the needed codecs for me, and once it’s finished (which typically takes all of 20 seconds), it will play the video. That’s really nice, and sure beats having to manually work out the right video codec to install. Nice one.

But the migration of data from Outlook 2000 to Evolution was shockingly, appallingly bad. There was a lot of misinformation on the web about approaches that should work, but which actually had data loss, and in the end, it took me four full frustrating days to get most of my personal data moved across. The whole migration was so painful that it has left me quite annoyed. When I hear people bandy around phrases like “the year of the Linux desktop”, I simply makes me think: Dream on! This is not the year of the Linux desktop. This is not even the fucking decade of the Linux desktop. Come back in 2011, at the earliest. And when I hear Evolution described as “an Outlook killer”, I can only laugh. To be an “XYZ killer”, you have to do everything that XYZ did, but better, AND you have to be able to import XYZ’s data. Microsoft Word was a WordPerfect killer, because it did what WordPerfect did, but it did it better (in a WYSIWYG way), and it imported WordPerfect’s data. Same for Excel versus Lotus 1-2-3. Same for Firefox versus Internet Explorer. But Evolution does NOT import data from Outlook (for any meaningful definition of the word “import”), so BY DEFINITION it simply cannot be an Outlook killer. And that’s before even getting to the fact that Evolution is not as feature-rich, nor as mature, nor as user-friendly, nor as bug-free as Outlook.

Understand that I’m no Microsoft apologist – I will happily use FOSS, if it’s as good or better.

However, if you are a masochist, if you enjoy pushing hot needles under your fingernails, then it is possible to make the transition. I evidently fall into this category, because I was too stupid or too stubborn to just give up. So here’s how you too can do the same, but I’m warning you straight up, it is not pretty, and it is not easy, and it is not quick.

And for anyone who says it is easy, allow me to enumerate some of the relevant facts:

  • I have/had 4 PST files, not one, like most people (i.e. the data is spread across 4 files, because Outlook barfs when a PST’s size approaches 2 Gb, so I needed to spread it out across multiple files to prevent this).
  • I have/had around 10 years worth of data. All of that data, every single bit of it, needs to come with me. This point is non-negotiable.
  • I use/used all the features of Outlook, apart from Journaling. That’s Email, Calendar, Contacts, Notes, and Tasks. Five categories of data, every single one of which I need. More details about each:
  1. Email. A total of between half a million and 600,000 emails, spread across a 692 nested email folders. These folders are categorized in a hierarchy to keep like mail grouped together. That’s 10 years of work/personal/hobby emails, sent and received and drafts, and this includes a collection of 40,000 spam emails received, kept to help with Bayesian training of spam versus ham.
  2. Contacts (roughly 550 contacts, some of which are just an email address, and some of which have complete details, spread across 9 nested folders).
  3. Notes (being used to store check-lists, passwords, etc., with 400 notes spread across 4 nested folders).
  4. To-do lists (being used to store information, check-lists, and list of bugs or wish-list items in various bits of software that I maintain, with 410 tasks spread across 26 nested folders).
  5. Calendar items (10 years of past events, and additional future events, recording both what happened, and predicted dates and deadlines for things that are going to happen).

So that’s the background. Here’s what does not work for migrating this data from Outlook to Evolution:

  1. Does not work: Export from Outlook, or getting Outlook to export it’s own data into some industry-standard file format. Ha! Have you ever looked at the File -> Import and Export section of Outlook? Remember, Microsoft are arrogant monopolist pricks, who have no vested interest in helping you move to anything else. So, we get just two export options, both of which are basically useless. Next option please.
  2. Does not work: Getting Evolution to read the PST files and import the data directly. It just doesn’t. Various feature requests for this has been open for the past 6 or 7 years, without any visible sign of progress. Move along, nothing to see here.
  3. Does not work: Readpst, which is an Ubuntu package, and which is apparently derived from libpst: On the very first PST I tried this with, it gave a series of warning about NULL pointers, gave a series of messages indicating that it wasn’t going to transfer everything anyway, and then proceeded to segfault. Clearly that’s not going to work.
  4. Does not work: Moving data from Outlook to Outlook Express, and then moving from Outlook Express into something else. Outlook Express only imports data from the main PST (ignoring the other files), and it loses data (converts all Contacts to mail items, converts all Tasks to mail items, losing many or most emails in the process). That’s right folks, even two Microsoft teams, who presumably work in the same building, can’t get their own email products to import data correctly from each other. Forget this.
  5. Does not work: Import into Thunderbird, which uses MBOX format, and then move the MBOX files onto the Linux box, and point Evolution at those. To do start this, you Install Thunderbird, and when you run it for the first time, choose “import from Outlook”, which will import the address book and your mail. I had high hopes for this option, it was very easy to use, and it seemed to work great … at first. However, it has a major problem: severe data-loss. Here’s an example: I have a folder that contains every bit of spam email I was ever sent. It’s useful for training spam detectors, and I found out, it’s also incredibly useful for testing migration tools for data integrity (because spammers send all kinds of weird formats, weird attachments, they ignore standards with impunity, etc.). In short, spam makes the perfect test case. This spam folder has 40,877 pieces of spam mail. How may bits of email do you think Thunderbird imported? The answer is 494 mail messages. That is a 99% data loss rate. Amazing. And there was not a single warning, not a single error – just completely silent 99% data loss. Now, I don’t care about losing my spam, but I do care very much about losing real data, and I had zero faith in Thunderbird at this point to migrate my data without data loss. So, ditch Thunderbird.
  6. Does not work: Migrate from Outlook PST to IMAP. Migrate using IMAP. Connect Outlook to an IMAP server on your LAN, copy everything there, connect Evolution to the same server, and copy or move everything from the IMAP server into Evolution. In theory, this should work great, and with a few test folders, it does. But the issue here is one of scalability: it seems to work fine on the simple stuff, but falls apart on the bigger stuff. Moving a single email would work fine. A single folder would work usually fine. But moving a hierarchy of folders with half a gigabyte of email would cause Outlook to start copying data, and then after about 10 minutes it would usually just get stuck, and then about 20 minutes later it would give a dialog box saying that the copy operation had failed. As a result, this option is unusable if you have substantial data, due (I suspect) to an Outlook IMAP bug. Other versions may work fine, but Outlook 2000 was buggy in this regard – for me, it kept hanging and could not completely transfer all of my data – and therefore it was, unfortunately, unsuitable for migrating data.

So what does this leave? At this stage, I thought I was out of options, and was tempted to just give up on Linux, and stick with Windows. Outport would move some of my data, but it would not move email, which is the largest and most complicated single component that I needed to move. Eventually I found the answer: O2M (which is a US$10 commercial product) for moving email + calendar + contact data, and Outport for tasks + notes, and 2 custom scripts I had to write to massage the O2M and Outport data into the correct format. Disclaimer: I don’t have any financial interest in O2M, I don’t know the people involved, and so forth – it simply the best tool that I could find for the job, and O2M does have problems too, but it’s problems are far less severe than the problems with the other methods.

Here is the link for the step-by-step details of how to migrate from Outlook 2000 to Evolution.

Decent browsers on mobile phones: Are we there yet?!

Brion Vibber summarises from OSCON on the future of browsers on mobile phones.

Some quick thoughts – capable smartphones are expensive (e.g. $350 to $1000), and the basic phones below that price point tend to be pretty limited and have small screens (but they’re cheap and fairly tough, so as an actual phone they work fine, but as an internet-enable communication device, they suck).

The good news that is that people turn over their phones relatively quickly (e.g. in Aus approx 11 million phones were sold for the last few years to a population base of 21 million, so average active phone lifespan presumably is around 1.9 years). So even if everyone bought only capable smartphones from this point onwards, it would take most of 2 years to get to sufficient market saturation that a phone with a capable browser could be assumed. But the fact is that people won’t all start buying smartphones (without a truly compelling reason to), and people who have smartphones won’t all sign up to mobile internet packages (it’s better in the US I think, but in Aus you usually have to pay extra for this, and you typically get an allowance of anything from 100 Mb to several gigs per month of bandwidth, and if you go over that you get slapped hard with extra usage charges – I’ve heard up to $1 per megabyte, but that’s so scary I hope it’s not true). So yeah, it puts people off. Realistically, probably 4 or more likely 5 years before this mess is sorted out and most people have a decent enough phone with a reasonable browser with mobile internet.

And for things like GPS, I say “BOO!” to only native apps being able to access that. GPS badly needs a standardised JavaScript interface, that can do stuff like say “do you have GPS?” and get a boolean answer, “do you have a signal?” and get a boolean answer, and then ask “what is the long + lat?” and get back an array of two decimal numbers. When this is native and works and runs without throwing errors in all browsers (both on phones and on desktops), then it’s going to be fricken awesome (e.g. walk around and have your phone display the Wikipedia article for the nearest landmark, walk around the city/go skiing and see where your mates are on a map on your phone and so be able to easily meet up with them for lunch/coffee, go to a new city and get a tour on your phone that knows where you are and tells you the most interesting tourist highlights that are closest to your location and that you haven’t visited yet, and so on and so forth). When it happens it’s going to be heaven-on-a-stick, but getting there feels like it could be painful and slow.

Pronunciation of Dutch names vs anglo-Australian naming

Gosh, Dutch names are hard to pronounce correctly. I just finished a business phone call to South Africa, and the person I was after had the surname of “Van Wyk”. My British/Australian upbringing tells me to pronounce that as “Van Wick”. So I did, and was met with complete and utter bafflement as to who I was after. On realising that I was a) not making a crank call b) calling from overseas c) after a person who did work there, the kindly secretary gave me a brief pronunciation lesson, and it’s pronounced “Van Veek”, as far as I could tell. The secretary also assured me that they probably couldn’t pronounce Australian names properly either… but I didn’t have to heart to tell them that actually no, they’d probably be fine, as anglo-Australians use surnames fairly sparingly, and often like to name people with very short words (example first names/nicknames: “Sharon”, “Kylie”, “Gazza”, “Mark”), and to name places in a relatively unimaginative fashion (e.g. Sydney has a harbour, so it’s called “Sydney Harbour”; there’s a bridge over that water, so it’s called the “Sydney Harbour Bridge”; there’s a tunnel under that water, so it’s called the “Sydney Harbour Tunnel”; the other side of the water is to the north, so it’s called the “North Shore”; the suburbs to the east/south/west of the city are called the eastern/southern/western suburbs; and so on and so forth). Personally, I quite like the shortness and simplicity of this style of naming, but as a consequence, I suspect it results in many Aussie names being comparatively easy to pronounce. Of course, Aboriginal place names are harder, but these generally use phonetic spelling.

Firefox T-shirt shipping madness

So the Mozilla store has released new T-shirts for Firefox 3. I don’t particular like the new T-shirt logo, and I thought the old logo was better. What’s really insane though is the cost of international shipping.

Cost of one Firefox T-shirt: US$17 = AUS$17.80

Cost of one Firefox T-shirt with delivery to Australia: US$110.64 = AUS$115.65

My guess on the total number of resulting Firefox T-shirts sales shipped to Australia at those prices: approximately zero.

NRMA feedback fail

It’s that time of year to renew my car registration, and buy the accompanying compulsory third-party insurance. So I tried the NRMA, and price-wise their quote was okay, but I wanted it mailed to me in the post so that I can pay it closer to when I actually need it. Trying to tell the NRMA this proved to be impossible:

… and then clicking the submit button gives this:

Feedback rejected

… forbidding all English punctuation – that’s a really nice touch! So I removed all commas, full stops, apostrophes, and question marks, leaving one continuous string of text, and clicked submit again. The result is this:

Wow, that’s impressively crap. Being that bad at listening to people’s feedback doesn’t just happen, it takes serious dedication and practise and commitment.

Response to “Where did all the PHP programmers go?”

Ok, I’ll bite in response to this “Where did all the PHP programmers go?” blog post:

What I cannot understand is why people with more than one Bachelor Degree in Computer Science recommend using bubble sort.

Sounds wrong but harmless, as you don’t write a sort implementation from scratch in PHP. You write the comparators used for the sort order, but the actual sort implementation is provided for you by language. I presume it uses qsort internally, but don’t know for sure. I have a degree in CS, and I can scarcely even recall the bubble sort algorithm (or even most of the sort algorithms for that matter), for the simple reason that it doesn’t matter in the real world (in 99% of cases) for web developers using scripting languages. That may sound (gasp) shocking, but it’s true – PHP is not a performance-orientated language, and it’s a fairly high-level language with a decent library of native functions, so you don’t generally write sort algorithms (rather you use the library ones that are provided for you, unless you have an overwhelmingly good reason not to).

The question you need to ask is: are you running a Computer Science class on sorting algorithms, or are you looking for people who know PHP and can get your thing built?

“What is the difference between the stack (also known as FILO) and the queue (also known as pipe, also known as FIFO)?”

Maybe rephrase the question to “you want to store multiple bits of information in a data structure or an array or a collection of some sort. How would you add data to the beginning of that data structure, and how would you remove data from the end?”

I.e. focus less on the Computer Science theory, and more on the application of it.

“Using PHP programming language, create a list to store information about people. For each person you’ll need to store name, age, and gender. Populate the list with three sample records. Then, print out an alphabetically sorted list of names of all males in that list. Bonus points for not using the database.”

Here’s a trivial implementation just using arrays, I don’t claim it’s remotely pretty or elegant, and I wrote just to see what’s involved in the above task:

<?php
error_reporting( E_STRICT | E_ALL );

function sort_by_name( $a, $b ) {
       if( $a['name'] === $b['name'] ) return 0;
       return $a['name'] > $b['name'];
}

function printMales( $array ) {
        foreach( $array as $person ) {
                if( $person['gender'] != 'male') continue;
                print "Name: " . $person['name'] . "\n";
        }
}

$people = array( array( 'name' => 'Bob'      , 'age' => 36, 'gender' => 'male'   ),
                 array( 'name' => 'Alice'    , 'age' => 23, 'gender' => 'female' ),
                 array( 'name' => 'Doug'     , 'age' => 63, 'gender' => 'male'   ),
                );

print "Before:\n";
print_r( $people );
usort( $people, 'sort_by_name' );
print "After:\n";
print_r( $people );
print "\n";
printMales( $people );

?>

But you know what? I had to look up the PHP manual for usort because I couldn’t recall off the top of my head whether it was “u_sort” or “usort”, and I couldn’t recall the parameters and their order. Also I had 3 trivial syntax errors that I fixed in 15 seconds. Now, I really hope for this pen-and-paper test that you are giving people access to the PHP manual, or if you are not that you are being very tolerant of minor syntactical errors or people who can’t recall whether the function name has an underscore, or who can’t recall the exact order of the parameters, and so forth. Because the question is: Is this a test of whether someone has memorized the entire PHP manual, or is this a test of whether people who can do what you want? Because when they are working, then you will give them access to the PHP manual – right?! If you want to distress people in the interview, then sure, treat it as a rote memory test of the PHP manual and Computer Science theory, and make it awkward if they get anything wrong – but if you’ve want to solve the problem of finding people then there has to be some leeway for recollection of technical trivia that you can find through Google in a few seconds.

Look, I’ve been in a similar situation of looking for PHP people to hire (the candidates were from China in this case), and the approach we used was to give them a test beforehand that they could do (in 24 hours of their own time), and then if they looked okay then they could get called in for an interview. This allowed culling people who were very bad, or who gave code that didn’t run – as there really is very little excuse for code that’s invalid or that doesn’t work if you’ve got 24 hours and access to the internet and your own computer. Most of the people weren’t great, some were very bad, and some were okay. If it helps, that PHP test is here, and it’s only intended to be a very simple test.