Website Management

Following my initial discussion on using geo-targeting to predict timezones I've finally found time to play around with some of these ideas.

The idea

Simplify the user experience by predicting the timezone someone is in and auto-populating a registration form accordingly.

The solution

At the time I wasn't really looking for any solutions other than matching an IP address up to an actual location. This is certainly possible and I'll show you how below but first I'll highlight the alternative. If you are reasonably confident that the timezone will be set correctly on the users computer it is possible to access that value via javascript.

GeoIP to Timezone

If you want to avoid relying on javascript then you still have some options.

Maxmind: The PEAR pacakage supporting maxminds geoip databases doesn't support timezone lookup but the script offered by maxmind directly does support timezone lookup. The usage seems a little convoluted but it is there.

IP2Location: The timezone information is integrated directly in the database once you reach 'DB11', their 11th database offering. The usage should be straightforward. There is only one drawback - DB11 costs $649/year. Personally this is more than I would be willing to spend but their client list is impressive so if this is within your budget spread the moolah and use my affiliate link .

Free solution: There are still a few options here. Although I haven't tried it I believe the maxmind code should work with their GeoLite products. Alternatively once you have a location, either using maxminds free databases or hostip.info you could try getting to a timezone using the 'world time engine' class available from phpclasses.org. The drawback with this approach is that even after the database lookup you still need to query two web services. That is going to introduce a significant lack in response time.

My preference

Although I started off looking at GeoIP services I would much prefer to be able to tackle this problem using javascript. It's not an ideal solution, some people will have their timezone set incorrectly while others will have javascript disabled but on balance I think it is good enough. This is the icing on the cake rather than core functionality after all.

This blog has been quiet for a little over a month now as 'real world' events have consumed all my spare time. Overall it has been a month well spent.

PEAR bug triage

I was able to set aside a little time one weekend for the inaugural PEAR bug triage event. I learnt a lot even if I didn't really achieve much. Definitely something I want to set aside some more time for in the (near) future.

Edinburgh International Science Festival

I spent a couple of days (attempting) to exhaust those perpetual motion machines most commonly known as children. I believe I failed entirely. The department I'm based in ran a stall focusing on the heart and healthy living. The experience left me exhausted but with renewed hope that the next generation are not entirely the devils the media would have us believe.

Munich

I spent a long weekend in Munich. It seemed like a really nice city. Best of all the trip was free which always adds an extra delight to any experience.

Tai Chi seminar

For the second time I went to a weekend seminar on Tai Chi. I've been attending weekly classes for ~ 18 months now but find the focus of these weekend seminars to be valuable. Draining but valuable.

My next project

Over the past few days I've finally found time to start work on a new project I've been thinking about for a couple of months.

A new start

With a new project starting I feel it's a good time to reflect on experiences with past projects and take a look at what are currently considered to be 'best practices'.

Frameworks

The first decision I made was to take a closer look at the plethora of frameworks which have sprung up over the past year or so. I decided to give zend framework a try and so far have found it to offer what I need. My main concern was that it would be too inflexible. I've wanted to deviate from the default three or four times now in what I would consider to be non-trivial ways and found that by hitting my code base with the manual a couple of times it would eventually yield to my will. There is certainly a learning curve to master but on balance I believe the benefits will be worth it.

Source Control

Previously my source control has been embarrassingly bad. I still have backups which contain backups which contain old code archives to remind me. I suspect there is useful code in there somewhere but it has reached the point now where finding it is so demoralising that I prefer to pretend it doesn't exist and start again.

Recently I've been playing with subversion. It's definitely something I want to continue using. Currently I have it running from a slightly flaky old computer I turn on when needed. It still isn't really an ideal solution. What I really want is an always on service I can connect to from anywhere. To that end I've been looking around for subversion hosting.

Following the suggestions in an year old post by Jonathan Snook I've been comparing the offerings available. I've put together an excel spreadsheet which you can download here or view in Google docs here.

Hopefully that will save someone a little work. Personally I'm edging towards assembla which makes 500 Mb of svn space available for free and seems reasonably priced should my needs grow. I would be delighted to hear from anyone who currently uses them or has in the past.

I have a love hate relationship with geo-targeting. The web wasn't designed with making it easy to get the geographical location of connected computers in mind. A users geographical location is interesting and potentially valuable though and so methods have been developed to make it (almost) possible.

These methods typically involve something akin to a brute force attack. Figure out where enough IP addresses have been assigned and you can get a good idea of where a user is from their IP address. Other methods involve identifying the computers through which they are communicating with you and assuming the user is in the surrounding geographical area. Neither method is perfect but in the majority of cases you can know which country a user is in with reasonable accuracy.

What I hate about geo-targeting is how some sites think they can locate you more accurately than the country you are in. Maxmind, which is probably the commercial leader, thinks it can guess your location to your nearest city with an accuracy of 81% in the US. Outside of the US I suspect this drops considerably. I'm seeing fewer sites than I once did trying to tell me where I'm connecting to the internet from (and getting it wrong) so I'll skip forward to what I love about geo-targeting.

Continue reading "Geotargeting in forms"

Over a full month after the fact I present my summary of the BarcampScotland2008 event.

The event was split over two days. It kicked off on the Friday evening (1st Feb) in the main room of Alison House at the School of Architecture. It was just the one room with a decent communal feeling. I think most people were holding back with their presentations for the following day. Despite this two presentations did take place. James Littlejohn made the first presentation after the welcome session and talked about data portability.



It was a good summary of the current situation. I also happen to agree with most of his positions. He has taken the decision of making his homepage the hub of his social network. Although I think he has perhaps taken things a little too far my main criticism is the implementation. Looking at the site it took me about 20 seconds to figure out that aboyne is where he is based and not his surname. I couldn't find his surname anywhere on his homepage. After navigating around for a while I found it in the byline for his blog. It wasn't an easy process. I highly doubt that data was machine readable despite the importance he attached to this during his talk.

Ewan Spence was next up with an improvised talk he largely made up on the spot. This rapidly migrated to a conversation with some interesting points raised.

Following on from this Dave McClure set a small competition going. A excessive number of random words were gathered from the audience which then broke up into 5 groups to brainstorm company ideas around any pair of words. Somehow the team I was in won with sexydyslexia.com, a couple that takes standard prose and converts it into netspeak and vice-versa. I notice that the domain name is still available so although rated as the best apparently no one in the audience wanted to run with it.

All the details on the second day after the jump . . .

Continue reading "BarCampScotland2008 Roundup"

The second BarCampScotland event finished yesterday. This was the first BarCamp event I've attended and I have to say I was impressed.

Almost without exception I would describe the talks I attended as interesting or very interesting. I plan to post summaries and link to slides where possible in a later post.

The facilities were also quite impressive. Appleton Tower always looks to me to be a rather drab building from the outside but the concourse, where BarCamp was based, was a deceptively impressive space and certainly met our needs. Having said that I don't know whether they turn the heating off on the weekend but it was definitely chilly.

There were five large (easily seating 100+ people) lecture theatres available on two levels which were again well equipped. For the number of laptops I saw out on display the power sockets beneath every other seat would have been invaluable. It was only in the penultimate session I realised they were there but as that was when I started being concerned about power it all worked out well.

The Slides

I spoke in the morning about contact importers and where I felt they were going in the future.

I've embedded the slidecast below. Feel free to link to it or embed it elsewhere.

As mentioned earlier this is the second, and final, post going into greater detail on one of the sessions at the recent (it's been sitting in my drafts folder for a week) FOWA conference in London.

Matt Biddulph discussed interacting with 3rd party sites and services and the portable social graph. This session was particularly interesting to me given my interest in handling contacts from MSN messenger, Gmail, AOL messenger (AIM) and Yahoo!.

Matt's talk focused around how we can move beyond the use of such scripts with their multitude of risks to a situation were a user can join a new site, release the minimum data necessary and quickly identify their friends already using the service.

For those wondering what risks I am talking about the current situation means that for a site to view your contacts in a service they need total access. This means they could view your mail and send mail through your account. Where a username/password combination is used to access other services on a site these would also be compromised, so to take Google as an example you also grant access to Google Checkout, Adsense, Analytics etc. This is the situation today. Even Dopplr, the contact importing functionality of which the conference chairs spoke about with nothing but praise, requires your username and password to import your Gmail contacts.

Luckily, this situation is on the cusp of improving. For services where you are happy to make your list of friends publicly viewable marking them up with microformats is a simple way to allow other services to make sense of your friends list. Matt mentioned that twitter was also thinking of taking this a step further and supporting openid so that a user could prove that a friends list really was their friends list. I'm not terribly familiar with openid but I assume this wouldn't lead to once again proliferating login details and you simply delegate to your actual provider. Correct me if I'm wrong.

This works well when you're happy for your contacts to be public but you're probably going to want to keep at least some of the contacts in your email address book private so another solution needs to be found. Thankfully a standard way of achieving this is being developed. In fact, Matt talked about five.

Yahoo! BBAuth
Google AuthSub
flickr authentication
AOL OpenAuth
OAuth

I've also talked previously about Windows Live Contacts Control which although more limited in scope for the purposes of this discussion it does much the same thing.

All these services, and hopefully there will be convergence in the standards, open up parts of that glorious social graph and they do it in a safe manner. They also work without an absolute demand for javascript which was one of my main criticisms of Windows Live Contacts when I first spoke about it. Even the Microsoft service now has a RESTful API, either I missed that when I first took a look or it's new.

Hopefully it won't be too long before most of the code on this site is nothing more than an historical curiosity. I don't think we are there yet but soon. . .

The slides from Matt's talk are also now available.

I've previously mentioned the work Steve Souders is doing evangelizing high performance websites at Yahoo! and I was very pleased to be able to hear him speak at the FOWA conference. Sadly the video from that conference isn't going to be freely available and although the audio and slides will be freely available they are not yet available.

Luckily he has previously recorded a video of a very similar (though not identical) presentation and I've embedded it below.



It's 37 minutes so for those in a hurry here are my notes from his presentation at FOWA.
The key points have also been discussed in a series of blog posts so where possible I'll link out to the relevant posts.

Importance of the backend

The first point raised is that the user perception of load time is more important than the actual load time. This means that the relevant metric is not how fast can the html document be returned to the browser but how quickly that html is rendered in the browser. In measuring how quickly a page renders it was quickly realised that the backend performance, returning the html for the page, accounted for only about 5% of the overall time it took to render the page. Even with a full cache the backend was still only 13% of overall time.

Of the top 10 sites in the US only the backend for Google accounted for more than 20% of the load time with a full cache. The Google homepage is so spartan that with a primed cache only two HTTP requests need to be made.

Cache Usage

The importance of the cache was then discussed with data presented discussing how many people at a site had a primed cache. They inserted a one pixel image on the Yahoo! homepage and then monitored the number of HTTP requests with a 200 header (empty cache) and a 304 header (primed header).

It was found that 50% of daily users have an empty cache which accounts for 20% of daily pageviews. This varies depending on the type of site, for example an empty cache accounts for fewer pageviews in a webmail site where each user will view multiple pages, but is broadly accurate. The data highlights the importance of catering for those users without a primed cache. Excessive use of images can't be justified by the assumption that once they are loaded the cached versions can be used. 50% of your users every day will be arriving at your site with an empty cache.

iFrame

Next he talked briefly about iFrames and how they can cause a 40-50 ms delay. onLoad doesn't work until the iFrame source responds which can cause a problem with 3rd party content.

YSlow

Next he discussed YSlow which grades a website based on the 14 rules developed through their research. YSlow is an extension for Firebug the popular development extension for the Firefox browser. It looks at how the page was built. Despite looking at the content rather than the response time its score correlates well with the rendering time. As such it could be a valuable tool during development to predict the speed of a site prior to its launch.

Another issue which YSlow apparently solves is a bug in how Firebug charts HTTP requests. Apparently Firebug will show queries to the cache as HTTP requests and YSlow patches this.

That's all I made notes on. I've got vague memories of stepping HTTP requests to increase download speed and cookies are always worth considering but I picked up a nasty cold in London and it's all a bit fuzzy.

Following on the back of my recent posts looking at the (hopefully) best and worst of benchmarks I thought it would be useful to finish off with some genuine tips for creating 'lightning fast' websites. I probably lack the experience and insight to bring anything new to the table though so instead I'll point you to a selection of interesting articles.

Firstly, let's lay the benchmarking issue to rest. Ben Ramsey, who after his initial outrage at my 7 tips post felt it "actually is really humorous" (probably unjustified praise but thanks anyway!), has a nice post highlighting the code in the PHP source confirming the lack of any difference I demonstrated in my follow up post. Wez Furlong commented on my 7 tips post and highlighted a post he made on benchmarking back in 2005. For anyone feeling my method was excessive his approach gives speedier results. Personally I'd like to see it run in triplicate though.

Next, as far as the minute differences the 'lightning fast PHP'-style posts are too often built around Ilia Alshanetsky probably has the best write-up.

Please keep in mind that these are not the 1st optimization you should perform. There are some far easier and more performance advantageous tricks, however once those are exhausted and you don't feel like turning to C, these maybe tricks you would want to consider.

Absolutely.

Getting to articles with tips for that 1st round of optimisations you may want to make there are 13 tips for high performance websites on the Yahoo! developer network. These were written by Steve Souders who, in addition to writing the book 'High Performance Web Sites,' is speaking at the FOWA conference next week. That's one session I definitely want to catch. Hasin Hayder has a follow up post which is definitely worth reading.

Hasin goes into more detail than the Yahoo! article and provides some sample code. A three part series of posts at the IBM developerWorks site takes a PHP focused look at high performance websites and provides some useful instructions on setting up your sites to use the XCache opcode cache, Xdebug and memcache.

Three rules for high performance web sites

For those wanting the abridged version here are my 3 tips for high performance.

1) Fast environment - Start from a position of strength. I didn't post the average speeds in the better benchmarks post because I was looking at the difference rather than the absolute values but the benchmarks were running ten times faster on my web host than on my desktop. There are various reasons why this may be the case, Linux vs Windows XP, system specs, PHP 5.2.3 vs 'evil' PHP 5.2.1, but it doesn't really matter beyond illustrating the need for a good server and host. Other things to consider include an optimizer/opcode cache and gz compression.
2) Cache everything - Database and web service queries, blocks of content and even your entire page are all fair game.
3) Test everything - Time your code. Profile your code. Test your assumptions (including tips 1 & 2).

Speed doesn't matter

Finally an alternative take because playing devils advocate is fun. Download speed is not how users determine the speed of a site. To the user a site is fast if they can quickly achieve their goal. Steven O'Grady at Red Monk also raises some interesting points contrasting the perspective of the developer and the user.

As always further suggestions, alternative viewpoints and discussion are welcome in the comments below.

After learning about the future of web design conference back in January and then deciding I was just going to be too busy in April when the conference was scheduled to be held I've been looking for something else. All the more so after watching some of the presentations.

Future of Web Apps . . . I'll be there

Well the Future of web apps conference is being held in London on the 3rd and 4th October and this one I can attend. I've already booked my place and I'm not too busy this time around.

I'm flying down late on Tuesday and then leaving as soon as the conference ends but if you're in the area, or attending the conference, and want to meet up during the conference get in touch.

Roadtrip

The team behind the conference is also organising a roadtrip were they plan to go out and visit 12 European cities. They are visiting Edinburgh so I'll catch up with them then.

For a couple of weeks now I've been intending to release two new tools for this site. For one week they've been live on the site (though not linked to) and still I haven't mentioned them.

It could be said I'm procrastinating. This is a shame because as updates go this is even more significant than the dedicated pages for the scripts I've put together.

css/js compaction

The first tool is a port of the original js/css compaction tool and represents the penultimate migration of code from the old domain. All that is left now is the msn contacts web service which I currently have no plans to move across. This tool is basic and if you're working with several javascript or css files and making modifications frequently then there are probably better tools available. For the majority of people, who don't alter their javascript or css files very often and just want to save a little bandwidth or reduce their page loading times, then this tool could be ideal.

Check your spam status

The second tool is the most interesting and certainly required the most work. It's taking the ideas developed in these two posts about checking your rbl/spam status and automating the process to their logical conclusion. This second tool allows scheduled checking and automated updates of any change. At present it checks 75 real-time blacklists of varying degrees of importance. Furthermore it will check these lists up to once a week on your behalf.

It's a simple tool but it's easy to use and should be a big time saver. Expect more of the same . . . soon.

p.s. - I already have some ideas for additional tools but if you are having trouble finding a tool you think will make your life simpler then get in touch as I may be interested in building it as a part of this site. Easiest way to contact me is via a comment. If you don't want it to be public just state you want it to remain private in the message.

Back in March I posted about amberjack which allows you to create tours for your site. A couple of days ago Duncan Riley, over on techcrunch, highlighted another service. Demofuse also allows you to create site tours and has some nice features I didn't see with Amberjack.

The main advantage of demofuse is it allows you a lot more interaction with the page, highlighting sections, giving tips etc. Where I think it is weak is in moving around the site. This functionality just doesn't seem as smooth as with amberjack.

Beyond the functionality the main difference is that amberjack is a downloadable javascript library licensed under the LGPL and demofuse is a hosted service. There is no mention of a fee for using it but this is something to be aware about.

My personal preference would still be amberjack - I would prefer the control which a downloadable library allows and for my tastes the added functionality which demofuse offers would be better integrated more fully into the design of a site rather than being limited to a site tour. It's a tough call though and I can certainly see situations where demofuse would be the better option.

Things have been quiet around here in terms of entries so to keep things rolling here are a few interesting links I've discovered over the past few days . . .

Florian posted a comment over on the msn contact grab entry highlighting a warning error if you use the script in a PHP environment configured with safe_mode enabled. It's a good reminder to switch off error reporting (and switch to logging instead) in a production environment.

Blogsecurity posted a short article were they looked at the wordpress version being used on 50 blogs. They found that all but one of them were using vulnerable outdated versions. Personally I would have liked to see a larger sample size and some discussion about how the samples were picked but I highlight it here because of the followup posted by Vidyut Luther at phpcult.com. I've previously suggested that developers should be more aggressive in attempting to get users signed up to a mailing list highlighting critical updates but Vidyut takes it further and suggests that an application should go into "read only mode.. no new posts, no comments, until the user acknowledges the threat, and does something about it." I like it.

Compete recently opened their API meaning we now have an alternative to paying for Alexa data which is more than a little questionable (Yet another post showing inconsistencies in the data).

Dave Thomas (via Travis Swicegood) highlighted the charity fund-raising efforts of RailsConf which apparently rasied $33k and suggests that we "see if we can make all industry conferences into fund raising events." Now I give monthly to several charities whose work I support (so don't think the worst of me) but I'm not a particular fan of this idea. Granted I would much rather receive a pile of crap than another burlap bag but I don't like the idea of being 'encouraged' to support the work of a charity I know nothing about.

Dave asks us to, "Imagine what could happen if a conference with 5,000 attendees raised just $20 per attendee. Then imagine $50, or $100. It starts to get serious." I'll ask you to imagine what these charities would be able to do with a regular stream of income they could rely on over the long term. Having said that I'll be wandering through the streets of Edinburgh wearing a bra in June in support of breast cancer support charities. If you think the cause is important then sponsorship would be welcome via justgiving. It's conceivable that this makes me something of a hypocrite.

There were going to be more links but as I've already written a small essay I'll stop for now. Incidentally, the reason for the lack of posts is that I've been working to move the js/css file compression/compaction tool I set up on the old domain over to this site as well as creating a totally new tool. Should be ready to go live soon.

As I write this http://staples.co.uk is reporting that it is down. There are probably quite a few people interested in shopping at one of their stores going elsewhere. If the site was actually down that would be fair enough. It isn't though. http://www.staples.co.uk works perfectly.

You quite often see 'server not found' warnings when visiting a site without www., for example my university, but to get through to the site and have it tell you the site is down is even worse. It demonstrates that whoever manages the server knows what they are doing but carelessly hasn't set things up correctly.

It doesn't matter which way round you have things but all traffic to www. should be redirected to the www free version or vice versa.

There are three main reasons why this is important:
- All your visitors actually reach your site
- No duplicate content issues in the search engines (this is assuming that both versions of the site actually work)
- Strengthens search engine positions by concentrating all incoming links on one site.

All it takes is a couple of lines in a .htaccess file if you're on an Apache server

%{HTTP_HOST} ^www.domain.com$ [NC]
RewriteRule ^(.*)$ http://domain.com/$1 [R=301,L]

It's not difficult so please sort this out.

Since July last year (10 months) this blog has received over 20,000 spam comments. It's an interesting statistic but fairly meaningless. Far more important is what happened to them all.

From those 20,000 spam comments I've only actually seen 2. A simple combination of Akismet and the blogg.de blacklist have caught all the rest. That's a 99.99% success rate. For the 1-2 minutes of work it took to install and configure the spam protector plugin for serendipity I find that really very impressive.

Although these tools prevent spam creating a visible impact it still remains a problem. So many spam comments are going to use up a fair amount of bandwidth. Also, things are only getting worse. On average I've received 2,000 spam comments a month but I've received 1,000 in the past week alone. The pace is increasing.

I've discussed Bad Behaviour before but I've never felt that I personally need it. I think it is getting towards the point where I need to take another look.

I'm currently subscribed to the developershed newsletter. I mainly get them for the programming articles but having just read the emails from the past few months and it was a couple of SEO articles that really caught my attention.

The SEO field is great in that a lot of people are willing to share what they know. It's possible to take a sceptical perspective and say they're only looking to build their mind-share and brand but there is still good information out there. Brands are being made by offering real value.

This is why these two articles stuck out. These articles were not about valuable content. They were about self-promotion all the way. Continue reading "DeveloperShed - All about self-promotion?"