Misc

Following my initial discussion on using geo-targeting to predict timezones I've finally found time to play around with some of these ideas.

The idea

Simplify the user experience by predicting the timezone someone is in and auto-populating a registration form accordingly.

The solution

At the time I wasn't really looking for any solutions other than matching an IP address up to an actual location. This is certainly possible and I'll show you how below but first I'll highlight the alternative. If you are reasonably confident that the timezone will be set correctly on the users computer it is possible to access that value via javascript.

GeoIP to Timezone

If you want to avoid relying on javascript then you still have some options.

Maxmind: The PEAR pacakage supporting maxminds geoip databases doesn't support timezone lookup but the script offered by maxmind directly does support timezone lookup. The usage seems a little convoluted but it is there.

IP2Location: The timezone information is integrated directly in the database once you reach 'DB11', their 11th database offering. The usage should be straightforward. There is only one drawback - DB11 costs $649/year. Personally this is more than I would be willing to spend but their client list is impressive so if this is within your budget spread the moolah and use my affiliate link .

Free solution: There are still a few options here. Although I haven't tried it I believe the maxmind code should work with their GeoLite products. Alternatively once you have a location, either using maxminds free databases or hostip.info you could try getting to a timezone using the 'world time engine' class available from phpclasses.org. The drawback with this approach is that even after the database lookup you still need to query two web services. That is going to introduce a significant lack in response time.

My preference

Although I started off looking at GeoIP services I would much prefer to be able to tackle this problem using javascript. It's not an ideal solution, some people will have their timezone set incorrectly while others will have javascript disabled but on balance I think it is good enough. This is the icing on the cake rather than core functionality after all.

The web service for fetching contacts from MSN messenger / Windows Live Messenger has been hit by a series of problems over the past few days. First I hit the bandwidth limit for the account used to host the service. Next, the server used was hit by a DDoS attack. Finally after all that was sorted out the bandwidth limit again caused problems.

Hopefully everything should now be back to normal and the service will be stable.

As always, if you do have any problems, my contact details are available under the 'About' tab at the top of the page.

I have a love hate relationship with geo-targeting. The web wasn't designed with making it easy to get the geographical location of connected computers in mind. A users geographical location is interesting and potentially valuable though and so methods have been developed to make it (almost) possible.

These methods typically involve something akin to a brute force attack. Figure out where enough IP addresses have been assigned and you can get a good idea of where a user is from their IP address. Other methods involve identifying the computers through which they are communicating with you and assuming the user is in the surrounding geographical area. Neither method is perfect but in the majority of cases you can know which country a user is in with reasonable accuracy.

What I hate about geo-targeting is how some sites think they can locate you more accurately than the country you are in. Maxmind, which is probably the commercial leader, thinks it can guess your location to your nearest city with an accuracy of 81% in the US. Outside of the US I suspect this drops considerably. I'm seeing fewer sites than I once did trying to tell me where I'm connecting to the internet from (and getting it wrong) so I'll skip forward to what I love about geo-targeting.

Continue reading "Geotargeting in forms"

Over a full month after the fact I present my summary of the BarcampScotland2008 event.

The event was split over two days. It kicked off on the Friday evening (1st Feb) in the main room of Alison House at the School of Architecture. It was just the one room with a decent communal feeling. I think most people were holding back with their presentations for the following day. Despite this two presentations did take place. James Littlejohn made the first presentation after the welcome session and talked about data portability.



It was a good summary of the current situation. I also happen to agree with most of his positions. He has taken the decision of making his homepage the hub of his social network. Although I think he has perhaps taken things a little too far my main criticism is the implementation. Looking at the site it took me about 20 seconds to figure out that aboyne is where he is based and not his surname. I couldn't find his surname anywhere on his homepage. After navigating around for a while I found it in the byline for his blog. It wasn't an easy process. I highly doubt that data was machine readable despite the importance he attached to this during his talk.

Ewan Spence was next up with an improvised talk he largely made up on the spot. This rapidly migrated to a conversation with some interesting points raised.

Following on from this Dave McClure set a small competition going. A excessive number of random words were gathered from the audience which then broke up into 5 groups to brainstorm company ideas around any pair of words. Somehow the team I was in won with sexydyslexia.com, a couple that takes standard prose and converts it into netspeak and vice-versa. I notice that the domain name is still available so although rated as the best apparently no one in the audience wanted to run with it.

All the details on the second day after the jump . . .

Continue reading "BarCampScotland2008 Roundup"

The second BarCampScotland event finished yesterday. This was the first BarCamp event I've attended and I have to say I was impressed.

Almost without exception I would describe the talks I attended as interesting or very interesting. I plan to post summaries and link to slides where possible in a later post.

The facilities were also quite impressive. Appleton Tower always looks to me to be a rather drab building from the outside but the concourse, where BarCamp was based, was a deceptively impressive space and certainly met our needs. Having said that I don't know whether they turn the heating off on the weekend but it was definitely chilly.

There were five large (easily seating 100+ people) lecture theatres available on two levels which were again well equipped. For the number of laptops I saw out on display the power sockets beneath every other seat would have been invaluable. It was only in the penultimate session I realised they were there but as that was when I started being concerned about power it all worked out well.

The Slides

I spoke in the morning about contact importers and where I felt they were going in the future.

I've embedded the slidecast below. Feel free to link to it or embed it elsewhere.

I plan on attending the second BarCamp event in Edinburgh at the start of next month. If you haven't heard of Barcamp before take a look at their site.

The first BarCampScotland even took place last year but I didn't know about it at the time so this will be the first I have attended. The intention with BarCamp seems to be to generate as much online content as possible. Videos on youtube, photos on flickr, presentations on slideshare. Wanting to know what I was letting myself in for I decided to set some time today to see what has previously been posted.

There isn't really much content, except photos, from the first BarCampScotland so I started to expand my search to BarCamp in general. There has been some decent content presented by various people. Below are some I found particularly interesting (though in no particular order . . .

DIY User Research
Leisa Reichelt

This isn't really an area I know much about though though have the uncomfortable feeling that I should. It is a nice introduction though be prepared for the slides running out before the 'story' is complete.

Leisa also presented at FOWA. Her slides and audio are also available.



How to Scale
George Palmer

This is perhaps the most complete overview of scaling a web application I've seen. I suspect the actual presentation would have felt something like a whirlwind going overhead but as we can take our time over each slide it's manageable, and very informative.



Introduction to SlideShare
Kapil Mohan

An introduction to SildeShare at BarCamp and posted to SlideShare. Naturally.



In my opinion they were the best 3 of the first 60. There were 275 in total. If you want to see more you can continue the search.

BarCampScotland

There were not many but there were some . . .







p.s. SlideShare supports audio. When did that happen? Lack of audio support was my main objection to the idea. Having said that none of the presentations have audio which is a shame.

The big facebook story this week has been the decision to boot Robert Scoble for running a scraper over his account so that he can move his 'social graph', more commonly know as his friends list, to other services. This was against their terms of service - naughty Scoble!

I'm not going to get into the moral issues about whether he breached the privacy of his friends by attempting to view their email addresses which they had granted him permission to view. My work on a MSN messenger contact grabber and highlighting gmail contact and yahoo contact grabbers probably speaks for itself.

Instead let's see if we can make sense of the terms of service Robert Scoble decided didn't apply to him.

I've taken relevant extracts from the terms of service at 3 websites. Try and guess which belongs to facebook.

Site 1

In addition, you agree not to use the Service or the Site to:

- harvest or collect email addresses or other contact information of other users from the Service or the Site by electronic or other means for the purposes of sending unsolicited emails or other unsolicited communications;

Provided that you are eligible for use of the Site, you are granted a limited license to access and use the Site and the Site Content and to download or print a copy of any portion of the Site Content to which you have properly gained access solely for your personal, non-commercial use, provided that you keep all copyright or other proprietary notices intact. Except for your own User Content, you may not upload or republish Site Content on any Internet, Intranet or Extranet site or incorporate the information in any other database or compilation, and any other use of the Site Content is strictly prohibited. Such license is subject to these Terms of Use and does not include use of any data mining, robots or similar data gathering or extraction methods.

Site 2

In using the service, you may not:

[...]
* use any automated process or service to access and/or use the service (such as a BOT, a spider, periodic caching of information stored by Microsoft, or "meta-searching");

Site 3

You agree not to access the Service by any means other than through the interface that is provided by Yahoo! for use in accessing the Service.

Social networks are built on scraping content

The first site is facebook. Robert Scoble clearly breached the terms of service. But here is where it gets interesting.

Site 2 is hotmail for which facebook provides a automated service, which could also be called a BOT, to grab contacts.

Site 3 is yahoo for which facebook provides a automated service to grab contacts.

Terms of service be damned! Mark Zuckerberg has a fortune to make and if that means being an accessory to millions of users breaching their contracts with other companies then so be it.

Many of the companies which facebook provides bots for are moving to support porting users data between services. On Thursday facebook was invited to do the same. I would ask that they give it serious thought.

As mentioned earlier this is the second, and final, post going into greater detail on one of the sessions at the recent (it's been sitting in my drafts folder for a week) FOWA conference in London.

Matt Biddulph discussed interacting with 3rd party sites and services and the portable social graph. This session was particularly interesting to me given my interest in handling contacts from MSN messenger, Gmail, AOL messenger (AIM) and Yahoo!.

Matt's talk focused around how we can move beyond the use of such scripts with their multitude of risks to a situation were a user can join a new site, release the minimum data necessary and quickly identify their friends already using the service.

For those wondering what risks I am talking about the current situation means that for a site to view your contacts in a service they need total access. This means they could view your mail and send mail through your account. Where a username/password combination is used to access other services on a site these would also be compromised, so to take Google as an example you also grant access to Google Checkout, Adsense, Analytics etc. This is the situation today. Even Dopplr, the contact importing functionality of which the conference chairs spoke about with nothing but praise, requires your username and password to import your Gmail contacts.

Luckily, this situation is on the cusp of improving. For services where you are happy to make your list of friends publicly viewable marking them up with microformats is a simple way to allow other services to make sense of your friends list. Matt mentioned that twitter was also thinking of taking this a step further and supporting openid so that a user could prove that a friends list really was their friends list. I'm not terribly familiar with openid but I assume this wouldn't lead to once again proliferating login details and you simply delegate to your actual provider. Correct me if I'm wrong.

This works well when you're happy for your contacts to be public but you're probably going to want to keep at least some of the contacts in your email address book private so another solution needs to be found. Thankfully a standard way of achieving this is being developed. In fact, Matt talked about five.

Yahoo! BBAuth
Google AuthSub
flickr authentication
AOL OpenAuth
OAuth

I've also talked previously about Windows Live Contacts Control which although more limited in scope for the purposes of this discussion it does much the same thing.

All these services, and hopefully there will be convergence in the standards, open up parts of that glorious social graph and they do it in a safe manner. They also work without an absolute demand for javascript which was one of my main criticisms of Windows Live Contacts when I first spoke about it. Even the Microsoft service now has a RESTful API, either I missed that when I first took a look or it's new.

Hopefully it won't be too long before most of the code on this site is nothing more than an historical curiosity. I don't think we are there yet but soon. . .

The slides from Matt's talk are also now available.

After flying back to Edinburgh after attending the Future of Web Apps conference in London Thursday night and spending Friday catching up with work it's time for a round up of what happened. There are a couple of topics I'm going to go into greater detail on in future posts but here I present to you the exhibitors, speakers, sites and ideas worthy of mention.

The conference kicked off with a keynote from Om Malik discussing 'What is the Future of Web Apps?' Mike Arrington from Techcrunch decided to gatecrash 15 minutes or so into the keynote. The conversation that followed was interesting though with the pessimism from Om working well with Mikes optimism. I've been following Techcrunch for a while but have now added GigaOm for the potentially balancing effect.

Ben Forsaith then demoed '10 Real-world apps' in 10 minutes. Surprisingly 9 of the 10 worked without a problem. The most interesting one was probably Buzzword which is a word processing app. Online office products have been getting a lot of attention recently with available anywhere functionality playing against the more basic options. Buzzwords really grabbed my attention because during the, admittedly short, demo it looked like it could wipe the floor with Microsoft Word when it came to handle images and altering the layout of the page. I frequently have to break reports into 5 or more sections to maintain the layout so if buzzwords performs as well with large files as it did during the demo then it may be goodbye Word. I haven't tried it yet but I've bookmarked it to try later.

Site Speed and User Experience

As I mentioned in the Benchmarks, Site Speed and User Experience post the first speaker of the day following the keynotes was Steve Souders discussing 'High Performance Websites'. Watch for a post discussing this in more detail later.

The quality of speakers stayed high throughout. I think on the first day the most informative/interesting speakers came at the end with Heidi Pollock discussing mobile applications which is an area I haven't previously looked at and John Resig who talked about some of the really interesting things coming up in Firefox.

On the evening of the first day Diggnation was filmed on the keynote stage in front of a packed audience. I've not watched diggnation before but it was absolutely hilarious live. I think it is only available for premium members at the moment but if you know different let me know as I would like to see what the video version was like.

Day Two

From reading the schedule I wasn't as excited by the second day as I was by the first but there was no need for worry. Simon Wardley got through 300 slides in 30 minutes with a highly engaging talk about commoditisation and utility computing. John Aizen and Eran Shir discussed the semantic web from their work at dapper. Matt Biddulph from dopplr discussed smart integration with third party sites. I'll be going into more detail on this later as well. The final session I went to was with Dick Costolo from feedburner and focused more on the business side but was interesting all the same.

Unfortunately I had to leave before the final keynotes to catch my flight but overall I felt it was a very good conference.

Expo

In addition to the conference there was also the expo hall with some interesting exhibitors.

Fav.or.it may just have what it takes to lure me away from google reader. It hasn't been officially launched yet but from what I saw during a demo it's a very interesting product. It is also built on the Zend framework which makes it worthy of note from a PHP viewpoint.

Widr sounded very promising. It's a geolocation service for the internet. It's going to potentially be more accurate than relying solely on the IP. If I understand the product correctly though I suspect it will always be a niche product as the user needs to install software for it to work. I suspect they also made a mistake in going for a .co.uk domain name rather than .com. The product has global appeal so to me a .com makes more sense.

Xcalibre launched their new flexiscale product which is probably best described as competitor for Amazon S3 and EC2. It looks like a very interesting product and from a technical perspective I suspect the better between the two but I worry that the strength of the British pound will make it less competitive on pricing.

Finally I'll highlight soup.io which is a blogging platform for less serious content. Probably best described as occupying the market between wordpress.com et al and twitter et al. It's not something I plan on using myself but it looked like a nice product which I could recommend to less web savvy family and friends.

After learning about the future of web design conference back in January and then deciding I was just going to be too busy in April when the conference was scheduled to be held I've been looking for something else. All the more so after watching some of the presentations.

Future of Web Apps . . . I'll be there

Well the Future of web apps conference is being held in London on the 3rd and 4th October and this one I can attend. I've already booked my place and I'm not too busy this time around.

I'm flying down late on Tuesday and then leaving as soon as the conference ends but if you're in the area, or attending the conference, and want to meet up during the conference get in touch.

Roadtrip

The team behind the conference is also organising a roadtrip were they plan to go out and visit 12 European cities. They are visiting Edinburgh so I'll catch up with them then.

Things have been quiet around here in terms of entries so to keep things rolling here are a few interesting links I've discovered over the past few days . . .

Florian posted a comment over on the msn contact grab entry highlighting a warning error if you use the script in a PHP environment configured with safe_mode enabled. It's a good reminder to switch off error reporting (and switch to logging instead) in a production environment.

Blogsecurity posted a short article were they looked at the wordpress version being used on 50 blogs. They found that all but one of them were using vulnerable outdated versions. Personally I would have liked to see a larger sample size and some discussion about how the samples were picked but I highlight it here because of the followup posted by Vidyut Luther at phpcult.com. I've previously suggested that developers should be more aggressive in attempting to get users signed up to a mailing list highlighting critical updates but Vidyut takes it further and suggests that an application should go into "read only mode.. no new posts, no comments, until the user acknowledges the threat, and does something about it." I like it.

Compete recently opened their API meaning we now have an alternative to paying for Alexa data which is more than a little questionable (Yet another post showing inconsistencies in the data).

Dave Thomas (via Travis Swicegood) highlighted the charity fund-raising efforts of RailsConf which apparently rasied $33k and suggests that we "see if we can make all industry conferences into fund raising events." Now I give monthly to several charities whose work I support (so don't think the worst of me) but I'm not a particular fan of this idea. Granted I would much rather receive a pile of crap than another burlap bag but I don't like the idea of being 'encouraged' to support the work of a charity I know nothing about.

Dave asks us to, "Imagine what could happen if a conference with 5,000 attendees raised just $20 per attendee. Then imagine $50, or $100. It starts to get serious." I'll ask you to imagine what these charities would be able to do with a regular stream of income they could rely on over the long term. Having said that I'll be wandering through the streets of Edinburgh wearing a bra in June in support of breast cancer support charities. If you think the cause is important then sponsorship would be welcome via justgiving. It's conceivable that this makes me something of a hypocrite.

There were going to be more links but as I've already written a small essay I'll stop for now. Incidentally, the reason for the lack of posts is that I've been working to move the js/css file compression/compaction tool I set up on the old domain over to this site as well as creating a totally new tool. Should be ready to go live soon.

Via phpdeveloper.org I track the latest phpclasses releases and given my work on contact grabbing scripts I was interested to see that a new contact grabber script has been posted.

I haven't tried it yet but apparently it can connect to hotmail, yahoo, gmail, orkut, rediff and myspace. It is an impressive collection of scripts. It uses libgmailer to get the gmail contacts just like I chose to do.

It also uses a file called msn_contact_grab.class.php to get the hotmail contacts which rings some bells. Sure enough on closer inspection it's the file I created to fetch a MSN messenger contact list. Except it's outdated.

Generally speaking I wouldn't have a problem with the script being included in another project. In fact I would encourage it. In this instance though there are a few problems.

Inaccuracies: This package is being promoted as fetching hotmail contacts for which it presumably relies on my script. My script does not fetch hotmail contacts. It fetches MSN messenger contacts. Although frequently similar they are not the same.

Updates: When you're releasing third party scripts as part of your project updates are going to be slow to filter through. It's a shame that just days after the package was released it already needs updating.

Licensing: The whole package has been released under the GNU General Public License (GPL). Given that the msn contact grabbing class has never been released under such terms is this licensing appropriate? I would argue that it isn't.

I spent quite a bit of time considering licensing when I first released the script. Rather than go with the GPL I decided to stick with simply 'free' to reflect the licensing of the original script from which my script was derived. I didn't feel it was appropriate to decide on a licence even though the script was now significantly different to the original version.

Contact: If the author of the new phpclasses package had made contact all the above problems could have been avoided. In fact I would probably have promoted the package here on the blog. It's good to talk.

These are all small things but I highlight them because each one could easily have been dealt with. I've now sent off an email and suspect that everything can be easily resolved. It's just a shame it has to be done after the fact.

Has anyone had similar experiences? How have you made improvements to prevent similar problems emerging? I think the first thing I need to do is make it easier for people to contact me. Secondly, I need to take another look at the licensing.

After mentioning in January the FOWD Conference which took place a few days ago on the 18th April it's safe to assume from the lack of coverage on the blog that I was sadly unable to attend.

All is not lost though as presentations and MP3s of the various presentations are now available on the official website. If you were unable to attend why not take a look?

As some of you have already noticed over the past day comments have been disabled. During this time the site has been moved to a new server. This should allow me to better handle the growth of the site.

The move seems to have been relatively pain free but if you spot any problems or anything you think is a bug please let me know in the comments of this post.

The big news of the day appears to be eBay acquiring StumbleUpon. A fair question is to ask why. To me it doesn't really make sense. Om Malik at GigaOM has an interesting take though

By marrying the [StumbleUpon] toolbar to Skype client, eBay can do an end run around Google’s dominance of the search business. A simple search box inside Skype client is all it would take.

I haven't used Skype so I couldn't comment on whether this would be a workable strategy but I do know that I have never used StumbleUpon for search. If I want to search for something I generally use google. If I want to kill 5 minutes I use the StumbleUpon toolbar.

If eBay genuinely was looking to get into the search business then this deal doesn't really make sense to me. Especially with technorati flaunting its statistical finery.

The second part of this news story is the response of google. Supposedly a potential suitor for StumbleUpon until very recently google has released a new feature for the Google Toolbar, on the same day as the acquisition, which closely resembles the StumbleUpon functionality.

With Microsoft complaining about Google acquiring DoubleClick and demanding an anti-trust investigation after presumably being priced out of the running itself and now the response of Google to the acquisition of StumbleUpon does anyone else feel that the big players have abandoned the idea of innovation and decided just to squabble amongst themselves? Their conduct seems to be increasingly petty.