Scratch Bikes Data

July 13th, 2014 by Alistair MacDonald No comments »

Scratch Bikes in Newcastle

In this post I reveal a little secret about Newcastle’s now defunct cycle hire scheme and make the cycle location data collated over two years available.

Until the end of 2013 Newcastle had a cycle hire scheme called Scratch Bikes operated by Grand Scheme (formally both WhipBikes Ltd and Scratchbikes Ltd). Back in 2011 I decided to use the cycle location information on the Scratch Bikes web site to make an amazing animation like the animation done using data from the Boris Bikes in London. Sadly I found that the data was anything but live so I could not.

When I pointed out the web site and phone app were inaccurate Scratch Bikes were quite honest and came clean that the numbers were in fact entered manually when the bicycles were checked at the start of the day. I did offer to help link the systems together but sadly they did not respond to my offer. They did however try and hire a friend instead but I guess they had their reasons.

So the data…. In 2011 I set up a server intermittently checking the Scratch Bikes web site to find out how many cycles were at each station. As mentioned the data only changed once a day and was not enough for my needs, but I had already coded up the scraper to discover this so decided to leave it running just in case it was useful. This ran continually for 2 years. I do not know if it is useful.

You can download the Scratch Bikes Usage Data here. The time stamp is from the site and appears to be the time when the data was entered on to there system. There is a separate table with the location information in. One note is that I do not have the official data for Malcolm Street and Cuthbert House (presumed to be 3400 and 3405) so I have made a best guess. If you can fill in the blanks then please get in touch. Please take the data and do cool things with it, and let me know what you do with it.

I will just close by saying that reimplementing the phone service and building a web service is not hard for someone like me to do. In fact I can have it up and running within a couple of weeks, the web site and app will be live and accurate, and you would be able to book out cycles by text, voice, from a smart phone, and a PC as well. If someone wants to team up make this happen again in Newcastle then get in touch.

Where are the comments?

February 16th, 2014 by Alistair MacDonald No comments »

Recently this blog has been the target of a silly amount of spam comments and I am having trouble keeping it up because of that, so I have disabled comments for now.

4 years ago my blog received about 200 spam comments a month caused me to install Akismet. The spam continued to increase slowly until it got to an average of 2,000 comments a month. Even with this number Akismet was doing a good job and blocking almost all of it, leaving just a handful of comments for me to moderate manually.

Annoyingly things have rocketed now and last month I received 68,897 spam comments. That is around 90 spam comments every hour. This I could still cope with easily, but they tend to come in short waves and I am having trouble keeping my site responsive when that happens. If you have experienced an unexpected wait for up to 20 seconds and them the site just works again, then this is why. Ironically it is using Akismet that is causing the delay, but not using something like this would be clearly be impractical.

The truth be told I do not blog as much as I did, and that most people interact with my blog externally on Twitter and Facebook instead of the comments, so I have just disabled all comments for now. I am considering using nginx instead of Apache to better cope with this kind of traffic, or using a service such as Cloud Flare, but for now I hope disabling comments will keep the site fully responsive all the time.

Update: I am experimenting with some WordPress plugins to block some bot nets before Akismet is called. Fingers crossed this will reduce the spam to a level that my server and Akismet can cope with again. Very few users should be blocked in error but if they are then they can click back, wait a minute, and try again.

Fancy another laugh at my expense?

January 30th, 2014 by Alistair MacDonald No comments »

Bright ClubWell this is your opportunity on Tuesday evening.

A couple of year ago I preformed at Bright Club Newcastle, a “geek” style stand up comedy night. Well for some odd reason I have been asked back again. I guess they are desperate. Anyway, I will talking about my pet plastic duck on Tuesday 4th February.

If you want to come along there are details on the Centre for Life web site.

SnapCam to Close

December 21st, 2013 by Alistair MacDonald No comments »

snapcamlogoSnapcam is a simple Windows web cam application and web service I produced in 2003 as a pet project. It simply took an image at a set interval and uploaded it to the web service. The (almost) live image could then be embedded in the user’s web page. It is now outdated and sadly I need to close the free service down.

It was designed to be very easy to install without a complicate signup process, and it initially ran entirely on a free hosting service available at the time. Use has fallen away over the years as most people have move on to a more modern web cam solutions, or stopped completely. Today I don’t feel it worth updating the software or moving the service on to new hosting that I need to do. I am really pleased people found and used the service useful over the last decade and part of the code does live on in some time lapse experiments.

New users have not been able to start using the service for a while, and current users should have been receiving a warning message through the software. The service will stop functioning on or after December 24th 2013. If you wants to keep using the service after that date then please get in touch and I will see if I can come up with a low cost solution for you.

Farewell Rails Playground

December 21st, 2013 by Alistair MacDonald 1 comment »

Most of my web sites, including this blog, and various experiments have been hosted on Rails Playground that I migrated to 6 years ago. On the whole things have gone well with only one short outage in the first 5 years, but sadly over the last year there have been a lot of problems including outages, data loss, and security issues. This post is a quick summery of the problems. I will be posting about some other related changes in the very near future.

It all started when another of their customers started relaying spam email, perhaps because of a security floor, or perhaps they were just a spammer. Because of this all outgoing email was blocked without notice. I have to say I am not at all happy about the way this was done and it stopped all my services that use email dead without any prior notice or notification. I was also not best pleased at the attitude that this was not a critical part of the service and I was initially expected to adapt my services to another email system.

After this happened for a second time and a lot of moaning my sites were migrated to a new server. They did all the migration for me although it would have been easer to do myself. They inconsistently screwed up the dates when porting the databases. Being a US focused company with technicians in Asia migrating UK sites between servers in goodness knows where I can see how it happened, but it is not an excuse for the error.

Another problem is this new server was running the security mod called suEXEC that will allow email to be blocked per user, but unless some effort is put in it opens up a gaping great security hole. The way suEXEC was set up, and in fairness this is becoming quite common with the lower end hosting companies, will allow someone who can find a small exploit in any code to completely take over my sites. I like to think my production code is secure, but no one can 100% promise this, and holes in WordPress and PHPBB are being found all the time.

My general advice is if your ISP uses suEXEC, and the account you use to upload files is the same account that the web server uses to access the files, then just go to another ISP. If you have a separate user account for the upload and the server then this is the prefect setup, but if not you are better without suEXEC at all.

Anyway, the new server was also resulting in a huge percentage of cron jobs failing and I can not login using SSL most of the time. After a few more significant outages things got a little out of control and I decided I had to move away.

I have now moved everything that I am not culling to a new virtual machine hosted with Digital Ocean (and that is a referral link so if you are thinking of signing up then please use the link). Although I am a fan of managed shared hosting (despite it not being popular in some circles) I am having trouble finding a good and reliable ISP at a decent price. As a result I have had to go the self hosted route that will take a little more effort than I would like, but having done it before I feel I have the sys admin skills to keep things secure, efficient and a lot more stable than things have been recently.

I am truly am sad to have to be moving away from Rails Playground as they really have been good in the past and I have been happy to say so, but I need an ISP I can rely on and sadly I can not do this any more. :-(

Upgrading to Ubuntu 12.04 LTS and the MATE Desktop

November 12th, 2013 by Alistair MacDonald No comments »

I have had a lightweight “nettop” computer on my desk running Ubuntu 10.04 LTS for quite some time. I use this almost exclusively for accessing the web with Chrome, and doing some admin tasks that are just more simple to do on a Linux machine. Unfortunately Ubuntu 10.04 LTS has not been fully supported since earlier in the year, and more recently Google stopped updating Chrome for it so I decided it was time to move on to something more modern.

One reason I have not updated yet is that the desktop has changed from Gnome 2 to Unity that is like moving from Windows XP to Windows 7. I don’t dislike the new interface like many, but it does take up more memory and processing power, and this little machine does not have much of either. I was originally intending to migrate it to Linux Mint using the MATE desktop but decided to give Ubuntu 12.04 LTS a try first as it was a simple auto upgrade. Granted it took a few hours in the end but did mention this machine is low on processing power.

The update went smoothly but needed a lot of confirmations over the process so needed checking on frequently. Unfortunately it was a little laggy as predicted and used a large percentage of the small machine’s memory.

I then had the mad idea of trying to install the MATE desktop on top of the Ubuntu installation I already had instead of a complete reinstall. I followed the install instructions on the MATE web site and it all went very smoothly. My exact script is below but I would recommend checking back to the original web site in case the distribution changes.

sudo add-apt-repository "deb http://repo.mate-desktop.org/ubuntu precise main"
sudo apt-get update
# Ignore the error. We were expecting it.
# The following line will fix it but we needed to do the update first.
sudo apt-get --yes --quiet --allow-unauthenticated install mate-archive-keyring
sudo apt-get update
sudo apt-get install mate-core
sudo apt-get install mate-desktop-environment

After logging out I needed to select MATE from the cog icon by my name before logging in again, and then I was up and running with MATE. I did need to change almost all the desktop colour settings to be able to read all the text, but most of the other menu setup has been brought across from the old setup. I still need to tweak the font settings to improve the general look and feel further, and some of the apps still have colour settings so there is some tweaking to be done, but all this is a lot less hassle than a complete reinstall.

Also most importantly it did the trick with only 230MB RAM being used (approximately the same as before give or take a few meg) and the boot time is only a little slower at 100 seconds. Shutdown is about the same at 15 seconds. Chrome also works as well as it did and that is what I was after.

Granted it is annoying to have to do this at all, but to keep the security patches coming I needed to do something, and I would find it tricky to argue that it would have been sensible for anyone to support such old systems. If you are in the same position then give it some thought and I hope this proves useful.

Off road points on the new Google Maps mobile apps

September 1st, 2013 by Alistair MacDonald 1 comment »

A few years ago I blogged about locating on off road point on Google Maps. This apparently proved quite useful to a number of Geocachers and ramblers. Recently the Google Maps apps has change on most (or perhaps all) platforms and this hack will not longer work on mobile devices. :-(

The only solution I have found to date is to use the original hack on the Google Earth app instead. Other than using more battery this works well for me. You can turn on roads to simulate the hybrid mode, and I have started saving locations to a map on My Maps and then they are shown automatically on the mobile. I hope this helps (again). If you can figure out a better way then please let me know.

Update: It appears that searching for Lat Long coordinates on there own (without the original hack) will now drop a marker on that exact spot. This is working on the old and new maps online, and the iPhone app, so I presume it will work on all platforms. Let’s hope this continues working from now on.

Goodbye Television Centre

March 22nd, 2013 by Alistair MacDonald No comments »

tvc_from_above

This is a sad time for those who have fond memories for the good old BBC Television Centre. BBC News moved out last week, the last broadcast from the building happening as I write this, and the building closing at the end of the month. I am one of those people with fond memories.

When I was young my father, formally an electrician in the the theatre, became a broadcast engineer for the BBC and this is why we moved to the south of the country when I was at school. The coincidence of this is I gained access to one of the most interesting places in the world, that being TVC.

At first I thought I must have only been to the building half a dozen times, but writing this I realised just how many more times I have been there and how many memories I have. I have seriously lost count and have seen so much to look back on.

I have been on the Top of the Pop set (in TC2 I think), in the 1980′s Tardis, and watched the (slow) filming of many random programmes from the viewing galleries. I remember walking through the sets of Johnny Briggs, Rolland Rat, and the back of the Noels House Party (that at the time was more of a doughnut than the binding it was contained in).

I have sat in the old news studio when the automated cameras were new and subtitling was done from the corridor. I have seen the turntable the christmas penguins rotated on, I have played with the old COW (Computer Originated World). I was even the first to see the Kenny Everett title sequence when VT was in the basement.

I have walked the corridors and past so may doors. I have eaten in the canteen, drunk the tea, chatted in the BBC club. I have explored the old OB units. I have relaxed in the Blue Peter garden. I have played with the galleries and must be one of the few people who can use a vision mixer that were self tort. I have visited three generations of network control and discovered how stupid all the security keys expiring on new year is.

I have even helped with the TV7 refit in the 90s. I have also briefly been on kids TV but not filmed at TVC. There are so many other memories but if I keep going I will not stop for months.

The most amazing thing is I never got lost in the building. Well, yes, okay, that last fact is a complete lie.

One odd thing is most of my experiences of the place were as a child. Even in the 2000s it was starting to feel a bit foreign and like the next generation had taken over. The atmosphere was still there but I was no longer part of it.

I have not been back since the great Manchester exodus started to drain the life out of the building. I don’t really want to because now there is just a building. The life has gone. I guess the memories will still be there but the atmosphere and the community is what the BBC has really lost, and I think it has gone because I can not find is elsewhere.

So that is it. Goodbye Television Centre. I have some fond memories. They are old and at times they feel like I was a different person, and in many ways I was, but I treasure them never the less.

Just in case you were wondering anything I did within TV was pure nepotism, but my radio career was all my own work.

Goodbye Posterous, Goodbye Dopplr

February 20th, 2013 by Alistair MacDonald 1 comment »

This month I am saying goodbye to a couple of services that I have been using for a while.

First Posterous. At the end of last week it was announced that the micro blogging platform Posterous will be closing down at the end of April 2013. This has been anticipated for a while since Twitter bought it in March last year and assured users that they will not close down without notice and will create an official data backup path.

I started using Posterous when I moved my “Vox Blog” to it (actually migrating to TypePad and then Posterous as it was easer) after Vox was closed at the end of September 2010.

As this is a second blog of “silly stuff” that I do not upload much to it nowadays I have decided to just kill it off. I did consider moving it to another service such at Tumblr but it is not simple to get the archive in there and I have other things to do that are more important. I will however place a full copy of my Posterious blog on my web site and redirect from the original URL so the content will not vanish from the Internet.

There is a small bug in the export where the original Import from Vox is listed backwards, and the Image URLs are mashed, but I fixed the URLs with a little .htaccess hacking. If you need help to do the same please do ask.

Next is Dopplr. Dopplr is a service where you log where you will be travelling to and it allows you to see if your friends are going to the same location. When you attend a lot of events like I do it can actually be useful. The service has been running unchanged since it was bought by Nokia in September 2009. I give it a lot of credit for running apparently untouched for almost 3½ years. Sadly it has just started to play up by not allowing trips to be created, edited, or deleted. It is clearly not being maintained and about to die.

Rather than move to another system (I only know of Tripit and it is not quite the same) I have imported my last Dopplr backup in to my Google Calender. Clearly this is not going to duplicate Dopplr’s social element, but will help with my planning and this is what I have mostly used it for in recent times.

So that is two services gone and I suspect there will be more going before the end of the month. The Internet is always exciting because of the constant flow of new and exciting things on it, but most will eventually vanish or be swallowed up by other services regardless if they are successful or not. It is sad but the way it is.

UKHost4U caught posting fake positive comments

December 18th, 2012 by Alistair MacDonald 2 comments »

In 2007 my company had a bad experience with the company ukhost4u and I blogged about it. In summery I needed a service, called them to check if they included that service in there package, they said it did, after signing up it turned out they did not actually supply that service, and then they refused to refund the subscription. I blogged about it to warn other and them wrote the money off as one of those things.

After the original post I was getting a lot of comments on it. At first negative and then a few positive. Now as I have pointed out in the past a blog post about a negative experience tends to attract a lot of negative comments from users in the same situation. Satisfied customers will rarely be searching for the words that list such a blog post and hence will not get the chance to comment. I for one remember this when reading negative posts and comments. On my post however there were a few positive posts that struck me as odd.

Anyway, time passes and last Thursday a gentleman called David Eadie posted a comment on my blog post. It was a bit strong and I don’t wish for any company to be made bankrupt but I can understand how Mr Eadie feels.

Then, less than 25 minutes later I receive the following comment in my moderation queue….

I have been nothing but happy with the service provided by UKHost4u. They have always been very friendly and have actually taken the time to fix issues with my web site code which would have normally have cost money and a developers time.

Unlike other providers I have used the telephone calls are answered and most emails that I have sent in have had a reply within 1hr which is much better than the 48hrs that my last provider took to even advise me that it had been assigned to support.

I understand some people here don’t seem to have been happy with them, but I think they will find a lot of much worse providers and when you go with someone with NO telephone support, 48hrs email reply times and 80% MAX up time you will be more than happy to use UKHost4u.

Now, it is odd to get this kind of post because of the afore mentioned reasons, it does sound a bit like marketing spiel, and the web site they entered is for sale. I really do like comments of multiple views on my blog but I also don’t want to be a patsy delivering a fake review for a service I feel is bad. As a result I contact the poster via the gmail address they gave and asked if they can provide a link to there Facebook, LinkedIn, or other profile to help convince me they were a real person. I did not get a reply.

Then I decided that I might as well call the owner of the domain and they were kind enough to take my call and confirmed that they are the owner of the domain and they had not posted the comment. So now I have confirmation the details entered were fake, and logic dictates that is was posted by UKHost4u or an agent acting on there behalf, but there is no proof yet.

So, next on to the access logs for the site. It turns out that the post came from 149.6.122.114, and guess what, it is a UKHost4u IP address. In fact it appears to be the address used by the UKHost4u server.

I have contacted UKHost4u to give them a chance to comment and perhaps even apologise. They replied and implied it would have been one of there customers using an IP address registered to them. I sent the information they requested and stated I did not believe a customer of theirs would create a fake persona. I have not has a new response.

So, if you are looking for a new ISP and read a positive review of UKHost4u online from someone you don’t know and respect then I would take it with a pinch of salt.

If you are from UKHost4u, and I suspect you will be reading this, then there are a couple of lessons here. First being that putting so much effort in to scraping a few quid from your customers is not only bad customer service but will give a justified bad image online. Second that you are not as anonymous as you think so being dishonest will eventually come back and bite you. If you want to work with me to fix your ways and change your company attitude and image than I am available to help, but I suspect you are more interested in diminishing what I have said than actually fixing your problems.

css.php