A Cycle Log to Fareham

This is as much a log as anything else, so bear with me. After my post the other day I decided to make a note of all the ‘tiny inconveniences’ and notes on a recent shopping trip to local market town, Fareham.

The theory goes like this: every situation that unduly burdens the cyclist is a disincentive to use the track (and, possibly, to cycle at all).

I’m not a huge cycling fan. It’s as much for utility as anything else, but my purpose for logging this is to capture all the little bugbears that steadily erode my willingness to cycle over public transport or (more likely) the car.

Trip started off at the north edge of Lee-on-Solent, towards Fareham. I was keen to get onto the dedicated bus route as Newgate Lane, the most direct route, is frustratingly narrow and busy.


Screenshot 2014-11-26 11.23.49
Lee at the bottom; Fareham at the top. I took the rightmost line to Fareham and returned on the left-hand one.

New Daedalus Junction – Shared path on northbound side. The pedestrian/cyclist crossing is staggered, so there are two sets of lights to press and wait at, just to get across one road. The estate is largely unbuilt so the road is quiet. With few cars turning in or out, there’s little point in waiting. In reality, as I was simply going ahead it would have been easier to rejoin the road and come back off to the cycle track again.

Peel Common Roundabout – To get round this roundabout from Lee to Gosport you need to cross two fairly busy roads. Both have dedicated signals for crossing, and these seemed reasonably quick.

There’s quite a deep trench in the path. Clearly somebody has come along and dug across the path, and the tarmac has subsequently sunk. At about an inch deep and a sharp edge I can’t imagine that does tyres any favours.

Recreation ground. Rarely used (except for roadsweepers?). The humps don't meet the kerbs making for a bumpy ride. [Source; Google Streetview]
Recreation ground. Rarely used (except for roadsweepers?). The humps don’t meet the kerbs making for a bumpy ride. [Source: Google Streetview]
After coming off Peel Common Roundabout the cycle path crosses the Recreation Ground entrance. This is normally closed – I’m not sure if it’s even used any more, but cyclists must give way to any traffic potentially coming in. There are humps on the entrance road where the path crosses (with an island inbetween), but these don’t extend to the edges giving the cyclist four nasty bangs if they’re not careful.

I came off the cycle path and round the Peel Common estate to connect to Tukes Avenue. Not sure if this is quicker, but I know the cycle path has several priority issues (cars over cyclists) which put me off following that path.

The new 'busway' - actually a really nice stretch for cycling.
The new ‘busway’ – actually a really nice stretch for cycling. [Source: Google Streetview]
At Wych Lane I join the new dedicated bus road. At the Wych Lane junction turning into the road is prohibited except for buses and cycles. On the road itself, there’s a “no motor vehicles” sign applied ‘except for authorised buses’. There’s also a ‘no pedestrians’ sign. It would have been reassuring to see a cycle sign just to make sure I was supposed to use the road. The signs technically allow it but with so many prohibition signs and cameras about I had residual doubt. That said, I’m overly cautious anyway so for most I doubt this matters.

Further down, by the Palmerston Business Centre, there are even signs for ‘Cyclists Dismount’. I’m not sure if this is intended for path-users or not but I figured I could continue using the road.

The bus road itself is fantastic to cycle down. Dead straight, very little traffic (naturally) and smooth.

At the end of the bus route, just before Redlands Lane a little cycle path deviated to the right. I know there’s a path on the far side of the A32, but there were no direction signs anywhere in sight. It would be useful to know the ‘recommended’ route to Fareham from here.

I turned right here, onto the cycle path. Pedestrians crossing the path had right of way, which left me waiting in the middle for a bit. Not the worst junction by a long way, but might scare a few novices.
I turned right here, onto the cycle path. Pedestrians crossing the path had right of way, which left me waiting in the middle for a bit. Not the worst junction by a long way, but might scare a few novices. [Source: Google Streetview]
I stuck with Redlands Lane and headed towards a path I knew, starting at the junction with St Michael’s Grove. This was a little tricky as several cars (and a bus) were trying to negotiate the junction and pedestrians were walking across the path entrance completely unaware of me. I waited in the middle of the road for them to cross but I can imagine this manoeuvre wouldn’t be to everybody’s liking.

The cycle path crosses a small residential road and crosses one of the streams leading to the Creek. Here, pedestrians are granted a fairly substantial bridge while cyclists follow the hill down and back up again. Unfortunately this stream also seems to be a favourite destination for dog walkers who naturally need to use the same (narrow) path. I had one nearly walk out on me, which – having just come down a fairly steep hill – was a good test for the brakes. There is good visibility here so realistically cyclists should see any potential hazards well in advance.

This path then turns under the railway and Western Way, with a rather blind bend caused by the bridge brickwork. This is also quite a narrow section, so cyclists and pedestrians alike have to be ready for anything.

Finally the path dumps the cyclist rather unceremoniously into a residential street, Crescent Road behind West Street. Poor signage again leaves the cyclist wondering which way is recommended, although the main street is clearly nearby and there aren’t that many options. Caution needs to be applied here as a garage is operational – I stopped to let a van reverse but wasn’t entirely sure he ever saw me.

Good bit of segregation here to protect the cyclist, although I suspect most don't bother waiting. [Source: Google Streetview]
Good bit of segregation here to protect the cyclist, although I suspect most don’t bother waiting. [Source: Google Streetview]
Once on West Street a cycle strip alongside the road takes the cyclist towards the town centre. A set of traffic lights protects the cyclist (continuing straight) as traffic is forced left onto Trinity Street. I had to wait a while for the lights to change, and imagine most cyclists don’t bother waiting. That said, the protected turn was nice to have.

Finally, to Fareham Town Centre where there was a bike stand waiting for me. Nice and straightforward.

On the way back, I decided to take a slightly different route and followed the journey as far as Redlands Lane. Here, I continued down St Michael’s Grove all the way to the bottom. Halfway down there was a sign directing Stubbington cyclists to turn right, but by the time I spotted this it was too late and traffic was passing me.

Sharp turn to join the cycle path, and judging by the grass damage it looks like a common issue. [Source; Google Streetview]
Sharp turn to join the cycle path, and judging by the grass damage it looks like a common issue. [Source; Google Streetview]
Once on Longfield Avenue I cycled on the road until a sign directed me onto the pavement. This was nearly a ninety-degree bend – quite sharp, but in hindsight (and with Google Maps) I realise this was intended for cyclists crossing from the other side – presumably the route I’d missed earlier.

From here it was a straight run up to the roundabout with Peak Lane, which I took to head towards Stubbington. This is a great path, nice and straight with a fairly smooth ride. However, once I’d built up speed I had to put on the brakes again to cross a small side road – I believe this is a restricted road to Newgate Lane. Again, cyclists must give way to traffic and despite nearly no traffic it is a blind corner so the cyclist must slow right down before crossing. I would bet most cyclists risk it – having built up speed. Also worth noting if I’d cycled on the main road, of course, I’d have no such priority issues.

Road crossing halfway up Peak Lane. The cyclist must slow right down and give way (blind corner). Google proves here that at least somebody uses the road! [Source: Google Streetview]
Road crossing halfway up Peak Lane on an otherwise long, straight, flat section. The cyclist must slow right down and give way (blind corner). Google proves here that at least somebody uses the road! [Source: Google Streetview]
In Stubbington the path crosses another small side road (again with priority to crossing traffic) before merging the cyclist onto a strip on the road. This is a bit of an awkward junction, since the cyclist must watch both the side road and the traffic behind them to ensure both a cross and a merge can be performed safely.

Cyclists are directed to turn onto Windermere Avenue which passes Meoncross School. As luck would have it it was kicking-out time and the road was congested with school-run drivers. I got a little satisfaction from passing them all.

At the bottom end, this turns into Burnt House Lane where the same school-runners were queuing to get onto the Gosport Road. The lane is narrow here and I had trouble passing the traffic. Admittedly I could’ve been a little more aggressive but many of the cars were too close to the kerb to pass on the inside.

Once through Stubbington I followed Gosport Road all the way back to the Mark’s Road junction, where a shared path re-emerges. This is a tricky junction and I’m sure it used to be signposted better.

Essentially the cyclist must take a left turn into the adjacent layby and move onto the pavement. The left turn is sharp but quite doable, however not signposted at all. They must then cross Mark’s Road (newly traffic light controlled) and cross the Gosport Road (again, with lights) to end up on the cycle path. I pressed the button for the first set of lights but after about twenty seconds the ‘please wait’ light disappeared. I ended up just waiting for a quiet time to cross, but this seemed fairly idiotic.

The second set of lights worked rather better and changed almost immediately. I crossed the main road and met the cycle path to take me back to Lee.

it’s worth adding that this section of Gosport Road (between Stubb and Peel Common Roundabout) is quite narrow and has a lot of motor traffic, yet at least two cyclists passed me on the road, while I used the dedicated path. I can’t help but wonder if the sheer hassle of crossing those junctions is enough to discourage them from bothering.

The Demise of Home Delivery?

I can’t help but feel today’s announcement that Amazon will offer collection from Post Offices is the tipping point for home delivery – actually for its demise. It seems logical to view home delivery as a critical mass. There needs to be a minimum density of deliveries per mile per day to justify the running of the vehicles, the staff, the infrastructure. If the deliveries for a parcel company are too far apart, the costs must go up and home delivery loses appeal. It’s a self-perpetuating route to failure.

Couple this with the sheer inconvenience of home delivery – taking time off or avoiding leaving the house to wait for a parcel seems ludicrous. I’ve encountered many businesses which ban employees from sending parcels to work. Home delivery is, in many ways, inconvenient.

Just yesterday on BBC News two ‘click and collect’ business owners were interviewed. Both companies operate those lockers you sometimes see next to supermarkets and in petrol stations. A code on your mobile phone unlocks the right door and your package is revealed. Business is booming, and growth rate is impressive.

All of this is eating away at the delivery companies’ model, and in a substantial way. Again, a critical mass model will fall apart when that mass is no longer maintained – i.e. when enough people switch away from home delivery – and the model spirals into unsustainability.

There are clearly opportunities remaining for home delivery companies: B2B and rural locations are two that come to mind, although I suspect prices might rise for both. For the rest of us, the appeal of picking up our packages at a local Post Office (or increasingly it seems, the Co-op) at our convenience must surely be greater than waiting for the parcel to come to us..

Surface Pro 3

Back in September I bought a Surface Pro 3 as they were being released in the UK. The first dilemma was choosing the device. I wanted one powerful enough not to feel sluggish, but the price escalation was quite significant so I had to choose well.

My choice: the mid-range i5, 4GB RAM, 128GB disk. For a general purpose work device, this appears to have been the perfect choice.

There are plenty of reviews online, describing the pros and cons of the device. Some daft sods have even pulled it apart to see what’s inside. I won’t go into too much detail, but let’s cover the basics.

The Screen

The screen is lovely. It’s nice and sharp, with a high-density screen meaning fonts look good. It’s possible to adjust the resolution and fit more on the screen, but I found the touch screen (which is useful) quickly loses its appeal as your fat finger becomes more of a clumsy liability than a helpful pointer.

At the default settings, I get an effective screen size of 1440×960, which is a reasonable workspace.

I haven’t yet hooked up a monitor (you need to go buy extras to convert from their diddy DisplayPort to something compatible with most monitors). Supposedly this device could run with a huge 4K screen, so the options are there if you need more ‘real estate’.

The screen is bright. I’ve used it outside in September sun with few problems. Yes, it’s reflective but at its maximum setting the screen easily outshines most local light sources.

I’ve found mine has a small but irritating brightness wobble. I’m not sure if this is by design (adaptive lighting and whatnot) but it’s distracting and I find myself turning up the brightness to avoid its effects.

The Disk

My business usually revolves around Word, Excel and online databases, so my disk requirements aren’t huge. If you’ve a lot of photos or video you probably ought to consider an external disk or online options. The Surface Pro has a slot for MicroSD cards but I haven’t had a reason to use this yet. It might give a helpful capacity boost but I’m not aware SD cards are known for their speed.

Many cloud services like Dropbox will dump all your files on the device when you add a new computer, which annoyingly eats up space unless you selectively choose to ignore some folders. Microsoft’s OneDrive offers online-only sync, which seems eminently more sensible on portable devices. Roughly speaking, your files appear but don’t take up any space until you choose to open them or keep them offline. It’s a sensible option and makes a big difference on these devices.

If you’re sensible you ought to have no problems with a smaller disk, but individual requirements will clearly matter. I’m still evaluating cloud services but will likely settle on one which offers online-only modes to save space.

The Keyboard

Let’s get one thing out of the way first. I really resent paying extra for the keyboard. It’s £100-odd on top of an already premium device. The problem is that it’s pretty well necessary to make the most of the Surface Pro.

I spent about two weeks with the device before acknowledging I would need to spend out more. WIthout the Type Cover Keyboard designed for the Pro you’re left with two choices: use a third-party USB/bluetooth keyboard or use the on-screen one. The former comes with the obvious obligation to carry yet another device. The latter is straightforwardly annoying, with the on-screen keyboard taking nearly half of the screen. Windows becomes letterbox-sized and is basically unusable.

I started with the on-screen keyboard but couldn’t get anything done without constantly scrolling. I moved to a bluetooth keyboard but became frustrated with the device overhead. So, I took the plunge and bought the Type Cover.

Skeptical at first, I rather like the keyboard. I’m a decent touch typist and can rattle out a document nearly as quickly as I can at home. The keyboard is comfortable enough and the keys feel well laid out. Usual ergonomics apply: as a flat laptop-style keyboard this is not something you want to use for hours on end, but it’s certainly effective and productive.

The keyboard has a backlight which is handy if you enjoy typing in darkness. The function keys are a little frustrating. Home, End, etc. are shared with function keys, so a secondary Fn keypress needs to be used to switch from one to the other. Annoying, but it’s something I’m getting steadily accustomed to – and seems to be fairly common nowadays anyway.

The touchpad is okay. I’ve found mine a little flimsy – enough to wonder if it’s a defect – but reviews elsewhere have made similar comments and the thing works. Realistically as more programs, web sites and the OS are gearing up for touch screens I find myself increasingly tapping on the screen to get something done, rather than move the cursor about. It saves time and works well.

The keyboard is magnetically attached, which is fun to snap on and off – and convenient if I’m working on something which doesn’t need it. Once attached it folds over the screen and offers some protection when the device is being carried about.

The Pen

As a fan of OneNote and an idle doodler, I use my pen quite a lot. It’s responsive, accurate and comfortable to use. My handwriting is usually recognisable but since I type faster than I write, it’s rarely put to the test. The Surface Pro has been an impressive companion at meetings and conferences, where I can quickly wake it up, jot some notes and shut it off again. OneNote’s workflow for new notes can be a little cumbersome but the hardware is in place.

Battery

I’ve managed day trips without my charger, firing it up at the airport at 6am, working through a conference and with still enough juice to see me through the return. This is with a reasonably conservative use and dimmer screen. If I’m working remotely at a desk I’ll take my charger for assurance but as-is the battery is quite capable of seeing me through most days.

Unlike my laptop, I get no indication of battery life remaining. This is supposedly to do with Microsoft’s smart standby technology, which I’ve mixed feelings about. The idea is that the device continues to run in a low-power mode, receiving emails and keeping itself ticking over – like a phone. In reality, my phone already fulfills the notification role and on a few disappointing occasions I’ve opened the Surface to discover the whole thing had failed to fall asleep and the battery nearly depleted.

As a result, I’ve now taken to shutting the device down completely when I’m likely to leave it for an hour or more. This is – I suspect – a problem with software, but it’s incredibly irritating to have to form counter-productive habits around this sort of stuff.

The charger is quite small, so easy to carry about, with a handy USB slot included. The power cable is magnetically attached and reversible, although I could do with the cable being a wee bit longer.

I’ve some concerns about having a battery sealed in the device. My laptops have never fared too well from constant usage, and batteries wear easily with constant recharges. My fear is that – after two years’ usage – the battery in this will be nigh-on useless. It’s a shame technology hasn’t improved around this as a fixed battery is likely a compromise for such a trim device, and I wonder what I’ll be thinking once its usefulness has deteriorated.

The Result

I bought this device because I needed a capable companion at meetings and on the move. I think it’s fulfilled that role rather wonderfully. As a ‘power’ user of Office and desktop apps, I’m still tied to the Windows ecosystem and the Surface Pro, being Microsoft’s flagship device, demonstrates its vision of how operating systems and devices can work together effectively.

My eventual goal is to shift to the Surface Pro entirely. I really feel productive on this device, and as my work patterns are changing (long story) I ought to be its ideal user – a mix of Windows apps, Office and the browser, all in a portable setting. This will undoubtedly form the basis for another post.

I’m not disappointed by the capabilities, but the quality needs some work. The Type Cover Keyboard came with a special fabric loop for the pen (which has nowhere else to live). This is left to the customer to attach, and whichever way you place it it looks tacked on. For a £100 keyboard and a £40-odd pen it feels like a cheap afterthought.

Windows 8.1 continues to be a clunky bastard amalgamation of desktop and tablet, but early reviews of Windows 10 are impressive. I can’t wait to use it – it seems made for Surface – but am not yet ready to risk installing a development version on this device.

The i5 4GB model has suited me, and reviews have suggested the i3 runs well enough – although I’d be reluctant to run more than the basic Office suite on that one. The i7 could offer a useful boost for the high-end requirements, although I question whether such a powerful processor is ready to be put into this form factor yet – my fan has rarely kicked in but I’ve read i7 reviews which complain of constant fan noise, heat issues and poor battery performance.

I would recommend the Surface Pro 3 (particularly this model) to those looking for a portable WIndows device with minimal fuss and good build quality. I’d strongly recommend the Type Cover keyboard as well – out of necessity. It’s worth checking with i3 owners to see how they’re finding things but performance has not once been an issue for me (that might change as I plan to install Adobe Creative Suite soon…)

If you’re not bound to Windows, a Nexus or Chromebook might fulfill your needs. Apple equivalents are, of course, available. I used a Nexus tablet for a long while and really liked it, but my productivity was limited by lack of split screen and mediocre Office apps. My requirements are specific; others’ mileage may vary.

Cycling and Whatnot

Argh! Why is there a cyclist on the road when there’s a perfectly fine cycle track right next to them?

I used to get annoyed at the cyclists: perhaps too lazy or ignorant to switch across to the track. Now I realise I was wrong. It’s the tracks themselves which ought to attract the blame.

There are a few roundabouts near our home. The council has invested goodness-knows many thousands of pounds installing traffic lights at almost every junction. They are bike-activated. The cyclist must negotiate each set of lights in turn, cross to the wrong side of the road, use a prepared track and cross back over several hundred yards later.

It’s quicker to cycle on the road. Despite all the efforts of the council, cyclists do just that: they keep to the road. Mission failed.

As a cyclist I couldn’t bear to break the rules at first. I’d stop, wait and follow the lines all the way through. It took a long time but I felt good for being good. Over time though, my patience slipped and I’d follow the quicker route on the road. I felt a little dirty, irresponsible and guilty, but by goodness it was nice to be able to get somewhere without faffing about.

Nowadays the British highway engineers are increasingly looking across the North Sea to our Dutch neighbours for inspiration. Cycling is a national passion over there, right up with pancakes and windmills.

Having now had the pleasure of nearly a decade in the Netherlands, both as a regular cyclist and a car driver, I can attest to the sheer brilliance of the Dutch cycle network.

It creates an equal footing between the driver and the cyclist: drivers yield to cyclists to offer the latter some continuity; cyclists face penalties for breaking the rules. Both are expected to offer and receive respect in equal doses.

Above all else, the Dutch cycle network offers continuity. The cycle paths, be they dedicated, shared or by road, are uninterrupted. Bar few exceptions, traffic lights exist only where they also exist for motor users – you can pretty well cycle to most places without having to get off.

Similarly, the small matters count a great deal: cycle paths have priority over side road traffic. If a car turns into a side road they must yield to cycle traffic before completing their turn. Compare that to the UK where the convention is squarely in favour of the driver: cyclists on tracks must yield when crossing side-roads.

In my mind, these priorities are all wrong.
In my mind, these priorities are all wrong.

In fact, in a nearby estate, I noticed the shared footpath/cycle track had regular give way signs. The cyclists were being instructed to give way to cars from driveways. This is a completely counter-intuitive priority system and reinforces the driver’s perceived rule of the road.

It comes back to my first point, why would a cyclist willingly use a path where they must yield to every turning car when, by cycling on the road, they are afforded greater priority? The whole system is daft from the outset.

Of course, one could argue the Great British driver is unaccustomed to this. They’ll turn into a side road or driveway assuming priority. Bang goes the cyclist; accidents shoot through the roof. Fear often inhibits progress, and I’d suggest this is one such occasion. We need to be more confident about re-thinking our strategies around cycling.

While I accept the UK roads are often irregular, too narrow or too congested to offer widespread benefits to cyclists, I do hope that more can be made to make an integrated, continuous cycle path system. This autumn, I’m hoping to get my UK bike back in order. There are cycle paths all the way to the nearest two major towns but the priority system is a mess. It’s a genuine dissuasion to ride.

I hope that one day, riding a bike in the UK is as convenient to all road users as they’ve already shown to be in the Netherlands and that our many miles of investment in cycle infrastructure can be fully utilised, rather than ignored and bypassed in the ways that they seem to be currently.

Automating Press Clippings

One of my projects is running a journal devoted to Business Improvement Districts (BIDs) in the UK. These are organisations formed around town & city centres to improve the local environment and trading conditions.

We’ve just started recording mentions of BIDs in the press, which is a useful way to track industry updates. At first, I was running this manually, but it because apparent pretty quickly that this would be unsustainable, so I looked at how to automate as much as possible.

I think I have a solution, dipping into various methods I’ve been meaning to use for a while.

The system starts by receiving an update from Google Alerts on a particular topic. This email normally contains 20-30 links to various worldwide news articles.

Email to script

This was something I’ve used before but the basic gist is that an email address is piped directly to a script. I created a new account on a domain I manage and set it to pipe to a script. In Postfix the virtuals file tells the server that username@domain.tld is mapped to a local alias, let’s say mailcatcher. In aliases, this alias is mapped to a script. The line looks like this:

mailcatcher "|/var/mailscripts/script.php"

Run all the mappers, reload Postfix and the email address is set up. Now, provided that script is executable, the entirety of the email will be sent via STDIN to that script.

Message Queueing

I’ve dabbled but never got too far into message queueing. The high-level idea is that pieces of information (‘messages’) are pushed around and progressively worked on by a number of scripts. These scripts are small – they usually do only one job, but they manipulate that message in some way or do something in response to it. Once a script’s work is done, the result is usually passed on to another. Depending on the complexity of the system, multiple scripts might work on a single queue of items.

The outcome is a chain of incremental processing, much like a factory line. For this task it works extremely well as I can break a fairly complex chain of actions into digestable pieces, so once an email is received the following chain is followed:

Receive email via mail script (above). This adds the whole email contents as a message to a ‘mail’ queue.

A script watches the ‘mail’ queue and takes any new messages. Its job is to find all the links in the email and create one new message for each link. These go into a ‘link’ queue.

Another script on the ‘link’ queue checks the database for known links to see if the link is a duplicate. Any message that isn’t a duplicate goes into the ‘retrieve’ queue.

The ‘retrieve’ script follows the URL and grabs a copy of the page in full. This then goes into the ‘parse’ queue.

The ‘parse’ script parses the HTML and looks for common elements (headline, publish date, source, etc.). Once it has enough information, these elements end up in a message in the ‘publish’ queue.

The ‘publish’ script creates a new database entry for the article and publishes it to WordPress. The article clipping is now live.

There are several good reasons for breaking up a procedural process into separate scripts and queues. For one, it’s very easy to scale up. Parsing might be a great deal harder (require more processing power) than anything else, so could need more machines working on it. Grabbing the contents of a URL is notoriously unreliable, so if a script fails here the queue entry is merely postponed until later when it’ll try again.

Debugging is also trivial. I can simply watch a particularly (/troublesome) message work its way through the system and see exactly where any errors are coming up.

This is also good for development. The parser only knows how to read a pre-defined selection of news pages, so if a new page arrives which can’t be parsed I need to add new rules to the parser. By holding the message in a queue I simply postpone the message until the parser is upgraded and the message can continue through the system.

There are a fair few messaging systems out there which handle this stuff well, but I needed something quick & dirty, so wrote something up to handle this myself. Down the line, I’m planning to move to a more robust, distributed system.

Web Interface

Many of the news articles are rejected because they cover regions we don’t care about (US, New Zealand, Canada, etc.) so I use domain blacklisting to disregard foreign news sources. This isn’t fool-proof. Some news aggregators are international, and might carry articles we care about.

Furthermore, once a UK-based article is picked up, I need to figure out which BID it relates to. Doing a simple search for town names is not enough (‘Rugby’ was pretty popular because of the sport-related mentions on pages), so I look for ‘{town name} business’ and phrases like that. It gets about 75% of the results satisfactorily.

For the rest, I need to manually intervene and provide a BID association myself.

This is the bit I need to work on. Currently the parsers and labelling systems are all hardwired. No good. It’s manageable for now, but in the near future I’m going to need a web interface to handle all the messages that don’t get through.

Overview

The whole system is running automatically and probably saving me at least an hour a day. It took around 8 hours to build so I’m happy with the returns. Having now put together a basic framework I expect to be able to use this for other tools as well.

Automation was one of the key lessons I learnt a decade ago with Blogwise, a blog directory that was easily attracting 100+ new submissions per day. The labour involved in handling submissions was phenomenal and automation scaled up in ways labourers could not.

It’s now something I advise others to do as well, particularly with repetitive tasks. A relatively small development investment can replace many man-hours of activity, and the economies of scale are often much more impressive.