Category: networks

TYRchive! on Google’s European lobbying blitz

The case of Google bullying the New America Foundation into sacking the Internet regulation team has gone a million miles around Twitter and the world by now, but I thought I’d share something from the vault, the TYReserve if you will. This POLITICO piece, with its attendant technical appendix, was always meant to be followed up by another piece rating the Brussels lobbyists.

The outstanding result I got was that Google’s lobbying effort in Brussels was a vast transformative onslaught that was bending the usual lobbying ecosystem out of shape. But nobody at the Pol wanted to believe this, and during the edit row that followed I found I’d made a major mistake, assuming one lobbyist per lobby. I rewrote much of the processing script and came back with the result that not that much had changed – Google was still lobbying the EU Commission to such an extent they had revolutionised the PR business. But by then everyone was seriously gun shy of anything involving that guy and charts, and they would much rather have a thumbsucker where I predicted the UK would stay in the EU or such. Oy.

Anyway, here’s the final version off the spike. You’ll want to read the first piece’s conclusions first.

Preview: TYR @ OpenTech 2017

One thing I’ve recently noticed knocking about the industry is what I’m beginning to call the “post-Internet” or maybe the “Afternet”. This is the conjunction of maybe three trends.

The first of these is the way 5G is panning out over in mobile.

To start with, pretty much everything in consumer and most things in enterprise are mobile these days so there’s no excuse for ignoring those boring 3GPP suits. Also, it looks very much as if the first deployments of the 5G New Radio are going to be fixed-wireless, as a replacement for DSL and a substitute for point-to-point microwave or maybe some fibre. So rather than being a “mobile thing”, they’re going to be another access network medium for the general public Internet service.

I’m not going to go into the radio stuff here, fascinating though it is. What interests me is that the 5G builders are increasingly keen on integrating general purpose computing capacity into the network, and importantly, putting it within the radio-access network or at least the mobile core network. So you might host an app inside T-Mobile or Orange Armenia or whoever’s RAN, which would obviously be lovely from a webperf point of view as a way of getting latency down. And, you know, distributed systems design is fun! Latency is a huge deal in 5G; 3GPP wants a round-trip time from the User Equipment (yes, they still call it that) to the carrier’s edge router facing the Internet of 10ms. The important point to bear in mind here is that if this is delivered, the competitive imperative to get the full benefit of it will be powerful. We’ve known for years that in all kinds of applications, latency and its standard deviation have a direct impact on user experience metrics and indeed on business KPIs.

This brings us to the next trend. I recall not so long ago Jamie Zawinski saying on his blog that Instagram was barely on the Internet at all, in any meaningful sense. At the time they didn’t have much of a Web site, so it was almost entirely an app store/smartphone experience. You couldn’t just download or upload stuff from whatever computer you happened to have with you. And you know, if they could host at least an upload accelerator reverse-proxy inside the mobile networks they certainly would. CDN nodes for the pix would also be valuable. Akamai will kinda-sorta do this for you but the actual coverage is pretty thin compared to what it is for fixed eyeball ISPs. Seeing as Google already has an absolutely huge global caching/CDN infrastructure, I’m surprised it’s not an Android API already.

Now for our third trend. If there’s one product that goes out supremely well in the enterprise these days, it’s Layer 2 Ethernet service, as opposed to IP-VPN or whatever T/E carrier grandad’s still using. One of the major applications for this is that some of the providers now have L2 interconnection (called an E-NNI for External Network to Network Interface, if you want people to think you’re a Bellhead for some strange reason) with the big clouds. So you sign up and suddenly MS Office 365 is sooo much snappier. From their point of view, this is really great if you’re a huge US telco or cableco that can get all its enterprise customers and three AWS Regions on-network, but not so good for everyone else. So the smaller operators – and some not-so-small but multinational ones – are trying to sign up as many E-NNI agreements as they can with each other, standardise their internal IT, all in the aim of being able to increase their L2 footprints.

The dark side of this is that in doing so they’re shifting emphasis from the Internet’s Layer 3, multilateral peering interconnection model to a new L2 private peering ecosystem. Personally I’m not sure it can work – one of the underestimated features of the Internet is that its federal nature means it has clear administrative demarcs, and this certainly won’t, as it’s going to be some sort of turtles-all-the-way-down but with VXLANs protocol tunnelling nightmare, which will make debugging it pure hell. A preview of this is the way a lot of the big clouds really struggle to do networking at all well.

The opposition, meanwhile, seems to be companies like Google who run proper IPv6 networks, but increasingly own everything but the last mile and increasingly rely on their own in-house fibre for everything. Google specifically has every reason to obsess about this after that time the NSA tapped all their inter-datacentre WANs and they bought the world’s supply of FASTLANE linerate encryption boxes all in a week.

Hence the Afternet – an oligopolistic collection of semi-autonomous clouds overlaying politicians’ demands for censorship and digital protectionism. Will your blog be on it? So, where are we going with this and who should we subject to remorseless public shaming? Well. That’s why I’m going to be speaking at this year’s OpenTech on the subject of 5G and Your Website. You may have noticed there’s a link to “5G Resources” at the top of the blog – there is nothing there at the moment but I promise it will be full of links before the 13th of May.

The place, the time, the artefact, the culture

Andrew Gordon’s The Rules of the Game: Jutland and British Naval Command is pretty special. In essence, this is a deep history of a mentalité, a way of thinking and seeing, and it’s a history that spirals outwards from a very specific time and place and a very specific artefact.

The time, the place: Windy Corner, a specific but only roughly defined bit of the North Sea that existed in reality for about 15 minutes from 1815, British summer time, on the 31st of May 1916, and for many more years as an purely spectral object of historical inquiry, after-action operational analysis, and cynical self-justification. That was the name given to the chaos ensuing as the British fleet deployed from column into line and the destroyers, cruisers, and battle cruisers tried to keep out of its way and each other’s, under fire. It’s just south of marker 2 on this chart from Wikipedia.


The artefact would be an Inglefield clip, a gadget for hanging signal flags on their halyards quickly invented in 1895 and still in use today. Again per Wikipedia:


An artefact is an ideology. In this case, the clip embodies the Royal Navy’s doctrine of command of the time. This was implemented in the technology of signalling, like the clip, the flags, and the codebook, and jealously guarded by a powerful career mafia of signals specialists, who both gave the ideology its political punch, and were themselves shaped by it.

The content of this ideology was drawn from a wider culture. It incorporated what is known in German as Befehlstaktik – the idea that leaders set down their detailed, specific instructions in written orders which followers execute without interpretation – and a looser idea of hyper-obedience which Gordon roots in a certain idea of manhood based on a revived ideal of chivalry. He argues that this ideology, so drastically different from the practice of Nelson’s time, succeeded because it permitted the RN to adapt to the Industrial Revolution. The freedom of manoeuvre steam power provided, and the changing mix of trades aboard ship, would be channelled into a tactical doctrine that let commanders manoeuvre as if on a parade ground, by incredibly detailed signalling.

The defenders of the ideology were also very interesting. The book is in part a group biography of key players. Gordon notes that their careers had key elements in common, and that in fact these elements were common across the signalling specialisation. Those were Royal patronage, Masonic networking, neochivalrous culture, and weirdly, association with the Royal Geographical Society. Less weirdly, RUSI also played a big role. Scandals involving the princes serving with the Navy frequently seem to have helped the men involved progress in their careers, not least because a flag lieutenant’s job included acting as a social secretary for the admiral, and quite a bit of celebrity management was all in the day’s work. Also, a ridiculous number of people in real jobs in 1914 had spent most of their sea time in the Royal Yacht.

Before we drift into laughing at quaint Victorians – and if that’s what you want, Gordon can deliver in spades, this book is packed with seriously weird characters and the pen-portrait footnotes can be brilliant – it’s worth pointing out that efficient modernising technocrats didn’t exactly help. The new signal book was itself a product both of the new technology of steam power, and of the new science of comparative linguistics. It was literally designed as a language. Also, there were complicated links between the development of analogue computing fire-control and signalling.

The effect of radio was ambiguous. Although officers whined about having to listen to signals from London, they were the same people who fervently believed it was literally better to lose the ship and everyone in her than to turn a blind eye to a signal. Also, where the signal book encouraged verbosity in tactical communications, radio encouraged it in administrative and strategic communications. Gordon points out that in some ways the real revolution in radio was the move from radiotelegraphy to voice radio; W/T was closer to e-mail, while voice was immediate.

There was always opposition to the signals culture. In part this was inevitable due to its role as a mafia for career advancement; its members couldn’t help making enemies as they scrabbled up the hierarchy. But in part it was rooted in deep scepticism that the incredibly complicated manoeuvres, lengthy hoists, and huge teams of professional signalmen would ever be practical in war. Gordon brings us George Tryon as a sort of John Boyd figure, who proved it was possible to do better with a vastly simplified but therefore more expressive signals language, and by delegating much more responsibility to ship’s captains and squadron commanders.

This brings us to the fateful accident when Tryon cocked it up and collided with his second-in-command’s ship. The 2iC argued at the inquiry afterwards that his duty was to obey whatever flag was hoist, even if it meant ramming the flagship. It didn’t matter that Tryon had been operating under the standard signal book at the time, not his tactical code. A fairly shameful cover-up – another – was organised, and the ideas went down with the man. The future would be Tryonite, but the RN would have to learn it the hard way.

One reason why the opposition failed was that the establishment was such a diffuse concept. The school of signals in Portsmouth, which gave it the apparatus of a profession, such as exams, credentials, a hallowed home ground, and a leader, was founded quite late in the day. Without it, they benefited from the tyranny of structurelessness – it’s difficult to criticise something that has no official existence.

10 insights on the Biryani Project

So, to summarise this post a bit, here are some insights:

1) The Curzon Foundation website has been up since February 2010. The various Curzon companies have existed for the same period of time, one succeeding the other as they successively failed to file accounts and got struck off. It looks very much like each one was intended to replace the previous one. They share addresses and directors. Although the Initiative and Foundation don’t have Afzal Amin as a director, we know from Mohammed Hanif’s LinkedIn profile that there is no distinction between them and the Institute.

2) From 2013 on, Afzal Amin and friends were operating parallel commercial and charitable entities with the same address, website, and field of operations. The £120,000 Government grant was paid to a nonprofit entity, but it would have been very easy to have it contract with one of the commercial companies for services, or buy assets from them. The scandal at the Kings Science Academy in Bradford is an example of this kind of related-party transaction abuse.

3) The network of directors around Amin includes local property developers, people associated with both Home Office and DCLG counter-radicalisation projects, a school governor accused (controversially) of being an Islamic extremist, and either a Labour councillor, or a serving police officer, or both. What was a Labour councillor doing trying to get a Tory elected? What was a cop doing getting involved in party politics? What was he doing holding outside directorships? If it was the councillor, not the cop, why didn’t he mention the Curzons on his declaration of interests?

4) Calling it the Curzon Institute was simply illegal. So was offering to pay EDL activists to canvass.

5) Far from being a purely hypothetical discussion, it seems that the Biryani Project actually became operational on the 26th of February, when the EDL did indeed march in Dudley. Local news reporting at the time quotes Councillor Hanif, Afzal Amin, and Chief Supt Johnson (also mentioned by Amin in the Yaxley-Lennon tape) expressing their profound satisfaction. The image of racists, jihadis, aldermen, cops, and a Tory converging on a provincial mosque by their common accord is deeply surreal.

6) The same story also mentions “community stewards” organised by the head of the Dudley Private Hire and Taxi Association. Seems legit…

7) If the Curzon entities were functioning as far back as February 2010, it seems logical to suppose they were getting money from somewhere, and further that they used it for something. That somewhere was presumably the PREVENT/CONTEST programme, but I don’t think anyone intended that to fund either Councillor Hanif or would-be Tory MP Afzal Amin’s electioneering or even some guy’s buy-to-let empire.

News stories about Jahan Mahmood at the time suggest that the “new” counter-radicalisation programme, which wasn’t meant to talk to anyone who might be too radical under the influence of Michael Gove, cut off his funding. Is the point here that Afzal Amin got DCLG to restore the flow of money into the pre-existing Curzon network, in exchange for its support to get elected?

8) It seems very, very likely that Afzal Amin exaggerated his military career substantially. We know that he was in Iraq as an education officer, not some sort of commando, and that he gave the impression to the BBC that he was a tutor or personal adjutant to Prince Harry when in fact he was the education officer attached to his regiment. He doesn’t seem to have left the Adjutant-General’s Corps Education & Training Branch throughout his career.

9) A source tells me that the leadership of the Defence Academy are “extremely angry” about Amin and are actively trying to get DCLG to cancel the grant and recover any money that hasn’t disappeared.

10) And there’s still another company – UKS3 Ltd – and another director – Michelle Clayton – to look into.

#rugbyleague tries streaming on the web. it doesn’t go well

Oh Rugby League, must it always be so? The answer is always yes. The FFR XIII, the French governing body, had the great idea of streaming their match with Wales today on the web, presumably because TV wasn’t interested and there are plenty of weirdos who would get up for the England/Samoa and Australia/New Zealand who would also watch the French game.

But tell me, having made the momentous decision, did they do a good job? Did they ask people who knew how to do a good job? You know the answer.

It ended up on Dailymotion, in really terrible quality, with no score, but not before they’d also knocked over their own WordPress site by putting the embedded video on the front page and handing out the link, so the thundering herd hit whatever VPS they bought for their website first rather than Dailymotion’s CDN infrastructure. Not surprisingly the database got its knickers in a twist. Why involve a database when what you really need is a cache?

So a good idea that our amateurish execution turned into a humiliating fiasco. Where have we heard that one before?

Genuinely evil

I have been reading the Leeds Teaching Hospitals report on Jimmy Savile. Obviously, it couldn’t really be any more grim, and you’ll have heard the latest shocking revelations via the mainstream media and Jamie Kenny.

But what about really sick and perverted behaviour? Here’s some for you.

A hugely important theme in the report is the remarkably vicious competition between the consultant doctors for status, resources, and career advancement. At a higher level of abstraction, this appeared as competition between LGI and other medical institutions for prestige, high technology, and investment. The huge scale of NHS infrastructure in Leeds meant that this was a seriously big deal politically and economically. This was dramatised as a rivalry with St James’s across town, a rivalry that was rooted in class distinction, as LGI historically emerged from the university medical school and St James’ from the workhouse.

The 1960s-70s board of governors, who were personally closest to Savile, were especially exercised by the perceived need to keep up with the expansion of St James’s as the light of NHS investment shone down on the previously benighted casual wards. This was why they wanted the publicity Jimmy Savile drummed up so badly.

This had direct practical consequences for Savile’s MO. One of the reasons why there were so many people crammed into the Nightingale wards, and so many children mixed into the general population, under minimal supervision by a thin scattering of student nurses overnight, was because they were desperate to retain their accreditation as a teaching hospital and needed to save elsewhere, something of enormous financial and career significance.

As time went on, his engagement with the top management shifted, in their telling at least. Rather than being personally associated with the bigwigs, he was increasingly in touch with the ambitious middle layer who used his services as a kind of broker to contact private donors, thus getting around the NHS’s procurement rules. I suspect there is more than one sordid story about money in here*.

However, this narrative is somewhat tendentious, when you look at the astonishing attempt by cardiology professor Alistair Hall, PR director Karl Milner, and others to get their hands on Savile’s estate. To recap, Hall, who was Savile’s doctor as well as his friend, gave a eulogy for him at the preposterous pseudo-state funeral in which he claimed that Savile had left much of his estate to endow a heart institute at LGI. Savile’s will contained nothing of the sort. It turns out from the report that Hall, Milner, and company had a get-together between his death and the funeral to decide how best to get their (institutional, rather than personal) hands on his money, like some gang of scheming relatives in a 19th-century French novel.

Actually, many of the recent top management demonstrate huge amnesia about him, which is astonishing when you realise one of them (former CEO Stuart Ingham) remembered Savile directly threatening him, saying that he could have him “dealt with” by people he “knew”, although he also remembered nothing else.

Which reminds me. Far below this exalted social level, Savile’s only official role was as a volunteer porter, and he made a great deal of effort to be everybody’s best mate at that level. It’s not hard to see a parallel with his career in the Northern club trade. Famously, gangsters love to control the door staff because anything that goes in or out of the club goes through the door. Savile had his own door staff to ensure that he controlled the door. Similarly, influencing the porters and security guards gave him what he wanted: access. To put it another way, he muscled in on the door at LGI with the assistance of the new chief porter, his close friend and apparent accomplice Charlie Hullighan, in the same way as he might have imposed a new Lithuanian bodybuilder on an uncooperative dance hall.

In fact, it would probably make sense to break this post up into sections by social class, but there’s only so much Savile I’m up for wading through.

And making this an underworld story is interesting, but I think it’s worth pointing the finger firmly at the cynical and ambitious medical bureaucrats driving in from Harrogate or Ilkley. They thought they were using him; he was using them; now they remember only that he’d always been hanging about and it was somebody else’s problem. They are the Schreibtischtäter of the story, and there’s something frankly Prussian about the report’s description of LGI in the postwar era.

The parallel, not-quite-equal nursing hierarchy was as bad for different reasons. They were just as ambitious but even more authoritarian, and where their medical colleagues cared about their science, they cared mostly about whether any of the students had snuck a boyfriend into halls or been seen in a pub. Nobody seems to have cared about the patients, who were somehow beside the point. The world of Orwell’s How the Poor Die was still with us.

Speaking of How the Poor Die and workhouses, the one place in Leeds where you were relatively safe from Jimmy was…Jimmy’s. Out of thirty-odd allegations on Leeds Teaching Hospitals premises, only one is recorded at St. James’s. Clearly, he had fully entered into LGI’s tribal identity. A real LGI man didn’t set foot in Jimmy’s, and neither did he.

*One that didn’t happen was the appeal for an MRI scanner in the early 90s, where Savile made it a condition of the donation – which wasn’t actually his money or even money he collected – that the machine be procured from a specific Japanese manufacturer, not named in the report. The medics refused to accept that particular model and the deal fell through.

daft IP addressing choices

This is only one of the reasons why squatting in other people’s netblocks is a bad idea. To understand the point, you’ve got to go back to the BT 21CN project, which was one of those “the Internet is just another service over our private network” ideas telcos tend to love. Although a lot of it didn’t work, like the weird ethernet-level multiservice router, they did build a huge MPLS core network that carries all the other stuff – i.e. mostly the Internet – as encapsulated traffic.

Because they did it this way, they also didn’t do IPv6, which left them with a problem. One of the advantages of doing it the way they did was that they could trivially have a parallel management network. But that meant finding at least two addresses per device for the whole of the UK. So they had the bright idea of picking a big netblock that doesn’t appear in the Internet routing table, and “borrowing” that.

Sensibly, they looked for one that would be very unlikely to ever be announced. Some organisations who got huge IP allocations back in the day, like MIT with its 3 /8 blocks, have been prevailed on to give at least some of them back for public use. The classic case is the trade show Interop, which used to own 45/8 and only use it one week a year.

The US Department of Defense, however, has a hell of a lot of address space, and usually doesn’t route publicly for fairly obvious reasons. And if they don’t want to give it up, who’s going to make them? So they peeked into the DODNIC allocation and picked 30/8. This is quite common; one day somebody will audit it all and there will be surprises.

Fun with network utilisation

Something really interesting in here, a map of DISA’s European network by link utilisation.


I choose to read this as showing everything that runs into Sigonella running red hot, therefore showing in the most literal possible way the swing of US political/military interest from the Middle East into North Africa. That said, they seem to have a very shaky idea of the dates involved and this might be much older, in which case the link out to Bahrain is doing the work.

Lib Dems: not quite useless

So, Wired writes up three West Point professors and their algorithm to decide which members of a terrorist network to zap. Apparently they implemented it in 30 lines of Python. The paper is here, with some pseudocode and the tantalising hint that they used NetworkX, but no Python. However, even the Wired piece tells us enough to reverse engineer it.

The key idea here is that whacking terrorist leaders is often stupid, because it causes the enemy to adopt a flatter, more decentralised, and therefore less vulnerable network structure. Also, they point out, the leaders are often forces for restraint and points of contact for negotiation.

Being who they were, they decided that they could fix this with a better optimisation. They looked at the network-wide degree centrality, a measurement of the centralisation or otherwise of the whole network which is defined as the fraction of total nodes in the network an average node is connected to. They then asked how this changed when they removed a node from the network. And they reasoned that increasing it was desirable, as it rendered the network overall more fragile and unstable.

Now, the Lobster Project uses weighted betweenness centrality – the fraction of the shortest routes through the network that pass through a given node, with more important nodes being accounted for as such – as its centrality metric. There is no particular reason to think that this would work differently.

So I thought I’d implement it. Their implementation used 30 lines, but I presume that includes the test harness to generate or load a specimen network as well as the analysis. Here goes:

def greedy_fragile(mgraph, month, mini, nodes):
...network_wide_centrality = float(sum(nodes.values())/len(nodes.values()))
...n = centrality_nodes(mgraph)
...nwc = float(sum(n.values())/len(n.values()))
...mgraph.add_node(mini[0], mini[1])
...return {'Minister': mini[0], 'Title': mini[1]['Title'], 'Department': mini[1]['Department'], 'Date': month, 'Greedy_Fragile': network_wide_centrality - nwc}

mgraph is the NetworkX graph object, month is the month, mini is the minister (or lobby), nodes is the precomputed list of nodes and their centrality values. Obviously, if it wasn’t for the weird datastore thing I’d have done this recursively and made it return the values for the whole network rather than calling it for each node.

And it works. The first result was that one particular minister was slightly reducing the overall centralisation (and therefore fragility/instability) of the system as a whole. And he’s Ed Davey. As the point of having Lib Dems is meant to be reducing the centrality of Dave from PR and paddock-boy in the system, this suggests that we shouldn’t get rid of him yet.

Going round the country, stirring up apathy

So, that PCCSpoil blog. To begin with, it was a collection of spoiled ballots from the police commissioner elections, a large (>75%) proportion of which seemed to add the hashtags #PCCSpoil or #policespoilballot. I had the impression that this suggested a campaign of some sort. After all, why the hashtag if you weren’t planning to put it on the web?

Since then, the blog has vanished, briefly shown a Mike Giggleresque student politics video, and now points at a petition to explain one’s spoiled ballot to No.10 Downing Street. Someone on Twitter thought the slogan “Don’t politicise the police”, which many of the spoilers used, might be a Police Federation internal line. But it’s not.

There was a policeman who was supporting it, CynicalBobby on twitter. Also, a well-known libertarian. But these were late to the party. Much earlier (9th November) a facebook page had appeared. Following up people who liked it or contributed in any way, I found no cops, but a variety of people who were by turns Green, sceptics, atheists, pro-Palestinian, and from Yorkshire, Wales, or the Isle of Wight, in more than one case. was registered by someone in Wrexham on the 9th of November, the same day the facebook page appeared. On the way, Plaid Youth glommed on. But many of the people involved were in comments to this Guardian piece, on their Northerner blog, from the 5th of November.

That, in turn, was from this lady, a regular contributor from God’s and Jimmy’s own county, a green-minded pro-Palestinian ex-Lib Dem who even introduced the slogan, on the 3rd. We have a patient-zero. Magic darts!

A special note. Tumblr, like Facebook, doesn’t delete your photos if you kill your account or even delete stuff from it. They remain in whichever content-delivery network they used. I know this because, after the PCC blog vanished, I noticed I still had a copy open in a browser tab, and I was able to wget all the images and the HTML wrapper into an archive.

Update: One of the people in this post is now claiming to be me! As a note to the TV producers who are asking me for copies of the spoiled ballots, PCCSpoil is not my blog and has nothing to do with me. My bet is the Plaid guy.