All power to the PC

I was intrigued yesterday by the claim in the paper that last Sunday our windmills were providing 40% or so of the electricity being consumed. This was, it seems, a record.

So off to google to check to fairly quickly come up with reference 1 which tells me all I could possibly want to know and more about the big picture on power in the UK. Furthermore, I can download the stuff to my own PC and I now have a spreadsheet containing a reading every 5 minutes since the middle of 2011, some 375,000 rows in all (see reference 2). Excel is almost as impressive as google!

The catch is that one quickly finds that turning this amount of data, which one might have thought was simple enough, into information was not that simple.

There are, for example, some odd outliers, with big spikes in demand. I have not checked but I suppose these spikes are carried by pumped storage or by the interconnectors (to France, for example). Alternatively, can one trust the data? I dare say there are some notes at templar but I have not yet looked for them.

Next I decide that while 375,000 rows might be fine for a power engineer or perhaps a power transmission engineer, they are not fine for me. So knock up a bit of VB to sample the rows, first 1 in 20 then 1 in 50. To find that running this VB on my PC takes some minutes. Clearly a rather inferior PC for this sort of thing and probably also true that VB is a lot faster on native data arrays than at pretending that a worksheet is a data array. I suspect the latter of actually being some sort of a database, some precursor of SQL, and coming with all the overheads that implies.

But we get there and I start doing graphs. To find that doings graphs of what is now some 7,500 rows does not work that well for data which moves around in the way that this stuff does. The graph illustrated is what Excel calls a 100% stacked line. Interpreting the graph is left as an exercise for the reader.

I think the first bottom line is that, as things stand now, coal and gas each do a third, nuclear and wind each do a sixth.

And the second bottom line is that, if you have a lot of statistics, you do need a statistician to give them some quality time if you want to get some quality information. To think that I was one a long time ago.

Reference 1: http://www.gridwatch.templar.co.uk/.

Reference 2: https://dl.dropboxusercontent.com/u/8152054/power-2.xlsm.