I have been known to attend one or two hack events. What is the most common feature mentioned? Of course it is data quality.
I was at one event a couple of years ago where one of the developers’ presentation was purely about the quality of the data supplied. Our friend Pareto would probably say that 80% of effort is cleaning up data, 20% is building something.
This is where I can hear my high horse trotting around the stable, pawing the ground, getting ready to be saddled up.
Clearly there are some transitory problems. Very few organisations are looking at processes involving data from end to end and considering wider data consumers as part of their data ecosystem.
Often this is because data is being generated as the by product of delivering services (‘exhaust fumes’), not delivering transparency or data. Clearly this needs to change.
If these processes change will this solve everything? Probably not. The whole point about hack events and external consumers needs is that often they are about being innovative and people wanting to do things differently. Unless there is a data crystal ball lying around this will always throw up challenges to the way data is presented.
However maybe some of these issues can be mitigated?
How about an internal data manifesto?
Data is our biggest asset
We are a data driven organisation/society
All our data is valuable
We should value our data
It is not our data as we hold it in trust for others
We are the custodians of our data for future generations
We must know how society wants to use the data we generate
Data is at the heart of what we do, how we operate and how we benefit society
I gave a short internal presentation this week about open data. As I was writing a few thoughts came to mind based on chats with data users.
One of the intriguing questions is – how does the public sector spend on data? This could be data being bought from other public sector bodies or the private sector.
For example the BBC buy weather data from the Met Office. How much do bodies pay in licence fees to Ordnance Survey every year? Someone suggested that it might be several million pounds a year. Of course the Trading Funds have a financing model to follow and are expected to generate income.
What about private sector data? Apparently it is quite common to pay for Dun numbers (Dun and Bradstreet) in government. How much does this cost each year? Do we know how much is going to be paid to the Royal Mail for postcodes address information?
Has anyone ever investigated this topic and done a map looking at the flow of data/money in and out of the public sector?
Just curious and whether there is something here about the wider benefits of open data to the public sector.
Ok I am making an assumption that there is going to be a CDO but bear with me on that.
Will a CDO be the saviour of data in the UK?
Here is a starter for ten list:
Work out who is responsible for data publishing and transparency – GDS (The Government Digital Service) or the Transformation Team in the Cabinet Office?
Do something with data.gov.uk – a great idea but a bit like a marathon runner it seems to have hit the wall.
Please sort out public sector data publishing – are central and local government working to the same standards?
Give some oomph to the National Information Infrastructure, even a new name. Data is the life blood of the economy it should be sexy not boring.
Mandate data registers for every public sector body as will now be the case in the US (and why not throw in the private sector as well why you are at it.)
Data quality is appalling and a perennial issue – it should be everybody’s responsibility not just the person who pushes the publish button.
Open, open, open – yes with the relevant caveats around privacy and security everything should be open.
Education – so we are teaching coding to kids where is data in all this? Most organisations cannot function without data and data management but this is not entirely clear when you talk to public sector bodies. So education, education, education – about data.
Make sure the UK stays at the top of the table for open data – but by using better metrics than the current lightweight assessments.
Create a network of Chief Data Officers.
So that’s ten points written in ten minutes – what else is missing?