Bringing the science to open data

Open data isn’t exactly a high profile issue for the public, but MPs on the Public Administration Select Committee (PASC) have got their teeth into the subject. The publication this week of its Statistics and Open Data report shows that its members are not happy with the progress of campaign, and believe it’s falling far short of its promise to boost business and provide better government through making most government data sets freely available.

One shortcoming that stands out is that public servants themselves don’t have the skills to do much with the data, and the statisticians who do often hide away when they should be champions for the cause. It follows up with a call on the Government to encourage training of more data scientists, with a timetable and target numbers, and include data skills and open data awareness in the Civil Service policy training.

It reflects the fact that open data is often messy data, and while there is a case for standardisation, it is unlikely the public sector will have the resource or the inclination to make it more user-friendly. For the foreseeable future it’s take it or leave it, and taking it will only be a serious option for those with someone with the ability to extract knowledge from the mess.

It ties in with a call that I’m hearing more often for an increase in the number of data scientists. Businesses are being told that if they want to take advantage of big data and the internet of things they need people with more the mathematical skills to grab insights from the torrent of information, but the investment isn’t an easy sell at the moment. It’s still a vague idea for many, and the benefits seem a long way off.

There’s a need for more degree courses and vocational training, and the Government could give it a push by developing some of its statisticians into the role of data scientists. It’s not just about the training, but beefing up the role, and giving it the type of profile that it gave to CIOs a few years ago. If it’s something for people in the Civil Service to aspire to, it could raise the profile of the role and set a trend for others to follow.

The immediate effect would be limited to a few Whitehall departments, but it could provide some longer term momentum for the profession and do something for the cause of open data.

Personal data in the app and API economies

Europe’s app economy made some headlines in February with the publication of a report by research firm Giacom, as part of the EU backed Eurapp project, which estimates developer revenues at €17.5bn in 2013 and forecasts they will rise to €63bn by 2018.

It throws the spotlight on the growing economic importance of apps, predicting that there will be 2.8m developers’ jobs and 4.8m people in support and marketing roles by 2018, and has some interesting observations and hard stats on the business. But it doesn’t touch on the influence of personal data, and how it’s managed by organisations that develop apps and the application programme interfaces (APIs) on which they run.

A growing number of consumers are sensitive about personal data, and their choices are going to be influenced by whether they feel it is held in a neutral and safe place. This places it among the factors that API and app developers will have to take into account in building their markets.

It’s a fractured picture. The app economy has been built on developers finding different ways to make money, including charging for the download of apps, offering  them as a medium for advertising or charging when they are used for transactions. The API economy is harder to define – there’s no strong consensus – but it revolves around the provider making the API available to developers as a base for their apps. You can see this in Apple’s App Store and Google’s Play Store, on platforms such as Facebook and Twitter, and among providers of personal data services such as Mydex that provide APIs.

These economies are growing on their appeal to people’s desire for convenience; apps give them an easy route into online services without having to provide their details each time around. But users often have no idea how this is creating revenue for the developer or API provider, even though it often involves the use of their personal data.

Different models are used. Some companies use the data to target their own advertising; others make it available to third party advertisers; it can be used to identify consumer habits and assess risks; to feed into business intelligence; and in developing new services. The potential is growing as more devices are able to provide information on people’s location and movements, and this is going to increase as the data on those apps is hooked up with the internet of things.

There are also some that are not trying to monetise the data but use it to make their own processes more efficient. But the key point for all cases is that there is a degree of intrusion, and it’s all done without the knowledge of most users.

By law API providers and app developers have to tell people how their data could conceivably be used, but in practice it is usually buried in those lengthy terms and conditions that few people are going to read before they click their approval. They only begin to understand further down the track, when in most cases it’s too late to withdraw approval and the only remedy is to stop using the app.

Put this alongside the sporadic horror stories in the press about breaches of personal data held by large organisations, and a growing number of people are going to think twice about signing up to use apps, and will often decide that they can live without them.

If businesses want to keep those people on board they need to find a consent model that gives users more control and feeds a sense of trust; which comes back to the need for a neutral and safe place for personal data that is detached from any commercial process.

There are possibilities around the provision of stores for personal data, supported by a credible process to verify an identity and verify the data, which individuals would populate or have populated by those they connect with and make available as they see fit. They would be most effective if they provide some flexibility; the individual might be able to give blanket permission for access to relevant data for every app they download; or have some a simple short form of consent for different scenarios under their own terms or ask for requests for specified data each time they sign up; or even make data available for one time use on apps for a single transaction.

They would surrender some of the convenience they currently enjoy, and they wouldn’t be the most attractive customers for the most commercially aggressive app developers, but it would give them the control that they want.

This could be done by an API provider or by a third party with access to the API. The latter might prove more attractive to the privacy savvy consumer wanting a service provider that isn’t looking for new ways to make money from the data. He or she would give up some privacy, but with a stronger sense of trust, especially if they were already using it for website-based services.

There is also a case for it assisting API and app providers by reducing the risk from false data. If they are given a false identity or inaccurate details it undermines the quality of what they can offer, and can lead to unhappy customers, reputational damage and, in the worst cases, legal problems from providing the wrong data about individuals. Taking the data, and proof of identity, from a third party with a more stringent verification process could provide a degree of security that they won’t obtain from someone quickly jabbing details into a smartphone or tablet.

It’s likely that some companies would be reluctant to give up free access to the data on which they’re building a commercial model, but it offers a degree of compensation that could be more highly valued as customers become more guarded.

Concerns over personal data are beginning influence the market for apps. This is one possible solution and there could be others out there. Overall, the companies that are willing to take it seriously could find a competitive edge in appealing to those customers who are not going to be more demanding in providing their trust.

This article was written in collaboration with Mydex.

BBC4 deserves its place more than BBC2

There's been a fresh round of rumblings in recently about BBC4 being closed/merged with BBC2. Michael Grade, David Dimbleby and Roger Mosey have all spoken about wielding the axe over 4 in recent months.

I can understand the financial pressures that lead to the Beeb asking whether it needs all its TV channels, but I don't buy the argument used to justify this, that there's no clear distinction between the two.

I'd argue that BBC2 is actually much closer to BBC1 in its programming. Just looking at what's on 2 this evening (4 March) between 6.00 and 10.30 – Revenge of the Egghead, a dogs’ intelligence test as part of Sport Relief, a documentary about planners, a sowing competition, another documentary about emergency medics and Sarah Millican’s chat show – I would say all but one (the programme about planners) would fit comfortably into BBC1's schedules.

By contrast, BBC4 is on the ground that once would have been firmly occupied by BBC2 – Michael Portillo looking at Irish railways, a documentary about crabs, another on the role of bones in evolution, and later documentaries on World War One and Captain Cook. True, it has the movie Billy Elliot on at 9.00, but overall it is aimed at a more distinct audience and fulfils a crucial part of the brief of a public service broadcaster - that bit about informing and educating (and plenty of us do find it entertaining).

I’d argue that the content of 4 is much further from the Freeview commercial channels than that of its BBC siblings. And if you close/merge it with BBC2 the overall effect will contribute to the dumbing down of TV.

So if they have to cut something, why not BBC2? Or shift the BBC4 brief, and all the staff from 4 to 2 and see how much of 2 can be absorbed into 1. That would do more to ensure that the BBC continues to play a role in catering for a minority audience, and carries on meeting its brief for public service broadcasting.

Mark Say is a UK based writer who covers the role of information management and technology in business. See www.marksay.co.uk

Lego, Raspberry Pi and computing skills

I’m happy to cheer on the government’s plan to include computer coding in the national curriculum for primary schools from next September. I also have concerns about its chances for success, but I think these could be improved by learning a few lessons from the history of Lego. Yes, those toy building blocks that a lot of us played with as kids. I’ll get to the reason in a while.

Placing computer coding on the curriculum makes sense. We need a population of coding savvy youngsters to lay the ground for computing innovations in the future. And it’s a fair argument that even those who don’t follow careers in IT will benefit from coding, learning to think with a combination of logic and creativity. It will be good for them and good for the UK.

But I can’t help remembering a previous effort to do something admirable in schools, when John Major’s government wanted to ensure that all secondary students learned a foreign language to proficiency. That would have corrected a long term failing in British education; but it foundered, largely because there weren’t enough people with the existing language skills who liked the idea of teaching for a living.

The effort to ingrain computing skills faces a similar barrier; anyone with any degree of expertise can earn a lot more money in a different environment, and they won’t take the stick that is often aimed at teachers by politicians the press. It’s easy to see why secondary schools are struggling to provide decent courses in computer science.

Hopefully, the plan for primary schools will get over this by taking a different approach.  The availability of Raspberry Pi, the tiny single board computer that can be used to explore the basics of coding, should enable unskilled teachers to learn with the first group of children. It won’t require the existing knowledge needed at secondary level.

But you need to make sure the kids want to learn. I listened to an interesting talk at Cass Business School last week, when Alex Klein, founder of start-up Kano Computing, warned against a learning by rote approach that would be no fun and dampen the youngsters’ interest. They are more likely to learn if they can play in doing so.

That prompted a thought about the previous week’s episode of ‘The Culture Show’ on BBC2, which looked at the influence of Lego on architecture. Some of the top architects around the world grew up messing about with with those plastic, clip-on bricks, indulging their imaginations in weird and wonderful constructions, and getting a feel for symmetry and design.

Lego provided a tool for the nurturing of lively minds, and this is what Raspberry Pi can be. I hope that teachers don’t use it as a tool to drill a bunch of dos and don’ts into the kids’ heads, but as something that’s fun. Treat it as they do art or storytelling, something that lets their imaginations run riot. In the long term it can give a generation of red hot computer scientists, and people who can turn creative minds to many other lines of work.

Mark Say is a UK based writer who covers the role of information management and technology in business. See www.marksay.co.uk

Risks v benefits in the Care.data row

Privacy activists are giving themselves a big cheer following NHS England’s  announcement that it’s putting the launch of Care.data, the central database of NHS patient records, on hold for a few months. It’s acknowledged that it did a poor job of telling the public about the scheme and their right to opt out, and is now stepping back to do it properly.

It wasn’t just the privacy groups that had protested – the British Medical Association, the Institution of Engineering and Technology and the Information Commissioner’s Office had all expressed their concerns. Given a few months, NHS England might take steps that will make these bodies happy. But I doubt if it will ever get the more militant privacy campaigners on its side, as they’ll still be able to make a noise about the more important issue – whether the data can be used to target individuals.

When the data becomes available for research, by public or private bodies, it will be anonymised. But there are some big question marks over whether that really protects people from someone digging into their medical records. There is evidence that if someone really wants to break down anonymised data they can do it by cross-referencing with other data sets; a report by the World Economic Forum has said the ‘triple identifier’ of birthday, gender and postal code is the giveaway for most people.

It’s a scary thought but one that should be kept in perspective. Identifying individuals takes time, and who would want to get at their healthcare data, and what would they do with it? It’s in the interests of the organisations that get access to the data to vet whoever works on it and place some heavy duty security controls in place. There is a risk, but it’s miniscule, and on a par with a lot of others we accept in our lives.

Against this are the benefits of making the anonymised data available for healthcare research, something that most people would understand and go along with. The data isn’t being collated just for the sake of it.

So NHS England could do worse than presenting it as risks v benefits. The more militant privacy campaigners would continue to object because they're concerned solely with the risk. But I expect that with an honest assessment of both the great majority of patients will be happy to part of Care.data.

Seduction by vinyl

Vinyl LPs have always had their champions. Ever since the point in the 80s when CDs became the dominant medium for listening to music, there have been people who swore the swore the sound was always better on a 33 RPM LP, and vinyl albums have often been priced at a premium to those little silver plastic discs.

This seems to have stepped up a gear since digital downloads knocked CDs off their perch, and there have been plenty of magazine articles, radio and TV documentaries exploring the enduring appeal of vinyl. A couple of weeks ago BBC4 gave Danny Baker three hour long shows to talk with various mates about what made the old LPs so great.

So far it’s been a minority interest, but this week I saw a sign that it’s a love affair that has obtained some weight in the mainstream media. On a platform at Baker Street Underground there was a large poster for the online dating service match.com, conveying a sense of expectancy with the words “I listened to her favourite album before the date so I could understand why she loved it so much”.

Alongside the words was an image of a vinyl LP, the stylus on the grooves of the first track. It was a surprising choice, as for the vast majority of people that moment would mean slipping a CD into a slot or pressing a button on an MP3 player, but it was obviously meant to convey that there was something special about the man, the woman and the prospects for their relationship. The message was that it would produce something better than most first dates, or whatever any other dating service could offer, because playing an LP produces something CDs or downloads can’t match.

You could argue over whether it’s a message that stands up to scrutiny, but when the advertising industry stars to use an idea it believes it is sufficiently widespread to seduce a large number of people. It’s betting that there’s a demographic with money to spend and a readiness to accept the association of ideas: in this case that a guy who listens to vinyl is worth a serious relationship.

It might amount to a load of old tosh, but it shows that listening to music on vinyl – or at least the idea of it – has become seductive to more than a few music geeks.

Mark Say's collection of fiction, Perversities of Faith, is available on amazon.co.uk and amazon.com. Also check out www.marksaywriter.com.

Credibility and kiddie hackers

I can’t be the only person who wasn’t completely surprised by the news that, according to IT security firm AVG, kids as young as 11 are beginning to write malicious hack code to wreak some havoc and steal some data in the cyber world. We’ve heard plenty about hackers in their mid to late teens, but this fits neatly with the image of delinquent behaviour by junior geeks who are happier staring at a screen than going out in the real world, and is something you could believe of those who would like to but don’t have the nerve to spray graffiti or smash up a bus shelter.

But while the story sounds credible, the evidence seems limited. AVG came up with one solid example, and a lot of talk about patterns that suggest there may be a growing number of kids barely of secondary school age who are up to no good in the cyber world. It’s not clear if it’s a genuine trend or speculation based on a handful of cases.

It’s worth remembering that a story like this can do some good for the company that raises the alarm. It says it is paying attention to serious issues and concerned about the wide world in which it works. That’s why they carry out research and publish studies aimed at asserting their credentials as thought leaders; and when it makes the national news it’s a result for the PR team.

But it has to be remembered that this is all part of the marketing effort, and that the overall aim of such efforts is to boost a company’s sales. It would need a lot of time studying the data, and probably a lot of expertise, for anyone to know if the assertions are correct, and I don’t know whether the evidence behind AVG’s warnings is as strong as it claims. But when something like this comes from the private sector you know there’s a commercial element to it.

It might be a real phenomenon, in which case it’s a genuine worry, but it might be just a storm in a cyber tea cup.

Mark Say's collection of fiction, Perversities of Faith, is available on amazon.co.uk and amazon.com. Also check out www.marksaywriter.com.