Category Archives: care.data

AI, care.data, children, communications, datasharing, engagement

Data-Driven Responses to COVID-19: Lessons Learned OMDDAC event

A slightly longer version of a talk I gave at the launch event of the OMDDAC Data-Driven Responses to COVID-19: Lessons Learned report on October 13, 2021. I was asked to respond to the findings presented on Young People, Covid-19 and Data-Driven Decision-Making by Dr Claire Bessant at Northumbria Law School.

[ ] indicates text I omitted for reasons of time, on the day.

Their final report is now available to download from the website.

You can also watch the full event here via YouTube. The part on young people presented by Claire and that I follow, is at the start.

—————————————————–

I’m really pleased to congratulate Claire and her colleagues today at OMDDAC and hope that policy makers will recognise the value of this work and it will influence change.

I will reiterate three things they found or included in their work.

  1. Young people want to be heard.
  2. Young people’s views on data and trust, include concerns about conflated data purposes

and

3. The concept of being, “data driven under COVID conditions”.

This OMDDAC work together with Investing in Children,  is very timely as a rapid response, but I think it is also important to set it in context, and recognize that some of its significance is that it reflects a continuum of similar findings over time, largely unaffected by the pandemic.

Claire’s work comprehensively backs up the consistent findings of over ten years of public engagement, including with young people.

The 2010 study with young people conducted by The Royal Academy of Engineering supported by three Research Councils and Wellcome, discussed attitudes towards the use of medical records and concluded: These questions and concerns must be addressed by policy makers, regulators, developers and engineers before progressing with the design, and implementation of record keeping systems and the linking of any databases.

In 2014, the House of Commons Science and Technology Committee in their report, Responsible Use of Data, said the Government has a clear responsibility to explain to the public how personal data is being used. 

The same Committee’s Big Data Dilemma 2015-16 report, (p9) concluded “data (some collected many years before and no longer with a clear consent trail) […] is unsatisfactory left unaddressed by Government and without a clear public-policy position.”

Or see

2014, The Royal Statistical Society and Ipsos Mori work on the data trust deficit with lessons for policymakers, 2019  DotEveryone’s work on Public Attitudes or the 2020 The ICO Annual Track survey results.

There is also a growing body of literature to demonstrate what the implications are being a ‘data driven’ society, for the datafied child, as described by Deborah Lupton and Ben Williamson in their own research in 2017.

[This year our own work with young people, published in our report on data metaphors “the words we use in data policy”, found that young people want institutions to stop treating data about them as a commodity and start respecting data as extracts from the stories of their lives.]

The UK government and policy makers, are simply ignoring the inconvenient truth that legislation and governance frameworks such as the UN General Comment no 25 on Children in the Digital Environment, that exist today, demand people know what is done with data about them, and it must be applied to address children’s right to be heard and to enable them to exercise their data rights.

The public perceptions study within this new OMDDAC work, shows that it’s not only the views of children and young people that are being ignored, but adults too.

And perhaps it is worth reflecting here, that often people don’t tend to think about all this in terms of data rights and data protection, but rather human rights and protections for the human being from the use of data that gives other people power over our lives.

This project, found young people’s trust in use of their confidential personal data was affected by understanding who would use the data and why, and how people will be protected from prejudice and discrimination.

We could build easy-reporting mechanisms at public points of contact with state institutions; in education, in social care, in welfare and policing, to produce reports on demand of the information you hold about me and enable corrections. It would benefit institutions by having more accurate data, and make them more trustworthy if people can see here’s what you hold on me and here’s what you did with it.

Instead, we’re going in the opposite direction. New government proposals suggest making that process harder, by charging for Subject Access Requests.

This research shows that current policy is not what young people want. People want the ability to choose between granular levels of control in the data that is being shared. They value having autonomy and control, knowing who will have access, maintaining records accuracy, how people will be kept informed of changes, who will maintain and regulate the database, data security, anonymisation, and to have their views listened to.

Young people also fear the power of data to speak for them, that the data about them are taken at face value, listened to by those in authority more than the child in their own voice.

What do these findings mean for public policy? Without respect for what people want; for the fundamental human rights and freedoms for all, there is no social license for data policies.

Whether it’s confidential GP records or the school census expansion in 2016, when public trust collapses so does your data collection.

Yet the government stubbornly refuses to learn and seems to believe it’s all a communications issue, a bit like the ‘Yes Minister’ English approach to foreigners when they don’t understand: just shout louder.

No, this research shows data policy failures are not fixed by, “communicate the benefits”.

Nor is it fixed by changing Data Protection law. As a comment in the report says, UK data protection law offers a “how-to” not a “don’t-do”.

Data protection law is designed to be enabling of data flows. But that can mean that when state data processing rightly often avoids using the lawful basis of consent in data protection terms, the data use is not consensual.

[For the sake of time, I didn’t include this thought in the next two paragraphs in the talk, but I think it is important to mention that in our own work we find that this contradiction is not lost on young people. — Against the backdrop of the efforts after the MeToo movement and lots said by Ministers in Education and at the DCMS about the Everyone’s Invited work earlier this year to champion consent in relationships, sex and health education (RSHE) curriculum; adults in authority keep saying consent matters, but don’t demonstrate it, and when it comes to data, use people’s data in ways they do not want.

The report picks up that young people, and disproportionately those communities that experience harm from authorities, mistrust data sharing with the police. This is now set against the backdrop of not only the recent, Wayne Couzens case, but a series of very public misuses of police power, including COVID powers.]

The data powers used, “Under COVID conditions” are now being used as a cover for the attack on data protections in the future. The DCMS consultation on changing UK Data Protection law, open until November 19th, suggests that similarly reduced protections on data distribution in the emergency, should become the norm. While DP law is written expressly to permit things that are out of the ordinary in extraordinary circumstances, they are limited in time. The government is proposing that some things that were found convenient to do under COVID, now become commonplace.

But it includes things such as removing Article 22 from the UK GDPR with its protections for people in processes involving automated decision making.

Young people were those who felt first hand the risks and harms of those processes in the summer of 2020, and the “mutant algorithm” is something this Observatory Report work also addressed in their research. Again, it found young people felt left out of those decisions about them despite being the group that would feel its negative effects.

[Data protection law may be enabling increased lawful data distribution across the public sector, but it is not offering people, including young people, the protections they expect of their human right to privacy. We are on a dangerous trajectory for public interest research and for society, if the “new direction” this government goes in, for data and digital policy and practice, goes against prevailing public attitudes and undermines fundamental human rights and freedoms.]

The risks and benefits of the power obtained from the use of admin data are felt disproportionately across different communities including children, who are not a one size fits all, homogenous group.

[While views across groups will differ — and we must be careful to understand any popular context at any point in time on a single issue and unconscious bias in and between groups — policy must recognise where there are consistent findings across this research with that which has gone before it. There are red lines about data re-uses, especially on conflated purposes using the same data once collected by different people, like commercial re-use or sharing (health) data with police.]

The golden thread that runs through time and across different sectors’ data use, are the legal frameworks underpinned by democratic mandates, that uphold our human rights.

I hope the powers-at-be in the DCMS consultation, and wider policy makers in data and digital policy, take this work seriously and not only listen, but act on its recommendations.

2024 updates: opening paragraph edited to add current links.
A chapter written by Rachel Allsopp and Claire bessant discussing OMDDAC’s research with children will also be published on 21st May 2024 in Governance, democracy and ethics in crisis-decision-making: The pandemic and beyond (Manchester University Press) as part of its Pandemic and Beyond series https://manchesteruniversitypress.co.uk/9781526180049/ and an article discussing the research in the open access European Journal of Law and Technology is available here https://www.ejlt.org/index.php/ejlt/article/view/872.

care.data, children, deregulation, National Pupil Database

Data Protection law is being set up as a patsy.

After Dominic Cummings’ marathon session at the Select Committee, the Times published an article on,”The heroes and villains of the pandemic, according to Dominic Cummings”

One of Dom’s villains left out, was data protection law. He claimed, “if someone somewhere in the system didn’t say, ‘ignore GDPR’ thousands of people were going to die,” and that “no one even knew if that itself was legal—it almost definitely wasn’t.”

Thousands of people have died since that event he recalled from March 2020, but as a result of Ministers’ decisions, not data laws.

Data protection laws are *not* barriers, but permissive laws to *enable* use of personal data within a set of standards and safeguards designed to protect people. The opposite of what its detractors would have us believe.

The starting point is fundamental human rights. Common law confidentially. But the GDPR and its related parts on public health, are in fact specifically designed to enable data processing that overrules those principles for pandemic response purposes . In recognition of emergency needs for a limited time period, data protection laws permit interference with our fundamental rights and freedoms, including overriding privacy.

We need that protection of our privacy sometimes from government itself. And sometimes from those who see themselves as “the good guys” and above the law.

The Department of Health appears to have no plan to tell people about care.data 2,  the latest attempt at an NHS data grab, despite the fact that data protection laws require that they do. From September 1st (delayed to enable it to be done right, thanks to campaign efforts from medConfidential et supporters) all our GP medical records will be copied into a new national database for re-use, unless we actively opt out.

It’s groundhog day for the Department of Health. It is baffling why the government cannot understand or accept the need to do the right thing, and instead is repeating the same mistake of recent memory, all over again. Why the rush without due process and steamrollering any respect for the rule of law?

Were it not so serious, it might amuse me that some academic researchers appear to fail to acknowledge this matters, and they are getting irate on Twitter that *privacy* or ‘campaigners’ will prevent them getting hold of the data they appear to feel entitled to. Blame the people that designed a policy that will breach human rights and the law, not the people who want your rights upheld. And to blame the right itself is just, frankly, bizarre.

Such rants prompt me to recall the time when early on in my lay role on the Administrative Data Research Network approvals panel, a Director attending the meeting *as a guest* became so apoplectic with rage, that his face was nearly purple. He screamed, literally, at the panel of over ten well respected academics and experts in research and / or data because he believed the questions being asked over privacy and ethics principles in designing governance documents were unnecessary.

Or I might recall the request at my final meeting two years later in 2017 by another then Director, for access to highly sensitive and linked children’s health and education data to do (what I believed was valuable) public interest research involving the personal data of children with Down Syndrome. But the request came through the process with no ethical review. A necessary step before it should even have reached the panel for discussion.

I was left feeling from those two experiences, that both considered themselves and their work to be in effect “above the law” and expected special treatment, and a free pass without challenge. And that it had not improved over the two years.

If anyone in the research community cannot support due process, law, and human rights when it comes to admin data access, research using highly sensitive data about people’s lives with potential for significant community and personal impacts, then you are part of the problem.  There was extensive public outreach in 2012-13 across the UK about the use of personal if de-identified data in safe settings. And in 2014 the same concerns and red-lines were raised by hundreds of people in person, almost universally with the same reactions at a range of care.data public engagement events. Feedback which institutions say matters, but continue to ignore.

It seems nothing has changed since I wrote,

“The commercial intermediaries still need to be told, don’t pee in the pool. It spoils it, for everyone else.”

We could also look back to when Michael Gove as Secretary of State for Education, changed the law in 2012 to permit pupil level, identifying and sensitive personal data to be given away to third parties. Journalists. Charities. Commercial companies, even included an online tutoring business, pre-pandemic and an agency making heat maps of school catchment areas from identifying pupil data for estate agents — notably, without any SEND pupils’ data. (Cummings was coincidentally a Gove SpAd at the Department for Education.)  As a direct result of that decision to give away pupils’ personal data in 2012, (in effect ‘re-engineering’ how the education sector was structured and the roles of the local authority and non-state providers and creating a market for pupil data)  an ICO audit of the DfE in February 2020 found unlawful practice and made 139 recommendations for change. We’re still waiting to see if and how it will be fixed.  At the moment it’s business as usual. Literally. The ICO don’t appear even to have stopped further data distribution until made lawful.

In April 2021, in answer to a written Parliamentary Question Nick Gibb, Schools Minister, made a commitment to “publish an update to the audit in June 2021 and further details regarding the release mechanism of the full audit report will be contained in this update.”  Will they promote openess, transparency, accountablity,or continue to skulk from publishing the whole truth?

Children have lost control of their digital footprint in state education by their fifth birthday.  The majority of parents polled in 2018 do not know the National Pupil Database even exists. 69% of over 1,004 parents asked, replied that they had not been informed that the Department for Education may give away children’s data to third-parties at all.

Thousands of companies continue to exploit children’s school records, without opt-in or opt-out, including special educational needs, ethnicity, and other sensitive data at pupil level.

Data protection law alone is in fact so enabling of data flow, that it is inadequate to protect children’s rights and freedoms across the state education sector in England; whether from public interest, charity or commercial research interventions without opt in or out, without parental knowledge. We shouldn’t need to understand our rights or to be proactive, in order to have them protected by default but data protection law and the ICO in particular have been captured by the siren call of data as a source of ‘innovation’ and economic growth.

Throughout 2018 and questions over Vote Leave data uses, Cummings claimed to know GDPR well. It was everyone else who didn’t. On his blog that July he suggested, “MPs haven’t even bothered to understand GDPR, which they mis-explain badly,” and in April he wrote,  “The GDPR legislation is horrific. One of the many advantages of Brexit is we will soon be able to bin such idiotic laws.” He lambasted the Charter of Fundamental Rights the protections of which the government went on to take away from us under European Union Withdrawal Act.

But suddenly, come 2020/21 he is suggesting he didn’t know the law that well after all, “no one even knew if that itself was legal—it almost definitely wasn’t.”

Data Protection law is being set up as a patsy, while our confidentiality is commodified. The problem is not the law. The problem is those in power who fail to respect it, those who believe themselves to be above it, and who feel an entitlement to exploit that for their own aims.

Added 21/06/2021: Today I again came across a statement that I thought worth mentioning, from the Explanatory Notes for the Data Protection Bill from 2017:

“Accordingly, Parliament passed the Data Protection Act 1984 and ratified the Convention in 1985, partly to ensure the free movement of data. The Data Protection Act 1984 contained principles which were taken almost directly from Convention 108 – including that personal data shall be obtained and processed fairly and lawfully and held only for specified purposes.”

“The Data Protection Directive (95/46/EC) (“the 1995 Directive”) provides the current basis for the UK’s data protection regime. The 1995 Directive stemmed from the European Commission’s concern that a number of Member States had not introduced national law related to Convention 108 which led to concern that barriers may be erected to data flows. In addition, there was a considerable divergence in the data protection laws between Member States. The focus of the 1995 Directive was to protect the right to privacy with respect to the processing of personal data and to ensure the free flow of personal data between Member States. “

care.data, change management, engagement, National Pupil Database

The National Data Strategy. Rewiring State power.

The National Data Strategy is not about “the data”.  People need to stop thinking of data only as abstract information, or even as personal data when it comes to national policy. Administrative data is the knowledge about the workings of the interactions between the public and the State and about us as people. It is the story of selected life events. To the State it is business intelligence. What resources are used, where, by whom and who costs The Treasury how much? How any government is permitted to govern that,  shapes our relationship with the State and the nature of the governance we get of people, of public services. How much power we cede to the State or retain at national, local, and individual levels over our communities and our lives matters.  Any change in National Data Strategy is about the rewiring of state power, and we need to understand its terms and conditions very, very carefully.

What government wants

“It’s not to say we don’t see privacy and data protection as not important,” said Phil Earl, Deputy Director at DCMS, in the panel discussion hosted by techUK as part of Birmingham Tech Week, exploring the UK Government’s recently released National Data Strategy.

I sighed so loudly I was glad to be on mute. The first of many big watch outs for the messaging around the National Data Strategy was already touted in the text, as “the high watermark of data use set during the pandemic.” In response to COVID “a few of the perceived barriers seem to have melted away,” said Earl, and saw this reduced state of data protections is desirable beyond the pandemic. “Can we maintain that level of collaboration and willingness to share data?” he asked.

Data protection laws are at their heart protections for people, not data, and if any government is seeking to reduce those protections for people we should pay attention to messaging very carefully.

This positioning fails to recognise that data protection law is more permissive in exceptional circumstances such as pandemics, with a recognition by default that the tests in law of necessity and proportionality are different from usual, and are time bound to the pandemic.

“What’s the equivalent? How do we empower people to feel that that greater good [felt in the pandemic] outweighs their legitimate concerns about data being shared,” he said.” The whole trust thing is something that must be constantly maintained,” but you may hear between the lines,  ‘preferably on our [government] terms.’

The idea that the public is ignorant about data, is often repeated and still wrong. The same old mantras resurfaced. If people can make more informed decisions, understand “the benefits”, then the government can influence their judgments, trusting us to “make the decisions that we want them to make [to agree to data re-use].”

If *this* is the government set course (again), then watch out.

What people want

In fact when asked, the majority of people both who are willing and less willing to have data about them reused, generally want the same things. Safeguards,  opt in to re use, restricted distribution, and protections for redress and against misuse strengthened in legislation.

Read Doteveryone’s public attitudes work. Or the Ipsos MORI polls or work by Wellcome. (see below). Or even the care.data summaries.

The red lines in the “Dialogues on Data” report from workshops carried out across different regions of the UK for the 2013 ADRN (about reuse of deidentified linked public admin datasets by qualified researchers in safe settings), remain valid today, in particular with relation to:

  • Creating large databases containing many variables/data from a large number of public sector sources

  • Allowing administrative data to be linked with business data

  • Linking of passively collected administrative data, in particular geo-location data

“All of the above were seen as having potential privacy implications or allowing the possibility of reidentification of individuals within datasets. The other ‘red-line’ for some participants was allowing researchers for private companies to access data, either to deliver a public service or in order to make profit. Trust in private companies’ motivations were low.”

The BT spokesperson on the panel, went on to say that their own survey showed 85% of people say their data is important to them, and 75% believe they have too little control.

Mr. Earl was absolutely correct in saying it puts the onus on government to be transparent and show how data will be used. But we hear *nothing* about concrete plans to deliver that. What does that look like? Saying it three times out loud, doesn’t make it real.

What government does

Government needs to embrace the fact it can only get data right, if it does the right thing. That includes upholding the law. This includes examining its own purposes and practice.

The Department for Education has been giving away 15 million people’s personal confidential data since 2012 and never told them. They knew this. They chose to ignore it. And on top of that, didn’t inform people who were in school since then, that Mr Gove changed the law. So now over 21 million people’s pupil records are being given away to comapnies and other third parties, for use in ways we do not expect, and is misused too. In 2015, more secret data sharing began, with the Home Office. And another pilot in 2018 with the DWP.  And in 2019, sharing with the police.

Promises on government data policy transparency right now are worth less than zero. What matters now is government actions. Trust will be based on what you do, not what you say. Is the government trustworthy?

After the summary findings published by the ICO of their compulsory audit of the Department for Education,  the question now is what will the Department and government do to address the 139 recommendations for improvement, with over 60% classified as urgent or high priority. Is the government intentional about change?

What will government do?

So I had a question for the panel: Is the government serious about its point in the strategy, 7.1.2 “Our data regime should empower individuals and groups to control and understand how their data is used.”

I didn’t get an answer.

I want to know if the government is prepared to build the necessary infrastructure to enable that understanding and control?

  • Enhance and build the physical infrastructure:
      • access to personal reports what data is held and how it is used.
      • management controls and communications over reuse [opt-in to meet the necessary permissions of legitimate interests or consent as lawful basis for further data processing, conditions for sensitive data processing, or at very least opt-out to respect objections].
      • secure systems (not just excel, and WannaCry resistant)
  • Enable the necessary transparency tools and create demonstrable accountability through registers of algorithms and data sharing with oversight functions and routes for redress.
  • Empower staff with the necessary human skills at all levels in the public sector on the laws around data that do not just consist of a sharepoint on GDPR — what about privacy law, communications law, equality and discrimination laws among others?
  • Empower the public with the controls they want to have their rights respected.
  • Examine toxic policy that drives bad data collection and re-use.

Public admin data is all about people

Coming soon is the publication of an Integrated Review, we were told, how ‘data and security’ and other joined up issues will feature.

A risk of this conflation is seeing the national data strategy as another dry review about data as ‘a thing’, or its management.

It should be about people. The people who our public admin data are about. The people that want access to it. The people making the policy decisions. And its human infrastructure. The controls on power about purposes of the data reuse, data governance is all about the roles and responsibilties of people and the laws that oversee them and require human accountability.

These NDS strategy missions, and pillars and aims are all about “the data”.

For a national data strategy to be realised and to thrive in all of our wide ranging societal ‘data’ interests,  it cannot be all about data as a commodity.  Or all about government wants. Or seen through the lens only of research. Allow that, and they divide and conquer. It must be founded on  building a social contract between government and the public in a digital environment, and setting the expectations of these multi-purpose relationships, at national, and local levels.

For a forward thinking data strategy truly building something in the long term public interest, it is  about understanding ‘wider public need‘. The purpose of ‘the data’ and its strategy, is as much about the purpose of government behind it. Data is personal. But when used to make decisions it also business intelligence. How does the government make the business of governing work, through data?

Any national data strategy does not sit in a vacuum of other policy and public experience of government either.  If Manchester‘s second lockdown funding treatment is seen as the expectations of how local needs get trumped by national wants, and people’s wishes will be steam rollered, then a national approach will not win support. A long list of bad government engagement over recent months, is a poor foundation and you don’t fix that by talking about “the benefits”.

Will government listen?

Edgenuity, the U.S. based school assessment system using AI for marking, made the news this summer, when parents found it could be gamed by simply packing essays with all the right keywords, but respondents didn’t need to make sense or give accurate answers. To be received well and get a good grade, they were expected simply to tell the system the words ‘it wanted to hear’.

“If the teachers were looking at the responses, they didn’t care,” one student said.

Will the government actually look at responses to the National Data Strategy and care about getting it right? Not just care about getting what they want? Or about what commercial actors will do with it?

Government wanted to and changed the law on education admin data in 2012 and got it wrong. Education data alone is a sin bin of bad habits and complete lack of public and professional engagement, before even starting to address data quality and accuracy and backwards looking policy built on bad historic data.

“The Commercial department do not have appropriate controls in place to protect personal data being processed on behalf of the DfE by data processors.” (ICO audit of the DfE , 2020)

Gambling companies ended up misusing learner records.

Government wanted data from one Department to be collected for the purposes of another and got it wrong. People boycotted the collection until it was killed off.

Government changed the law on Higher Education in 2017 and got it wrong.  Now  third parties pass around named equality monitoring records like religion, sexual orientation, and disability and it is stored forever on named national pupil records. The Department for Education (DfE) now holds sexual orientation data on almost 3.2 million, and religious belief data on 3.7 million people.

What could possibly go wrong?

If the current path is any indicator, this government is little interested in local power, or people, and certainly not in our human rights. They are interested in centralised power. We should be very cautious about giving that all away to the State on its own terms.

 

The national data strategy consultation is open for submissions until

Samples of public engagement on data reuse

 

 

care.data, change, change management, children, datasharing, empowerment, human rights

A fresh start for edtech? Maybe. But I wouldn’t start from here.

In 1924 the Hibbert Journal published what is accepted as the first printed copy of a well-known joke.

A genial Irishman, cutting peat in the wilds of Connemara, was once asked by a pedestrian Englishman to direct him on his way to Letterfrack. With the wonted enthusiasm of his race the Irishman flung himself into the problem and, taking the wayfarer to the top of a hill commanding a wide prospect of bogs, lakes, and mountains, proceeded to give him, with more eloquence than precision, a copious account of the route to be taken. He then concluded as follows: ‘Tis the divil’s own country, sorr, to find your way in. But a gintleman with a face like your honour’s can’t miss the road; though, if it was meself that was going to Letterfrack, faith, I wouldn’t start from here.’

Ty Goddard asked some sensible questions in TES on April 4 on the UK edTech strategy, under the overarching question, ‘A fresh start for edtech? Maybe. But the road is bumpy.’

We’d hope so, since he’s on the DfE edTech board and aims “to accelerate the edtech sector in Britain and globally.”

“The questions now being asked are whether you can protect learning at a time of national emergency? Can you truly connect educators working from home with their pupils?”

and he rightly noted that,

“One problem schools are now attempting to overcome is that many lack the infrastructure, experience and training to use digital resources to support a wholesale move to online teaching at short notice.”

He calls for “bold investment and co-ordination across Whitehall led by Downing Street to really set a sprint towards super-fast connectivity to schools, pupils’ homes and investment in actual devices for students. The Department for Education, too, has done much to think through our recent national edtech strategy – now it needs to own and explain it.”

But the own and explain it, is the same problematic starting point as care-data had in the NHS in 2014. And we know how that went.

The edTech demands and drive for the UK are not a communications issue. Nor are they simply problems of infrastructure, or the age-old idea of shipping suitable tech at scale. The ‘fresh start’ isn’t going to be what anyone wants, least of all the edTech evangelists if we start from where they are.

Demonstrators of certain programmes, platforms, and products to promote to others and drive adoption, is ‘the divil’s own country‘.

The edTech UK strategy in effect avoided online learning, and the reasons for that were not public knowledge but likely well founded. They’re mostly unevidenced and often any available research comes from the companies themselves or their partners and promoter think tanks and related, or self interested bodies.

I’ve not seen anyone yet talk about disadvantage and deprivation from not issuing course curriculum standard text books to every child.  Why on earth can secondary schools not afford to give each child their text book home? A darn sight cheaper than tech, independent of data costs and a guide to exactly what the exams will demand. Should we not seek to champion the most appropriate and equitable learning solutions, in addition to, rather than exclusively, the digital ones? GSCE children I support(ed) in foreign languages each improved once they had written materials. Getting out Chromebooks by contrast, simply interfered in the process, and wasted valuable classroom time.

Technology can deliver most vital communications, at speed and scale. It can support admin, expand learning and level the playing field through accessible tools. But done wrongly, it makes things worse than without.

Its procurement must assess any potential harmful consequences and safeguard against them, and not accept short term benefits, at the cost of long term harm. It should be safe, fair, and transparent.

“Responsible technology is no longer a nice thing to do to look good, it’s becoming a fundamental pillar of corporate business models. In a post-Cambridge Analytica world, consumers are demanding better technology and more transparency. Companies that do create those services are the ones that will have a better, brighter future.”

Kriti Sharma, VP of AI, Sage, (Doteveryone 2019 event, Responsible Technology)

The hype of ‘edTech’ achievement in the classroom so far, far outweighs the evidence of delivery. Neil Selwyn, Professor in the Faculty of Education, Monash University, Australia, writing in the Impact magazine of the Chartered College in January 2019 summed up:

“the impacts of technology use on teaching and learning remain uncertain. Andreas Schleicher – the OECD’s director of education – caused some upset in 2015 when suggesting that ICT has negligible impact on classrooms. Yet he was simply voicing what many teachers have long known: good technology use in education is very tricky to pin down.”

That won’t stop edTech being part of the mainstay of the UK export strategy post-Brexit whenever that may now be. But let’s be very clear that if the Department wants to be a world leader it shouldn’t promote products whose founders were last most notably interviewing fellow students online about their porn preferences. Or who are based in offshore organisations with very odd financial structures. Do your due diligence. Work with reputable people and organisations and build a trustworthy network of trustworthy products framed by the rule of law, that is rights’ respecting and appropriate to children. But don’t start with the products.

Above all build a strategy for education, for administrative support, for respecting rights, and for teaching in which tools that may or may not be technology-based add value; but don’t start with the product promotion.

To date the aims are to serve two masters. Our children’s education, and the UK edTech export strategy. You can if you’re prepared to do the proper groundwork, but it’s lacking right now. What is certain, is that if you get it wrong for UK children, the other will inevitably fail.

Covid19 must not be misused to direct our national edTech strategy. I wouldn’t start from here isn’t a joke, it’s a national call for change.

Here’s ten reasons where, why, and how to start instead.

1. The national edTech strategy board should start by demonstrating what it wants to see from others, with full transparency of its members, aims, terms of reference, partners and meeting minutes. There should be no need FOI to ask for them. There are much more sensitive subjects that operate in the open. It unfortunately emulates other DfE strategy, and the UK edTech network which has an in-crowd, and long standing controlling members. Both would be the richer for transparency and openness.

2. Stop bigging up the ‘Big Three’  and doing their market monopolisation for them, unless you want people to see you simply as promoting your friends’-on-the-board/foundation/ethics committee’s products. Yes,” many [educational settings] lack the infrastructure” but that should never mean encouraging ownership and delivery by only closed commercial partners.  That is the route to losing control of your state education curriculum, staff training  and (e)quality,  its delivery, risk management, data,  and cost control.

3. Start with designing for fairness in public sector systems. Minimum acceptable ethical standards could be framed around for example, accessibility, design, and restrictions on commercial exploitation and in-product advertising. This needs to be in place first, before fitting products ‘on top’ of an existing unfair, and imbalanced system, to avoid embedding disadvantage and the commodification of children in education, even further.

5. Accessibility and Internet access is a social justice issue.  Again as we’ve argued for at defenddigitalme for some time, these come *before* you promote products on top of the delivery systems:

  • Accessibility standards for all products used in state education should be defined and made compulsory in procurement processes, to ensure access for all and reduce digital exclusion.
  • All schools must be able to connect to high-speed broadband services to ensure equality of access and participation in the educational, economic, cultural and social opportunities of the world wide web.
  • Ensure a substantial improvement in support available to public and school library networks. CILIP has pointed to CIPFA figures of a net reduction of 178 libraries in England between 2009-10 and 2014-15.

6. Core national education infrastructure must be put on the national risk register, as we’ve argued for previously at defenddigitalme (see 6.6). Dependence such as MS Office 365, major cashless payment systems, and Google for Education all need assessed and to be part of the assessment for regular and exceptional delivery of education. We currently operate in the dark. And it should be unthinkable that companies get seats at the national UK edTech strategy table without full transparency over questions on their practices, policy and meeting the rule of law.

7. Shift the power balance back to schools and families, where they can trust an approved procurement route, and children and legal guardians can trust school staff to only be working with suppliers that are not overstepping the boundaries of lawful processing. Incorporate (1) the Recommendation CM/Rec(2018)7 of the Committee of Ministers to member States on Guidelines to respect, protect and fulfil the rights of the child in the digital environment  and (2) respect the UN General comment No. 16 (2013) on State obligations regarding the impact of the business sector on children’s rights, across the education and wider public sector.

8. Start with teacher training. Why on earth is the national strategy all about products, when it should be starting with people?

  • Introduce data protection and pupil privacy into basic teacher training, to support a rights-respecting environment in policy and practice, using edTech and broader data processing, to give staff the clarity, consistency and confidence in applying the high standards they need.
  • Ensure ongoing training is available and accessible to all staff for continuous professional development.
  • A focus on people, nor products, will deliver fundamental basics needed for good tech use.

9. Safe data by design and default. I’m tired of hearing from CEOs of companies that claim to be social entrepreneurs, or non-profit, or teachers who’ve designed apps, how well intentioned their products are. Show me instead. Meet the requirements of the rule of law.

  • Local systems must stop shipping out (often sensitive) pupil data at scale and speed to companies, and instead stay in control of terms and conditions, data purposes, and ban product developments for example.
  • Companies must stop using pupil data for their own purposes for profit, or to make inferences about autism or dyslexia for example, if that’s not your stated product aim, it’s likely unlawful.
  • Stop national pupil data distribution for third-party reuse. Start safe access instead.  And get the Home Office out of education.
  • Establish fair and independent oversight mechanisms of national pupil data, so that transparency and trust are consistently maintained across the public sector, and throughout the chain of data use, from collection, to the end of its life cycle, including annual data usage reports for each child.

10. We need a law that works for children’s rights. Develop a legislative framework for the fair use of a child’s digital footprint from the classroom for direct educational and administrative purposes at local level, including commercial acceptable use policies.  Build the national edTech strategy with a rights’ based framework and lawful basis in an Education and Privacy Act. Without this, you are building on sand.

care.data, human rights, surveillance, transparency, trust

Thoughts from the YEIP Event: Preventing trust.

Here’s some thoughts about the Prevent programme, after the half day I spent at the event this week, Youth Empowerment and Addressing Violent Youth Radicalisation in Europe.

It was hosted by the Youth Empowerment and Innovation Project at the University of East London, to mark the launch of the European study on violent youth radicalisation from YEIP.

Firstly, I appreciated the dynamic and interesting youth panel. Young people, themselves involved in youth work, or early researchers on a range of topics. Panelists shared their thoughts on:

  • Removal of gang databases and systemic racial targeting
  • Questions over online content takedown with the general assumption that “someone’s got to do it.”
  • The purposes of Religious Education and lack of religious understanding as cause of prejudice, discrimination, and fear.

From these connections comes trust.

Next, Simon Chambers, from the British Council, UK National Youth Agency, and Erasmus UK, talked about the programme of Erasmus Plus, under the striking sub theme, from these connections comes trust.

  • 42% of the world’s population are under 25
  • Young people understand that there are wider, underlying complex factors in this area and are disproportionately affected by conflict, economic change and environmental disaster.
  • Many young people struggle to access education and decent work.
  • Young people everywhere can feel unheard and excluded from decision-making — their experience leads to disaffection and grievance, and sometimes to conflict.

We then heard a senior Home Office presenter speak about Radicalisation: the threat, drivers and Prevent programme.

On Contest 2018 Prevent / Pursue / Protect and Prepare

What was perhaps most surprising was his statement that the programme believes there is no checklist, [but in reality there are checklists] no single profile, or conveyer belt towards radicalisation.

“This shouldn’t be seen as some sort of predictive model,” he said. “It is not accurate to say that somehow we can predict who is going to become a terrorist, because they’ve got poor education levels, or because necessarily have a deprived background.”

But he then went on to again highlight the list of identified vulnerabilities in Thomas Mair‘s life, which suggests that these characteristics are indeed seen as indicators.

When I look at the ‘safeguarding-in-school’ software that is using vulnerabilities as signals for exactly that kind of prediction of intent, the gap between theory and practice here, is deeply problematic.

One slide included Internet content take downs, and suggested 300K pieces of illegal terrorist material have been removed since February 2010. That number he later suggested are contact with CTIRU, rather than content removal defined as a particular form. (For example it isn’t clear if this is a picture, a page, or whole site). This is still somewhat unclear and there remain important open questions, given its focus  in the online harms policy and discussion.

The big gap that was not discussed and that I believe matters, is how much autonomy teachers have, for example, to make a referral. He suggested “some teachers may feel confident” to do what is needed on their own but others, “may need help” and therefore make a referral. Statistics on those decision processes are missing, and it is very likely I believe that over referral is in part as a result of fearing that non-referral, once a computer has tagged issues as Prevent related, would be seen as negligent, or not meeting the statutory Prevent duty as it applies to schools.

On the Prevent Review, he suggested that the current timeline still stands, of August 2020, even though there is currently no Reviewer. It is for Ministers to make a decision, who will replace Lord Carlile.

Safeguarding children and young people from radicalisation

Mark Chalmers of Westminster City Council., then spoke about ‘safeguarding children and young people from radicalisation.’

He started off with a profile of the local authority demographic, poverty and wealth, migrant turnover,  proportion of non-English speaking households. This of itself may seem indicative of deliberate or unconscious bias.

He suggested that Prevent is not a security response, and expects  that the policing role in Prevent will be reduced over time, as more is taken over by Local Authority staff and the public services. [Note: this seems inevitable after the changes in the 2019 Counter Terrorism Act, to enable local authorities, as well as the police, to refer persons at risk of being drawn into terrorism to local channel panels. Should this have happened at all, was not consulted on as far as I know]. This claim that Prevent is not a security response, appears different in practice, when Local Authorities refuse FOI questions on the basis of security exemptions in the FOI Act, Section 24(1).

Both speakers declined to accept my suggestion that Prevent and Channel is not consensual. Participation in the programme, they were adamant is voluntary and confidential. The reality is that children do not feel they can make a freely given informed choice, in the face of an authority and the severity of the referral.  They also do not understand where their records go to, how confidential are they really, and how long they are kept or why.

The  recently concluded legal case and lengths one individual had to go to, to remove their personal record from the Prevent national database, shows just how problematic the mistaken perception of a consensual programme by authorities is.

I knew nothing of the Prevent programme at all in 2015. I only began to hear about it once I started mapping the data flows into, across and out of the state education sector, and teachers started coming to me with stories from their schools.

I found it fascinating to hear those speak at the conference that are so embedded in the programme. They seem unable to see it objectively or able to accept others’ critical point of view as truth. It stems perhaps from the luxury of having the privilege of believing you yourself, will be unaffected by its consequences.

“Yes,” said O’Brien, “we can turn it off. We have that privilege” (1984)

There was no ground given at all for accepting that there are deep flaws in practice. That in fact ‘Prevent is having the opposite of its intended effect: by dividing, stigmatising and alienating segments of the population, Prevent could end up promoting extremism, rather than countering it’ as concluded in the 2016 report  Preventing Education: Human Rights and Countering terrorism in UK Schools by Rights Watch UK .

Mark Chalmers conclusion was to suggest perhaps Prevent is not always going to be the current form, of bolt on ‘big programme’ and instead would be just like any other form of child protection, like FGM. That would mean every public sector worker, becomes an extended arm of the Home Office policy, expected to act in counter terrorism efforts.

But the training, the nuance, the level of application of autonomy that the speakers believe exists in staff and in children is imagined. The trust between authorities and people who need shelter, safety, medical care or schooling must be upheld for the public good.

No one asked, if and how children should be seen through the lens of terrorism, extremism and radicalisation at all. No one asked if and how every child, should be able to be surveilled online by school imposed software and covert photos taken through the webcam in the name of children’s safeguarding. Or labelled in school, associated with ‘terrorist.’ What happens when that prevents trust, and who measures its harm?

smoothwall monitor dashboard with terrorist labels on child profile

[click to view larger file]

Far too little is known about who and how makes decisions about the lives of others, the criteria for defining inappropriate activity or referrals, or the opacity of decisions on online content.

What effects will the Prevent programme have on our current and future society, where everyone is expected to surveil and inform upon each other? Failure to do so, to uphold the Prevent duty, becomes civic failure.  How is curiosity and intent separated? How do we safeguard children from risk (that is not harm) and protect their childhood experiences,  their free and full development of self?

No one wants children to be caught up in activities or radicalisation into terror groups. But is this the correct way to solve it?

This comprehensive new research by the YEIP suggests otherwise. The fact that the Home Office disengaged with the project in the last year, speaks volumes.

“The research provides new evidence that by attempting to profile and predict violent youth radicalisation, we may in fact be breeding the very reasons that lead those at risk to violent acts.” (Professor Theo Gavrielides).

Current case studies of lived experience, and history also say it is mistaken. Prevent when it comes to children, and schools, needs massive reform, at very least, but those most in favour of how it works today, aren’t the ones who can be involved in its reshaping.

“Who denounced you?” said Winston.

“It was my little daughter,” said Parsons with a sort of doleful pride. “She listened at the keyhole. Heard what I was saying, and nipped off to the patrols the very next day. Pretty smart for a nipper of seven, eh? I don’t bear her any grudge for it. In fact I’m proud of her. It shows I brought her up in the right spirit, anyway.” (1984).

 

The event was the launch of the European study on violent youth radicalisation from YEIP:  The project investigated the attitudes and knowledge of young Europeans, youth workers and other practitioners, while testing tools for addressing the phenomenon through positive psychology and the application of the Good Lives Model.

Its findings include that young people at risk of violent radicalisation are “managed” by the existing justice system as “risks”. This creates further alienation and division, while recidivism rates continue to spiral.

care.data, change management

Shifting power and sovereignty. Please don’t spaff our data laws up the wall.

Duncan Green’s book, How Change Happens reflects on how power and systems shape change, and its key theme is most timely post the General Election.

Critical junctures shake the status quo and throw all the power structures in the air.

The Sunday Times ran several post-election stories this weekend. Their common thread is about repositioning power; realigning the relationships across Whitehall departments, and with the EU.

It appears that meeting the political want, to be seen by the public to re-establish sovereignty for Britain, is going to come at a price.

The Sunday Times article suggests our privacy and data rights are likely to be high up on the list, in any post-Brexit fire sale:

“if they think we are going to be signing up to stick to their data laws and their procurement rules, that’s not going to happen”.

Whether it was simply a politically calculated statement or not, our data rights are clearly on the table in current wheeling and dealing.

Since there’s nothing in EU data protection law that is a barrier to trade doing what is safe, fair and transparent with personal data it may be simply be politically opportunistic to be seen to be doing something that was readily associated with the EU. “Let’s take back control of our cookies”, no less.

But reality is that either way the UK_GDPR is already weaker for UK residents than what is now being labelled here as EU_#GDPR.

If anything, GDPR is already too lenient to organisations and does little especially for children, to shift the power balance required to build the data infrastructures we need to use data well. The social contract for research and other things, appropriate to  ever-expanding technological capacity, is still absent in UK practice.

But instead of strengthening it, what lies ahead is expected divergence between the UK_GDPR and the EU_GDPR in future, via the powers in the European Union (Withdrawal) Act 2017.

A post-Brexit majority government might pass all the law it likes to remove the ability to exercise our human rights or data rights under UK Data protection law.  Henry VIII powers adopted in the last year, allow space for top down authoritarian rule-making across many sectors. The UK government was alone among other countries when the government created its own exemption for immigration purposes in the UK Data Protection Act in 2018. That removed the ability from all of us,  to exercise rights under GDPR. It might choose to further reduce our freedom of speech, and access to the courts.

But would the harmful economic side effects be worth it?

If Britain is to become a ‘buzz of tech firms in the regions’, and since  much of tech today relies on personal data processing, then a ‘break things and move fast’ approach (yes, that way round), won’t protect  SMEs from reputational risk, or losing public trust. Divergence may in fact break many businesses. It will cause confusion and chaos, to have UK self-imposed double standards, increasing workload for many.

Weakened UK data laws for citizens, will limit and weaken UK business both in terms of their own positioning in being able to trade with others, and being able to manage trusted customer relations. Weakened UK data laws will weaken the position of UK research.

Having an accountable data protection officer can be seen as a challenge. But how much worse might challenges in court be, when you cock up handling millions of patients’ pharmaceutical records [1], or school children’s biometric data? Save nothing of the potential implications for national security [2] or politicians when lists of millions of people could be open to blackmail or abuse for a generation.

The level playing field that every company can participate in, is improved, not harmed, by good data protection law. Small businesses that moan about it, might simply never have been good at doing data well. Few significant changes have been of substance in Britain’s Data Protection laws over the last twenty years.

Data laws are neither made-up, bonkers banana-shaped standards,  nor a meaningful symbol of sovereignty.

GDPR is also far from the only law the UK must follow when it comes to data.  Privacy and other rights may be infringed unlawfully, even where data protection law is no barrier to processing. And that’s aside from ethical questions too.

There isn’t so much a reality of “their data laws”, but rather *our* data laws, good for our own protection, for firms, *and* the public good.

Policy makers who might want such changes to weaken rights, may not care, looking out for fast headlines, not slow-to-realise harms.

But if they want a legacy of having built a better infrastructure that positions the UK for tech firms, for UK research, for citizens and for the long game, then they must not spaff our data laws up the wall.

Duncan Green’s book, How Change Happens is available via Open Access.

Updated December 26, 2019 to add links to later news:

[1]   20/12/2019 The Information Commissioner’s Office (ICO) has fined a London-based pharmacy £275,000 for failing to ensure the security of special category data. https://ico.org.uk/action-weve-taken/enforcement/doorstep-dispensaree-ltd-mpn/

[2] 23/12/2019 Pentagon warns military members DNA kits pose ‘personal and operational risks’ https://www.yahoo.com/news/pentagon-warns-military-members-dna-kits-pose-personal-and-operational-risks-173304318.html

care.data, change, confidentiality, data, datasharing, empowerment, ethics, GDPR, National Pupil Database

Can Data Trusts be trustworthy?

The Lords Select Committee report on AI in the UK in March 2018, suggested that,“the Government plans to adopt the Hall-Pesenti Review recommendation that ‘data trusts’ be established to facilitate the ethical sharing of data between organisations.”

Since data distribution already happens, what difference would a Data Trust model make to ‘ethical sharing‘?

A ‘set of relationships underpinned by a repeatable framework, compliant with parties’ obligations’ seems little better than what we have today, with all its problems including deeply unethical policy and practice.

The ODI set out some of the characteristics Data Trusts might have or share. As importantly, we should define what Data Trusts are not. They should not simply be a new name for pooling content and a new single distribution point. Click and collect.

But is a Data Trust little more than a new description for what goes on already? Either a physical space or legal agreements for data users to pass around the personal data from the unsuspecting, and sometimes unwilling, public. Friends-with-benefits who each bring something to the party to share with the others?

As with any communal risk, it is the standards of the weakest link, the least ethical, the one that pees in the pool, that will increase reputational risk for all who take part, and spoil it for everyone.

Importantly, the Lords AI Committee report recognised that there is an inherent risk how the public would react to Data Trusts, because there is no social license for this new data sharing.

“Under the current proposals, individuals who have their personal data contained within these trusts would have no means by which they could make their views heard, or shape the decisions of these trusts.”

Views those keen on Data Trusts seem keen to ignore.

When the Administrative Data Research Network was set up in 2013, a new infrastructure for “deidentified” data linkage, extensive public dialogue was carried across across the UK. It concluded in a report with very similar findings as was apparent at dozens of care.data engagement events in 2014-15;

There is not public support for

  • “Creating large databases containing many variables/data from a large number of public sector sources,
  • Establishing greater permanency of datasets,
  • Allowing administrative data to be linked with business data, or
  • Linking of passively collected administrative data, in particular geo-location data”

The other ‘red-line’ for some participants was allowing “researchers for private companies to access data, either to deliver a public service or in order to make profit. Trust in private companies’ motivations were low.”

All of the above could be central to Data Trusts. All of the above highlight that in any new push to exploit personal data, the public must not be the last to know. And until all of the above are resolved, that social-license underpinning the work will always be missing.

Take the National Pupil Database (NPD) as a case study in a Data Trust done wrong.

It is a mega-database of over 20 other datasets. Raw data has been farmed out for years under terms and conditions to third parties, including users who hold an entire copy of the database, such as the somewhat secretive and unaccountable Fischer Family Trust, and others, who don’t answer to Freedom-of-Information, and whose terms are hidden under commercial confidentilaity. Buying and benchmarking data from schools and selling it back to some, profiling is hidden from parents and pupils, yet the FFT predictive risk scoring can shape a child’s school experience from age 2. They don’t really want to answer how staff tell if a child’s FFT profile and risk score predictions are accurate, or of they can spot errors or a wrong data input somewhere.

Even as the NPD moves towards risk reduction, its issues remain. When will children be told how data about them are used?

Is it any wonder that many people in the UK feel a resentment of institutions and orgs who feel entitled to exploit them, or nudge their behaviour, and a need to ‘take back control’?

It is naïve for those working in data policy and research to think that it does not apply to them.

We already have safe infrastructures in the UK for excellent data access. What users are missing, is the social license to do so.

Some of today’s data uses are ethically problematic.

No one should be talking about increasing access to public data, before delivering increased public understanding. Data users must get over their fear of what if the public found out.

If your data use being on the front pages would make you nervous, maybe it’s a clue you should be doing something differently. If you don’t trust the public would support it, then perhaps it doesn’t deserve to be trusted. Respect individuals’ dignity and human rights. Stop doing stupid things that undermine everything.

Build the social license that care.data was missing. Be honest. Respect our right to know, and right to object. Build them into a public UK data strategy to be understood and be proud of.

Part 1. Ethically problematic
Ethics is dissolving into little more than a buzzword. Can we find solutions underpinned by law, and ethics, and put the person first?

Part 2. Can Data Trusts be trustworthy?
As long as data users ignore data subjects rights, Data Trusts have no social license.

AI, care.data, change management, data, datasharing, engagement, ethics

Ethically problematic

Five years ago, researchers at the Manchester University School of Social Sciences wrote, “It will no longer be possible to assume that secondary data use is ethically unproblematic.”

Five years on, other people’s use of the language of data ethics puts social science at risk. Event after event, we are witnessing the gradual dissolution of the value and meaning of ‘ethics’, into little more than a buzzword.

Companies and organisations are using the language of ‘ethical’ behaviour blended with ‘corporate responsibility’ modelled after their own values, as a way to present competitive advantage.

Ethics is becoming shorthand for, ‘we’re the good guys’. It is being subverted by personal data users’ self-interest. Not to address concerns over the effects of data processing on individuals or communities, but to justify doing it anyway.

An ethics race

There’s certainly a race on for who gets to define what data ethics will mean. We have at least three new UK institutes competing for a voice in the space. Digital Catapult has formed an AI ethics committee. Data charities abound. Even Google has developed an ethical AI strategy of its own, in the wake of their Project Maven.

Lessons learned in public data policy should be clear by now. There should be no surprises how administrative data about us are used by others. We should expect fairness. Yet these basics still seem hard for some to accept.

The NHS Royal Free Hospital in 2015 was rightly criticised – because they tried “to commercialise personal confidentiality without personal consent,” as reported in Wired recently.

“The shortcomings we found were avoidable,” wrote Elizabeth Denham in 2017 when the ICO found six ways the Google DeepMind — Royal Free deal did not comply with the Data Protection Act. The price of innovation, she said, didn’t need to be the erosion of fundamental privacy rights underpinned by the law.

If the Centre for Data Ethics and Innovation is put on a statutory footing where does that leave the ICO, when their views differ?

It’s why the idea of DeepMind funding work in Ethics and Society seems incongruous to me. I wait to be proven wrong. In their own words, “technologists must take responsibility for the ethical and social impact of their work“. Breaking the law however, is conspicuous by its absence, and the Centre must not be used by companies, to generate pseudo lawful or ethical acceptability.

Do we need new digital ethics?

Admittedly, not all laws are good laws. But if recognising and acting under the authority of the rule-of-law is now an optional extra, it will undermine the ICO, sink public trust, and destroy any hope of achieving the research ambitions of UK social science.

I am not convinced there is any such thing as digital ethics. The claimed gap in an ability to get things right in this complex area, is too often after people simply get caught doing something wrong. Technologists abdicate accountability saying “we’re just developers,” and sociologists say, “we’re not tech people.”

These shrugs of the shoulders by third-parties, should not be rewarded with more data access, or new contracts. Get it wrong, get out of our data.

This lack of acceptance of responsibility creates a sense of helplessness. We can’t make it work, so let’s make the technology do more. But even the most transparent algorithms will never be accountable. People can be accountable, and it must be possible to hold leaders to account for the outcomes of their decisions.

But it shouldn’t be surprising no one wants to be held to account. The consequences of some of these data uses are catastrophic.

Accountability is the number one problem to be solved right now. It includes openness of data errors, uses, outcomes, and policy. Are commercial companies, with public sector contracts, checking data are accurate and corrected from people who the data are about, before applying in predictive tools?

Unethical practice

As Tim Harford in the FT once asked about Big Data uses in general: “Who cares about causation or sampling bias, though, when there is money to be made?”

Problem area number two, whether researchers are are working towards a profit model, or chasing grant funding is this:

How data users can make unbiased decisions whether they should use the data? We have all the same bodies deciding on data access, that oversee its governance. Conflict of self interest is built-in by default, and the allure of new data territory is tempting.

But perhaps the UK key public data ethics problem, is that the policy is currently too often about the system goal, not about improving the experience of the people using systems. Not using technology as a tool, as if people mattered. Harmful policy, can generate harmful data.

Secondary uses of data are intrinsically dependent on the ethics of the data’s operational purpose at collection. Damage-by-design is evident right now across a range of UK commercial and administrative systems. Metrics of policy success and associated data may be just wrong.

Some of the damage is done by collecting data for one purpose and using it operationally for another in secret. Until these modus operandi change no one should think that “data ethics will save us”.

Some of the most ethical research aims try to reveal these problems. But we need to also recognise not all research would be welcomed by the people the research is about, and few researchers want to talk about it. Among hundreds of already-approved university research ethics board applications I’ve read, some were desperately lacking. An organisation is no more ethical than the people who make decisions in its name. People disagree on what is morally right. People can game data input and outcomes and fail reproducibility. Markets and monopolies of power bias aims. Trying to support the next cohort of PhDs and impact for the REF, shapes priorities and values.

“Individuals turn into data, and data become regnant.” Data are often lacking in quality and completeness and given authority they do not deserve.

It is still rare to find informed discussion among the brightest and best of our leading data institutions, about the extensive everyday real world secondary data use across public authorities, including where that use may be unlawful and unethical, like buying from data brokers. Research users are pushing those boundaries for more and more without public debate. Who says what’s too far?

The only way is ethics? Where next?

The latest academic-commercial mash-ups on why we need new data ethics in a new regulatory landscape where the established is seen as past it, is a dangerous catch-all ‘get out of jail free card’.

Ethical barriers are out of step with some of today’s data politics. The law is being sidestepped and regulation diminished by lack of enforcement of gratuitous data grabs from the Internet of Things, and social media data are seen as a free-for-all. Data access barriers are unwanted. What is left to prevent harm?

I’m certain that we first need to take a step back if we are to move forward. Ethical values are founded on human rights that existed before data protection law. Fundamental human decency, rights to privacy, and to freedom from interference, common law confidentiality, tort, and professional codes of conduct on conflict of interest, and confidentiality.

Data protection law emphasises data use. But too often its first principles of necessity and proportionality are ignored. Ethical practice would ask more often, should we collect the data at all?

Although GDPR requires new necessary safeguards to ensure that technical and organisational measures are met to control and process data, and there is a clearly defined Right to Object, I am yet to see a single event thought giving this any thought.

Let’s not pretend secondary use of data is unproblematic, while uses are decided in secret. Calls for a new infrastructure actually seek workarounds of regulation. And human rights are dismissed.

Building a social license between data subjects and data users is unavoidable if use of data about people hopes to be ethical.

The lasting solutions are underpinned by law, and ethics. Accountability for risk and harm. Put the person first in all things.

We need more than hopes and dreams and talk of ethics.

We need realism if we are to get a future UK data strategy that enables human flourishing, with public support.

Notes of desperation or exasperation are increasingly evident in discourse on data policy, and start to sound little better than ‘we want more data at all costs’. If so, the true costs would be lasting.

Perhaps then it is unsurprising that there are calls for a new infrastructure to make it happen, in the form of Data Trusts. Some thoughts on that follow too.

Part 1. Ethically problematic

Ethics is dissolving into little more than a buzzword. Can we find solutions underpinned by law, and ethics, and put the person first?

Part 2. Can Data Trusts be trustworthy?

As long as data users ignore data subjects rights, Data Trusts have no social license.

Data Horizons: New Forms of Data For Social Research,

Elliot, M., Purdam, K., Mackey, E., School of Social Sciences, The University Of Manchester, CCSR Report 2013-312/6/2013
care.data, children, datasharing, ethics, privacy

Leaving Facebook and flaws in Face Recognition

This Facebook ad was the final straw for me this week.

I’m finally leaving.

When I saw Facebook’s disingenuous appropriation of new data law as-a-good-thing I decided time’s up. While Zuckerberg talks about giving users more control, what they are doing is steering users away from better privacy and putting users outside the reach of new protections rather than stepping up to meet its obligations.

After eleven years, I’m done. I’ve used Facebook to run a business.  I’ve used it to keep in touch with real-life family and friends. I’ve had more positive than negative experiences on the site. But I’ve packed in my personal account.

I hadn’t actively used it since 2015. My final post that year was about Acxiom’s data broker agreement with Facebook. It has taken 3 hours to download  any remaining data, to review and remove others’ tags, posts and shared content linking me. I had already deactivated 18 apps, and have now used each individual ID that the Facebook-App link provided, to make Subject Access requests (SAR) and object to processing. Some were easy. Some weren’t.

Pinterest and Hootsuite were painful circular loops of online ‘support’ that didn’t offer any easy way to contact them.  But to their credit Hootsuite Twitter message support was ultra fast and suggested an email to hootsuite-dpa [at] hootsuite.com. Amazon required a log in to the Amazon account. Apple’s Aperture goes into a huge general page impossible to find any easy link to contact.  Ditto Networked Blogs.

Another app that has no name offered a link direct to a pre-filled form with no contact details and no option for free text you can send only the message please delete any data you hold about me — not make a SAR.

Another has a policy but no Data Controller listed. Who is http://a.pgtb.me/privacy ? Ideas welcome.

What about our personal data rights?

The Facebook ad says, you will be able to access, download or delete your data at any time. Not according to the definition of personal data we won’t.  And Facebook knows it. As Facebook’s new terms and condition says, some things that you do on Facebook aren’t stored in your account. For example, a friend may have messages from you after deletion. They don’t even mention data inferred. This information remains after you delete your account. It’s not ‘your’ data because it belongs to the poster, it seems according to Facebook. But it’s ‘your’ data because the data are about or related to you according to data protection law.

Rights are not about ownership.

That’s what Facebook appears to want to fail to understand. Or perhaps wants the reader to fail to understand. Subject Access requests should reveal this kind of data, and we all have a right to know what the Facebook user interface limits-by-design. But Facebook still keeps this hidden, while saying we have control.

Meanwhile, what is it doing?  Facebook appears to be running scared and removing  recourse to better rights.

Facebook, GDPR and flaws in Face Recognition

They’ve also started running Face Recognition. With the new feature enabled, you’re notified if you appear in a photo even if not tagged.

How will we be notified if we’re not tagged? Presumably Facebook uses previously stored facial images that were tagged, and is matching them using an image library behind the scenes.

In the past I have been mildly annoyed when friends who should know me better, have posted photos of my children on Facebook.

Moments like children’s birthday parties can mean a photo posted of ten fun-filled faces in which ten parents are tagged. Until everyone knew I’d rather they didn’t, I was often  tagged in photos of my young children.  Or rather my children were tagged as me.

Depending on your settings, you’ll receive a notification when someone tags a photo with your name.  Sure I can go and untag it, to change the audience that can see it, but cannot have control over it.

Facebook meanwhile pushes this back as if it is a flaw with the user and in a classic victim-blaming move suggests it’s your fault you don’t like it, not their failure to meet privacy-by-design, by saying,  If you don’t like something you’re tagged in, you can remove the tag or ask the person who tagged you to remove the post.

There is an illusion of control being given to the user, by companies and government at the moment. We must not let that illusion become the accepted norm.

Children whose parents are not on the site cannot get notifications. A parent may have no Facebook account.  (A child under 13 should no Facebook account, although Facebook has tried to grab those too.) The child with no account may never know, but Facebook is certainly processing, and might be building up a shadow profile about, the nameless child with face X anyway.

What happens next?

As GDPR requires a share of accountability for controller and processing responsibilities, what will it mean for posters who do so without consent of the people in photos? For Facebook it should mean they cannot process using biometric profiling, and its significant effects may be hidden or, especially for children, only appear in the future.

Does Facebook process across photos held on other platforms?

Since it was founded, Facebook has taken over several social media companies, the most familiar of which are Instagram in 2012 and WhatsApp in 2014. Facebook has also bought Oculus VR [VR headsets], Ascenta [drones], and ProtoGeo Oy [fitness trackers].

Bloomberg reported at the end of February that  a lawsuit alleging Facebook Inc. photo scanning technology flouts users’ privacy rights can proceed.

As TechCrunch summarised, when asked to clear a higher bar for privacy, Facebook has instead delved into design tricks to keep from losing our data.

Facebook needs to axe Face Recognition, or make it work in ways that are lawful, to face up to its responsibilities, and fast.

The Cambridge Analytica scandal has also brought personalised content targeting into the spotlight, but we are yet to see really constructive steps to row back to more straightfoward advertising, and away from todays’s highly invasive models of data collection and content micro-targeting designed to to grab your personalised attention.

Meanwhile policy makers and media are obsessed with screen time limits as a misplaced, over-simplified solution to complex problems, in young people using social media, which are more commonly likely to be exacerbating existing conditions and demonstrate correlations rather than cause.

Children are stuck in the middle.

Their rights to protection, privacy, reputation and participation must not become a political playground.

AI, care.data, change management, transparency

The power behind today’s AI in public services

The power behind today’s AI in public services

Thinking about whether education in England is preparing us for the jobs of the future, means also thinking about how technology will influence it.

Time and again, thinking and discussion about these topics is siloed. At the Turing Institute, the Royal Society, the ADRN and EPSRC, in government departments, discussions on data, or within education practitioner, and public circles — we are all having similar discussions about data and ethics, but with little ownership and no goals for future outcomes. If government doesn’t get it, or have time for it, or policy lacks ethics by design, is it in the public interest for private companies, Google et al., to offer a fait accompli?

There is lots of talking about Machine Learning (ML), Artificial Intelligence (AI) and ethics. But what is being done to ensure that real values — respect for rights, human dignity, and autonomy — are built into practice in the public services delivery?

In most recent data policy it is entirely absent. The Digital Economy Act s33 risks enabling, through removal of inter and intra-departmental data protections, an unprecedented expansion of public data transfers, with “untrammelled powers”. Powers without codes of practice, promised over a year ago. That has fall out for the trustworthiness of legislative process, and data practices across public services.

Predictive analytics is growing but poorly understood in the public and public sector.

There is already dependence on computers in aspects of public sector work. Its interactions with others in sensitive situations demands better knowledge of how systems operate and can be wrong. Debt recovery, and social care to take two known examples.

Risk averse, staff appear to choose not to question the outcome of ‘algorithmic decision making’ or do not have the ability to do so. There is reportedly no analysis training for practitioners, to understand the basis or bias of conclusions. This has the potential that instead of making us more informed, decision-making by machine makes us humans less clever.

What does it do to professionals, if they feel therefore less empowered? When is that a good thing if it overrides discriminatory human decisions? How can we tell the difference and balance these risks if we don’t understand or feel able to challenge them?

In education, what is it doing to children whose attainment is profiled, predicted, and acted on to target extra or less focus from school staff, who have no ML training and without informed consent of pupils or parents?

If authorities use data in ways the public do not expect, such as to ID homes of multiple occupancy without informed consent, they will fail the future to deliver uses for good. The ‘public interest’, ‘user need,’ and ethics can come into conflict according to your point of view. The public and data protection law and ethics object to harms from use of data. This type of application has potential to be mind-blowingly invasive and reveal all sorts of other findings.

Widely informed thinking must be made into meaningful public policy for the greatest public good

Our politicians are caught up in the General Election and buried in Brexit.

Meanwhile, the commercial companies taking AI first rights to capitalise on existing commercial advantage could potentially strip public assets, use up our personal data and public trust, and leave the public with little public good. We are already used by global data players, and by machine-based learning companies, without our knowledge or consent. That knowledge can be used to profit business models, that pay little tax into the public purse.

There are valid macro economic arguments about whether private spend and investment are preferable compared with a state’s ability to do the same. But these companies make more than enough to do it all. Does it signal a failure to a commitment to the wider community; not paying just amounts of taxes, is it a red flag to a company’s commitment to public good?

What that public good should look like, depends on who is invited to participate in the room, and not to tick boxes, but to think and to build.

The Royal Society’s Report on AI and Machine Learning published on April 25, showed a working group of 14 participants, including two Google DeepMind representatives, one from Amazon, private equity investors, and academics from cognitive science and genetics backgrounds.

Our #machinelearning working group chair, professor Peter Donnelly FRS, on today’s major #RSMachinelearning report https://t.co/PBYjzlESmB pic.twitter.com/RM9osnvOMX

— The Royal Society (@royalsociety) April 25, 2017

If we are going to form objective policies the inputs that form the basis for them must be informed, but must also be well balanced, and be seen to be balanced. Not as an add on, but be in the same room.

As Natasha Lomas in TechCrunch noted, “Public opinion is understandably a big preoccupation for the report authors — unsurprisingly so, given that a technology that potentially erodes people’s privacy and impacts their jobs risks being drastically unpopular.”

“The report also calls on researchers to consider the wider impact of their work and to receive training in recognising the ethical implications.”

What are those ethical implications? Who decides which matter most? How do we eliminate recognised discriminatory bias? What should data be used for and AI be working on at all? Who is it going to benefit? What questions are we not asking? Why are young people left out of this debate?

Who decides what the public should or should not know?

AI and ML depend on data. Data is often talked about as a panacea to problems of better working together. But data alone does not make people better informed. In the same way that they fail, if they don’t feel it is their job to pick up the fax. A fundamental building block of our future public and private prosperity is understanding data and how we, and the AI, interact. What is data telling us and how do we interpret it, and know it is accurate?

How and where will we start to educate young people about data and ML, if not about their own and use by government and commercial companies?

The whole of Chapter 5 in the report is very good as a starting point for policy makers who have not yet engaged in the area. Privacy while summed up too short in conclusions, is scattered throughout.

Blind spots remain, however.

  • Over willingness to accommodate existing big private players as their expertise leads design, development and a desire to ‘re-write regulation’.
  • Slowness to react to needed regulation in the public sector (caught up in Brexit) while commercial drivers and technology change forge ahead
  • ‘How do we develop technology that benefits everyone’ must not only think UK, but global South, especially in the bias in how AI is being to taught, and broad socio-economic barriers in application
  • Predictive analytics and professional application = unwillingness to question the computer result. In children’s social care this is already having a damaging upturn in the family courts (S31)
  • Data and technology knowledge and ethics training, must be embedded across the public sector, not only post grad students in machine learning.
  • Harms being done to young people today and potential for intense future exploitation, are being ignored by policy makers and some academics. Safeguarding is often only about blocking in case of liability to the provider, stopping children seeing content, or preventing physical exploitation. It ignores exploitation by online platform firms, and app providers and games creators, of a child’s synthesised online life and use. Laws and government departments’ own practices can be deeply flawed.
  • Young people are left out of discussions which, after all, are about their future. [They might have some of the best ideas, we miss at our peril.]

There is no time to waste

Children and young people have the most to lose while their education, skills, jobs market, economy, culture, care, and society goes through a series of gradual but seismic shift in purpose, culture, and acceptance before finding new norms post-Brexit. They will also gain the most if the foundations are right. One of these must be getting age verification right in GDPR, not allowing it to enable a massive data grab of child-parent privacy.

Although the RS Report considers young people in the context of a future workforce who need skills training, they are otherwise left out of this report.

“The next curriculum reform needs to consider the educational needs of young people through the lens of the implications of machine learning and associated technologies for the future of work.”

Yes it does, but it must give young people and the implications of ML broader consideration for their future, than classroom or workplace.

Facebook has targeted vulnerable young people, it is alleged, to facilitate predatory advertising practices. Some argue that emotive computing or MOOCs belong in the classroom. Who decides?

We are not yet talking about the effects of teaching technology to learn, and its effect on public services and interactions with the public. Questions that Sam Smith asked in Shadow of the smart machine: Will machine learning end?

At the end of this Information Age we are at a point when machine learning, AI and biotechnology are potentially life enhancing or could have catastrophic effects, if indeed “AI will cause people ‘more pain than happiness” as described by Alibaba’s founder Jack Ma.

The conflict between commercial profit and public good, what commercial companies say they will do and actually do, and fears and assurances over predicted outcomes is personified in the debate between Demis Hassabis, co-founder of DeepMind Technologies, (a London-based machine learning AI startup), and Elon Musk, discussing the perils of artificial intelligence.

Vanity Fair reported that, “Elon Musk began warning about the possibility of A.I. running amok three years ago. It probably hadn’t eased his mind when one of Hassabis’s partners in DeepMind, Shane Legg, stated flatly, “I think human extinction will probably occur, and technology will likely play a part in this.””

Musk was of the opinion that A.I. was probably humanity’s “biggest existential threat.”

We are not yet joining up multi disciplinary and cross sector discussions of threats and opportunities

Jobs, shift in needed skill sets for education, how we think, interact, value each other, accept or reject ownership and power models; and later, from the technology itself. We are not yet talking conversely, the opportunities that the seismic shifts offer in real terms. Or how and why to accept or reject or regulate them.

Where private companies are taking over personal data given in trust to public services, it is reckless for the future of public interest research to assume there is no public objection. How can we object, if not asked? How can children make an informed choice? How will public interest be assured to be put ahead of private profit? If it is intended on balance to be all about altruism from these global giants, then they must be open and accountable.

Private companies are shaping how and where we find machine learning and AI gathering data about our behaviours in our homes and public spaces.

SPACE10, an innovation hub for IKEA is currently running a survey on how the public perceives and “wants their AI to look, be, and act”, with an eye on building AI into their products, for us to bring flat-pack into our houses.

As the surveillance technology built into the Things in our homes attached to the Internet becomes more integral to daily life, authorities are now using it to gather evidence in investigations; from mobile phones, laptops, social media, smart speakers, and games. The IoT so far seems less about the benefits of collaboration, and all about the behavioural data it collects and uses to target us to sell us more things. Our behaviours tell much more than how we act. They show how we think inside the private space of our minds.

Do you want Google to know how you think and have control over that? The companies of the world that have access to massive amounts of data, and are using that data to now teach AI how to ‘think’. What is AI learning? And how much should the State see or know about how you think, or try to predict it?

Who cares, wins?

It is not overstated to say society and future public good of public services, depends on getting any co-dependencies right. As I wrote in the time of care.data, the economic value of data, personal rights and the public interest are not opposed to one another, but have synergies and co-dependency. One player getting it wrong, can create harm for all. Government must start to care about this, beyond the side effects of saving political embarrassment.

Without joining up all aspects, we cannot limit harms and make the most of benefits. There is nuance and unknowns. There is opaque decision making and secrecy, packaged in the wording of commercial sensitivity and behind it, people who can be brilliant but at the end of the day, are also, human, with all our strengths and weaknesses.

And we can get this right, if data practices get better, with joined up efforts.

Our future society, as our present, is based on webs of trust, on our social networks on- and offline, that enable business, our education, our cultural, and our interactions. Children must trust they will not be used by systems. We must build trustworthy systems that enable future digital integrity.

The immediate harm that comes from blind trust in AI companies is not their AI, but the hidden powers that commercial companies have to nudge public and policy maker behaviours and acceptance, towards private gain. Their ability and opportunity to influence regulation and future direction outweighs most others. But lack of transparency about their profit motives is concerning. Carefully staged public engagement is not real engagement but a fig leaf to show ‘the public say yes’.

The unwillingness by Google DeepMind, when asked at their public engagement event, to discuss their past use of NHS patient data, or the profit model plan or their terms of NHS deals with London hospitals, should be a warning that these questions need answers and accountability urgently.

As TechCrunch suggested after the event, this is all “pretty standard playbook for tech firms seeking to workaround business barriers created by regulation.” Calls for more data, might mean an ever greater power shift.

Companies that have already extracted and benefited from personal data in the public sector, have already made private profit. They and their machines have learned for their future business product development.

A transparent accountable future for all players, private and public, using public data is a necessary requirement for both the public good and private profit. It is not acceptable for departments to hide their practices, just as it is unacceptable if firms refuse algorithmic transparency.

“Rebooting antitrust for the information age will not be easy. It will entail new risks: more data sharing, for instance, could threaten privacy. But if governments don’t want a data economy dominated by a few giants, they will need to act soon.” [The Economist, May 6]

If the State creates a single data source of truth, or private Giant tech thinks it can side-step regulation and gets it wrong, their practices screw up public trust. It harms public interest research, and with it our future public good.

But will they care?

If we care, then across public and private sectors, we must cherish shared values and better collaboration. Embed ethical human values into development, design and policy. Ensure transparency of where, how, who and why my personal data has gone.

We must ensure that as the future becomes “smarter”, we educate ourselves and our children to stay intelligent about how we use data and AI.

We must start today, knowing how we are used by both machines, and man.

First published on Medium for a change.