This week we discovered that Manchester City football club employs 11 statisticians and a professor of physics to analyse “every step each player takes” generating over 1500 statistics for each player in each match. When you look at the combination of all this data and compare it to the price paid for each player, the information is clearly highly valuable to football clubs and to the players themselves who are even now wearing devices that tell them when their performance in any aspect is dropping below their ‘target’.

Is this to be the future of education? Multiple cameras in every classroom tracking mastery of content verses cost efficiency of the teacher? Students getting a buzz on their smart phone whenever their performance in a piece of ‘tailored’ content is not matching expectation?

Undoubtedly there are ways in which such technology can assist education but there are also a whole range of ways in which it could be a living nightmare. This week, for example, we also discovered that 47,000 teachers left the profession in the UK last year with the most common reason being the ‘obsession with targets, monitoring and data’. So where is such data valuable and to whom?

Four Simple Tests for Data Systems Designed for Personalisation

The designers of data systems can unknowingly make assumptions based on how they were educated themselves. Such assumptions tend to get built in to the system and can go unchallenged. To prevent this happening, here are four simple tests to apply to any data system designed to improve personalisation.

Test 1 – Does it work without using examples from Maths? Maths is almost unique in having higher level questions that can be assessed as right or wrong by a computer. Unfortunately in the last few conferences I have attended on this subject presenters used the example of a successful project in Maths and then, without acknowledging how Maths is unique extrapolated out to other areas of study. To avoid this issue I would advise caution when maths examples are used. If you conduct the debate without considering maths and come up with a solution then you can be absolutely certain that Maths will slot back in just fine.

Test 2 - Does the system lead to students collaborating more? The ability to work and learn collaboratively has been identified as the most vital 21st century skill. Research evidence is very strongly in favour of collaborative 'student centred' learning and very much opposed to individualized learning. Does the data system place greater weighting on activities that are collaborative? Is it likely to drive collaboration and reduce the amount of individualized learning?

Test 3 - Is most of the data intended to be used by the learner themselves? In the 19th century, education started with the state and the church and next most important was the school, then the teacher then the student. In the 21st century we need a personalised approach that starts with the student and works out from there. I know that most commercial companies still have to deal primarily with the school or education system as their client but student voice and the world of the app is rapidly changing this. Even the football clubs that treat players as commodities pass all data collected on a player back to them so that they ultimately can be engaged in their own improvement regime. So we should begin by considering the value of student data for the student themselves and work outwards from there.

Test 4 - Are student opinions as important as other forms of data? It amazes me that we don't regularly ask students about how confident they feel or how well lessons helped them understand. When students become the true client of the education system this will change enormously and companies will be falling over themselves to gather feedback from the students. Good data systems should treat survey data and peer assessment data as equal in value to test data and teacher assessment.

Data For the Student

My daughter is learning the Highway Code in preparation for her driving test and on her phone she has a detailed breakdown of which types of questions she is good at and which ones she needs more practice in. The insurance company has given us cheaper insurance because we have allowed them to fit a GPS tracker in her car which gives her a monthly report on 10 key indicators including how close to the speed limit she has been and how often she has applied the brakes severely. All this data has undoubtedly increased the efficiency of her learning and her confidence. It won’t help her develop the actual skill of driving but it will deliver all the low level learning content that is required for her to progress.

Monitoring and feedback apps that can assess short answer response and factual questions as well as taking simple measurements from GPS or simple verbal responses, are available across the world at relatively low cost. Some such as Lingua can analyse your pronunciation and help you learn languages whilst others can buzz you if your concentration drops while driving for example. Some of these apps analyse your response to questions through the use of highly complex advanced data processors that are able to gauge emotions, and modify their feedback automatically to match the self-confidence of the user. These supercomputers are otherwise known as people!! Peer learning and peer assessment is the most powerful growth area in online education for example giving feedback to your photography and social wit through ‘likes’; helping you break through a difficult point in a game or guiding you through how to stay calm while making an omelette! Even soft skills development and 21st century skills can appear on a dashboard following online peer assessment with software such as PbyP (Personalisation by pieces).

MOOCs (Massive Open Online Courses) are no longer just providing content and simple feedback tests but are beginning to facilitate communities of peers who can support each other without the need for costly online tutors. Companies that have attempted to use the same systems in schools however have found that they only work with students who already possess the skills of motivation, determination and resilience. Nobody, for example has developed a teacher data dashboard for an Xbox game because the ‘students’ already have the determination and motivation they need so are entirely self-directed. The data and game stats are uniquely provided for the ‘student’ themselves because they are the only people who need them. Peer assessment is trusted in these cases because it is low status compared to formal qualification. Serious gaming lab are attempting to apply the same psychology to learning History or Science for example but have noticed that students are only driven by such motivation for a month or so because it is developing success in the game that is motivating them and not the underlying content. Also, only around 20% of students are driven by games to this extent anyway, and whilst this is a large market for games it is not the complete solution for education.

The school of one and such schemes have similarly noticed that for most students, just providing courses that respond to your data and give you a ‘personalised playlist’ have limited success outside of maths teaching and research has now clearly demonstrated the risk of social isolation that can occur. If the motivation, resilience and determination isn’t there then no amount of clever online programming appears to work.

It seems there are two missing ingredients for self-directed learning of traditional school subjects. One is to enable every learner to have access to the internet whenever they need it, the second is frequent face to face help and praise from an expert who can link the understanding to things you already know and challenge you out of your comfort zone while nurturing your confidence and helping you build your own motivation, determination and resilience. In other words, a teacher!

Data for the teacher

Sugata Mitra noticed that if he employed a student to move between self-directed learners with the sole role of taking an interest in their work and praising them then, irrespective of how much the software praised them, the students were more motivated and maintained effective learning for longer. The Tripod project in the US also found that if a student believed that a teacher liked them and wanted them to excel then they were significantly more likely to be successful. Learning is inherently a social activity and most ‘higher level’ comprehension and skills development occurs through debate, discussion and exploration. Excellent data in the hands of an uninspiring teacher is not going to inspire a love of learning whilst the best parents and teachers in the world do an excellent job with no data at all.

So let’s imagine the most amazing data system we can think of which analyses every key stroke and reaction a learner makes and feeds back analysis suggesting what can be done to improve further. The key question for me is, who does this data go to? I can think of four possibilities

1. The student and the teacher get it and then the teacher’s main role is to help the student interpret and act on the information
2. OR the teacher gets it and ‘tailors learning’ for the student.
3. OR the student, their teacher and a group of their peers get the data. The teacher’s main role is to ensure that the team collaborate to help each other support the learner.
4. Just the student gets the data and can ask for support from the teacher if they wish.

Currently it is model 2 that is happening most often. The teacher feels they have to tailor the solution that allows the student to pass the exam even if the student has no motivation, determination or resilience themselves. Won’t this just produce a generation of students who lack the basic fundamental self-direction they will need to sustain their learning into adulthood? Should this be the goal of data in learning?

Data for the school

The school needs a dashboard of key indicators so it can check if its decisions are having the required impact. Like good football clubs it must nurture talent in its students and teachers and, also like football clubs it is likely that the most effective way to do this is when the teacher has control of the data and takes an active role in improving it. MacKinsey found that the most rapidly improving education systems in the world were those that engaged teachers as active researchers in charge of the data and actively sharing it with students. In the case of the highest performer, Finland, teachers are trusted to assess in the absence of national test data. In the worst examples, governments use national data to set sweeping national policy that is then applied to all contexts disempowering teachers and learners. Large scale research projects in the US and UK have found that such imposition of data led targets leads to a gradual deskilling of the profession and wider variation in the quality of education overall.

If extremely high quality data is available to schools then they will almost always use it to focus more heavily on the targets set by government. If a national indicator is the number of students who pass proficiency in maths, for example, then schools will use the data to accurately target additional resources on those students who are failing to hit this threshold and those teachers who are filing to convert enough students to examination success. In short – what can be measured will absorb even more of the funding and what can’t be measured will be side lined. This has already been identified as an issue by the OECD who have shown that subjects such as Art, Music and Physical Education are being downgraded as more funding is channelled to improving international comparison data. Nowhere on the planet is this more bizarre than in the UK in which the country leads the world in film, music and the creative industries from the special effects of Gravity through to the directing talent of Steve McQueen yet the UK government has just downgraded art, music, film studies and most of the creative qualification routes that were available to students because they were afraid that standards were not being measured accurately enough for comparison of data!!

In summary

We will be hearing from a range of suppliers working in this field at the Global Forum in Barcelona. The acid test to apply to any solution proposed is simple. If you swap the word ‘student’ for ‘employee’ and then read through the proposal imagining that you are the ‘employee’ being discussed, will the proposed system result in you being more empowered, will you be consulted about your learning and will it help you support your higher order learning such as creativity, collaboration and reflection as much as it takes into account lower order measurements?

How many of the following tick list does the data system satisfy?

• The system is primarily designed to feedback data to the learner
• The system gives equal weighting to lower level measures, peer assessment measures and ‘softer’ skills
• The system facilitates teachers as action researchers trying out new ideas and evaluating them
• The system facilitates learners working together collaboratively on problems
• The system is being applied to Art and Music with the same weighting as to Literacy and Numeracy
• Learners have the ability to challenge the conclusions drawn from the data held about them

How might you change this list to make it a better checklist?