Assessment Service for UK Primary School Pupils

Hi folks,

I’ve recently completed the training course and created this map to accompany a case study for a digital transformation I coached. This was a pioneering project as although the client organisation was used to coordinating paper-based assessments, this was their first assessment to be delivered digitally. Additionally, it was the first that targeted pupils so young.

It shows that:

• the organisation contracted out the building of key components of the value chain and, therefore, lost a good deal of proprietary knowledge gained. This is a policy of the UK Government as departments typically do not have the skills to take on these types of projects.
• the organisation outsourced the compute and storage utilities, as they did not have the internal capability.
• the School Management System does not contain accurate pupils records and so the Pupil DB has to rely on periodic school census data

If I had to run a business like this, I would:

• integrate the Pupil DB and Results DB into the School Management System to reduce operating costs and increase value within the system (i.e. tracking pupil progress in a more granular way)

As this was a pioneering service, the Pupil experience is in the novel practice space and the School Staff and Civil Servant experience is in the emerging practice space.

I’m happy to take feedback on this map.

Thanks in advance,

Andrew

Digital Assessment for Primary School Children1922×1732 239 KB

Good map, good insights.

It could be worth checking whether there are any components from the value chain available as products or commodity services that could lower the costs.

Thanks for the feedback, @chris.daniel …

As this was a pioneering service for the client, this meant a custom build for all components shown (plus a few more, omitted for clarity) … We did manage to outsource the compute and storage infrastructure though, as the client didn’t have that capability internally.

What became apparent to me on creating the map was the strategy in GOV.UK digital delivery to outsource the research, design and build experience. Although this does make some economic sense, since the GOV.UK departments lack the skills and talent to do it themselves, it does mean that the UK taxpayer is paying private sector organisations to learn how to do this for them. It would be an interesting exercise to sketch a causal loop diagram for the learned helplessness the economic paradigm creates.

I guess my challenge here would be about user needs, especially user needs for the children who are being assessed … is taking the assessment the pupils’ user need or is it really someone else who wants them to take the assessment?

and then if that’s the case, what are the children’s users needs, e.g. do they need to be assessed fairly or to have their data protected, etc.

Great challenge. The primary customer of the system is the Minister for Education, as an assessment instrument to indicate if the education policy is working. That said, the system is entirely dependent on several constituencies in order to succeed:

• School staff (users) can use the system effectively
• Pupils (users) can be assessed effectively
• Teacher and headteacher unions (stakeholders) are supportive of the assessment and don’t instruct their members to boycott it

In reality, the prime goal of the service is that it is a fair representation of a child’s mathematical ability, and not of their IT ability. Exhaustive user research was conducted, which informed the user’s experience, the interface design, as well as policies that guided the assessment that took into account the most extensive list of adjustments ever undertaken by the Department for Education.

Oh, and security was baken in from the start.

One of the things I like to do with my maps is look at each component and evaluate its overall health. How’s it doing? Is it in tip top shape? Is it middling? Is it weak af? or even DEAD?

And… how can you even tell? Are the components in your system observable enough to evaluate their health?

The other thing I might consider doing is looking for hidden dependencies… What else do each of those parts depend on… especially since the health of a component part varies with how effectively its dependencies are performing.

Another thing… follow the suck! Where does it hurt? Where is the pain! Increase the granularity of the map in those places (or make new maps to explore them more deeply).

Also echo @agvbergin on the competing needs. While it may be completely clear what the official line is, you may need to go see what’s actually going on in order to better design how the assessment fits into their world. While extensive user research may have been performed, it’s very easy for that stuff to get myopic in focus (e.g., preoccupied with which buttons someone clicks in which order instead of noticing the student’s actual lived experience surrounding the assessment itself). You may be right about its efficacy, but it never hurts to confirm! (At the very least, you could include more of that context on the map… unless that’s really not the point here! Which is okay too! You get to decide!)