In progress.

My feelings on this has changed a lot. Ethan Mollick’s Co-Intelligence second principle is to always keep humans in the loop.

An exceptionally disastrous case study of this in motion was when the UK’s Department for Education decided to use AI to grade A-level marks due to the inability of students to take their final year exams. This led to many students receiving marks that they felt were very low. Some took issue with unfair considerations in testing, including being graded down if their school had historically lower marks. Funnily, it looked like they had hoped to use the AI model as a second pair of eyes to the normal marking, perhaps to ascertain its accuracy over time for use in years to come. Just not, you know, untested in 2020. They even wrote, “Any future use of AI is likely to take some time and a lot of testing. We are not going to suddenly see AI being used at scale in marking high profile qualifications overnight.”

Funnily, it looked like they had hoped to use the AI model as a second pair of eyes to the normal marking, perhaps to ascertain its accuracy over time for use in years to come. Just not, you know, untested in 2020. They even wrote, “Any future use of AI is likely to take some time and a lot of testing. We are not going to suddenly see AI being used at scale in marking high profile qualifications overnight.”

(Side note: I attended high school in Queensland, which until recently, used the ‘OP score’ which actually was an algorithm that sounds very similar to the UK’s 2020 scoring. It was a percentile grading across the whole state, using a combination of our school marks with general aptitude and intelligence tests that everyone takes regardless of subjects. Then it grades people’s intelligence against other students in their school/class and each school against each other to grade people on a bell curve.

I definitely saw the benefits of the algorithm, scoring higher because I went to a prestigious high school that was extremely academically focused. I also accidentally gamed the system by taking 10 subjects.

In progress.

In progress.