Skip to main content

Daniel Stucke

Diagnostic questioning #Computing style

2 min read

Back in September I was keen to get the Computing teaching community to work together to write a bank of high quality multiple choice questions. In fact, rather poorly, it was the last thing I wrote on this blog! Head back to see why I think that good quality MCQs can be an invaluable part of assessment for learning. Some educators got in touch and sent me some questions, but a lack of common format and no central repository curtailed my enthusiasm for the idea.

Step in www.diagnosticquestions.com. This fantastic site is the brainchild of @mrbartonmaths.

There are lots of sites out there that support MCQs, there are a few reasons why I think www.diagnosticquestions.com stands out:

  • It’s free.
  • It’s really easy to add questions, including as images. So if people already have quizzes as html files, or moodle quizzes or powerpoint or whatever then they can get transferred in here pretty easily with a simple screenshot upload.
  • Students have to explain their answer for every answer. And then they get to compare their explanations to the best ones from students around the world. Fantastic for them, and also fantastic for teachers to delve into understanding, or lack there of.

It would be fantastic if some of the Computing teaching community could chip in with questions of their own. Whilst I’ve put most up so far credit is due to the brilliant @mrocallaghanedu who wrote and shared them originally on his blog.

I plan to use small quizzes each lesson on the run up to exams to ensure that students and myself know what they know, and more importantly know where the gaps are. They can revise to the gaps, and I can teach to them. Of course this will be supplimented with lengthier question types akin to those found in the exam. I also hope to get some of my more able students to write MCQs of their own. It’s no easy task and really tests your subject knowledge, a perfect task for those pushing A/A* grades this Summer in the GCSE.

The Diagnostic Questions site:

Daniel Stucke

Crowd-Sourcing Multiple Choice GCSE Computing Questions

6 min read

A plea to teachers of Computing GCSE, please join an effort to write a bank of high quality multiple choice questions to support the teaching of this course.

Why multiple choice questions?

There is an increasing body of research and writing showing that skilfully written multiple choice questions are an effective means of developing retention. They are easy to administer and easy to mark, particularly in an IT rich environment. My experience (and looking at OCR data, most other school’s experience) is that students perform poorly on the written examination part of the GCSE. Their retention and recall of the knowledge needed to succeed in the exam is poor.

Well written multiple choice questions could be used:
- as lesson starters / plenaries etc to revisit learning covered earlier in the course, helping to develop retention
- as hinge questions in the middle of a lesson as part of the AfL process
- as small assessments of units of work
- without the wrong answers, as flashcards for learner’s revision

Further reading

My thoughts on this have been shaped by some excellent writing and research online…
- Joe Kirby - How to design multiple-choice questions
- Daisy Christodoulou - Research on multiple choice questions
- Robert Bjork - Multiple-Choice Tests Exonerated, at Least of Some Charges: Fostering Test-Induced Learning and Avoiding Test-Induced Forgetting
- Dylan Wiliam - When is assessment learning orientated
- Dylan Wiliam & Caroline Wylie - Diagnostic Questions: Is there value in just one?
- David Didau - How effective learning hinges on good questioning

Interested?

Great! I have set up a Google Form for you to submit your questions and answers. I’ve split the Computing GCSE up into a few high level topic areas to categorise each question. Once you’ve shared some questions I will happily give you access to the spreadsheet behind the form and what will hopefully grow into a large bank of high quality questions. If you have lots that you’ve already prepared and want to send me an email with them in a different format that would be wonderful. I’ll do my best to add them into the rest without you having to copy each one in to the form.

7 Principles for Designing Multiple Choice Options

Quality questions that help develop retention need carefully crafted options for the answers. Please follow these guidelines when constructing your questions and answers.

With his permission, I’ve shamelessly stolen this list from Joe Kirby.

  1. The proximity of options increases the rigour of the question For instance, the question is, what year was the battle of Hastings? Options 1065, 1066, 1067, 1068 or 1069 are more rigorous than options 1066, 1166, 1266, 1366 or 1466. Of course, the question itself also determines the rigour: ‘80 is what percentage of 200?’ is much easier than ‘79 is what percentage of 316?’  
  2. The number of incorrect options increases rigour Three options gives pupils a 33% chance of guessing the correct answer; five options reduces the chances of guessing to 20%; always create five rather than three or four options for multiple choice questions. A ‘don’t know’ option prevents pupils from blindly guessing, allowing them to flag up questions they’re unsure about rather than getting lucky with a correct guess. With this in mind the form will accept questions of the form:
    • 1 correct answer from a total of 4
    • 1 correct answer from a total of 5
    • 2 correct answers from a total of 5
    • 2 correct answers from a total of 6
      The further down that list the less chance someone can guess the answer correctly.
  3. Incorrect options should be plausible but unambiguously wrong If options are too implausible, this reduces rigour as pupils can too quickly dismiss them. For instance, in the question: what do Charles Dickens and Oliver Twist have in common, an implausible option would be that they were both bank robbers. However, if answers are too ambiguously similar, this creates problems. For instance, in the question, ‘What happens in the plot of Oliver Twist?’, these options are too ambiguous: a) A young boy runs away to London b) An orphan falls in with a street gang of street urchins c) A poor orphan is adopted by a wealthy gentleman d) A criminal murders a young woman and is pursued by a mob e) A gang of pickpockets abduct a young boy
  4. Incorrect options should be frequent misconceptions where possible For example, if you know pupils often confuse how autobiographical ‘Oliver Twist’ is, create options as common confusions. These distractors flag up what pupils are thinking if they select an incorrect option: a) Both were born in a workhouse b) Both were separated from their parents and family c) Both were put in prison for debt d) Both had families who were put in prison for debt e) Both were orphans
  5. Multiple correct options make a question more rigorous. Not stating how many correct options there are makes pupils think harder. For example: Which characteristics of “Elegy Written in a Country Churchyard” can be seen as Romantic? a) It celebrates the supernatural. b) It is written in iambic pentameter. c) It emphasises emotion over reason. d) It deals with the lives of common people. e) It aspires to nature and the sublime.
  6. The occasional negative question encourages kids to read the questions more carefully. Once they get a question like ‘Which of these is NOT a cause of World War 1?‘ wrong, and realise why, they’ll work out they need to read questions again to double-check on what it is they’re asking.
  7. Stretch questions can be created with comparisons or connections between topics. What was common to both the USA and Germany during the Great Depression?
    a)     Jewish immigration increased
    b)     Membership of Ku Klux Klan increased
    c)     Public works projects were implemented
    d)     Government social programs were reduced

Still here? Then let us begin!

The form is available here http://bit.ly/multicompq. Don’t rush, fashion some great questions as you go throughout this year. Together we can make a powerful learning resource. And hopefully many hands will make light work of the task!

If you have a big list of questions to submit to the cause then please email them to me at dstucke [at] stretfordhigh.com and I will endeavour to add them to the master list. If you would like access to the bank of questions then leave me a comment here, email me on the address above or send me a tweet. I’ll collate them into a shareable format. Probably just a csv or txt file to start with that can then be used to import into your response system of choice be that Moodle quizzes, Socrative or ExitTicket questions, whatever you choose.

Daniel Stucke

Confidence in predicting attainment

3 min read

How accurately can we set targets and predict performance of pupil’s exam attainment?

At this time of year the pressure is on teachers and leaders in school to know exactly how their young people will perform in the GCSE exams that they are currently sitting.

There is an expectation from leaders, governors and of course Ofsted to accurately track, monitor and predict pupil progress and hence exam performance.

Our school, like every other in the country, set targets for individual pupils at the start of the year and then ask the teachers to measure the pupil’s progress in comparison to this target grade, and to predict their final exam grade. These are collated and a range of performance measures for classes, subjects and the whole school are calculated.

But how accurate are these and how confident can we be in these predictions and targets? Our school has 155 young people in year 11. And some subjects have as few as 15 learners. One or two students having a bad day in the exam, or realising too late that they picked a subject that they really don’t enjoy, can have a large impact on the results. How much should we take this into account when setting targets and holding teachers and middle leaders to account?

This train of thought has led me to look at the use of confidence intervals. I’ve used the binomial proportion confidence interval to calculate the 95% confidence interval on the number of students predicted to achieve a C+ in each subject. This takes into account the size of the class.

I’ve ended up with results such as: English C+ = 79% +/- 6%. So we’re 95% sure they will end up with results between 73% and 85%. Computing, with a smaller number of students, comes in at 70% +/- 16%.

Is this an appropriate statistical measure to use?

The confidence intervals are quite large due to the small number of students involved. I’m not sure if this appropriate though? This measure is normally used when sampling a small part of a larger population. Is that what we are doing here? In that our Computing class is a small sample of the national group sitting that examination? Or are we actually sampling the whole population where that population is simply all the students studying it at our school?

If this is not that right measure of confidence to use in this case then what is?

Lots of questions that I’m hoping some of the data / statistics community can help answer. The stakes are high in schools now and the targets that departments and schools are set have high levels of accountability attached to them. It’s only fair that targets and predictions come with a suitable statistical window attached. How should that be calculated?

Daniel Stucke

2 min read

[gallery]

Simple Things - 5 Minute Lesson Plan - iPad Edition.

I love the 5 Minute Lesson Plan, the wonderful TeacherToolkit should be very proud of it. It’s all but superseded our school lesson plan proforma for most staff and I can understand why.

The best medium I’ve found to date for using it is the good old fashioned piece of paper. It just works best when you’re scribbling down ideas. Typing onto it is just unpleasant.

But I don’t like paper, I lose paper, and next year when I want to see it again the paper is gone.

Step in the iPad, Penultimate and a stylus.

To set this up you can follow these simple instructions:

  • Save the image above to your iPad camera roll. (Press the right arrow to get the blank image.)
  • Inside any Penultimate Notebook, tap the Papers icon on the top right corner of your screen. Hit the (+) sign, then Continue to add a new Paper.
  • Select the image you’d like to create your Paper from, and name the Paper whatever you would like.Click on your custom Paper to start writing or drawing, just as you normally would.
  • It will now be the default Paper when you create new pages in your current notebook. If you switch Papers or Notebooks and then want to return to the last selection, you can find it in My Papers.

Hope you find this useful. The updated edition of Penultimate is great, and this means I can now quickly scribble down lesson plans and keep a permanent record of them in Evernote.

PS: I’m enjoying using the iPad more and more for my note-taking. I’ve ordered the new Adonit Jot pen, looking forward to it arriving and seeing if it means that I can finally move my notes out of my trusty moleskin and straight onto the iPad.

Daniel Stucke

GCSE English, seconds out, Round 2!

5 min read

It’s hard to believe it’s a year since Round 1 of the English GCSE fiasco. Is Round 2 about to kick off? Don’t be surprised if it does!

Will teachers and students be knocked out again?

Just to recap:

  • Accountability pressures from league tables etc led to lots of schools entering students for their GCSE English in January and/or July.
  • This skewed the January results and led to exam boards / Ofqual setting boundaries that made it a bit too easy to get a C grade in the January exam.
  • In July, exam boards / Ofqual realised their mistake and moved the grade boundaries for students that sat the exam in July. This made the headline C+ figure fit with Government expectations, but meant that students who sat the July exam sat an unfairly harder exam than those who sat it in January.
  • Thousands of schools and young people cried foul.
  • Results stood.
  • Ofqual & Gove announced changes to try and stop this happening in future. January results haven’t been released this year, they will be moderated in the Summer at the same time as the July entries. Going forward early entry is being squeezed out as an option. Controlled Assessments will be wound back in due course to alleviate alleged (but not to my knowledge, evidenced) cheating by teachers.

The impact of this is still being felt across the Secondary sector. Schools dropped below floor targets, I’m sure people directly or indirectly will have lost jobs. English departments with many years of experience were left questioning their knowledge of the curriculum, not quite sure what a Grade C student looked like any more.

The pressure on schools to get students through English and Maths at a ‘pass’ grade of C has not diminished. Moves to focus on levels of progress in the future are welcome but I don’t believe they will impact on the tendency for %C+ in English & Maths being the headline measure that schools are held to account with.

But it’s not just the pressure on schools, it’s also the pressure on young people. The constant rhetoric of students achieving 'good’ GCSEs or 'passing’ their GCSEs from Government and through the press means that it’s never been more important for learners to leave with a C in Maths and in English. Not having these grades puts them at a huge disadvantage at every possible next (and future) step of their education or employment.

So schools have been looking for ways to increase the number of learners getting this magic grade. Early entry, multiple entry, iGCSE entry or combinations of all three have taken place. Ofqual have been getting in early this year with warnings that this will skew results next week. This feels slightly ominous to me.

Many schools have looked to enter students for the International GCSE or iGCSE. Entries are up from 18,000 to 63,000 this year, including 20,000 late entries. The logic is clear, it’s seen as an 'easier’ option, with clearer controlled assessment requirements, clearer exam questions and a more reliable structure. Some schools have switched entirely to the qualification, some have moved key cohorts to the qualification and some have entered students in addition to the normal English Language GCSE as a 'second chance’.

The Independent have a sensationalist article claiming the 'gamble’ has failed. It’s poorly thought out. Yes the percentage of students getting A* and C+ grades have fallen, but that fails to take into account the makeup of the population sitting the qualification this year. iGCSEs have historically been the preserve of Independent Schools (who have clearly been wise to it’s charms way before state schools). Talented, well supported learners who have been paying for very expensive educations tend to get pretty good results. The extra 45,000 entries this year are very different, almost all will be state school students, and in particular the majority of these will be C/D borderline students. Ring any bells? This is exactly what happened with the January GCSE entries last year.

We entered our borderline students for the iGCSE. They seem to have done incredibly well. Excitement will be reserved until all the results are in next week.

If lots of schools have entered borderline students for the iGCSE and if lots of them have done well then the next week could be interesting.

iGCSE results are out and set in stone now. Combined with the effects of multiple entries etc., they could positively influence the overall national figures for English C+ and 5A*-C with English & Maths. Which makes me nervous. Ofqual & Gove will not want to see any dramatic jumps in performance. The only other way to influence this would be to tamper with the grade boundaries for the normal English GCSE again. And they have form.

As long as the league tables and press / Government obsession with grade C’s remains, this merry go round will continue. Schools have to look for every opportunity to get the best figures they can, they would be letting the students down if they didn’t. We should of course be looking to give students a rounded education and a love of both English and Mathematics but that ignores the results driven culture that pervades every aspect of Britain today.

Image from Kikfoto on Flickr.