Thursday 21 September 2017

Joining the dots - and no copying!

Standardised digital tests, tailor-made for students in Irish second-level schools, are available now for the first time
Screen test: Teacher Deirdre O'Donoghue, who manages the student induction and testing process at Lucan Community College, working with pupils Photo: mark condren
Screen test: Teacher Deirdre O'Donoghue, who manages the student induction and testing process at Lucan Community College, working with pupils Photo: mark condren
ERC team: Dr Eemer Eivers says DOTS is tailor-made for Irish students
Katherine Donnelly

Katherine Donnelly

One student was unashamedly honest and admitted that he prefers paper tests "because it's easier to copy what that person next to you is doing".

Apart from that, reactions to a suite of new, computer-based standardised tests for Irish post-primary schools were generally positive, ranging from "a lot more interesting than a normal paper exam" to "fun and different" and "10/10".

The comments are among those gathered from students who acted as 'guinea pigs' for the new online tests, devised by the Educational Research Centre in Drumcondra, Dublin. The ERC is already well known for its so-called 'Drumcondra' standardised tests, used in primary schools.

It recently launched DOTS (Drumcondra Online Testing System), which will eventually host a large range of standardised tests for both post-primary and primary students.

The three tests currently available represent a first in a number of respects: the first such tests produced in Ireland for Irish post-primary students; they are digital, rather than paper-based; they provide instant, computer-generated results.

Standardised testing is a common assessment tool in education systems.

Standardised tests compare a student with other students at the same class level or of the same age, using the same bank of questions, scored in the same way.

The tests are typically divided into ability (to help schools gauge the capability or potential of students) and achievement/curriculum-linked (to help schools gauge how well different aspects of the curriculum have been absorbed by students).

Some differ from the usual school-based exam or the state exam, in that they are not assessing specific student knowledge of a particular curriculum, but rather whether students are meeting certain standards in key areas, such as literacy and numeracy.

Others are very closely linked to the curriculum, and provide teachers with a picture of how a student is doing relative to the rest of his or her class, and relative to students in other schools, nationally.

When done on an international basis, such as the periodical OECD PISA assessment, it allows the Department of Education to see how Irish students are performing against their peers in other countries.

In Ireland, standardised testing has been the practice in primary schools for many years, but less so at second-level and, what tests have been used, came from the UK.

According to Dr Eemer Eivers, research fellow at the ERC: "many post-primary schools do administer standardised tests, but they tend to be general ability tests, not achievement tests. There hasn't been a strong tradition of standardised achievement testing in post-primary schools."

Post-primary schools most commonly use ability tests at the pre-entry stage.

Dr Eivers says that the ERC's new achievement tests, for the end of second year, would indicate how well a student was absorbing the content of the maths and English curriculum and, if it was in maths, could reveal whether students were having difficulty with, for instance, geometry.

She says that, in Irish post-primary schools, assessment linked to learning of the curriculum has always tended to be via teacher-made tests or via the state certificate exams, and "while we are not suggesting that our standardised achievement tests replace either of these, they definitely can complement them both".

Dr Eivers points to their value in allowing teachers to gauge how their students compare against a large, nationally representative sample of their peers. "It is simply not possible for a teacher to do that without using standardised tests."

Curriculum-linked standardised testing at second-level is a feature of high-performing education systems around the world. In Ireland, it has been the subject of discussion for years, but, like a lot of other ideas in the education field, progress is slow, if at all.

When the junior cycle reforms were announced in 2012, compulsory standardised testing of students at the end of second year was envisaged as part of the bold plan to phase out the Junior Cert as we know it and put a greater focus on ongoing school-based evaluation.

Those reforms have been diluted substantially in the face of teacher opposition and, in the process, the idea of compulsory standardised testing disappeared from that platform. But, it didn't go away.

Other recent Department of Education initiatives, such as the Numeracy and Literacy Strategy and the School Self Evaluation programme, place an increasing onus on schools to deliver on performance targets and continually assess how they are doing.

Dr Eivers is part of the ERC team that developed the DOTS tests, which are tailor-made for Irish students. For example, teachers can gauge their pupils' strengths and weaknesses on specific aspects of the Junior Cycle curriculum.

The ability test measures the general reasoning ability of students during the transition from primary to post-primary, and during the early years of post-primary. It is not directly linked to the curriculum. The maths and reading tests are curriculum-based, and designed for the end of second year.

As well as carrying out pilot projects to see how well DOTS worked, the ERC also "standardised" the tests on a very large, nationally representative sample of students in 2016, so comparisons against the national norm are immediately available to schools.

Dr Eivers says the online dimension of the tests has engaged students, which makes it more likely that they provide accurate measures: "If a student is bored or thinks the test lacks credibility, you get a poor measure of what they know."

The ERC also worked to ensure that technology does not overshadow what is being assessed: "Students require only minimal IT skills. The questions are very simple. We've deliberately avoided the 'bells and whistles' type of questions - drag this, drop that, circle the other - which may look impressive, but don't generally add anything to the accuracy of measurement. In fact, they can sometimes add unwanted variance - interfering with what you are trying to measure."

And yes, the student who commented that paper tests were easier if you wanted to copy what someone else was doing gets full marks for that observation.

Each test is available in multiple, parallel versions and DOTS randomly assigns one to a student. The student's password shows which version of the test a student has, so a teacher can sit that student beside someone with a different version. "Additionally only one question at a time is displayed on-screen. These features mean that students have very limited opportunities to copy," says Dr Eivers.

Apart from being designed for Irish students - the software was also developed by an Irish company, Vidappt - DOTS is a lot cheaper than UK tests.

'Students more relaxed doing computerised test'

One of the things that struck Diane Birnie when she introduced her students to the Educational Research Centre's (ERC) new computer-based tests was the ease with which they approached them.

"They were more relaxed with the computer than if they were doing it with paper and pencil," says the principal of the 875-pupil Lucan Community College, Co Dublin.

The school was one of those that piloted DOTS for the ERC, and has been won over by it. It recently used the ability test for incoming first years and is using the achievement tests for second years.

Up to now, the school used standardised tests from a British company, and Ms Birnie regards DOTS as a "really positive" development.

She counts the benefits of DOTS in terms of the link to the Irish curriculum, the fact that they are computerised and that they are cheaper.

Ms Birnie says when students did pen and pencil standardised tests she would have noticed that "they might skip a line, or be looking for things on the wrong line", but that is not possible with DOTS.

The instant, computer-generated results are also a big boon for the school, particularly when dealing with incoming first years.

"Previously, it could take two to three weeks to get the results. Now within a couple days of the test, I can be working on forming classes," says the Lucan principal.

Irish Independent

Editor's Choice

Also in Irish News