Developing
an ARB resource
Assessment
resources originally were developed to cover the New
Zealand English, Mathematics, and Science curricula for
levels 3-6. This was later changed to levels 2-5. Further resources are added to provide support for
teachers as areas of need are identified.
The resources
are designed for a variety of assessment purposes,
including:
- diagnostic
- formative
- peer
- self
- summative.
The development process
Creating a
resource for the Assessment Resource Banks is a rigorous
process subject to numerous checks and reviews.
1.
The first draft
The resources
and scoring guides are initially drafted by:
- ARB
development team members at NZCER and/or
- writing
panels consisting of classroom teachers.
2.
Review process
Two or three
of the ARB team review the draft for:
- curriculum
fit
- suitability
for level
- content,
including teacher information
- clarity of
instructions
- layout
- surface
features, e.g., punctuation, paragraphing, spelling
- appropriateness
of graphics and photographs
- relevance
of context.
Changes are
made based on review feedback before being passed on to
the next reviewer.
The resource may be rejected if reviewers decide it is
unsuitable.
3.
Pre-trial (pen-and-paper resources)
Four to seven
resources are grouped into a trial pack.
The trial pack is trialled with a sample of 6-10 students.
Adjustments are made to the resources as a result of
responses made by the trialling students.
Particular
attention is paid to:
- vocabulary
used
- the clarity
of instructions
- whether
students can successfully complete the task.
The resource
may be rejected at this point if students' work demonstrates it is
unsuitable.
4.
One of two types of trial are carried out.
- Pencil-and-paper tasks are sent to approximately eight
schools selected from all over New Zealand and
completed by about 200 students. The scripts
are returned to NZCER. ARB team members code student
responses, and results are analysed and collated.
This provides diagnostic data about the students'
responses to the task.
- Practical
tasks are trialled face-to-face with groups of
four to ten students. Feedback from students and
observations of them completing the task may lead to
changes.
5.
Final review
Three people
review the resource and associated trial data to decide
whether:
- tasks are
too hard or too easy
- the task is
valid and reliable
- the order
of the questions should be altered
- wording and
formatting should be adjusted.
The resource
may then:
- have
diagnostic information, gathered from analysis of
the returned scripts, added
- be accepted
and published on the Banks after final changes
- undergo
further development and be retrialled in the future
- be rejected
as unsuitable.
Revising and
refreshing older ARB resources
Since the
first ARBs were developed there have been changes in the
focus for assessment, with emphasis placed on assessment
for learning as well as of learning. There
have also been advances in technology that allow for
more interesting resources.
Older
resources are now being revisited to:
- check their
relevance to current directions in teaching and
learning
- check that
the context relates to current students' experiences
- update
their format.
These
resources may:
- remain unchanged
- undergo minor
changes that do not require retrialling
- undergo major
changes that require retrialling
- be removed
from the banks.
ARB
Home |
New Zealand Council for Educational Research. All rights reserved 
