Kirkpatrick’s model of evaluation has been around for many
years. I have answered many evaluations
about whether I enjoyed the training, what I had learnt, and if I would put
into practice what had been learnt. These questions address the first three
levels on Kirkpatrick’s model. Dick (2007, p.150) says that even Kirkpatrick
acknowledges the difficulty of measuring the fourth level of changes in the
organisation due to the training. Even the behaviour level can be difficult to
measure in the organisation. Despite the most comprehensive learning and best
intentions, there can be other factors which influence whether the learning can
even be implemented.
In the case of my project, I have designed some learning for
using a program called WebDewey. In order to use this, students must also understand
the process of number-building in the Dewey Decimal System. Sadly, if a student
doesn’t go directly from TAFE to a workplace which uses this program, evaluating
for level 4 is unnecessary. Currently some libraries still use Dewey in print,
but eventually they will all move online, so it is not unnecessary training.
My introduction to learning Webdewey is at http://librarydiploma.weebly.com/
if anyone would like to see how they go. It is for diploma level students whom
we assume have a basic understanding. I would still appreciate feedback from
people who may have no experience. You might even learn a bit about Dewey! I
encourage you to have a go at the quizzes. I have also created a video to demonstrate how to use the website. This package is only an introduction. I realise there will need to be many more exercises and explanations.
One response to the quizzes was that the student did not have
the schedules to answer some of the questions. I have since included the
relevant information. This was a bad oversight, since accessing the schedules is what
WebDewey is mostly about, and it had not been introduced at that stage.
Otherwise the formative evaluation for levels 1 and 2 of Kirkpatrick’s model
indicates this foray into online teaching is heading in the right direction
with regard to reaction and learning. I’m confident that if the students have
learnt successfully and their workplace endorses it, they will use what they
have learned (level 3).
Overall:
Level 1: Reaction - Students enjoyed the learning experience.
Level 2: Most people said they learned something, even when they did badly in the quizzes.
Level 3: the students will be applying what they have learnt in further lessons
Level 4: This will depend if the students get work in an appropriate workplace.
Overall:
Level 1: Reaction - Students enjoyed the learning experience.
Level 2: Most people said they learned something, even when they did badly in the quizzes.
Level 3: the students will be applying what they have learnt in further lessons
Level 4: This will depend if the students get work in an appropriate workplace.
References
Dick, W. (2012). Evaluation
in instructional design: The impact of Kirkpatrick’s four-level model. In R.
Reiser & J. Dempsey (Eds.), Trends
and issues in instructional design and technology. 3rd
edn. Merril, New Jersey
Further reading
Chapman, A.
(2012) Evaluation in Instructional Design
- Kirkpatrick's 4 Level Model . Businessballs.com (online accessed 18 Aug 2012)
Oh dear, Narelle I will never get a job as a librarian after taking a few quizzes! I do have to say that I like your resource as it is very explicit and easy to use. Even though I was very tired when I had a go at the quizzes and really did not read all the information I still felt motivated to continue which is saying something for me. I have to evaluate my resource using Kirkpatrik's model and adapt it somewhat as it is for primary children and not adults. I can still use similar assessment i.e. formative and summative assessment. Will try and get my evaluation up by Wednesday evening.
ReplyDeleteThanks Kate - your comments are really encouraging. Like I said in my blog, there is a lot of assumed knowledge, so don't feel bad - maybe I should do something for Cert 3! Kirkpatrick's model is a lot about the workplace, so I had to adapt it too - we mostly assess in a classroom, so just have to anticipate that the skills will be applied in the workplace, and what effect they may have.
ReplyDeleteWow Narelle, your resource is fantastic. You have obviously put in a lot of work, you are very inspiring!!!
ReplyDelete