10 January 2007
Writing in The Observer, John Naughton caricatures another vignette….
There’s a surreal quality to it, conjuring up images of kids trudging into ICT classes and being taught how to use a mouse and click on hyperlinks; receiving instructions in the creation of documents using Microsoft Word and of spreadsheets using Excel; being taught how to create a toy database using Access and a cod PowerPoint presentation; and generally being bored out of their minds.
Then the kids go home and log on to Bebo or MySpace to update their profiles, run half a dozen simultaneous instant messaging conversations, use Skype to make free phone calls, rip music from CDs they’ve borrowed from friends, twiddle their thumbs to send incomprehensible text messages, view silly videos on YouTube and use BitTorrent to download episodes of Lost. When you ask them what they did at school, they grimace and say: ‘We made a PowerPoint presentation, dad. Yuck!’
10 January 2007
The “top left” quadrant of Wiliam’s enhancement of Messick’s four-facet model of validity deals with within-domain evidential/interpretive validity. How is the assessment designed so as to provide constructs that evidence that which is to be assessed within the domain. He cites Embretson (1983) (1) as providing part of the conceptual model for this quadrant.
Embretson’s model distinguishes between construct representation and nomothetic span. In the former, assessment designed so that it is situated in tasks that represent that which is to be assessed. In the latter it is designed to correlate with other tasks deemed valid.
Mislevy et al (2002) (2) discuss the model in the context of the “psychometric principles” of validity, reliability and comparability. They relate the task model to three other models – the student’s learning, the assessment (or measurement) of this learning and the scoring. Their argument appears to be that the construct representation resonates more with the psychometric principles than does nomothetic span, but that both may be needed.
In the context of my research it would seem that I am doing some sort of comparison between the two parts of Embretson’s dichotomy. Construct representation – using what the students have learnt by way of ICT capability to provide an assessment. Nomothetic span – using some assessment that correlates to this as measured by other assessments.
Is use of the former inherently more engaging than the latter? Does it fit with student’s own constructs of what they have learnt?
(1) Embretson, S. E. (1983), Construct validity: Construct representation versus nomothetic span in Psychological Bulletin, 93, 179-197.
(2) Mislevy R, Wilson M, Ercikan K, Chudowsky N (2002), Psychometric Principles in Student Assessment CSE Technical Report 583, Centre for Studies in Evaluation, LA also available online at http://www.cse.ucla.edu/reports/TR583.pdf