More on methodology – the marketing approach

26 October 2008

I wrote recently about the methodology of triple hermeneutics as described by Alvesson and Stöcklund and how it might be relevant to my work. The trail that led to this started with my director of studies’ suggestion that I look at the world of marketing in respect of how it deals with perceptions. This has now led to the writing of  Chisnall (2005). Sure enough in the chapter on “Basic Techniques” there is a discussion of the place of reliability and validity in qualitative and attitude research . I quite like this word ‘attitude’. It helps frame a question ‘What is the attitude‘ of 16-year olds to ICT capability and its assessment. Chisnall says

“The measurement of behavioural factors such as attitudes… has been attempted by a variety of techniques… the ones that are the most reliable and valid from a technical viewpoint generally being the most difficult… to apply” (p234).

Oh well!

Validity for Chisnall consists of content, concurrent and construct validity – so fairly conventional there. One would have expected face validity to be mentioned too, perhaps. He also cites a pamphlet (sic) by Bearden et al (1993) that describes some 124 scales for measuring such things in the field of marketing, consumer behaviour and social research.

Bearden, W, Netemeyer, R & Mobley, M (1993), Handbook of marketing scales: Multi-item measures for marketing and consumer behaviour research. Newbury Park, CA: Sage (in conjunction with the ACR).

Chisnall, P (2005), Marketing research (7th ed). NY: McGraw Hill.

Theories of motivation and assessment of ICT – some questions

2 October 2008

One of the books that I have cherished in this process is Tombari and Borich’s on Authentic Assessment in the Classroom (1999). I have returned to it to analyse their categorisation of theories of motivation and used this to generate some questions that may be useful in data collection.

Attribution theory

How do students perceive success in ICT tasks and what do they attribute that success to?

Self-efficacy theory

How do students see their ICT capability and where does that view comes from?

Goal theory

What assumptions, if any, does ICT assessment make of students goal-theory characteristics and how do students themselves view their capacity to succeed?

General Teaching Council report calls for no school tests for under-16s

11 June 2007

Call to ban all school tests for under-16s | UK News | The Observer (10 June 2007)

“All national exams should be abolished for children under 16 because the stress caused by over-testing is poisoning attitudes towards education, according to an influential teaching body.

In a remarkable attack on the government’s policy of rolling national testing of children from the age of seven, the General Teaching Council is calling for a ‘fundamental and urgent review of the testing regime’. In a report it says exams are failing to improve standards, leaving pupils demotivated and stressed and encouraging bored teenagers to drop out of school.”

Demotivation, stress and, crucially for my work , poisoned attitudes. Will year 11s thoughts on the validity of assessment at 16 be coloured by their experience of testing (and other assessment) pre-16. Fairly inevitable I should think…

EPPI review (2005) Motivation and assessment

27 January 2007

Smith C, Dakers J, Dow W, Head G, Sutherland M, Irwin R (2005) A systematic review of what pupils, aged 11–16, believe impacts on their motivation to learn in the classroom. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

This EPPI review, cited by Gilbert, is focusing on motivation of 11-16 year olds. Its main findings identify six themes in the key to motivation. Each theme may have some relevance here. Italics represent direct quotes from the summary of the review.

  • The role of self : how is the learner’s own constructs represented in their view of learning? How does the role of the ‘group’ affect this?
  • Utility: Students are more motivated by activities they perceive to be useful or relevant.
  • Pedagogical issues: Pupils prefer activities that are fun, collaborative, informal and active.
  • Influence of peers: Linked to role of self
  • Learning. Pupils believe that effort is important and can make a difference; they are influenced by the expectations of teachers and the wider community.
  • Curriculum. A curriculum can isolate pupils from their peers and from the subject matter. Some pupils believe it is restricted in what it recognises as achievement; assessment influences how pupils see themselves as learners and social beings. The way that the curriculum is mediated can send messages that it is not accessible at all.

In this last point, the role of assessment is raised. So what does the review have to say about assessment in general?

The way that assessment of the curriculum is constructed and practised in school appears to influence how pupils see themselves as learners and social beings. (Summary, page 4)

… assessment [has a role] in nurturing or negatively influencing motivation (page 6 and page 63)

…the recent systematic review of the impact of summative assessment and tests on student’s motivation for learning acknowledges that ‘motivation is a complex concept’ that ‘embraces… self efficacy, self regulation, interest, locus of control, self esteem, goal orientation and learning disposition’ (Harlen and Deakin Crick, 2002:1) (page 8 of the EPPI review)

Students’ motivation is influenced by their ‘affective assessment’ (Rychlak, 1988) of events, premises and actions which are perceived as meaningful to their existence. (page 35, and linked to ‘logical learning theory’ (uncited))

Student satisfaction with their ‘academic performance tended to be influenced both by grouping, curricular and assessment practices and by its relationship to perceived vocational opportunities’ (Hufton et al., 2002:282). (page 45)

…learning situations that were authentic – in other words, appeared real and relevant to the pupils – could positively influence pupil motivation… ‘Sharing the assessment process with students is another way to capture students’ motivation…When students and teachers analyse pieces of writing together in an exchange of views, students can retain a sense of individual authority as authors and teachers convey standards of writing in an authentic context’ (Potter et al. 2001:53) (page 47 of EPPI)

Harlen W, Deakin Crick R (2002) A systematic review of the impact of summative assessment and tests on students motivation for learning. Version 1.1. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Hufton NR, Elliott JG, Illushin L (2002) Educational motivation and engagement: qualitative accounts from three countries. British Educational Research Journal 28: 265–289.

Potter EF, McCormick CB, Busching BA (2001) Academic and life goals: insights from adolescent writers. High School Journal 85: 45–55.