“So what’s your PhD about?”

30 December 2006

One of my supervisors, Karen, made the very helpful suggestion that I need to be able to sum up my PhD research to someone I might meet in a ‘lay’ setting – down the pub or on the bus if you like. This situation raised itself over Christmas during family gatherings. “So what’s you PhD about, Pete?”, I was asked. This led to some conversation about ICT and assessment and qualifications. Some from the point of view of professionals, some from parents (of school students), some from ‘lay’ observers of education.

It led onto a discussion at about 3 am as stuff poured from my head. I grabbed a piece of paper and jotted it down. A sort of concept map if you like.

What should students learn? What should be on the curriculum? What is 'should' anyway?

What engages students, learners? Can it be generalised?

What about validity of assessment per se? If teachers teach to tests and learners learn to pass them, does it invalidate the assessment? Is the learning of skills and facts held in to high regard (learning those facts and skills needed to pass)?

Are the previous questions concerns for all subjects not just ICT? What is the specialness about ICT?

How does ICT relate to other subjects? Is the D and T needs/solution/evaluation design cycle a key feature of all learning? Is problem solving at the heart of sustained learning?

What is the relationship of 'assessment objectives' to assessments? Do awarding bodies really reflect the former in the latter?

Whither creativity?

Just a bunch of ideas from the middle of the night but worth noting down maybe…

“Young people taking control of the technology”

23 December 2006

This is a quote from Laurence Boulter’s “I Can Teach ICT” site (via Naace mailing lists). It is exactly the type of vignette that got me started on this research route. How can we have an assessment system that allows for this sort of thing? How do we change the way we approach the assessment of ICT? Do we need to? Or is it about ‘validation’ of what has been learnt? is it about providing the opportunity for students to share their learning in exactly the way Laurence has done?

“Fixed in my mind was a conversation I had with a year 8 pupil a year or so ago who stayed after class to show me her Pixo site. I had known for some while that some pupils were dabbling with such sites but I had not really acknowledged, or even taken the time out to see what they were actually doing. Whatever it was they were doing with these sites it had to be trivial surely? This young lady, perfectly polite, just above average ability, steadily working towards level 5, was quite unexpectedly explaining to me how she had prepared in Photoshop (not available in school) an image she had obtained from the internet. She then exported the image from Photoshop in a suitable file format then posted it to her Pixo site. When posting it she added html code that had been emailed to her by a friend that produced a glitter effect on the image (a sort of My Little Pony horse if I remember correctly). She then went on to show me the rest of her site. She explained how she modified the html template and how she had created different categories on her site and linked between them. This was not a level five dialogue I was having with this pupil and as she showed how her collection of virtual friends left comments and html tips on her site, her tangible friends that had stayed behind with her made grabs at the mouse saying “can I show him mine?”. It became clear to me that this was not just the activity of a geeky isolate, this was a representative of a community of young people taking control of the technology. “How much time do you spend on this?” I asked. “Every night!” she answered. “My Dad is always complaining that he can’t get to use the computer. He keeps asking me what I am doing. I try to explain but he doesn’t understand””

Assessment methodologies at 16

22 December 2006

There has been much in the news about changing methodologies for assessment at 16. Most of this has been about the unsuitability of coursework as a valid tool. The solution, to reduce the amount of assessment outside of ‘exams’, seems to be a bit of a knee jerk. But then education is always at the forefront of politicians’ ‘need to do somthing’.

One thing that is also changing though is the introduction of e-portfolios at 16. The DIDA qualification requires this and maybe this helps to deepen the understanding of the application of ICT. Or maybe the e-portfolio becomes a barrier? Certainly it would help assess students’ ability to use an e-portfolio – but we need to be careful that this is the central thing being assessed. How might it be applied to the creative use of ICT demonstrated by students at BAFTA for example?

Validity and validation

20 December 2006

Is an assessment valid? What does this mean? Gipps and Murphy (1994) discuss this semantic issue. They relate validity with bias (or lack of it). If a test, or assessment, is valid it is free from bias (although the opposite is does not necessary follow). They cite Cronbach and Messick’s notions of a unitary model of validity based on the construct. How is the assessment constructed? Does it measure what it intends to measure, and is it free from bias. This construct validity is regarded as one of the three dimensions – the others being content and criterion. They argue that no content or criteria can ever be free from bias, and hence these are less dominant aspects when looking for validity.

On the other hand validity, or validation at least, has a very different meaning. It is used to mean the process of recognising (as valid) that which has been learnt in non-formal settings. See, for example, the ECOTEC project. In higher education this might be equated to the process of Accreditation of Prior Experiential Learning (APEL). In APEL, non-certificated learning is validated against assessment criteria that have been designed to assess formal learning. A judgement (assessment) is made to see if the learning claimed as APEL does equate to that which might be learnt formally. It is used to exempt learners from parts of programmes.

Over arching these two processes, ensuring validity and validation of non-formal learning, is a more thorny concept. That of peer or community validation of skills, knowledge and understanding. Someone regarded as a expert in a field by his or her peers probably has had a more valid assessment of their capability than someone who simply has the piece of paper. Even if that piece of paper has been awarded through scrutiny and validation of some APEL portflio.

Recognition of non-formal learning: EFVET

19 December 2006

The European Forum for Vocational Education and Training have a report that lists “Six key messages for lifelong learning”. The report appears to be date 2006 but I only divine that from the URL..

Key Message 4 is “VALUING LEARNING”. In here EFVET talk of valuing non-formal learning. There is talk of common recognition of ‘portfolios’. I agree with the sentiment but wonder about the effect of imposing standardised portfolios. Does the nature of informal learning make this difficult. Does the nature of ICT skills also work against it? You see a fully-functional spreadsheet – how do you know what skills the author actually possesses?

NB Although ICT is not mentioned explicitly in this section, it is a specified ‘skill’ elsewhere int he report. That it refers to ‘skills’ is another problem when thinking about education (as opposed to training). Are mere ‘skills’ enough?

Formal, informal and non-formal learning

18 December 2006

I have been musing around the nature of formality in learning and reading some interesting angles eg from Stephen Downes and Graham Attwell. Their arguments, respectively that informal learning is not formless and that informal and formal learning are equally valid, make good sense to me. In the context of my study what a students learn at school about ICT and what they outside of ICT lessons both contribute to their learning. Whether the formal can ever keep up with the informal though is a matter of conjecture. Formal means tested, assessed according to some external criteria (at least it tends to include those things, if not mean them exactly). These take ages to develop and standardise. They cannot keep up. Where is the GCSE that looks at use of wikis?

Anyway… I would like to add that there is a third way. Non-formal. That which is learnt in school but not in formal lessons. And then there is the argument… is that different to the informal learning out of school? I believe it is. In ICT I would categorise ot as the learning to use ICT to support learning (eg in learning English one might use ICT) but which is not assessed. It is incidental to the formal learning. It is not part of the learning objectives framework for the lessons, subjects, students. This comes from reading Michael Eraut from way back in 1994. But it is still relevant today. This is shown by the OECD’s website on the terms. Which is non-formal and which is informal is an interesting semantic sideline. It is interesting to note the OECD placing certification (ie assessment) at the heart of this divide.

Update (21/12) – literature review… and again on 14/01/07… see this blog post.

Towards a PhD, starting the long journey

3 December 2006

So… after several false starts I have just this week made a presentation of my PhD proposal, and am now truly on the road. I’ll use this space to explore ideas….


Assessment of ICT at 16: its validity and relationship to students’ formal and informal learning.


1. To critically analyse the ways in which students aged 16 construct their learning of ICT capability in formal and informal contexts.
2. To explore the relationship between formal and informal learning within the field of ICT.
3. To explore the methodologies of assessment of ICT capability at 16 and how this affects student perceptions of their capability.
4. To develop a theoretical base to evaluate the construct validity of assessment of ICT at 16.