Net Gen Nonsense

I came across this blog via George Siemens’ elearnspace ERN newsletter. It provides a balance (if not a balanced) to counter the oft made assertions about the generation of younger learners who are completely net-savvy, digital natives etc. I also came across similar balancing statements in a dissertation I have just examined… I’ll need to get the references!

Taking the ideas from the previous post and putting them into a diagram I get this

Some assessment uses ICT (or technology) - this is e-assessment (x axis).

Some assessment is designed to assess ICT capability (y axis).

Elliott’s Assessment 2.0 seems to be using ICT, not as e-assessment, but as a medium for allowing judgement to be made about the ICT capability (z axis).

Now of course, analysing any one particular assessment methodology one could locate it in this three-dimensional space. for example:

A traditional written paper would be on the y-axis. The NAA online assessment activities designed for KS3 would be in the space between all three axes (with perhaps a lower y- and z-values than x-value. Coursework would have an x-value of 0 but would have some components of y and z. Online assessments such as the driving test would be on the x-axis.

My questions here are “Where is the highest validity”? and “Where is the highest reliability?”. How does one use Elliott’s Assessment 2.0 to determine success in a certificated qualification?

Much as I dislike the nomenclature (Assessment 2.0), I found this paper by Bobby Elliott (and thanks to my colleague Bruce Nightingale and the ALT newsletter for bring the name to my attention) illuminating on many levels. Firstly here was someone making the links between theways in which technology is reportedly used by young people and the ways it could be used for technology. Secondly the author works for a government agency - the Scottish Qualifications Authority (SQA). Is this evidence of policymakers thoughts are changing to embrace the vicarious ways in which evidence of learning can be presented by technological opportunity?

My thoughts return though to the Macfarlane distinction between assessment of technology (eg the ICT curriculum) and assessment through technology (ie the methodology). This paper by Elliott seems to be moving a little away from the latter and perhaps towards the former. But perhaps, also, it is defining a third axis - assessment of technological capability through evidence presented through that technology. Maybe it is asking the question ‘What should be assessing?’ (ie the curriculum) rather than ‘How should we assess it’ (the methodology). But more than that it is saying can we assess the ‘what’ through the ‘how’.

The impressive list of tools that may be used for evidence presenting (and assessment) in Elliott’s paper also underlines my sceptcism of a one size fits all technological solution to assessment. And when I look down that list I am reminded of surveys presented by Terry Freedman (at a TDA conference in Nov 07) and others that show that young people’s use of tools is very diverse and very thinly spread. It is also very transient - MySpace here today gone tomorrow.

The very tool that Elliott uses to present his iPaper may well be a case in point. What if an awarding body decided Scribd was the thing to use. How long before it becomes the sliced bread superceded by the next best thing? How do we build in agility for assessment so that it does not become an exercise in rewarding the fashionable? (as opposed to the current system which rewards the old-fashionable).

PS Yes it’s been a long time… Higher Education management - ie my temporary role for 07/08 - and PhDs don’t easily mix… but I know that is also my excuse… and I’m sticking to it…

This week’s eLearning Resources and News from George Siemens draws attention to two recent reports.

Firstly from Deloitte a report on ‘media democracy in the US‘. (PDF)

71% of 13-24s read content created by others, with 56% claiming to be creators of online content.

Secondly, and more enlightening, a report from the US National School Boards’  Association (CREATING & CONNECTING//Research and Guidelines on Online Social — and Educational — Networking ) looks at the use of online tools by teens. It has a section on the educational uses of social netowrking - 59% report they use such channels for learning purposes.

It identifies 31% of teens as being non-conformists. Those who break rules (or maybe conventions) on access and safety. These non-conformists lead the way in creation, sharing and networking online.

Also interesting were the stats that show almost universal expectations across board districts that students should use online sources for learning, while at the same banning networking and communication tools in school. And also an very low percentage of teens reporting having met someone face-to-face having first met them through the Internet (but maybe they just aren’t saying). Still a very interesting picture of the educational dimensions of social networking.

Reading the Futurelab report Beyond the Digital Divide: Rethinking digital inclusion for the 21st century (Selwyn and Facer, 2007) (1) I am struck by their exposition of a number of digital divides - beynd the ‘traditional’ one of access. In particular, there is the divide caused by a misalignment of the culture of the context in which ICT is being used.

This leads me to address the question: How can we assess the ICT capability of a person without assessing the mediating effect of the context and culture in which they are operating? In particular the context of the school and the education system.

To quote the report (p16) “If the wider cultural context of use (such as the workplace,  school or home) does not fi t well with the culture of the ICT application, then  use will not easily follow. As such ICT use is not just based on the individual  being able to ‘understand’ the potential benefi ts of ICT use, but how well  ICT-based activity ‘fi ts’ with the wider contexts within which they are  operating.”

(1) Selwyn, N and Facer, K (2007)  Beyond the Digital Divide: Rethinking digital inclusion for the 21st century, Bristol: Futurelab available online at http://www.futurelab.org.uk/resources/documents/opening_education/Digital_Divide.pdf accessed 25/07/07

In a discussion in the Naace online community (1), Stephen Heppell comments on the ever-changing nature of tools and the permanence of ICT as a force for creativity. His comments resonate with the ‘practice’ side of my model in the previous post and how it can be counter to the theory/system side.

Heppell reflects on young people’s changing usage of tools - MySpace is so yesterday, Facebook today, something else tomorrow. Further, he points out that young people never e-mail now. My step daughter added a different lens to this - they do e-mail but only when communicating with adults or when they are passing on links, files etc.

Some selected quotes from Heppell’s comments are posted here (reproduced from the posting to the Naace mailing list by permission of Stephen Heppell, they should not be reproduced without citing this context)

“Most efforts that I can remember to establish standards in educational ICT have failed. And that is no loss. They hold everything back … The whole world of ICT is so organic and changes so rapidly - one minute MySpace is cool, the next it’s where your grandad goes. Just as adults get their blackberries finally emailing to each other, so children have stopped emailing altogether (”it’s what your dad does..”). And so on.”

And then, on the need for systems and policy to match students’ ambitions:

“Systems are never ambitious for children. Children are, so are their teachers, parents and others are too, but without a shared vision of just how good all this can be, it all founders into a generation of coasting kids delivering on dull targets. If you word search the “Higher Standards, Better Schools For All” white paper for example you will find the word “creativity” is entirely absent, as indeed is the word “ingenuity”.

“[I , and others,] constantly see, and are delighted by, just how ambitious children can be for their learning - especially where it is mixed age, project based, over a decent length of time, shared and not capped in any way. We need to lock that ambition into policy.”

On students a leaders of learning

“Last week I was in a school working with a group of young secondary children who were busy designing a CPD workshop to bring their teachers up to speed with the cool things they might do on Facebook, with why poking isn’t rude any more, with Bebo and myArtSpace and YouTube Comments and so on. They were very sanguine about what their teachers needed to know and were in turn interested as to the ideas their teachers might have about using these new places and spaces in learning. There is a rich irony in imagining that down the corridor their teachers might have been busy parsing a policy document to plan the ICT curriculum for those same children!”

On the need for assessment ot be relevant to this debate:

“I think if Hollywood can measure satisfaction as people leave the rough cut of a movie (and then fund the re-shoot of an ending as with Pride and Prejudice in the US version) then we can measure creative esteem, ingenuity, delight, satisfaction and so on. All or any would be more helpful measures than cohort aggregate exam passes.”

On his related projects:

“Have a look at www.learnometer.net or indeed at www.heppell.net/doctoral if you want to see where I imagine all this will be going. The learnometer project is already under way.”

(1) Naace is the association for the advancement of ICT in education. Its community has a private mailing list from which these comments are drawn (they should not be reproduced without citing this context)

I hadn’t come across this paper until today…

To quote from the executive summary…

“This paper is focused on exploring the inter-relationship between two key trends in the field of educational technologies. In the educational arena, we are increasingly witnessing a change in the view of what education is for, with a growing emphasis on the need to support young people not only to acquire knowledge and information, but to develop the resources and skills necessary to engage with social and technical change, and to continue learning throughout the rest of their lives. In the technological arena, we are witnessing the rapid proliferation of technologies which are less about ‘narrowcasting’ to individuals, than the creation of communities and resources in which individuals come together to learn, collaborate and build knowledge (social software). It is the intersection of these two trends which, we believe, offers significant potential for the development of new approaches to education. ”

These new approaches include those encapsulated in these quotes

“Today, the use of social software in education is still in its infancy and many actions will be required across policy, practice and developer communities before it becomes widespread and effective. From a policy perspective, we need to encourage the evolution of the National Curriculum to one which takes account of new relationships with knowledge, and we need to develop assessment practices which respond to new approaches to learning and new competencies we expect learners to develop. “

and

“A rigid curriculum inhibits the development of the knowledge and skills that may be useful in the 21st century. If we are to promote the benefits of problem solving and collaboration then they need to be validated and legitimated by the assessment system. This is the greatest challenge for education policy.“

There is some sort of mapping in my mind between the two trends identified in the first paragraph and the two aspects of my research.

The report has a change in the purpose of education and its interface with the change in technologies. I have the two views of ICT/assessment of ICT, (and maybe also views of the purpose of ICT in education). Putting these four onto a diagram I see that two are to do with system, theories etc and two are to do with learners, users, practice.

Derek Wenmoth has  recentlyposted two articles on his blog that have relevance to the area of my study.

Firstly, the reinvention of schooling in Knowsley - schools being replaced by learning centres with ‘rich tasks’ and enhanced possibilities of personalised learning. Does the very concept of school mitigate against the alignment of assessment with capability?

Secondly, the seemingly avant garde yet, in some way, strongly pragmatic map of future forces affecting education. Here the landscape of current trends and dilemmas is mapped out. Crucial for me are the aspects of schooling and the characteristics of “Generation Y” or “Millennials” (Howe and Strauss).

Following on from my tutorial at NTU I took the landscape to my ‘external advisor’, Peter Twining (of Schome fame). We spent an hour and a half in heated discussion. Heated to the extent that my brain fried but all very amicable! The outcomes were firstly a re-framing of my thoughts - and probably of my aims although that can wait for a while, and secondly a timeline for the project.

What emerged was a clarity of vision: I am looking at

A how year 11 students perceive ICT capability and
B how the assessment system (at 16) perceives it.

My project is to define the difference between A and B and to suggest ways in which the two may be aligned. This latter point, of a PhD thesis making recommendations, is one of the doctoral level learning outcomes that I hadn’t really paid attention to. Actually I hadn’t come across any of these outcomes before this month… I’ll post something about them if I can find an electronic copy or time to type them up!

I also came away with a timeline. The literature review that have embarked on will need to give way to a finding my way to a suitable methodology. This will require a change of focus of reading to look more at the methodology and methods I wish to adopt so that I may collect data in the coming academic year. Part of this discussion will be to look at the literature around ascertaining student’s perceptions and gathering the student voice.

I will also need to consider the impact of eliciting views from students in school situations as opposed to outside school. The choice of data collection instruments will also be subject to discussion - will interviews suffice, or will observation of their capability be necessary. It is likely that a piloting of a range of tools will be needed with a fuller data collection in 2008/09.

This data collection, together with the literature review, will yield information about A above. Further review of the literature, this time on policy, together with examination of assessment materials (exams, coursework assignments), will yield information on B and reveal the differences between them. This will then lead to the recommendation phase.

A rough timeline has been developed (click image to see it full size):

Whither my landscape in this simplified model? The landscape had four features - assessment, learning, policy and technology. These may be seen in the model, I believe:

• assessment is in A and B
• learning is in A
• policy is in B
• technology is in A and B

My previous post was with Anglia Polytechnic (now Anglia Ruskin) University. While there I served on the Education Faculty’s research degrees committee (RDC) and also attended the University RDC. One of the things that I was involved in was early steps to develop the use of personal development planning (PDP) tools.  It is interesting to revisit this two years later on their website “Planning your research training”

I was reminded of this by discussions at the PhD supervisors’ course at my current employer (Nottingham Trent University). There is a need for us to look at this aspect of PhD support and guidance we felt.

The APU (ARU) materials came were stimulated by papers from UKGrad. Their website contains a PDP database that lists many other case studies on the development of such support and training.