Rhizomatic Education

In this article Cormier discusses the ways in which the curriculum may need to change to reflect the new ways in which knowledge is constructed – through networks and communities of learners, as opposed to the traditional model of content transmitted by teachers. There is one reference to assessment where Cormier looks at how the traditional assessment is against that content.

To quote the article

“Information is the foundation of knowledge. The information in any given field consists of facts and figures, such as may be found in the technical reference manuals of learning; in a nonrhizomatic model, individual experts translate information into knowledge through the application of checks and balances involving peer review and rigorous assessment against a preexisting body of knowledge. The peers and experts are themselves vetted through a similar sanctioning process that is the purview, largely, of degree-granting institutions. This process carries the prestige of a thousand-year history, and the canon of what has traditionally been considered knowledge is grounded in this historicity as a self-referential set of comparative valuations that ensure the growth of knowledge by incremental, verified, and institutionally authorized steps. In this model, the experts are the arbiters of the canon. The expert translation of data into verified knowledge is the central process guiding traditional curriculum development.”

The other side of the coin is not discussed but I suspect that in the new ‘rhizomatic’ (and I’m no biologist so the metaphor is lost on me) model has peer and expert assessment. It’s the old thing about how do we know something is good – often because it is valued by those who need to use it (predictive validity as per the discussion at Cambridge – value <=> validity).

Linked from ERN (George Siemens)

Cormier, D (2008) Rhizomatic Education: Community as Curriculum in Innovate, 4, 5, July 2008 [online] available at http://innovateonline.info/index.php?view=article&id=550&action=article accessed 31/10/08

Digital Disconnect

This from Ray Tolley on the Naace mailing list http://www.eschoolnews.com/news/top-news/index.cfm?i=55665

The ‘digital disconnect’ is alive and well… “kids tell us they power down to come to school.”

More on methodology – the marketing approach

I wrote recently about the methodology of triple hermeneutics as described by Alvesson and Stöcklund and how it might be relevant to my work. The trail that led to this started with my director of studies’ suggestion that I look at the world of marketing in respect of how it deals with perceptions. This has now led to the writing of  Chisnall (2005). Sure enough in the chapter on “Basic Techniques” there is a discussion of the place of reliability and validity in qualitative and attitude research . I quite like this word ‘attitude’. It helps frame a question ‘What is the attitude‘ of 16-year olds to ICT capability and its assessment. Chisnall says

“The measurement of behavioural factors such as attitudes… has been attempted by a variety of techniques… the ones that are the most reliable and valid from a technical viewpoint generally being the most difficult… to apply” (p234).

Oh well!

Validity for Chisnall consists of content, concurrent and construct validity – so fairly conventional there. One would have expected face validity to be mentioned too, perhaps. He also cites a pamphlet (sic) by Bearden et al (1993) that describes some 124 scales for measuring such things in the field of marketing, consumer behaviour and social research.

Bearden, W, Netemeyer, R & Mobley, M (1993), Handbook of marketing scales: Multi-item measures for marketing and consumer behaviour research. Newbury Park, CA: Sage (in conjunction with the ACR).

Chisnall, P (2005), Marketing research (7th ed). NY: McGraw Hill.

Sir Mike Tomlinson lecture

Sir Mike Tomlinson, chair of the working group on 14-19 reform that led to the 2003 Tomlinson Report, came to NTU today to give the RSA Shipley Lecture. This year the lecture was also a memorial to former NTU and RSA stalwart Anne Bloomfield. The subject, dear to her heart and to Sir Mike’s, was “Vocational education should be a rounded education“.

With the backdrop of the history of attempted introductions of vocational education (the 1944 Butler Education Act with its tripartite system, TVEI, GNVQs, Curriculum 2000 and Diplomas), Tomlinson argued for the move away from debates about ‘parity of esteem’ towards a view of the ‘credibility and value’ of qualifications. Echoes here of the value and validity arguments of Monday’s seminar at Cambridge.

It was also notable that the lecture included issues of how ‘true’ vocational education must have

• relevance to 16-year olds (face validity),
• a knowledge base that is used in, and applied to, occupational areas – however broadly drawn (validty determined by use, not by the test itself)
• a theoretical content balanced with sector-based skills (content validity)

Again this echoes with Monday. Another thread running through was the role of assessment (systems) in undermining the vocational educational initiatives – TVEI assessment becoming ‘traditional’, GNVQ assessment being changed to equate to GCSE/A level, key skills being decoupled from GNVQs, Curriculum 2000’s insistence on equal numbers of units per qualification with a convergence of assessment types.

Also mentioned, although not in the same sense of ‘undermining’ was the persistence of the BTEC model and the way that NVQs were never envisaged to be other than accreditation mechanisms for on the job training.

The BTEC model of occupational credibility and general education was the model that was paramount in the description of vocational education with the caveat ‘what is general education’?

Throughout I was wondering where ICT fits into all this. Never mentioned as a ’subject’ nor even as a ’skill’ it was conspicuous by its absence. It is, of course, present in the specialised diplomas and as a functional skill although the former may be bedevilled by the wide diversity of the sector it is serving, I fear.

Tomlinson was upbeat about the Diplomas but focused especially on the need to get a true progression from level 1 through to 3. The custom of level 1 being what you get if you fail level 2 (GCSE grades D-G rather than A*-C) must not be repeated he urged. Also the need to get level 2 threshold systems so that learners who do not reach that threshold at , I would say the magical (and arbitrary, age of 16 could do so by subsequent credit points accumulation – rather than ‘repeating GCSEs’, a model that doesn’t serve well, he argued.

Another hour of useful insights.

Cambridge Assessment seminar

I attended  a seminar, on the subject of validity, one of a series of events run by Cambridge Assessment (CA). It was led by Andrew Watts from CA.

This was extremely informative and useful, challenging my notions of assessment. As the basis for his theoretical standpoint Andrew used these  texts 

• Brennan, R (2004), Educational Measurement (4th edition). Westport, CT: Greenwood
• Downing, S (2006) Twelve Steps for Effective Test Development in Downing, S and Haldyna, T (2006) Handbook of TEst Development. NY: Routledge
• Gronlund, N (2005), Assessment of Student Achievement (8th edition). NY: Allyn and Bacon [NB 9th edition (2008) now available by Gronlund and Waugh]

He also referred to articles published in CA’s Research Matters and used some of the IELTS materials as examplars. 

The main premise, after Gronlund, is that there is no such thing as a valid test/assessment per se. The validity is driven by the purposes of the test. Thus a test that may well be valid in one context may not be in another. The validity, he argued, is driven by the uses to which the assessment is put. In this respect, he gave an analagy with money. Money only has value when it is put to some use. The ntoes themselves are fairly worthless (except in the esoteric world of the numismatist). Assessments, analogously, have no validity until they are put to use.

Thus a test of English for entrance to a UK university (IELTS) is valid if, the UK university system validates it. Here then is the concept of consequential validity.  It is also only valid if it fits the context of those taking it. Here is the concept of face validity – the assessment must be ‘appealing’ to those taking it.

Despite these different facets of validity (and others were covered – predictive validity, concurrent validity, construct validity, content validity), Gronlund argues that validity is a unitary concept. This echoes Cronbach and Messick as discussed earlier. There is no validity without all of these facets I suppose would be one way of looking at this.

Gronlund also argues that validity cannot itself be determined – it can only be inferred. In particular, inferred from statements that are made about, and uses that are made of, the assessment.

The full list of chacteristics that were cited from Gronlund are that validity

• is inferred from available evidence and not measured itself
• depends on many different types of evidence
• is expressed by degree (high, moderate, low)
• is specific to a particular use
• refers to the inferences drawn, not the instrument
• is a unitary concept
• is concerned with the consequences of using an assessment

Some issues arising for me here are that the purposes of ICT assessment at 16 are sometimes, perhaps, far from clear. Is it to certificate someone’s capability in ICT so that they may do a particular type of job, or have a level of skills for employment generally, or have an underpinning for further study or have general life skills, or something else, or all of these? Is ’success’ in assessment of ICT at 16 a necessary pre requisite for A level study? For entrance to college? For employment? 

In particular I think the issue that hit me hardest was – is there face validity: do the students perceive it as a valid assessment (whatever ‘it’ is).

One final point – reliability was considered to be an aspect of validity (scoring validity in the ESOL framework of CA).

KS3 SATS scrapped in England

This somewhat unexpected announcement was made this week. Tests for 14 year olds in maths, English and science have been scrapped. Given that many schools start their GCSE/level 2 courses at 13 now, especially in ICT, this might change radically the ways in which the middle years of secondary are organised. It may also affect students’ perceptions of assessment as they will not have had those high stakes external tests at 14.

Tutorial part 2

I had a tutorial (by telephone) today with the other part of my supervisory team. An interesting model emerges that develops the earlier one:

What emerged was a clarity of vision: I am looking at

A how year 11 students perceive ICT capability and
B how the assessment system (at 16) perceives it.

My project is to define the difference between A and B and to suggest ways in which the two may be aligned.

What now emerges is the more sophisticated notion of a number of views of what ICT capability is, with some sort of Heideggeran absolute at the intersection. Thus there may be four views of what ICT capability is:

• the view of the awarding bodies
• the view of the students
• the view of the education system (policy)
• the observed view from research

Is there also then a Heideggeran absolute, autonomous view somewhere in the intersection of all these?

We also talked about the notions of perception and interpretation of the students view and came down to the question: How authentic and relevant does assessment feel to students? This, of course, has limitations as due to precisely because of the hermeneutical considerations of it the students’ view.

Building on the notion of the abstract view that would define assessment of ICT in absolute terms (and my stance which rejects this in favour of the diversity of views listed above), we then talked about the importance of the social cultural view in which students’ interpretations are coloured by their class, peer groups, families etc.

One final concept is the emergence of literature on assessment as learning and how the ‘teaching to the tests’ means that students are spoon fed and do not learn beyond the framework of assessment.

Connectivism and serendipity

In looking around for thoughts on Husserl I came across the WordPress blog ‘Between Husserl and Heidegger‘ – a blog as an adjunct to a taught face-to-face course. On clicking on one of the tags (Husserl) I was surprised to see a link to a post in another blog about connectivism. This is the theory of learning espoused by two of the leading lights in the technology and learning arena – George Siemens and Stephen Downes.

The surprise was not that this should turn up in a search (although the link to Husserl is pretty tenuous through a quoted marginalia). Rather it is the subject of an online course that one of my colleagues is attending and blogging about at this very time. Is that serendipity, coincidence or reticular activation?

Is that a milestone?

So another day’s study leave – another 3000 words or so committed to ‘paper’… at least when it is printed out it makes a thud on the desk!

This month I have restarted the PhD, many things have happened to enable this… a colleague reported that she will finish during this year… a research cluster meeting has signed me up to present a paper at  seminar (maybe I need to sign up for the one at ITTE in Cambridge too!)… I have booked the aforementioned study leave and set some targets (pretty basic ones like ‘write something proper’ but I now have 17000 proper words – although I am under no illusion: about half of them will probably go before I’m done)… I have restarted tutorials… I have built this PhD into my performance development review targets… I’m doing a different job, which seems to suit better… I am supervising three other candidates and see the process from the other side… all good… for now!

Putting the Ph into the PhD?

I have been struggling this week with the concept of ‘perception‘. After a tutorial my focus was on how I might apporach the capture and analysis of students’ ‘perceptions‘ about assessment. This word has been quite fundamental to my description of the research. In the tutorial, we got talking about marketing theories and perceptual analysis as a method in that discipline.

Needless to say I know little about marketing. So what is the perceptual analysis? It has proved an elusive hunt but I have travelled over some interesting territory. One laden with ontological considerations and debate.

First there is the hermeneutics of Husserl and Heidegger. For one the importance of the existence of the objects  of consciousness ONLY in the way in which they are perceived by the consciousness, for the other the autonomy of such objects irrespective of the sense we bestow on them. This perception is then reported linguistically and Wittgenstein’s concept of the language game filters any such sense.

Then there is the triple hermeneutics of Alvesson and Sköldberg (2000). Hermeneutics – the analysis of interpration, double hermeneutics – the analysis of interpreted interpretation (the dual lenses of researcher and respondent), triple hermeneutics – the analysis of interpreted interpretation and the context behind that interpretation (the three lenses of researcher, respondent and context).

Lowe et al (2005) propose a 4th hermeneutic in the context of marketing (and that was how I came in) but I am not sure yet how this applies!

Finally, and most pragmatically (*), Conroy (2003) examines interpretive phenomenology (or rather re examines it) and develops a  methodology and methods for doing something fairly similar to what I am proposing, albeit in the context of nursing (the usual context for this approach, it seems). Here is a model that I need to examine and critique for its use in my study as I move towards the primary research phase.

And in this phrase – interpretive phenomenology – hides the word I have been meaning when I have said perception: it is interpretation. So not ‘what is a student’s perception of….’ but what is a student’s interpretation of…’ The problem is, you see, that perception has a particular meaning in this philosophical arena. I had to go down the false road of Arnold Berleant to realise it… Thanks are due here to my colleague Kev Flint…

I have a fair chunk of literature review written, I have many ideas about methodology and method. Now is the time to crystallize this and move on to ‘action’. My director of studies agrees and this ‘permission’ is what I have been waiting for…

(*) pragmatic in the the sense of being related to action… but actually very philosophical in nature

History of assessment

Blindingly obvious I guess, but nevertheless not a field I have mined much. “History of assessment” needto be a significant contextual filter for my research.

I am attending some of the seminars of the Cambridge Assessment Network as and when I can (Kathryn Ecclestone on Ofqual, Harry Torrance on Policy and Practice). There are others that I have been unable to attend but would like to have done. Fortunately useful overviews such as this one from Helen Patrick are often put online.

MUPPLE (Mashup and PLE)

This report (Wild, Kalz and Palmer, 2008) on the proceedings of the first conference on Mashups and Personal Learning Environments (MUPPLE) is interesting to me for the underlying concepts.

While it is necessarily focused on the technological developments that have allowed mash ups to become part of users’ armoury, there is an interesting parallel (explicit in the title but maybe less so in the content) with learning. As more tools become available learners are able to combine them (mash them up) for their own purposes. Personalisation of the curriculum is another parallel and these all have interesting consequences for assessment and for learners perception of capability. If someone appropriates tools for their own use are they, de facto, demonstrating high levels of ICT capability?

Becta report into Web 2.0

Becta have published a research report (Crook & Harrison, 2008) on use of Web 2.0 by learners, teachers, at home, in school etc. This statement in the summary caught my eye:

Pupils feel a sense of ownership and engagement when they publish their work online and this can encourage attention to detail and an overall improved quality of work. Some teachers reported using publication of work to encourage peer assessment.

Where is the use of these tools in extrenal assessment? Come to that, where is the use of peer assessment in external assessment.

Also noticeable is the emergence of yet another Rogers’ adoption curve – with the earley adopters being the young ones etc… is this true? DO teachers really not use Web 2.0 tools? How does that square with the quote above? It is borne out in the research of Solheim (2007,  which cites OfCOM’s 2006 statistic of 40% of ‘adults’ having used social networking sites*, compared to 70% of 16-24 year olds and Comscore’s 2007 finding that over half (1.3 million) of Facebook’s new users in the previous year were 25 or older.

* OfCOM’2 2008 report tates that ‘only’ 21% have ’set up a profile’ on such sites.

Comscore (2007), Facebook sees flood of new traffic [online] available at http://www.comscore.com/press/release.asp?press=1519 accessed 06/10/08

Crook, C and Harrison, C (2008)  Web 2.0 Technologies for Learning at Key Stages 3 and 4: Summary Report, Becta [online] available at http://schools.becta.org.uk/upload-dir/downloads/page_documents/research/web2_ks34_summary.pdf accessed 06/10/08

Ofcom (2006), The communications market. [online] available at http://www.ofcom.org.uk/research/cm/cm06/ accessed 06/10/08

Ofcom (2008),The communications market report, [online] available at http://www.ofcom.org.uk/research/cm/cmr08/ accessed 06/10/08

Solheim, H (2007) Digital Natives versus Higher Education: who is ready for whom, MSc dissertation, University of Southampton.

Theories of motivation and assessment of ICT – some questions

One of the books that I have cherished in this process is Tombari and Borich’s on Authentic Assessment in the Classroom (1999). I have returned to it to analyse their categorisation of theories of motivation and used this to generate some questions that may be useful in data collection.

Attribution theory	How do students perceive success in ICT tasks and what do they attribute that success to?
Self-efficacy theory	How do students see their ICT capability and where does that view comes from?
Goal theory	What assumptions, if any, does ICT assessment make of students goal-theory characteristics and how do students themselves view their capacity to succeed?

Taxonomy of difficulties in the assessment of ICT

This paper, from the Assessment in Education Unit at Leeds, is again bang on the line of what I am doing. Albeit that they are looking at the KS3 onscreen ICT tests (the AEU were commissioned as part of the evaluation of that project). Nevertheless there are some very pertinent analyses of the didfficulties students, and the system, encounter in assessment of ICT. For example

… sources of difficulty that relate to the subject being assessed. The assessment of ICT brings particular risks. As McFarlane (2001) points out, assessment of ICT capability need not in itself be computer-based, but as in this case it was, the sources of difficulty in our set that are associated with this aspect all relate also to on-screen assessment, e.g.

• Pupils know enough to succeed in the tasks without using ICT for all the steps.

• The demands in the interaction between tasks and software on short-term memory and organisational skills are greater than the level of ICT capability that is being assessed.

Activity theory and ICT

Now that the new academic year is under way, and I am in a new job (again) I hope to be able to crack on with this project. If not.. well I need to decide one way or the other.

Anyway, I am in the middle of three days study leave and ma busily writing up what I have so far. In doing so I have also come across some useful things which I will include here.

Firstly a paper from the School of Education conference at Leeds University by Aisha Walker. This makes some interesting links between activity theory and attainment in ICT and provides a model that may be useful when I come to look at the data colleection and analysis. 

The title “What does it mean to be good at ICT” really caught my eye. That’s what it’s all about isn’t it?