Assessing #digitalstudies – grades or badges

What is #digitalstudies?

If you are reading this post, the chances are, you already know what #digitalstudies is; however, for a refresher take a look at the introduction on the wiki to see how it differs from the more ‘traditional’ subjects of ICT and Computing.  Here, you will see that a fundamental difference is the way in which the subject is assessed using blogging portfolios or bportfolios.

What is a bportfolio?

Traditionally pupils, students and adults would keep an online eportfolio of their work that they could ‘exhibit’ as a way of demonstrating their achievement and skills over a period of time.  The aim of this would be to show potential schools or employers what they were capable of doing in the hope that they could get a place at that school / company.

The problems with this eportfolio approach included:

  • Each portfolio piece is isolated, not showing how it links with other pieces
  • Eportfolios do not show progression from initial idea to solution as it is the completed work that is shown
  • Eportfolios do not include self, peer and teacher comments
  • Eportfolios do not encourage the owner to respond to comments during the progression of their work, favouring a response to completed work.
  • Eportfolios do not encourage the owner to comment on their own progression
  • Eportfolios do not show failure

By including the portfolio as part of a blog (i.e. a bportfolio) the owner is able to impart so much more information to the reader.  There is also a real sense of progress in the owners work.

Demonstrating #digitalstudies skills through a bportfolio

Rather than delivering isolated units that do not interrelate, #digitalstudies allows the pupil to look at a project that as a minimum will last a term, although could concievably last a school year.  By using a Problem Based Learning / Project Based Learning approach pupils are encouraged to seek solutions themselves rather than have the teacher show them first.  Projects will be structured in such a way that a solution will require the pupil to demonstrate skills from each of the available strands (authoring, literacy, society, technology).

As the student progresses through their solution they will be expected to post their thoughts and processes on their blog so that they can receive feedback both from readers inside and outside of the school.  This will also act as a means of showing how they progressed through the broad stages of analysis, design, implementation, testing and evaluation (and user documentation if appropriate).

Assessing #digitalstudies skills

Firstly, it should be stated that whilst bportfolios are the most significant part of assessment for #digitalstudies, it will also be supported by product / solution presentations and hack days.  A hackday (or morning / afternoon) allows a pupil to complete an integrated task rather than a traditional written paper.  It is possible that a Hack Day could also be supported with a written paper as obviously pupils still need to familiar with the requirements of GCSE exams.

Traditionally I would give a class a mark scheme, rubric and even an example of completed work so that they knew exactly what was expected of them.  Nothing really wrong with this as it provides the pupils with very clear expectations and there was no confusion over what they had to do.  I use this approach mainly because that is ‘always what I have done’.  The biggest problem;however, with being so explicit about what I expect the pupils to do is that I am forcing them to take the route that I want them to take to get to a solution as I have told them what they need to do to get each mark.  I have removed any incentive to explore different paths.  This traditional method also discourages learning through failure as they already know what I expect.

#digitalstudies must have a far more flexible way of assessing.  It cannot say to a pupil that they must take a certain route from the foothills of a mountain to its summit; indeed, it should allow them to choose their own summit.  We should put the mountain range in front of them, show them the different peaks, show them the different places that they can start from and then let them find their own way.  Sometimes they will go wrong and will have to come back down and try a different route; sometimes they can ask a climbing partner (pupil) for support or ask a guide (teacher) for help.  Ultimately though, they must find their own route and be responsible for their own successes and failures.

Assessment of their failures, successes and progress will be done continuously by internal AND external peers, teachers and essentially anyone who reads their blog.  They would obviously be encouraged to seek support outside of their normal environment by using email, twitter, etc to contact ‘experts’ who could help them – they would be expected to blog about this contact as way of showing progress towards a solution.

Assessment of most of the work done in #digitalstudies does not need to be AoL but MUST be AfL and should be continuous.  We should not present mark schemes telling pupils what steps we expect them to take to get to a solution.  We should let them find the best way appropriate to their skills and encourage them to broaden their investigations through AfL.

Grading #digitalstudies

Whether it is a good thing or a bad thing is not for this post; however, at my school we use ‘Challenge Grades’ that are calculated by pupils taking MIDYIS and YELLIS tests and a likely GCSE grade statistically attributed to them.  We then ‘boost’ this statistical prediction by another grade (or more) to take into account the Value Added that we expect to deliver through the pupil attending our school.  We then send a report home every half term to say whether the pupil’s work for that half term is + or – against their challenge grade; this also includes a 140 character ‘how to improve’ comment.  At the end of the school year we write a full report.

I have included this introduction as it demonstrates that whilst I believe that #digitalstudies must be flexible in the way pupils are assessed, I have to, as probably all of us do, report back on the pupils progress.  I do not see a conflict of interest between #digitalstudies and providing a grade, although I can see their being some difficulties.

I have to provide report grades between A* and D, so must find a way of mapping what the pupils are doing against this grade.  At the moment, I have mark schemes so their is no problem as x out of y equates to n grade but as I said earlier this means that all pupils have taken the same route to a solution and is therefore not appropriate for #digitalstudies.

My initial thoughts are to provide a rubric that assesses the actual blog rather than the computing / ICT work that they do. By setting a scheme that awards grades for such things as ideas, reflection, creativity, data gathering, post frequency, feedback to others, organisation, appearance of blog, etc I will be able to keep the pupil informed about how they are using their blog and that they are on the right track.  I am also able to complete my reports and provide informed feedback to parents and SMT.

#digitalstudies badges

I believe, as does @sharland in his post assessing pupils’ work for #digitalstudies – my examination of badges and levels that badges are the way forward.  Badges allow the pupil to progress by taking their own route towards a solution.  Rather like gamification they can quickly pick up badges for work that they are already proficient at, whilst also being encouraged to persevere for those badges that they do not yet have the skills for.

Using a PBL approach, solutions cannot be achieved unless a broad range of badges have been collected; however, the badge system must also differentiate.  By including badge levels (e.g. Novice, Intermediate, Advanced) pupils of all abilities will be able to reach solutions.  This is where the real skill in delivering #digitalstudies will come into play – Problems must be set so that all pupils have the chance of reaching a solution no matter their ability.  The weaker pupils should be encouraged to collect the Novice badges and aim for Intermediate where they are stronger, whilst the stronger  pupils should be aiming for Advanced status throughout their journey.

There is still some way to go to develop the badge system although @infernaldepart is looking at developing an independent system along the lines of Mozilla Open Badges.

In conclusion, the bportfolios used in #digitalstudies will allow for traditional grading as required by SMT, whilst open badges will allow a more flexible ‘grading’ system for the pupils using a type of gamification incentive.


10 comments on “Assessing #digitalstudies – grades or badges

  1. A superb post Nic and covers a lot of what the 3 of us have been thinking. The idea of digital portfolio that shows all aspects of the student work and allows feedback in the output i know im looking for.
    One of the challenges for schools adopting the #digitalstudies model is persuading SLT that this approach to assessment is valuable and useable in schools where data is gold!
    Onwards and upwards

    • Thanks Chris for taking the time to leave a comment. You are right, there is a challenge in place to convince SLT about the assessment opportunities with #digitalstudies; however, portfolios are so well geared to AfL that they can do lot of the marketing for themselves. We just need to make sure that come September everyone hits the ground running.

  2. Pingback: Diigo Links (weekly)

  3. Liking the idea of bportfolios a lot Nic, but wondering whether you feel they sit alongside or as a replacement for eportfolios? I suspect that some of the perceived problems with eportfolios (links to other work, comments) might be addressed in some systems, though wouldn’t it be fair to say that not ‘showing failure’ is actually a positive feature of an eportfolio i.e. they’re more about surfacing your successes? That said, ‘failure’ certainly shouldn’t be something to hide away and the ability to learn from and move beyond discarded approaches is worthy of recognition. Maybe a bportfolio supports the formative process, whilst the eportfolio leans closer to a summative end-product?

    Also wondering how you might resolve the tension between your school’s grading process and the way you want to assess bportfolios? To what extent do you feel the criteria you want to award grades for (data gathering, post frequency etc) map across to the criteria upon which GCSE grades are based? I guess you could award a mark based on the rubric you might be creating, then as you say x mark generates y grade. But wondering whether that’s valid. If as a Physics teacher, my class was researching Forces, would it be possible for me to map across the grades for that to a an investigation into Current Electricity … or do you feel the differences between the two sets of assessment criteria (GCSE & bportfolio) are not that distinct?

    Hope this doesn’t come across as me being critical here Nic as that’s certainly not my intention. Just interested in clarifying my thinking … or rather getting you to help me do that 😉
    I *do* think that the chronology, commenting options, ability to tag posts and perhaps choose to whom posts are visible(?) offer much potential in revealing student progression, so you post is much appreciated.

  4. Thanks Ian for taking the time to leave a comment and for raising such thought provoking questions.

    For me, the major advantage of a bportfolio is that an essential part for the student is to demonstrate the progress that they have made from analysing an initial problem through to final solution(s), showing their progress – successes and failures – thoughts and research along the way. As teachers, this will give us far greater opportunity to ‘get inside the pupil’s head’ and see how they work, which in turn gives greater opportunity to give really valid and formative feedback.

    My take on an eportfolio is that the pupil exhibits an end result without showing any progress or feedback along the way. I cannot see how this could provide the teacher or the pupil with adequate opportunities to evaluate working processes, progress or solution ideas. It does show what they can do, but not how they got there.

    With regards to ‘resolving the tension between my school’s grading process and the way I want to assess bportfolios’ I am sure that this will take several meetings with my Director of Studies. I am not yet sure how these will go; however, we already have in place where are minimum requirement is what is currently called a key assessment mark each half term and it is likely that the #digitalstudies / PBL approach will lend itself to a year’s project that has half termly stepping stones along the way – It is these stepping stones that ‘could’ be graded. Since writing the post, I have gone off the idea of actually marking the portfolio posts as I really want the pupils to concentrate on the comments and not on a grade.

    Your comments were definitely not regarded as cynical; they have helped me clear my own thinking a little more.

  5. Thanks for responding in such detail Nic; that’s really helped thanks.

    I’m couldn’t agree more that bportfolios present a real opportunity to bring progress and process to the surface, making it visible in a way that enables guidance, suggestions, encouragement and alternatives to be offered … formative feedback as you rightly say. I especially like the notion that others could be involved in that formative process and furthermore that it can become more discursive in the way that this post has i.e. it’s not just ‘hand in work – teacher gives feedback – done’

    Good luck with your meetings with your DoS – look forward to hearing how they go.

  6. Pingback: Y7 Mobile Phone Project for #digitalstudies | @teachesict

  7. Pingback: Y8 House Building Project using #digitalstudies and #badges | @teachesict

  8. “This is where the real skill in delivering #digitalstudies will come into play – Problems must be set so that all pupils have the chance of reaching a solution no matter their ability.”

    This is key part of this post for me…are you advocating that teachers set open ended problems? Or that students develop their own problems to solve? To be a truely PBL task, surely the student would need to decide on their chosen problem.

    Now logistically this would be difficult, as some pupils are not working at a such a level where they can define to themselves a problem. A lot of the problems they develop would be unfeasible.

    Or do we take the middle road and provide the focus or scope, and give them pathways to follow: A novice problem, Intermediate problem and Advanced problem. Is this not just differentiation?

    • Thank you for taking the time to leave a response and to raise a good question. For me, initially at least, I shall be setting a general problem where the student can work through different pathways according to their strengths and interests whilst creating their portfolio.
      I don’t think that this is ‘just’ differentiation as it allows for pupils to work at more than one different level depending on the exact task.
      The integration of the badges being developed by @infernaldepart into the bportfolios will also have a massive impact on how the pupil moves through the task. You may also want to look at @sharland ‘s description of the portfolios
      Also bear in mind that the ‘problem’ will require a solution that covers many skills and will lasts a school year.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s