More on threshold concepts and #ACRLILRevisions

The three threshold concepts in the new ACRL draft Framework for Information Literacy (Higher Ed) are noted as:

• Scholarship as a conversation
• Research as inquiry
• Format as process

From conversations on Twitter, Andy Burkhardt made a great post about how he has implemented "Research as inquiry" in his instruction. These practical examples are so helpful in understanding such a theoretical framework. Since I have been pushing research as conversation, or "Scholarship as conversation," in my own teaching, I thought I would share what I have done as well (for reference, I wrote about my initial thoughts on the new draft framework in a previous post).

Credit course and scaffolding

We have since paused our for-credit courses at the library, but in the last two sessions, I scaffolded research as conversation throughout the semester. I started off with introducing the concept, then made greater analogies to other modules, and in the end, had students create a short, animated video or comic strip (or script if they were not feeling visuals) illustrating a facet of research as conversation. (And this course is where we initially started using digital badges, as a side-note).

Searching online communities

In the fall, I had two additional opportunities of note to use scholarship as conversation, but also the other two threshold concepts. In a course in the UA's new eSociety program, my colleague, Leslie Sult, and I collaborated with the instructor to develop an in class activity and assignment. Students were researching a current event in a variety of formats/online communities (social media, local news, national/international news, news blogs, etc.), and the instructor wanted her students to do some critical thinking in groups to evaluate information and think about bias and point-of-view. I came up with the worksheet below, and we wound up having some great conversations as each group presented on their resource (YouTube and Twitter were especially interesting):

And additional questions we asked to coincide with the worksheet, after a brief lecture on related issues was delivered, included:

• Open versus closed community: impact? E.g., Facebook closed vs Twitter open – algorithms on Fbook and Google search when signed in (stay in the echo chamber)
• Primary versus secondary sources: what is the difference and when might you use either?
• How are messages changed/altered when they are retweeted or shared? Is anything lost? (like playing telephone), how do you account for this in searching? How do you know what part of the message is accurate? Methods for this
• Search strategies and tools: hashtags, groups, slang, memes, etc.
• Trolling: how does it affect communities and how might it change your search strategy?
• How do you know if someone is trolling a group or a topic discussion? Does trolling have significance in your search? Should you seek it out or ignore it?
• Back to whose voices are heard? What might the effect of being in the “echo chamber” do to whose voices you hear personally? What search strategies could you use to get out of the echo chamber?

The learning outcomes based on the instructor's course learning outcomes in conjunction with Leslie's and my goals for library instruction were the following:

1. Engage in a focused process of inquiry within an assigned online community in order to articulate the ways in which online communities function across contexts in contemporary life
2. Strategically access and evaluate information via search in an assigned online community in order to recognize various perspectives including rhetorical, philosophical, historical, sociological, and psychological viewpoints
3. Develop insight into the ethical aspects of information creation, use, access, and durability in order to be conscious of many group-related issues and practices relative to the use of computing technologies to facilitate group collaboration

Student athletes and avoiding plagiarism

When working with student athletes later in the semester, I more literally included scholarship as a conversation into my instructional design for a session I collaborated on with the Director of the athletes' writing center and my colleague, Niamh Wallace. I started the session talking about the process of research to frame positive uses of citations (how they help a conversation) and the negative effects of plagiarism, accidental or not (how they harm a conversation). To illustrate the concept in their minds first, I read them Burke's Unending Conversation Metaphor that I slightly adapted to more modern language they could relate to. I asked them to close their eyes and....

Imagine that you enter a party. You come late. When you arrive, others have been there long before you, and are engaged in a heated discussion, a discussion too heated for them to pause and fill you in. In fact, the discussion had already begun long before any of them got there, so that no one present is qualified to retrace for you all the steps that had gone before. You listen for a while, until you decide that you get the gist of the argument and join in. Someone answers; you respond; another comes to your defense; another aligns against you, to either the embarrassment or gratification of your opponent, depending on the quality of your ally’s assistance. However, the discussion is endless. It’s getting late, so you have to take off. And you leave, with the discussion still vigorously in progress.

Marisa (Director of writing center) incorporated a short lecture on how to write well when using other people's ideas, and as a hands-on activity had students write about their thoughts based on what we had presented to them (having them cite us). Then, using game design for this session, the theme was the "Citation Olympics," and we had students compete in groups for prizes as they learned content. Our format was introduce concept > practice > compete in the Citation Olympics at the end. Each module was a "sport," essentially. Here is a copy of the PPT we used to guide the session for a better idea (though much detail still gets left out from not including lecture notes).

Workshop on avoiding plagiarism for student athletes from Nicole Pagowsky


Anyhow, thought it might be helpful to share, and I hope to see how others have been teaching these concepts to gain a better understanding of how the new framework can be put into practice.

Adding another piece to the library + student retention puzzle

image from www.businesscontinuityjournal.com

Grit has become a big, flashy word over the last year or so, in regards to instilling adaptive learning in students and building resiliency. The idea is that students who are able to adapt and bounce back from failure are able to learn from mistakes, being more likely to stick around in school when they are faced with challenges.

In looking at educational psychology literature, and is becoming more widely known through research on positive effects of gaming, experiencing failure supports adaptive learning. Rohrkemper and Corno (1988) highlight the problematic duality of failure versus success, where failure is always bad and success is always good. This in fact is not true, where constant success can be detrimental and failure can improve performance (that is, learning from failure). Focusing on how students think, rather than what they know, is one step in the right direction, along with modeling adaptive behavior, and teaching students to understand that tasks and learning can be malleable. In library instruction, this reminds me of what Pegasus Librarian (I believe?) mentioned in regards to providing students with a "dirt-view" of research (I can't find this post, I'm thinking it could have possibly been an episode on Adventures in Library Instruction). But basically, where we show students research takes work and builds on failure, and it's almost hilarious when we show students practiced, perfect searches because that is not how research works at all. Kluger and DeNisi (1996) support this notion of learning through failure by arguing that after doing an enormous meta-analysis of feedback interventions research, the conclusion is that the feedback literature is inconclusive and highly variable based on situations and learners involved. They explain that learners are most successful in learning through discovery, rather than feedback, particularly controlling feedback (ahem, grades).

Brownell (1947) advocated for teaching meaning in arithmetic, typically a rote, "tool subject." You'd think the argument of teaching meaning would be quite clear, especially in 2013, but this debate continues in some ways. We can see the shift in information literacy, it seems there is more agreement now to move away from navigation and clicks ("bibliographic instruction") to teaching students a more holistic understanding of research in "information literacy." Barbara Fister, as always, is very eloquent in how she explains the importance of this. But really this is another avenue to instill resiliency in students, by focusing on higher order thinking (though, higher order thinking is not always appropriate in every context), we truly are looking toward students' process rather than having interest only in final product. As Brownell explains, teaching meaning provides a greater context for students to find value in the particular subject being taught. With all the difficulty librarians can have regarding one shots (this model could/should change) in building connections with students and improving motivation for students to learn aspects of the research process, providing deeper knowledge about why, and not just what and how, can improve the learning environment.

I was going to next talk about how to provide successful feedback, because it is important, but to avoid making this post so long that no one actually reads it, I just want to wrap up with whether in a class (credit-bearing, one-shot) or through more auxiliary approaches, libraries should be places for students to build grit and resiliency through exploring failure. We talk about how orientations are important for students to develop a social connection and feel comfortable somewhere on campus, and this is a very important aspect of retention, but these safe spaces should also provide opportunities for students to take safe risks and learn how to adapt to failure. This doesn't necessarily mean libraries need to gamify the whole library or offer badges as a panacea for solving student retention or student motivation concerns, but these are examples of methods that could prove useful. Setting up other opportunities in the library for students to test out ideas are ways in which to draw them in and instill adaptivity. Hopefully they are also getting opportunities for safe failure in their campus-wide courses, but it's certainly not a guarantee. Libraries should think about how we can provide opportunities for safe risk in a variety of ways, whether it's instruction, programming, collections, or UX. It's one step in figuring out how we can support student retention initiatives on campus and demonstrate value.

--
Brownell, W. A. (January 01, 1947). The Place of Meaning in the Teaching of Arithmetic.The Elementary School Journal, 47, 5, 256-265.

Kluger, A. N., & DeNisi, A. (January 01, 1996). The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory. Psychological Bulletin, 119, 2, 254-284.

Rohrkemper, M., & Corno, L. (January 01, 1988). Success and Failure on Classroom Tasks: Adaptive Learning and Classroom Teaching. The Elementary School Journal, 88,3, 297-312.

Grading and assessment, water and oil?

As I've been getting ready for ACRL Immersion 2013 Program Track (I leave in 1 week!), I've been finishing up a lot of readings on assessment. I've actually been really glad to read these articles, because as I've better solidified my notion of assessment through applying it to the instruction I am doing, I am finding my ideas are aligning with what I am reading.

The pattern I am finding in these readings is that assessment needs to be more holistic; assessment should be a method for students to learn rather than a focus on evaluation; and assessment should provide ongoing, meaningful feedback for students to practice instead of being judged.

We are wrapping up the summer semester with the badges pilot, and Purdue Passport incorporates assessment within earning a badge. Typically, a badge is given after a skill has been achieved, where assessment is more evaluative and judgmental rather than to provide feedback for improvement. This clashes with how I would prefer to teach and use the badges, so I've been using the feedback/assessment mechanism in Passport differently than it might be intended.

This is good because I think students are getting more out of the class, but also poses some conflicts:

1. If badges being awarded are not based on more rigid judgement of skill acquisition, how valuable are they?
2. On this note, how interoperable are they? Can their qualities be translated or compared to other institutions or libraries offering similar badges if desired evidence isn't as clearly enforced?

Because this is a credit class, grades need to be tied to student work. For this, the badges are essentially pass/fail. You either earn the badge or you don't. If a student is late in finishing badge work an exception is made to give them half off, but this is the only partial credit awarded. There are pros and cons to this as well:

Pros: Students can take risks in their responses and have less fear of failure (this positive aspect is rooted in game mechanics); I can focus more on the quality of my feedback rather than what level of good or bad the student's work falls into

Cons: How is good student work differentiated from bad work? Particularly if bad work is due to sloppiness or disinterest. Shouldn't a student who submitted excellent work (or evidence) for a badge be awarded the badge, where less stellar work would not be awarded the badge? Isn't the purpose of awarding badges to demonstrate that a skill was successfully acquired?

I have such mixed feelings on this. But one feature of Passport is to allow students a re-do. I use this often for sloppy work. I will leave feedback explaining exactly what I'm looking for and give the student a second chance (next semester I will be sharing specific rubrics for each badge with students so they have an even better concept of what level of understanding is desired). 

I am not a stickler on lower-level concepts like formatting a citation perfectly or memorizing exact steps on how to find an article when you only have a citation (these are specific assessments in the class to address more basic skills within learning outcomes). If a student has most of a citation right but forgets to italicize the journal title for MLA style, it's really just busy work for them to make them re-do it or for me to take points off. I leave feedback letting them know they mostly got it and to remember to double check these things for formal papers; and then I give them all the points. I love Barbara Fister's 2013 LOEX Keynote (in fact, my team read it as part of our strategic planning for the new fiscal year). I agree so strongly with her whole presentation, and using a specific example here, "very rarely outside of school are citations needed." I care way more about if students are able to understand what the purpose of a citation is and to incorporate this into their new understanding of "research as conversation" than about styles and how to format.

One assigned article that has been part of this class for a long time is a CQ Researcher article on cheating: why students cheat and how they cheat. It's interesting to see what students agree with in their reflections and a number do say that when a student doesn't feel course material has real application in their lives (or when an instructor provides little to no meaningful feedback), a student has no motivation or investment to put in quality work, and so cheating is easy. Focusing less on grades and more on understanding and a conversation between the students and us as instructors creates a richer experience for all. Their reflections resonate well with what we're doing in the course to make it apply to their lives, to attain better work from them, and in turn to provide more meaningful, continuous feedback. This also allows for continuous improvement on our end; the crux of assessment. 

Feelings and games

I'm currently enrolled in a MOOC (I am unsure how I currently feel about MOOCs overall in theory, but am liking this one so far). It's the Game Elements for Learning (#GE4L) MOOC offered through Canvas.

One of the readings we have been directed to is Karl Kapp's post, The emotional toll of instructional games.

(Kapp also wrote: The gamification of learning and instruction: Game-based methods and strategies for training and education. 2012).

I am finding this very intriguing because I have been having reservations about incorporating some game elements into the badge pilot I have designed for our 1 credit course. I wanted to have a leaderboard so students could feel good about doing extra work, but I did not want people to "shut down," as Kapp says, for not making it onto the leaderboard. The intent is to motivate, but in reality, it could have an opposite, unintended effect.

Kapp says, "If you decide to add game-elements (gamification) or if you decide to create a learning game with winners and losers, you need to find a way to deal with those who do not win. You need to help them avoid some of the negative feelings. You may even decide that a cooperative game is better than placing someone in a losing situation."

In the class, we are not offering enough rewards where certain people would feel like everyone but them is winning, and I am trying to mix it up so that a variety of people are included. I am also incorporating easter egg "mini-badges" for exceptional work, cooperative skills, or etc. be varied so the same people are not winning for excelling at particular skills while ignoring other skills. Students can unlock them without knowing. Still feeling it out as I go, and since this is the pilot it will be great to see what worked and what did not. I'm honestly a little worried that there is the reverse effect, that students might not care at all about the leaderboard. I'm still getting great work from them, but I'm not clear on if it is factoring into their submissions or not. They are very likely more motivated by the grade. And since this is a summer class, people are traveling and might have their thoughts elsewhere. Looking forward to survey data at the end of summer. I think I'm going to need to revisit the game + instructional design to make it more well-defined, include more motivational game elements, and find a way to make it not so heavily grade-focused (if possible since it is a credit course).


Kapp lists 12 ways to "mitigate losing in an instructional game or gamification situation," and I thought it might be useful to comment on each:

1) Forewarn the learners that they might become upset or frustrated if they find themselves losing and that is part of the learning process.
I did not do this, but I have at least tried to address affective learning outcomes and how research can cause frustration; that it's not easy or linear and you have to practice. I hope these skills can be applied to the game mechanics and course content as well, but perhaps it's better to come right out and say it.

2) Inform learners that they might lose the game and that is OK, learning will still occur.
I have done this in a way to allow students multiple attempts at a badge. This really can tie in to taking risks and having a decreased fear of failure. Since assessment should be to improve learning, not just judge work, I try to make the feedback really count.

3) Carefully brief all the learners on the instructional objectives of the game and de-emphasize winning.
Since the badging structure is tied directly to the course, the course objectives are the main focus. In this case, badges are more of a visual way to track progress.

4) Acknowledge the frustration or anger at losing.
Similar to my response for #1.

5) Ask learners to find lessons and reason within the lose. Have them dissect why they lost. Ask “can those insights lead to learning?”
Perhaps I should have students do more cognitive work here, but when they get it wrong when submitting work for a badge, I reiterate and clarify what exactly I'm looking for so they have a better opportunity to "win" the badge when they try again.

6) Don’t spend a great deal of time extolling the winners. Acknowledge winning and move right to the instructional lesson.
Exactly. I post a leaderboard in the news section and just leave it at that.

7) Provide a list of strategies that will help the learner win next time. (After the game.)
I try to do this in the assignment description so students know exactly what I'm looking for. It might even be better to post the actual rubric next semester.

8) Within the curriculum, follow the game activity with an activity where everyone can feel positive.
I'm not sure how this would play out in my scenario, but I think the participation points for discussion might work in this way. Rather than being graded on what is said, general points are given simply for being involved.

9) If in a classroom, allow people who did not win a chance to discuss why they didn’t win. Online, provide chat opportunities.
More chances for reflection would be very beneficial. The badges offered are so incremental, however, larger reflection might not be a good fit.

10) Consider if creating “winning” or “losing” is really what you want in the learning experience. Sometimes it is appropriate. Often it is appropriate but be prepared for unintended consequences and negative feedback if you don’t handle the situation properly.
Yes, definitely reconsidering even having a leaderboard. Maybe these easter egg "mini-badges" could instead be private between instructor and student.

11) Create different levels of winning, can a learner win a round, or one task, can small victories occur throughout the game. This is helpful because if a learner falls behind early, they may mentally drop out early in the learning process. Find ways to keep them engaged.
Love this.

12) Finally, you may want to consider building a cooperative rather than a competitive game. Working together is far more inclusive than competition.
Love this too. Just have to find a way to track each student's own work to tie to badge earning.

#ala2013 recap: Badges, student retention, and over-capacity parties

Wow #ala2013 went by so fast! This was hands down my favorite conference that I've been to over the last 3 years. Here's my brief recap of highlights:

I didn't attend as many sessions as I would have liked; I presented twice, led a discussion group, and reviewed people's resumes for NMRT's Resume Review Service, so a lot of my time was already nailed down, but it was all stuff I wanted to do so it worked out.

Gamification
On Saturday morning, I presented on the LITA: What to know before gamifying your library panel. We had a range of topics including: Bohyun Kim (moderator) giving an overview of gamification; Dave Pattern's use of Library Game / Library Lemontree at the University of Huddersfield (UK); Annie Pho covering the not-fun-but-very-important stuff on how to create institutional buy in and obtain grant money for these sorts of projects; and Young Lee explaining the technology aspects involved and how he plans to use badges in a law school library. My presentation was titled, "Anchoring the badge: Setting standards for game-based learning in library instruction." I discussed my current implementation of badges for instruction at The University of Arizona Libraries. You can see the Slideshare presentation with everyone's slides; though, since it was such a large panel not all of us contributed slides (myself included). So you won't get much from what I discussed in that link. Here is a very brief summary below; I am sure I will be speaking and writing about this project more as it progresses (have IRB approval!), so I plan to share more information in the near future.

Importance and benefits of using badges for instruction:

• Makes instruction more scalable, can ensure wider adoption of IL skills: trackable, measurable
• With trackability and assessment built in, this presents possibilities for customized learning ("microcredentialing," demonstrate specific skills; customization can greatly improve motivation and learning)
• Evidence is tied to the idea of competency-based learning (use specific outcomes to show criteria has been met for assessment, accreditation, program SLOs, other standards like the ACRL IL Standards, etc.)
• What we are doing at the University of Arizona: my overview was very brief since I'm still currently studying this and have gotten IRB approval to do so

I left those in attendance with some thoughts from Dan Hickey of Indiana University, via a Campus Technology article, How badges really work in higher education:

• "What sorts of claims will your badges make about the earners and what evidence will your badges contain to support those claims? 
• What assumptions about learning will frame your consideration and implementation of badges?
• How will your badges program be introduced? Will it be a centralized effort or pockets of innovation? "

You can read more about badges and gamification in academic libraries from what I have published in ACRL TechConnect on initial plans for badges at the UA Libraries, as well as our use of SCVNGR back in a pilot:

• Test-driving Purdue's Passport platform for library instruction

• Taking a trek with SCVNGR: Developing asynchronous, mobile orientations and instruction for campus

Char Booth also has a great post on badges at her blog, Info-mational, looking at badging in higher ed and discussing how she is using this form of micro-credentialing in the ACRL Immersion Teaching with Technology track. See her post, MYOB: Make your own badge.

More on Badges for Instruction
On Sunday, I presented a Conversation Starter with Annie Pho and Young Lee: Achievement unlocked: Motivating and assessing user learning with digital badges. Our hashtag was #alabadge, and you can see some helpful Tweets summarizing the session.

Tweets about "alabadge"

Student Retention
On Saturday, I also co-facilitated my and Jaime Hammond's ACRL Student Retention Discussion Group meeting.  You can also find the group on ALA Connect. Our topic for this meeting was:

How do we measure causation versus correlation in the library’s role in student success and retention? The ACRL Student Retention Discussion Group will be discussing the impact of a “culture of assessment” on libraries and demonstrating value on campus in regards to retention. We will discuss how effective demonstration of value in campus retention is through traditional methods and hope to explore ideas participants have for new initiatives.

To help guide the discussion, we used Megan Oakleaf's article on assessment strategies:

Oakleaf, M. (March 01, 2013). Building the Assessment Librarian Guildhall: Criteria and Skills for Quality Assessment. The Journal of Academic Librarianship, 39, 2, 126-128.  

We had some great discussions about what people are doing at their institutions, and seemed to have a good mix of academic librarians from community colleges and universities. The minutes should be posted within the next week or so; if this interests you, joining the Connect group will keep you up to speed. We also organize monthly article discussions during the regular academic year, with volunteers choosing articles and facilitating.


Misc
Other things included the Librarian Wardrobe + Every Library After Hours Party, which will have a solid recap on Librarian Wardrobe soon. We had a great time helping to raise awareness and $$ for Every Library, and so excited to plan more events with them at future conferences. Apologies to anyone who could not get into the party, it's very, very hard to find venues that allow for a large capacity without charging tons of money that neither LW or EL have to spare. We do have plans to accommodate more of everyone for #ala2014.


There was a lot of other great stuff but I'm going to stop there since this is already getting pretty long. I had a lot of fun spending time with friends and meeting new people at this conference. In the meantime, I am getting ready to go to ACRL Immersion in Seattle later this month for Program Track and have some other, exciting projects in progress as well. Check back here for more updates on badges and other stuff!

Library research expertise, collect them all

From Purdue Passport

Student motivation can be problematic in college courses, and particularly with auxiliary college work where skills are encouraged, but aren't necessarily required to be learned (ahem, library research skills). Some instructors are serious about students building a knowledge base in using the library and developing critical thinking in regards to information, but it's not across the board. As we know from the ERIAL Project, student perceptions are heavily influenced by their instructors' relationships with the library. When the library has a good relationship with an instructor, research assignment design tends to be strong and students get a better grounding in using library resources. As great as this is and as much as we'd hope to advertise this fact to faculty, we can't exactly force every instructor on campus to work with us and especially to incorporate a research-related or info lit-type of assignment if they either don't want to, or it doesn't fit with the course.

So, we wonder, how can we help students develop these skills even if we can't work with them through a class, or if we haven't yet become embedded where they are. I've been thinking about this a lot over the past year in relation to student retention and also gaming and motivation, and became very interested in Mozilla's Open Badges, which I discussed here back in January when exploring badge systems. These badges are tied to certain skills that can be earned through reading and completing certain tasks, which can then be displayed in a portfolio or on social networking sites.

Thinking about how this can be tied to education has been apparent in MOOCs, and just recently, Purdue has developed Passport to offer badges in a university setting. I have been approved to be a beta tester, which I am really excited about. We have been talking about incorporating gamification and a badge system here at the University of Arizona Libraries since I started and was particularly enthusiastic about it, but we run into issues with the programming side of the system since we have limited staff in that regard. We are hoping to develop a gamification layer over our existing tutorials and guides and will have badges tied to the ACRL Information Literacy Standards (as a very basic explanation of these ideas).

Now, let's be realistic, I think we all get it that most students aren't going to be persuaded to do extra work in learning library research skills just because they might get a PNG image after completing tutorials and quizzes (I certainly know I wouldn't have been convinced as an undergrad). However, I am hoping we are able to work with the career center, tutoring, and other areas on campus that might help give the badges more value so students feel they are meaningful. If only one unit on campus is offering these badges, what exactly do they even mean? However, if students can include a suite of them in an eportfolio or on a resume, that does have more value. On the flipside, from our analytics, we do see that students, and even non-students, complete our tutorials regularly without them being assigned, and for the ones offering a certificate upon completion, we have a large number of people submitting their information to receive one. So, there is clearly intrinsic motivation present, but we hope to use a combination of intrinsic and extrinsic to find the right balance in helping students build these skills.

I wrote a literature review on motivation in gamified learning scenarios for a gaming in education course I took this semester, which you can read here if you're interested. Applying these ideas to a badge system in libraries is more tricky than a classroom since we typically do one-shot sessions, and like I mentioned these skills are often treated as auxiliary to a class.

Anyhow, I will keep this blog more updated than usual as I beta test and incorporate badges into our resources! More next time...