Open educational resources evaluation and L&D dungeon and dragons

Continuing my recap of doing two conferences at once (go here to get filled in) October 22, AECT was in full swing and I had my first presentation of the week. This presentation was with a great group of people including Jennifer Maddrell, Jennifer Englund, Royce Kimmons, Jessica Resig, Emily Tolzmann, and Cherise Moore. One thing “different” about higher education presentations are the number of people on presentations 😀

My “glue” Anna Leach attended AECT with me and we went together to pick up our badges.

This was such a cool project and our presentation was titled: Evaluating Open Educational Resources: Results of a Collaborative Evaluation of Crowd-Sourced Lesson Plans

The presentation focused on an evaluation of a service-learning professional development opportunity for instructional design students and volunteers to develop 106 open educational resources (OER) for the benefit of adult basic education. We shared the findings of the OER evaluation and consider implications for the design, implementation, and evaluation of OER, as well as service-learning as a professional development opportunity for instructional designers.

To see the full presentation, you can view the slides here

My role in this was evaluating the OER. It was a really fun project and I enjoyed going through all of the resources. I speak more about this on the Praxis Pedagogy podcast and talk more about the work Jennifer and Royce are doing in this area. 

Presenting at AECT, thanks for the pic Anna Leach <3

I was able to get two selfies with awesome people at AECT

Valary Oleinik

Kari Word

After this, we went over to meet up with Tracy Shroyer who was arriving at the conference! After she got settled in, we then headed back to the Mirage. I had the opportunity to FINALLY meet after years Amanda Jackson. I also got some more pictures.

Ian Crook (who is ridiculously tall btw, not James McLuckie tall, but tall 🙂

Mike Taylor

Bianca Baumann

Dr. Clark Quinn

Josh Risser

 Here are my picks for the top 3 special moments up until Tuesday.

#1 L&D D&D with fire

Starting at TLDC18, Matt Pierce kicked off L&D D&D. Yes, it’s Dungeon and Dragons but you haven’t played D&D unless you have a Dungeon Master like Matt Pierce. We met up at Denny’s and Tracy Shroyer, Kristen Hayden-Safdie and others showed up to play a “deconstructed” game. I think we were all tired but it made for a fun game. Bonus is that Denny’s was across from the Mirage volcano which made me very happy (every time I play D&D I try to set everything on fire).

#2 Letting two special people know what they mean to me

For the past year, I’ve served as the Communication Officer for AECT’s Research and Theory division. This division was led by Dr. Ginger Watson and previously led by Dr. Enilda Romero-Hall. If I decide to stay in higher education, I want to be like these two ladies. They are wicked smart, kind-hearted, and I’ve really appreciated getting to know them better. AECT’s conference theme was Inspired so I took the opportunity to tell both of them how wonderful they are (and take a pic of course)

#3 LXD Illuminati

Before going to AECT for my presentation Ceren, Anna, Utkan and I went shopping at the Vegas outlets. We stumbled upon this defunct fountain and I knew I had to get a pic in front of it. If you want to understand more about the Illuminati, ask Alex Salas 🙂

That’s a wrap for Tuesday. Devlearn kicks off on Wednesday and it’s my first day going between both conferences. Coming up in the next installment are coffee mugs, Sophia the robot, and an award 🙂

Read More

2 identities: 2 conferences: 1 week

Looking at my professional growth over the past few years, two identities emerge: Cara the academic and Cara the practitioner. 

In October of this year, I decided to go on an epic quest: could I do two conferences at the same time? I traveled to Las Vegas at present at the Association for Education Communications and Technology (AECT) conference and Devlearn

Since starting my learning technologies academic journey in 2016, I’ve been involved with AECT attending the last 3 conferences (Jacksonville (2017), Kansas City (2018), and Las Vegas (2019)). AECT is a higher education focused organization whose members are faculty, staff, and graduate students in educational technology research. I learned about AECT through my former advisor Dr. Correia. Dr. Correia encouraged me, along with Anna Leach, Natalie Gintert & Tim Nunn to submit a class project we completed to the conference. When it got accepted, we had the opportunity to present and publish a proceedings paper. After that, Anna and I decided to continue to work together for future projects at AECT. Given this year was my third AECT conference, I had confidence in what to expect and how to best deliver my presentations.

Devlearn, on the other hand, was a whole different challenge for me. After learning about Devlearn years ago, it has been on my bucket list to speak at. As an L&D person, Devlearn has been a conference I’ve respected and feared. Despite attending ATD ICE earlier this year with 10,000+ people, Devlearn has intimidated me. In fact, when I hosted a co-working session to encourage others to submit speaking proposals, I admitted my own self-doubts and anxiety. All year up until the proposals were due I kept asking myself if I was good enough to speak at Devlearn. When I received my acceptance email, I was in class and I had to get up and leave to go Tigger bounce for 5 minutes (I wish this was a joke but that legit happened). 

Going to both of these conferences was special to me because I was able to reunite with some of my favorite people. My dear friend Ceren was able to go to Devlearn and was my travel buddy and then when we arrived in Las Vegas, we were reunited with Anna Leach 🙂

Also to make the most out of this experience, Ceren and I flew out on a Sunday because my first presentation at AECT was on Tuesday and I wanted to have a day to ease into the chaos. 

Given the full jammed packed experience, each blog post will highlight a few special moments. Here are my picks for the top 3 special moments up until Tuesday.

#1 A Surprise Reunion

All the way up to the conference, I was in contact with Ceren’s boyfriend Utkan in Turkey who was planning a surprise visit to Las Vegas for the week. He almost blew it by checking into the airport the night before and he did a nice job of keeping her guessing. I did hate to see Ceren all sad because she was convinced something bad was happening. After we landed in Vegas and got settled in, I was in contact with Utkan on where we were. When we went to the elevators at the hotel, he came up behind us and I was able to snap this picture. All of us were crying.

#2 A warm welcome and hearty breakfast

Ceren and I were at the airport at the garish time of 4:30AM. We arrived in Las Vegas around 7:30AM and by the time we got the rental car, it was close to 9:00AM. We were invited to breakfast by Jac Hutchinson and her husband and her friends. I’ve known Jac for several years now and she’s more like a family member to me than a peer who works in the same profession. We met the crew at Ellis Island and it ended up being a place I’d visit 2 more times before leaving Las Vegas. The breakfast was delicious and cheap with easy parking access. Don’t believe me, check out the Trip Advisor.

#3 Sharing a special moment with two of my closest friends

I don’t think I’ll ever be able to clearly articulate how it felt to pick up this with these people. I cannot tell you how important it is to have people who love and support you no matter what. Anna has been such a dear friend ever since my master’s program at Ohio State and I would have quit the Ph.D. program a long time ago if I didn’t have Ceren. Both of them knew how much this meant to me and both of them wanted to go with me to pick this up.

My Devlearn speaking badge
Anna, yours truly, and Ceren

My next blog post I’ll share my reflections from my first presentation at AECT about open educational resources.

Read More

You can rise above sunk cost effect

Sunk cost effect: How it keeps bad projects afloat

Today’s topic, I’d like to explore today through the lens of L&D is the sunk cost effect.  I first heard about this on one of my favorite podcasts Freakonomics.  The topic of the episode I heard it on was about losing.  Not the sexiest topic I know but it was what came up in the queue and it hooked me in. 

A quick note:

This blog post contains affiliate and referral links which may reward me in the event of a sale. I use these funds to feed my cat. Thanks for the cat food.

What is the sunk cost effect? Simply, it’s the tendency to continue investing in an endeavor because of past investments of time, effort, or money in that endeavor, even when the payoff or return on investment is no longer available. 

The sunk cost effect is the tendency to continue investing in an endeavor because of past investments of time, effort, or money in that endeavor, even when the payoff or return on investment is no longer available. Rationally speaking past investments, or sunk costs, should not influence decision making. Let me repeat this. If it worked previously, you shouldn’t automatically do it that way again or let that influence your decision making. Instead, looking at the current situation and the factors impacting the current project. Of course, it’s not always this simple and people don’t always have this line of thinking. If I had a dollar for every time I’ve heard, “We’ve done training like this for X years, why should we change it now?” I could retire. I think most of us could. Often this line of thinking impacts everything: design, project management, and daily life. How can we as L&D folks inspire change? 

For a class I took a few years ago, it was required to read Everett Rogers’ Diffusion of Innovation. This book may be old but it has a lot of great information about being a change agent and inspiring change. One takeaway for me is that change relies heavily on human capital. In other words, how much support an idea can impact its longevity. The importance of having champions for learning cannot be overstated. 

Let’s say you have an L&D consulting business and you want to make a super easy storyboard template to generate some side income. You’ve put a ton of work into it and the week before you are due to launch, an L&D professional development society offers a similar product for free, even to non-members. Do you keep going and launch yours at the cost you were going to sell at? Or back off? Let’s say you keep going and all the time and effort you put into this only netted you 5% of the income that you put into creating it. 

This is an example of the sunk cost effect. People are susceptible to errors in situations like these because they fear failure more than they desire success. And because they don’t want to feel or appear wasteful. We have all heard the idiom, “Winners never quit, “and quitters never win.” Well, it turns out this isn’t true at all. Sometimes the only way to win is to quit. 

So how can we avoid errors due to sunk cost effects? Normally, recognition is the first step to recovery. Let’s take a look at some research. Going back into the time machine to 1995, Tong & Yates found that sunk cost effects are multi-dimensional. In other words, there could be multiple factors at play in sunk cost effects including company culture and policies.

Often times a significant negative emotional event is required to disrupt the sunk-cost and force people to reevaluate their decisions. Accordingly, when such events occur they should be used as opportunities for reflection and reconsideration. So whether you use your knowledge of sunk cost effects to make better L&D planning decisions or to know when to end learning initiatives with no realistic prospects of success, remember: there is great power in knowing when to quit.

Read More

Applying Hick’s Law to L&D

I’ve been self-educating myself more in UX design and I came across a concept called Hick’s Law.  Maybe I was drawn to it because the name made me homesick, but I wanted to explore how we can apply this to our learning experiences.

Hick’s Law, also known as the Hick-Hyman Law is named after a British and an American psychologist team of William Edmund Hick and Ray Hyman. I’m not sure why Hick’s name continues to stick on this.  In 1952, they decided to examine the relationship between the number of stimuli present and an individual’s reaction time to any given stimulus. Hick’s Law predicts that the time, and by implication, the effort it takes to make a decision increases with the number of options. Behold my very non-fancy chart:

The formula for Hick’s Law is RT=a+b  log2 (n)

Breaking this down, RT is reaction time, n is the number of stimuli present, and “a” and “b” are constraints that depend on the task and condition.  A could be signing into the LMS and b could be finding the correct course to self-enroll in.

Hick’s Law is important when response times are critical. But it’s broader than that when applying this to L&D. For example, becoming the best widget maker takes time to learn how to build the components. By being able to deconstruct the components of the widget, it increases their ability as a widget maker. Hick’s Law works best for fast decisions without grave consequences. Hick’s Law does not apply to complex decision-making or decisions requiring reading, scanning, searching, or extended deliberation.

When response time is critical, keep it simple silly (KISS). Systems with fewer options are almost always rated as easier to use and more satisfying than similar products with more options. This reminds me of selecting faces for feedback versus filling out a survey. It turns out that abundant choice not only slows response time, it often leads to feelings of dissatisfaction and frustration.

Another implication of Hick’s Law is the need for emphasis. When a few items are designed to stand out, like a critical reporting function or frequently used option, you nudge users to where to select so response times can be quickened. This is why you don’t want critical functions like, let’s say, an ejection seat handle in an airplane mixed in with a bunch of other controls. If a pilot needs to eject, they need every millisecond we can give them. Making an option stand out achieves its objective.

Lastly, Hick’s Law underscores the need to reduce noise and distraction in decision-making contexts. This reminds me of Cammy Bean’s “clicky clicky bling bling”. When choices are presented with many distractions, the distractions effectively act like additional choices, slowing response time. This makes our courses full of sludge.

So, what are some instances you should apply Hick’s Law to in your learning experiences?

Read More

What can a past perspective tell us about learning experiences?

As many of you may know, in addition to my full-time gig in ID at The (trademark rejected) Ohio State University, I’m also a Ph.D. student in Learning Technologies.  As I’m finishing up my coursework and transitioning to candidacy, I’ve tried to take some time for reading. This reading is often targeted in the learning technology/education realm and in addition to research articles, I often like looking at books.

I thought I’d start with what I already have so this evening I took a browse in my living room looking at my bookshelf.   A former supervisor often would give me books from his office (he at one time had a Picasso painting on the floor, no joke).   While I appreciated these books, sadly, I often didn’t take the time to take a good look at them. This evening I pulled Basic Principles of Curriculum and Instruction by Ralph Tyler off my shelf and cracked it open.  I knew this was a gifted book from my previous supervisor due to the Half Price Books sticker on the cover (I’m pretty sure the place was his second home). I opened the cover to check the publication date, 1949. Flipping through the chapters I came across the chapter, How Can Learning Experiences Be Selected Which Are Likely to be Useful in Attaining These Objectives.  As I read the pages, not only did Tyler provide a definition for learning experiences but also general principles. I thought I’d share it with you in hopes of having a conversation about this.

According to Tyler, “the term learning experience refers to the interaction between the learner and the external conditions in the environment to which he can react. Learning takes place through the active behavior of the student; it is what he does that he learns, not what the teacher does.  It is possible for two students to be in the same class and for them to be having two different experiences.”  


A few things stand out to me in this definition:

  • There was no eLearning in 1949
  • I’m assuming “he” was a more general term for people

I’d also like to unpack this definition and share my internal inquisitive dialogue, specifically the external conditions the environment for the reaction.  Going back to my interest in learner engagement, I can see this translating the behavior involved in the learning experience. What are they doing? How is the learner an active participant?  What did these interactions look like? Reading further, Tyler suggests the teacher can “provide an educational experience through setting up an environment and structuring the situation so as to stimulate the desired type of reaction.”  Does this mean that learners can have different experiences in the same conditions?

Now let’s dive into Tyler’s general principles in selecting learning experiences:

#1: “For the given objective to be attained, a student must have experiences that give him an opportunity to practice the kind of behavior implied by the objective”

When I first read this one, I thought it may imply those pesky learning styles that some folks may still believe.  Tyler provides the context in authentic learning, how does this learning objective tie to a behavioral/ performance outcome? 

#2: “Learning experiences must be such that the student obtains satisfaction from carrying on the kind of behavior implied by the objectives”

This one was quite interesting too.  Again, I wonder what Tyler would think about all of the eLearning out there which has the learner constantly hitting next.  What Tyler meant about this one is to know your audience. Make the experience enjoyable. Don’t insult their intelligence. Don’t bore them to death.  

#3: “The reactions desired in the experience are within the range of possibility for the students involved”

Tyler’s intent with this one is the appropriate level.  How can you build experiences that are Goldilocks, just right for the learner? 

#4 “There are many particular experiences that can be used to attain the same educational objectives.

This one alludes to not doing the same thing over and over again.  Challenge the status quo in your learning experience. Keep the learner on their toes. 

#5: “The same learning experience will usually bring about several outcomes”

When learning experiences are crafted with care, outcomes are also multi-dimensional.  This is something I try to do with my work at OSU, I challenge faculty to think about the transferable skills students will get from their class.  What can they learn in this course that helps them in their future career path, no matter where they go?

This is the tip of the iceberg for this chapter as Tyler expands more on these concepts.  What do you think of this? Are Tyler’s perspectives still in line with instructional design?  Sound off below! 

Read More