BlendKit 2014 Reflection Statement

It’s done! I have completed my course BlendKit 2014: Becoming a Blended Learning Designer https://blended.online.ucf.edu/blendkit-course/

This course started out as a “good thing to learn and do” and quickly became a personal obsession! My original goal was to add a few more design elements into a course I teach. However, over time I kept asking myself more questions about the learning design. A passionate lifelong learner, I knew from previous experience and courses there was a lot to learn. It does feel like a Pandora’s box has been opened now!

The reason for this is that my primary orientation since 1997 (when I built my first online course) has been ‘how can I do this online’. Today, my question is, ‘what is the BEST way to teach this’? My initial question was right for the time and the results of my experiences were positive. The learning outcomes of the online teaching were the same or better than what we got in the classroom. Additionally, we were able to serve more learners across the country. It is helpful to note here that I coordinate and teach in a post-graduate college certificate that trains career counsellors across Canada. That being said, approximately two-thirds of our learners now live within a 3-hour radius of the campus and are open to travelling a few times per term to campus to attend class. The combination of the student profile plus by shift in question led me here.

The documents created for the portfolio review are all housed in a Google Drive folder titled: BlendKit Portfolio. Anyone with this link can view all of the files:

https://drive.google.com/folderview?id=0B7n5jOQLzhA-YnVMRVJnWUFpam8&usp=sharing

In the middle of this course, many design ideas and strategies came together. They are best represented in this presentation:

https://docs.google.com/presentation/d/1cKNzKz_hCpo3CmgNO8BXJ2Ete_TULAcbvHKTzFfsYGk/edit?usp=sharing

A ‘light bulb’ went off and I was able to get a gestalt forming in my mind of what the classroom, webinar and LMS elements and ratios needed to be. Once I had this formed, it was much easier to complete the course.

Course Expectations

The expectations are articulated in two documents:

A Guide to Getting Started: https://docs.google.com/document/d/1G8yzwoCF9Vbj67xAQcHZoY3SV4ettG2tFIaIBsiSna8/edit?usp=sharing

A Guide to Understanding How This Course Works: https://docs.google.com/document/d/1mnRtYIoZBxpgmikNTZRsun1zDKlBL_tTXTjf6Uyyn90/edit?usp=sharing

Feedback from students indicates that the second document is particularly important. Prior to entering the course, most students do not have an understanding of how it is possible to learn in this course. This simple sheet helps them out.

Learning Objective

The course “blueprint” was very difficult to create and took a lot of time. However, now that it is done I am excited about sharing it with students. It really helped me to understand how I integrate the course learning process. I also believe that students will find the MixMap helpful for understanding the course design:

https://docs.google.com/document/d/1351nYFxOnQqxnDgLeC5sHKbqpDVXN6LpHSs0mJtvhMA/edit?usp=sharing

The official course outline is here:

https://drive.google.com/file/d/0B7n5jOQLzhA-ZmpjQzdVMVZVMVU/edit?usp=sharing

This is an official course outline that meets college standards. Unfortunately, the layout and components have been simplified over time, I don’t find this current design helpful for students.

Learning Activities/Content

The Week 2 Module in this course is a useful example of the overall course design. It only includes the asynchronous online web content and the synchronous webinar.

Here is the core of the week’s activities: https://docs.google.com/document/d/1pq0mCrdTPzVsR0BpvBIJ0ZpxG02t6zT9qSLvNrzJPOE/edit?usp=sharing

Here is the interaction design:

https://docs.google.com/document/d/1_iI9HqCC3SuiFLtF1BmYuaDPmcczkf13sM7N2rFvbgo/edit?usp=sharing

Here are the Webinar slides for Week 2:

https://docs.google.com/presentation/d/11D0fkm3PuXEEIIAuFnSUhkjLa2m-Z6ZMSLOfNyZg7gI/edit?usp=sharing

The classroom pieces come later in the term and I am still redesigning those.

Learning Assessments

The Week by Week schedule is what students pay the most attention to, it drives the course learning and assessment: https://docs.google.com/document/d/1PynUGh9uRnId7tqtC1sWJ2ApgfB3Kfn5Ma7lCDd68Dg/edit?usp=sharing

The Instructions for the Course Assignments are located here: https://docs.google.com/document/d/13w6cjEEYzM8KKRNKjaWa8CB1_AcMuf0OjDjDmm-yBnY/edit?usp=sharing

The Marking Rubrics are here:

https://docs.google.com/document/d/1ZjyFOmh2po4nDVTTD5m0GmcmQDEzRBfPFcExj15QVwc/edit?usp=sharing 

Technology Tools

There are a variety of technologies used in this course. The main one is the learning management system: Desire2Learn (D2L). My greatest challenge over the years has become the LMSystems. In 1997 I started working in WebCT Beta and loved it. It was a simple design and layout that worked well from the student perspective. Over the past decades I believe that overall the LMS packages have become less about the learner and more about the institution and registration management. In my opinion, the LMS is a barrier to learning. On the longterm, I hope to free my courses of the LMS and go to a simpler CMS like WordPress.

I will be using the GoToTraining platform for the Webinar. This allows for presentation of slides, recording of webinars, text chat and voice interaction, polling and distribution of materials. Google Drive (Docs) will be used for students to sign-up for their learning partners (as well as for the Case Study Presentations later in the course). All of my personal files that I have created for the course are housed in DropBox. Students can directly link this folder with their own DropBox account (if they have one) or download the files directly to their device.

Ethical/Legal Practices

The Career Development Practitioner Certificate program runs in the Province of Ontario and is legally mandated to comply with the Accessibility for Ontarians with Disabilities Act (AODA): http://www.aoda.ca/ All of the documents that sit in the learning management and content management systems are based on templates that comply with the AODA standards.

Additionally, we have an Accessibility Office which provides excellent support to help ensure our students success by adapting the learning process to their specific needs

The CDP program educates professionals who are a part of a professional practice. The program is designed to ensure students learn the ethical foundations necessary to embrace the highest standard of professional ethics in the field in Canada. These standards are identified here: http://career-dev-guidelines.org/career_dev/

Evaluation and Review

At the time of writing, I have completed the Blended Course Implementation Checklist / Before the Course Starts form: https://docs.google.com/document/d/1qkhsxY1d5zvkoXQGuj5IzFfoVwW87jmuJt-Tn2BzmEo/edit?usp=sharing

I am working on using the Peer Review sheet for myself and have a commitment from a colleague who teaches in the CDP program to review my course using this template.

We just finished a review of the technology we use in the program, an extract is available here: https://docs.google.com/presentation/d/1w-JWFzh9SyzSn6TnbVFLgUHZ4ZmVrql9yUOELUitJfQ/edit?usp=sharing

Finally, we have done a variety of “Student Appraisals of Teaching” in the program. This process is being reviewed. I participated in a pilot study last winter and am looking forward to a program wide rollout to take place next year.

Conclusions:

I love lifelong learning and this course has been a joy to participate in! It has also been an excellent reminder of the ‘stress’ that comes with trying to complete all of the assignments in a short period of time when one has work commitments and children to parent. I have appreciated the perspective that I have gained from this standpoint. Most importantly, I believe I have a strong blended design to implement next fall!

Rob Straby



 What our learners are saying to us about the technology

In my previous post I noted that there is insufficient research that looks at whether the technologies being used do what we want them to do. How easy are they to access? Are they benefiting the students learning or getting in the way? Do they help us to connect or leave us feeling distance and cold? These and many more questions keep my up late at night. To understand what the experience of learning with technology is like in the Career Development Practitioner program, we have run surveys every few years. There are interesting insights from this information. You can view extracts of the survey here:

The first piece that stands out to me is that in all the surveys we have run, students’ access to faculty is, on average, same as or greater than a ‘traditional classroom’ delivery. Their connection to other learners is also higher than one would expect. When I think about it, this does make sense. Even when I took graduate seminar courses, I only got to know a few students well, there where many others whom I did not know anything about what they thought about the course. In the discussion forums, learners share more detailed and intimate information than I have witnessed in a classroom.

The feedback that has helped us to change what we do is the research about accessing specific technologies. Although I know there were Webinar access issues in our last term, I did not realize how far reaching they were. Over half of the students had problems accessing one of our webinar platforms. This is important feedback enabling change. At our most recent faculty meeting, we agreed to drop the weaker system and run all of our webinars in one system.

This is the type of information I feel can help us improve our delivery!

 



 Listening to the User: We need to do more of this!

One of the key elements that I think about a lot in blended learning is the quality of the technology we use to do things. In my review of the blended learning course material and external links I perceive there is a gap. There is insufficient research that looks at whether the technologies being do what we intend for them to do. In a similar vein, are the decisions we make to choose specific systems pedagogically driven? Do we know what students really think about the technology we require them to use in order to meet their own learning goals?

There are some instances of student feedback, an example of this is “Course and Learning Management System Project Report and Recommendations” prepared by the University of Texas at Austin (click here to access the file).

Prior to adopting a new LMS, they conducted surveys of five different LMSs. They used a variety of strategies for the review. What stands out for me, is the choice they made to survey students. The chart from page five is most enlightening. There were statistically significant differences in the user experience. For example, one of the systems reviewed was easy to navigate than the others. In my experience with my own students, finding out where things are in the first 2 weeks of a course (whether blended or fully online) is the most important factor affecting how long it takes for the person to start to learn the content (as opposed to learning the technology). I would like to see more research in this regard!

 



 Blending Content and Assignments

This week I am continuing on the journey of re-writing courses that I teach. This post provides a reflection on my insights to date. The challenge question for me is “What elements of my course would improve the learning outcomes if I put them back in the classroom?” For the past 5+ years, all the content and assessments are in a 100% online environment.  In reworking the course design, I have been drawing on the BlendKit Reader. The structure of learning activities suggested by Littlejohn and Peggler (2007) in “Preparing for Blended e-Learning” is helpful. They outline 5 learning activities based on Laurillard’s Conversational Model. I realized that I use four of these approaches. My goal is to enhance the learning activities by adding these classroom components to the course. The table below outlines this design.

Table 2: Assessment Typology

The formal assessments are online. This will continue to be the case for the blended course design. In the past, I have offered informal assessment online. My goal with the new design is to add informal formative assessment into the classroom delivery. The table below provides an overview of this structure.

Table 1: Type of Learning Activity

My goal is that this blended design will enhance the learning outcomes beyond either the previous classroom or online programs!



 Reflections on Assessments in Blended Learning Design

In reviewing the literature on the use of assignments in blended learning (see for example: https://blended.online.ucf.edu/blendkit-course-blendkit-reader-chapter-3/) I have affirmed a number preferences in my approach. The first part to be clear about is that at this time I prefer to engage a constructivist learning design for the counsellor education program I teach.

When I review post-secondary courses, whether classroom, blended or fully online, I am surprised at the amount of formal quizzes that are used. The rationale is clear in that these courses are scalable. You can have large numbers of students enrolled. My choice to focus on project based and reflective assessment approaches comes with a challenge. These assessment approaches take longer to mark.

My favourite approaches to assessment are client problem solving projects. These require the learner to integrate the majority of skills learned in the course through an applied counselling project with a client. Rather than simply write these assignments as a paper, they are required to present them to their peers. They learner must demonstrate both counsellor skills and the ability to reflect on practice.

Until I discover something better, I plan to continue with this approach, it works!

 



 Designing Blended Learning Interactions

Over the past week I have been reading and reflecting on the types of interactions I create in my courses. One of the reflection pieces is from my BlendKit2014 reading from Kevin Thompson:

https://blended.online.ucf.edu/blendkit-course-blendkit-reader-chapter-2/

In this reading there are two key types of interactions considered:

  • Student to student and
  • Student to instructor.

I find that this is an insufficient consideration for some of my courses. The learning outcomes are designed to help learners develop practical skills in career counselling. To help this I believe there is a third dimension required: student to community.

These have students apply skills learned in the class to people in their networks outside of class.  The people are voluntary clients. The set-up for this process runs through a scaffolding experience as follows:

  1. Students review reading material in preparation for a webinar;
  2. We have a webinar on a topic, introduce a skill and students are able to ask questions about the process;
  3. Over the following week, students work with peers in the class (we call these learning partners) to practice the skill.
  4. This leads to a reflection oriented ‘Action Learning Log’ posted to the class discussion forum.
  5. We have another class webinar where we debrief the Action Learning Logs.
  6. Students then apply the technique in the work with a volunteer client;
  7. This leads to a second reflection oriented ‘Action Learning Log’ posted to the class discussion forum.
  8. We have another class webinar where we debrief the Action Learning Logs.

This cycle continues through the course as the learners develop their knowledge and skill. I have found this layering approach helpful for teaching professional interpersonal skills and knowledge.



 Why Blended Learning Now?

In the early 1990’s I started teaching a traditional face-to-face classroom delivery for a Concordia University College. Following that I moved to Conestoga College. I set-up the Career Development Practitioner Post-Graduate Certificate. We evolved from the classroom to a hybrid environment and then to an online program. Here a point form history:

  • 1995: Program started with traditional in-class delivery
  • 1996: Survey assesses potential for distance education in Ontario
  • 1997: First online course using WebCT beta
  • 2000: All courses have a WebCT site
  • 2004: Pilot project delivery of Teleclasses
  • 2005: All courses available by distance education
  • 2008: Initial pilots of a synchronous web-based platforms
  • 2008: Program transition from WebCT to Angel
  • 2009: Adoption of synchronous eLearning platform
  • 2013: Program transition from Angel to D2L

The program has been 100% online for five years. I am considering bringing parts of the program back into the classroom for a few courses. There are two reasons for this. The first is research. The most profound research I have seen is a meta-analysis (a research study about research studies). The U.S. Department of Education published an “Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies” in 2010. They state “Students in online conditions performed modestly better, on average, than those learning the same material through traditional face- to-face instruction” (p. xiv) and, notably, “Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction” (p. xv).

I did not see any differences in student learning outcomes when I compared marks between those who took the program 100% online and those who took it in a blended format. I am wondering if there are things I can do in a blended format to increase the learning outcomes.

The second piece that has influenced my thinking in this regard is conversations with graduates. The graduates who had a face-to-face classroom component built lasting relationships with their peers that have carried on over many years. This is important, as they have been able to help one another with their career management.

I am facing a challenge right now. What elements of the online program would enhance the learning outcomes if they I returned them to the classroom. I have a different challenge than a lot of my peers.

Chapter One of the BlendKit reader cites the work of Carman (2002) who identifies five elements of learning. These are:

  • live events,
  • self-paced learning,
  • collaboration,
  • assessment and
  • support materials.

When I review these, it is clear to me that they can happened either online or in a classroom. Yet, there is no rationale that I can find for suggesting one gets superior outcomes in either mode.

My primary learning goal this spring it so sort out a rationale for how to discern what goes where!



 Reflections on Learning, eLearning and Blended Learning

This blog focuses on my passion for career development. It also concerns the education of career development professionals. I have been also impassioned about learning throughout my life. As technology has evolved, I have morphed my teaching from the classroom to the online world. The evolution of online learning has been of benefit to my students and it has enabled my to extend the depth of my own knowledge and skills.

I have participated in many online courses. Yet, my participation has always been ‘within’ the course walls. This spring I enrolled in BlendKit2014: Becoming a Blended Learning Designer. It is a MOOC focused on blended learning. For this course I decided I would ‘public’ my learning process in my blog. For those of you who follow this blog for the ‘career’ pieces, don’t worry, that will continue to be my primary focus!



 Through the Looking Glass ~ A Second Look at Assessments

Last week I suggested that it is helpful to distinguish between two types of assessments: formal and informal. Career professionals sometimes have a bias that formal tools are better than informal ones. This bias extends to the clients I have met as well.

There is a mystique about formal assessments and clients often assume that the “test must be right”. I have received hundreds of phone calls in my career from individuals asking for “a test that will tell them what they should do”. This is important to be aware of.  These perceptions about formal career assessment can get in the way of helping a person sort out what they want to do next in their career. A person desperate for an answer to sort out their life needs more than a ‘test’. In my experience there are a many other issues that need understanding first. This requires an interview to unearth the deeper issues going on that led to this disparate state.

If a career exploration process is necessary, there are a couple of questions that need to answered:

  • Is this the best way to assess their needs? Are there informal approaches that would be better suited to the client’s situations?
  • Does the instrument fit the client profile? Was it developed and normed for an adult client if they read at a Grade 8 level and came to Canada as a refugee 3 years ago?
  • How reliable and valid is the instrument? Does it mean the standard statistical measures? Does the research prove that the instrument will deliver the same results on different occasions? Can it predict satisfaction with a career choice?
  • Does the instrument address the types of questions the client has?
  • Is there adequate time to deliver this approach? The process requires three steps. This begins with an initial consultation, the administration and then an explanation and interpretation.

Do you have a positive response for ALL the questions above? Then a formal assessment may be helpful as a part of the career exploration process. If you do choose to apply a formal approach, in my experience it will be more helpful if you include one or more informal strategies.

Finally, we need to keep in mind that no assessment approach will answer the client’s questions alone. They will always have to go out into the world and research career opportunities.



 What Do Those Career Assessments Do?

I get bombarded with questions from both clients and practitioners about career assessments. Most of the time the words that are (unfortunately) used are “career tests”.  A discussion about this recently took place on LinkedIn group and I feel compelled to address that in more depth here.

The first thing to clear the air about is that there are no careers “tests”. The tools used in the career development process may provide an assessments of an  individuals approach to work. There are two types of approaches professionals in the field will use, either alone or in combination.

There are “formal” and “informal” approaches. Formal refers to the use of a tool that involves a questionnaire or survey and compares the results to others in the population. Informal approaches are often based on interview strategies. Formal approaches can include the assessment of many things such as interests, personality, values, abilities and more. Informal approaches include structured interviewing strategies, sharing of stories, visioning, journaling, etc.

When someone asks about “career tests” they are referring to the formal approaches. They are seeking one of two types of assessment tools as follows. What is important here it to understand what these tools do and do not do as they measure different things.

1. “Career Interests”: The Holland Self Directed Search and related material (e.g. Career Key, Strong Interest Inventory, Jackson Vocational Interest Survey, etc.) all assess career “interests”.

2. “Personality Type”: The Myers-Brigg Type Indicator, Personality Dimensions, True Colours, DISC, etc. assess personality “type or temperament”.

Career interests and personality type are *different* approaches and may be confusing to understand. Career interest assessments suggest career areas a person would enjoy, often stated as occupational titles (e.g. banker, baker, candle stick maker). Personality type will provide information about ‘how’ a person will approach a career area. Personality type cannot determine interests such as which occupational type a person would enjoy. Type and temperament tools sometimes provide suggestions to support this. They are not designed to do this. Please keep this distinction in mind. In a career centre that uses formal tools such as these, it is typical for an assessment process to use both interest and type tools.

In future posts I will address why the use of formal approaches is limited and what approaches can provide more help.