Calibrate Research Base
When implemented with fidelity, a sound instructional design for any training and development program can have a significant impact on outcomes.
The instructional design of Medallion Learning's intelligent learning platform, Calibrate, is built on a rich research base that has been validated by efficacy in a wide and varied set of implementations. Take a look at this research base summary to learn how to introduce a sound instructional design into your training and development program.
RESEARCH FOUNDATIONS OF CALIBRATE
Medallion Learning’s Calibrate instructional model combines best practices in both instruction and assessment with a “smart” process that continually modifies instruction to meet the unique needs of individual learners. The result is that all Calibrate learners achieve better results in less time, regardless of the nature of the content. The key to Calibrate’s effectiveness is in its sophisticated instructional model which is derived from the work of leading instructional designers and theorists.
THE CALIBRATE PROCESS
The Calibrate instructional model includes the following components:
Pre-Learning Assessment: An assessment is given to each learner to determine what he/she already knows using the most discriminating and reliable items drawn from a large test item bank. Test items are constantly adjusted and exchanged as more data is collected in order to build exchanged as more data is collected in order to build a robust and predictive Pre-Learning Assessment.
Personal Learning Path: A Personal Learning Path is prescribed for each learner based on the results of their Pre-Learning Assessment. This Personal Learning Path separates what a learner knows from what they still need to learn and allows the learner to focus on areas of need. The individualized nature of the instructional process allows the learner to proceed at a pace and in a location that best suits them.
Preparation for Learning: The learner is prepared and motivated to learn through the following critical steps: a. The learner is provided with an overview of the Unit (topic and objectives) as an advance organizer to learning b. The learner is provided with a context for the instructional material through the following methods: i. Recall of prior learning is stimulated ii. Relevant prior knowledge is discussed (e.g., prior experience, prior concepts) iii. Correlations between new information and prior knowledge are drawn c. The expectation for learning is established—a description of what the learner will be expected to do after instruction is provided
Instructional Delivery and Guided Practice: Content is presented in a systematic and sequential way through a variety of modalities (video, audio, and text) and is reinforced by a number of supporting tools to accommodate different learning styles. The learner is continually engaged with the material as new content is explained, modeled, or demonstrated and he/she is supported in encoding information for long-term storage through the use of examples, case studies, mnemonics, simulations, etc.
Independent Practice: Learners are provided with numerous and diverse tools to independently practice newly acquired knowledge and skills. Performance is directly elicited and learning is reinforced through immediate feedback and error correction. In addition, extension activities present the learner with opportunities to expand their knowledge and apply what they have learned.
Validate Learning: In the Calibrate instructional model, progress is continuously monitored and instruction is adjusted accordingly. Specific feedback on the learner’s performance is provided through opportunities for learners to check their understanding, and learning is formally measured through unit and final course assessments.
Instructional Design is the practice of creating “instructional experiences which make the acquisition of knowledge and skill more efficient, effective, and appealing” (Merrill et. al., 1996). As a field, instructional design is historically and traditionally rooted in cognitive and behavioral psychology. While sophisticated in detail, the process generally consists of (1) determining the current state and needs of the learner, (2) defining the end goal of instruction, (3) delivering instruction, and (4) measuring the outcomes. Instructional outcomes are directly observable and scientifically measured.
The Calibrate instructional process is informed by pedagogically (process of teaching) and andragogically (adult learning) tested theories of learning and takes place in a self- paced, computer-based environment.
The instructional model used in Calibrate is based on the ADDIE model (Reiser, R. A., & Dempsey, J. V., 2012) with the five phases: (1) analysis, (2) design, (3) development, (4) implementation, and (5) evaluation.
Driven by Subject Matter Experts
Analyze: This stage includes gathering or constructing information about the audience (learner); the tasks to be completed (content); and the overall goals of instruction. The author, a subject matter expert (SME), then organizes the information in preparation for writing.
Design: In the design phase, the information gathered through analysis is fitted to the instructional model -- the base document (e-book), learning objectives, and test items are written; instructional tasks and demonstrations are identified and scripted. Care is taken to ensure all material is compliant with Medallion's Principles of Instruction.
Led by Medallion Learning Instructional Designer
Develop: In the third phase, the work of the author (text, scripts, activities, and tools required to meet the goals identified in the Analyze phase) is entered into the learning platform using a templated authoring tool.
Implement: The developed content is then implemented, allowing the designer to informally test the material and determine its functionality for the intended audience.
Evaluate: The final phase ensures the material is effective for the targeted learner. The product evaluation, containing both formative and summative assessments, is critical to the design team because it provides data used to alter and enhance the design. The ADDIE model is an iterative process meaning at each stage, the designer can assess the project's elements and revise them as necessary.
Medallion Learning's Calibrate instructional model is grounded in the work of the most influential learning theorists of our time, including Robert Gagné, Robert Mager, Benjamin Bloom, and David Merrill.
While the work of several learning theorists is incorporated, the Calibrate instructional model is primarily designed around the cognitive information processing model of Robert Gagné (Gagné & Medsker, 1996). Gagné is perhaps best known for his "Conditions of Learning" where he postulated that different types of learning exist, and that different instructional strategies are required to elicit the various types of learning.
Gagné pioneered the science of instruction during World War II when he worked with the Army Air Corps training pilots. He went on to develop a series of studies and works that simplified and explained what he and others believed to be "good instruction." More recently, Gagné has applied his instructional theory to the design of computer- based and multimedia-based learning (Gagné, Robert M., et al., 1981). Gagné's work is considered by many to be the foundation for much of the science of effective instruction today .
Robert Gagné (Gagné & Medsker, 1996) identified four phases of instruction and nine instructional steps (events) that are essential for successful learning and knowledge retention. These steps (see below) should satisfy or provide the necessary conditions for learning and serve as the basis for designing instruction and selecting appropriate media (Gagné, Briggs & Wager, 1992).
Gagne's Four Phases of Instruction and Nine Instructional Steps
Preparation (the student is prepared to learn) 1. Getting the learner's attention (reception) 2. Stimulating recall of prior learning (retrieval) 3. Describing the expected outcome of the learning experience (expectancy) Presentation (the student is presented with the material to be learned) 4. Providing a context for the instruction (selective perception) 5. Delivering instruction and providing guidance (semantic encoding)
Performance (the student engages with the material and is provided with feedback on his/her performance through informal assessment) 6. Eliciting performance of the material (responding) 7. Providing feedback on the performance (reinforcement)
Validation (the student's performance is formally assessed to determine the effectiveness of the learning experience) 8. Assessing performance (retrieval) 9. Encouraging retention and transfer (generalization)
Sequencing of Instruction
Gagné further suggests a hierarchy of complexity for intellectual skills that provides a basis for the sequencing of instruction: stimulus recognition, response generation, procedure following, use of terminology, discrimination, concept formation, rule application, and problem solving. The implication here is that learning will be enhanced by the appropriate sequencing of instruction as some higher order functions may require prerequisite skills (be identified through the task analysis). The Calibrate instructional model closely follows Gagné's phases of instruction and instructional steps.
HOW CALIBRATE IMPLEMENTS GAGNE'S FOUR PHASES OF INSTRUCTION
Prerequisite knowledge is ensured through the sequential delivery of content.
Recall of prior learning is stimulated.
Relevant prior knowledge is discussed (e.g., prior experience, prior concepts).
Correlations between new information and prior knowledge are drawn.
The learner is provided with an overview of the Unit (topic and objectives) as an advance organizer to learning.
A list of Learning Objectives is provided.
The expectation for learning is established.
A description of what the learner will be expected to do after the lesson is provided.
The student is motivated to learn.
The learner is provided with a context for the instruction.
- Content is delivered and demonstrated.
- At a range of cognitive levels.
In a sequence that facilitates learning.
New content is chunked, explained, and demonstrated.
Material is delivered through a variety of modalities (audio, video, and graphics) to accommodate different learning styles.
The learner is engaged with the material.
Learners are helped to encode information for long-term storage.
Guidance strategies (case studies, examples, mnemonics) are used.
Learners are provided with frequent interactive activities to experience the content first hand.
- Performance is directly elicited.
- New skills are practiced.
Correct understanding is confirmed.
Labs allow for Independent Practice of learned skills.
Extension opportunities are provided to emphasize retention, retrieval, and generalization.
- Specific feedback on the learner's performance is provided through quizzes.
- Learning is reinforced.
Errors are corrected.
Formal Assessment is made through Unit Tests and the Course Post-test.
Progress is measured.
Arguably the most well-known and respected figure in the field of training and human performance, Mager is credited with revolutionizing the performance improvement industry with his groundbreaking work. His Criterion Referenced Instruction (CRI) framework has been used effectively in many professional and technical training programs and particularly in self-paced multimedia courses. CRI is aimed at adult learners who can pace themselves and evaluate themselves with assessment tools incorporated into the courses. Mager was influenced by Gagné, Knowles, and Rogers, and has incorporated concepts of their work into CRI.
The essence of CRI is the construction of Learning Objectives. A learning objective is a description of a performance that learners need to be able to exhibit to be considered competent. It describes an intended result of instruction, rather than the process of instruction itself (Mager, R.F., 1984).
Clearly defined objectives provide learners with a way to organize their learning efforts. With clearly stated objectives, learners are provided with a path and are consequently better able to decide what activities will help them achieve mastery of the content at hand. Without clear objectives there is no sound basis for the selection or designing of instructional materials, content, or methods. His point is that if you don't know where you are going, it is difficult to select a suitable means for getting there.
Clear objectives also make it possible to determine whether the objective has been accomplished. Tests are the mileposts along the road of learning that inform both instructors and learners about learning progress. Without clear objectives, tests can be misleading and uninformative. Further, valid test items can only be selected or created intelligently when instructional outcomes are explicit.
Calibrate's instructional model directly incorporates the work of Mager. While not always explicit, learning objectives guide the instructional process and are constructed to describe:
What the learner will do: This is stated in the form of a verb and is observable, either overtly or covertly.
Under what conditions: In order to clearly state an objective, it might be important to also state the conditions under which the learner must demonstrate their mastery of the objective. What will the learner be allowed to use? Or, what will the learner be denied?
To what degree of success: Specifying the target level of performance for an objective indicates precisely to the learner the quality of performance necessary to master the objective. Further, it provides the standard against which the (1) effectiveness of instruction and (2) the performance of the learner can be compared.
Bloom is perhaps the most pre-eminent educational psychology theorist, having made significant contributions in both instructional and cognitive processing theory. Calibrate integrates two of Blooms important theories: Mastery Learning and Bloom's Taxonomy of Learning Objectives.
The term mastery learning was coined by Benjamin Bloom in 1971. Mastery learning has little to do with specific content, but rather is a description of the process of mastering particular learning objectives. Mastery learning focuses on overt behaviors that can be observed and measured and requires well-defined Learning Units organized into smaller, sequentially organized Learning Objectives.
Mastery learning may be implemented as teacher-paced group instruction in a classroom setting, as one-to-one instruction in a tutoring setting, or as self-paced learning with programmed materials, as it is in Calibrate. It may involve direct teacher instruction, cooperation with classmates, or independent learning.
Individualized instruction has some elements in common with mastery learning but, as in the Calibrate model, individualized instruction eliminates group activities allowing more capable or motivated learners to progress ahead of others and thus maximize instructor interaction (when in a classroom setting) with learners who need the most assistance.
In a mastery learning environment, the learner is provided with frequent and specific feedback by using diagnostic, formative tests designed to correct mistakes along his/her learning path. Assessment in the mastery learning model is used less as a measure of accountability and more as a source of evidence to guide future instruction. Final course outcomes are determined using criterion-referenced assessments as learners are not competing against each other, but rather competing against themselves in order to achieve a standard.
While the Calibrate instructional model incorporates most of the important concepts of mastery learning, there is one significant difference. In mastery learning in its purest form, each learner must master each learning unit before proceeding to the next. While mastery is certainly desirable in the Calibrate model and every attempt is made to achieve mastery, it is not required. In this sense the Calibrate model is more "mastery teaching" than "mastery learning."
Bloom's Taxonomy of Learning Objectives
Bloom's Taxonomy divides educational objectives into three "domains": Cognitive (knowing), Affective (feeling), and Psychomotor (doing). Within the domains, learning at the higher levels is dependent upon having attained prerequisite knowledge and skills at lower levels.
The original intent of creating the taxonomy was to improve communication between educators who were designing curriculum and assessments. More recently, Bloom's Taxonomy has been used to (1) motivate educators to focus on all three domains, creating a more holistic form of education, and (2) to ensure delivery of instruction across the entire range of levels: knowledge, comprehension, application, analysis, synthesis, and evaluation.
While Calibrate instruction focuses exclusively on the cognitive domain, Bloom's Taxonomy is used to guide subject matter experts as they write content and instructional designers as they develop tasks. This ensures that instruction is delivered at the full range of cognitive levels, either in the base content or in extension activities that typically focus on the higher levels of synthesis and evaluation.
BLOOM'S TAXONOMY OF INSTRUCTIONAL OBJECTIVES (COGNITIVE DOMAIN)
Knowledge: Recall of information
- Arrange, define, duplicate, label, list, match, memorize, name, order, recognize, relate, recall, repeat, reproduce
Comprehension: Interpret information in one's own words
- Classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, sort, tell, translate
Application: Use knowledge or generalization in a new situation
- Apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, prepare, practice, schedule, sketch, solve, use
Analysis: Break down knowledge into parts and show relationships among parts
- Analyze, appraise, calculate, categorize, compare, contrast, criticize, diagram, differentiate, discriminate, distinguish, examine, experiment, inventory, question, test
Synthesis: Bring together parts of knowledge to form a whole and build relationships for new situations
- Arrange, assemble, collect, compose, construct, create, design, formulate, manage, organize, plan, prepare, propose, set up, synthesize, write
Evaluation: Make judgments on basis of given criteria
- Appraise, argue, assess, attack, choose, compare, defend, estimate, evaluate, judge, predict, rate, score, select, support, value
M. David Merrill
In The First Principles of Instruction (Merrill, 2010), Merrill identifies the fundamental principles of good instructional design-independent of pedagogic strategy. Merrill's model has been used both as an instructional design model and as an evaluation grid to judge the quality of a pedagogical design.
Merrill describes five principles of instruction (Merrill, 2006) for promoting learning. The author postulates that learning is promoted when learners:
Engage in a task-centered instructional strategy (Task-centered).
Activate prior knowledge or experience (Activation).
Observe a demonstration (Demonstration)**.
Integrate their new knowledge into their everyday world (Integration).
Apply the new knowledge (Application).
About Merrill's Five Points of Instruction
(1) Task-centered learning is Merrill's central principle of instruction. By Merrill's definition, a task represents a problem that may be encountered in a real-world situation. Stated performance objectives that learners will be able to solve at the end of the learning sequence constitute a problem, providing they are "real- world" examples. A progression through problems of increasing difficulty is used to scaffold the learning process into manageable tiers of difficulty.
In the Calibrate instructional model, the courseware demonstrates what is to be learned in several ways: * The task or the problem the learner will be able to do/solve is described. * Learners are engaged at the problem or task level not just operation or action levels. * Units include a progression of problems rather than a single problem.
(2) Activation of relevant previous experience promotes learning by allowing learners to build upon what they already know. This is accomplished in the Calibrate instructional model through the systematic and sequential delivery of content and in the introduction phase of the instructional process ensuring that:
- The courseware activates prior knowledge or experience.
- Learners have to recall, relate, describe, or apply knowledge from past experience as a foundation for new knowledge (when appropriate).
- Learners have an opportunity to demonstrate previously acquired knowledge or skill (when appropriate).
(3) Demonstration is included in instruction through simulations, visualizations, modeling, etc., that exemplify what's being taught. Demonstration includes guiding learners through different representations of the same content through extensive use of a media, highlighting variations, and providing key information.
In the Calibrate instructional model, the courseware demonstrates what is to be learned in several ways:
- The model employs both primary and secondary instructional tools to demonstrate the content. If a learner doesn't understand the content after viewing the primary tool, alternative explanations using different media are available.
- Both examples and non-examples of concepts are provided.
- Learner guidance techniques are employed. For example, learners are directed to relevant information, multiple representations are used for the demonstrations, and multiple demonstrations are explicitly compared.
- Many different forms of media are used to enhance learning.
(4) Application requires that learners use their new knowledge in a problem-solving task and have the opportunity to practice and apply acquired knowledge or skill.
The application phase should be accompanied by feedback and guidance. Merrill describes several different types of practice:
- Information-about practice requires learners to recall or recognize information.
- Parts-of practice requires the learners to locate, name, and/or describe each part.
- Kinds-of practice requires learners to identify new examples of each kind.
- How-to practice requires learners to do the procedure.
- What-happens practice requires learners to predict a consequence of a process given conditions, or to find faulted conditions given an unexpected consequence.
The Calibrate instructional model includes guided practice as a fundamental part of instructional delivery. Attention is paid to:
- Requiring learners to use new knowledge or skill to solve problems.
- resenting corrective feedback on learner performance.
- Providing access to context-specific help and guidance for learners who experience difficulties.
(5) Integration in effective instruction occurs when learners are given the opportunity to demonstrate, adapt, modify, and transform new knowledge to suit the needs of new contexts and situations. Reflection through discussion and sharing is important to internalizing new knowledge information and giving the learner a sense of progress. Collaborative work and a community of learners are helpful in this regard.
There are plenty of opportunities to demonstrate learning in the Calibrate instructional model;; in fact, learning must be demonstrated at every level. However, the model is by definition, an individualized learning platform. As such, it does not easily lend itself to ongoing collaborative and community-based activities such as discussion and sharing. That said, opportunities to discuss and share information are provided through instructor, peer-to-peer, and online community portals.
Bloom, B. (ed.) Taxonomy of educational objectives: the classification of educational goals;; Handbook I: Cognitive Domain New York, Longmans, Green, 1956.
Gagne, R. (1962). Military training and principles of learning. American Psychologist, 17, 263- 276.
Gagne, Robert M., et al (1981). Planning and Authoring Computer-Assisted Instruction Lessons, Educational Technology, v21 n9 p17-21, Sept 1981.
Gagne, R. (1985). The Conditions of Learning (4th.). New York: Holt, Rinehart & Winston.
Gagne, R. (1987). Instructional Technology Foundations. Hillsdale, NJ: Lawrence Erlbaum Assoc.
Gagne, R. & Driscoll, M. (1988). Essentials of Learning for Instruction (2nd Ed.). Englewood Cliffs, NJ: Prentice-Hall.
Gagne, R., Briggs, L. & Wager, W. (1992). Principles of Instructional Design (4th Ed.). Fort Worth, TX: HBJ College Publishers.
Gagne, R, & Medsker, K. (1996). The Conditions of Learning, Fort Worth, TX: HBJ College Publishers.
Mager, R.F. (1984). Preparing instructional objectives. (2nd ed.). Belmont, CA: David S. Lake. Merrill, M. D. (2002). First principles of instructions, Educational Technology Research and Development, 50(3), 43-59.
Merrill, M. D. (2010). First principles of instruction, in C. M. Reigeluth and A. Carr (Eds.). Instructional Design Theories and Models III. Lawrence Erlbaum Associates.
Merrill, M. D., Drake, L., Lacy, M. J., Pratt, J., & ID2ResearchGroup. (1996). Reclaiming instructional design. Educational Technology, 36(5), 5-7.
Reiser, R. A., & Dempsey, J. V. (2012). Trends and issues in instructional design and technology. Boston: Pearson.