Assessment - Foundations Curriculum FAQ
WEEKLY FEEDBACK QUIZ: What is a weekly feedback quiz?
Weekly feedback quizzes are short quizzes delivered through SofTest that students will complete at home each week. While low stakes, these will help students examine how well they have learned the week’s material. These quizzes will be limited to key concepts that must be understood.
WEEKLY FEEDBACK QUIZ: Does theme content go into the weekly feedback quiz (ethics, leader, etc.)?
Theme content can go into the weekly feedback quiz if appropriate.
WEEKLY FEEDBACK QUIZ: How do I access my weekly feedback quiz?
Weekly Feedback Quizzes will be available for download in the student's SoftTest account with the following schedules:
-For first year Foundations Curriculum students, weekly feedback quizzes will usually be released Thursdays at 5 p.m. and close on Monday at 8 a.m.
-For second year Foundations Curriculum students, weekly feedback quizzes will usually be released Tuesday at 12 p.m. and close on Fridays at 8 a.m.
For more information about SofTest, refer to the Foundations Curriculum assessment technology page.
WEEKLY FEEDBACK QUIZ: What if my computer crashes while I am writing the weekly feedback quiz at home?
SofTest creates multiple encrypted copies of the assessment every 60 seconds on the exam taker’s hard drive. If an exam taker should have to restart their computer during an assessment, they will be required to enter a resume code to be allowed to re-enter the assessment approximately where they exited.
There is a default amount of time (currently set at 7 mins) an exam taker can remain out of an assessment and be allowed to continue the assessment. If an exam taker is out of the assessment for longer than that time they will still require a resume code, but will be forced to end their exam immediately after re-entering.
If your keyboard and/or mouse stops responding while in SoftTest, do the following:
Turn off your computer (press and hold the power button, if that fails, pull the power cord and battery)
Turn the computer back on.
When the ‘SofTest Exam Restart’ window appears, select RESUME/YES and enter resume code to return to the exam. You will be returned to within 59 seconds of where you left off.
If an exam taker requires technical assistance when they are writing their weekly feedback quiz:
During regular office hours: Monday to Friday 9 a.m. to 5 p.m.
Contact MD Program ExamSoft Support
Outside of regular business hours:
Contact Exam Taker Support via e-mail OR call 866.429.8889 Monday to Friday from 7:30 a.m .to 8:30 p.m. ET and Saturday from 8:30 a.m. to 6:30 p.m. ET (US Toll-Free)
WEEKLY FEEDBACK QUIZ: How do I know that the quiz is submitted properly?
There will be a confirmation screen on the students’ device congratulating them on successfully uploading their exam. See below:
WEEKLY FEEDBACK QUIZ: Is there a limit time to complete the weekly feedback quiz?
While there is no "limit" of time to complete the quiz, if you start it then go away from the program or it crashes, you will need to go back in within 10 minutes to be able to continue the quiz. Therefore, while there is no a limit, it is best to do the quiz in one sitting.
WEEKLY FEEDBACK QUIZ: Do I get the results of the weekly feedback quiz right away?
Yes, you will be able to see your score as soon as you complete the quiz. Also, once your quiz has been uploaded you will receive a detailed feedback report in your ExamSoft portal account.
WEEKLY FEEDBACK QUIZ: Do I get the answers to the weekly feedback quizzes questions?
The MD Program test committee reviewed the literature and audited practices both nationally and internationally regarding the provision of answers to online tests. It was the recommendation of this committee that for the weekly feedback quizzes, the best practice would be to provide students with feedback regarding those topics they did not answer correctly and provide the actual question stems but not their correct answers. The report you receive in your portal after the quiz will contain the learning objective IDs attached to all questions on the quiz, the question stems, and your results for each question.
WEEKLY FEEDBACK QUIZ: Practice quiz attempts
Students have access to two additional attempts of each weekly feedback quiz. These attempts are for learning purposes only and will not appear in the Learner chart. Students will receive feedback reports in their ExamSoft portal accounts, showing the learning objectives attached to all questions, and showing the question stems, after completing the quiz and each subsequent attempt.
WEEKLY FEEDBACK QUIZ: What if I forgot to do a weekly feedback quiz, or am sick and cannot write the quiz?
For first year students, weekly feedback quizzes are usually released weekly on Thursday at 5 p.m., and close Monday at 8 a.m., which gives students a window of more than 3 days to complete the quiz. For second year students, weekly feedback quizzes are released at 12 p.m. on Tuesday and close Fridays at 8:00 a.m., which gives students a window of approximately 4 days to complete the quiz.
There is no make-up time or justification of absence or deferrals if students miss the quiz. Weekly feedback quizzes you have missed will be indicated as such in your learner chart, and you might be asked to explain why that quiz was forgotten when you meet with your Academy Scholar. Please see assessment section of your course Portal for more information on assessment dates and requirements.
MASTERY EXERCISE: What is a mastery exercise?
Mastery exercises are one-hour assessments which are completed in class. Students bring their own computer devices to take the assessment in class on SofTest software. These exercises take place every one to three weeks, with flexibility in spacing depending on course structure, and carry more weight in determining the student has been successful in a course. There are 20 to 40 multiple choice questions in each mastery exercise. Multiple choice questions examine how well concepts are integrated and applied. Some questions will be cumulative – asking students to apply concepts from various courses or contexts.
The multiple choice questions are a mixture of knowledge, comprehension, and application questions, with an emphasis on comprehension and application rather than on knowledge which is the focus of the weekly feedback quizzes. The content for the mastery exercises is drawn from the following areas: TOPIC lectures, pre-week materials, and self learning modules, theme lectures. Also, information from CBL cases may be part of the mastery exercise but usually only in order to highlight material addressed in one of the lectures. Anatomy and ICE lectures will not be covered on mastery exercises as they are assessed elsewhere.
MASTERY EXERCISE: Are mastery exercises cumulative?
No, mastery exercises are not cumulative. The mastery exercises will only address material covered in those specific weeks or block. However, concepts covered in earlier weeks will certainly recur in subsequent weeks and as such you will be expected to know and recognize them in the new setting.
MASTERY EXERCISE: How many details do I need to know for a mastery exercise?
As many as you think is appropriate to feel that you have mastered the points covered and the learning objectives for the week. For example, memorizing each step of a complicated biochemical pathway is of less value to you than understanding the function of the pathway, the key molecules involved, and the outcomes if the process is dysregulated in some manner.
You should use your performance on the weekly feedback quizzes to determine whether you are understanding the concepts adequately. Keeping up with weekly materials and performing well on quizzes will ensure you are well prepared for the mastery exercises.
MASTERY EXERCISE: Are mastery exercises all weighted equally?
No, mastery exercises are weighted according to the amount of material (number of weeks of material) they cover.
MASTERY EXERCISE: How much time will I have to complete the in-class mastery exercise?
Mastery exercises are scheduled in MedSIS and usually take place:
On Mondays from 8:00-9:00 a.m., for year 1.
On Fridays from 8:00-9:00 am (with some exceptions), for year 2.
You should aim to arrive 15 minutes before the hour, ready for your mastery exercise with your assessment downloaded on your computer.
MASTERY EXERCISES: Mastery Exercise question review
Faculty will provide a debrief or ‘teaching point’ for questions answered poorly (i.e. answered incorrectly by more than 20% of the class). Neither the question nor answers will be released. This is to maintain the integrity of the exam bank, and in order for students not to focus on the question but rather the learning objective or the learning gap.
For those who want a review of their responses, they will have the opportunity to meet the Course Director for a one-on-one secure review.
MASTERY EXERCISE: What if I forgot to bring a computer on mastery exercise day?
Backup laptops and/or lab computers have been reserved in case a student has forgotten their device or their computer crashes.
MASTERY EXERCISE: What are the instructions given on mastery exercise day?
- All bags, jackets, and electronic devices should be left at the front of the room.
- You should have your personal computer on your desk.
- Your mastery exercise should have been downloaded on your personal computer in your SofTest account.
- If you have not downloaded your mastery exercise, please download it now.
- We will not be checking IDs or taking attendance as SofTest records this information in detail.
- If you encounter a technical issue on your computer, please raise your hand to get technical support.
- When your mastery exercise is properly uploaded you will see a window that will clearly indicate that your mastery exercise has been successfully submitted. If you do not see this window, please come and see me to confirm.
- Nobody can leave the room in the last 10 minutes of the mastery exercise from 8:50-9:00. You will need to stay in the room seated with your personal computer closed after your mastery exercise is successfully uploaded. You may leave earlier if you wish if you have completed and uploaded your mastery exercise.
- You can raise your hand if you would like to have a piece of paper to work on. This paper must be handed in at the end of the mastery exercise.
- The access code to start the mastery exercise is: [...].
- You have 1 hour for the mastery exercise.
MASTERY EXERCISE: What if my computer crashes while I am writing the mastery exercise?
Same as with weekly feedback quizzes except for mastery exercises the invigilator or Discovery Commons tech support will input into the students’ computer a universal resume code to restart the exam.
MASTERY EXERCISE: What if I am sick and cannot write a mastery exercise?
Students are required to report absences for all activities that are scheduled assessments, including mastery exercises. To report an absence and for more information on general regulation on taking examinations as scheduled refer to the school absences page. Students will normally be informed that there is a pre-determined make-up time and date. Mastery exercises are typically rescheduled to the next self-learning day.
MASTERY EXERCISE: How are mastery exercises created?
Mastery exercise MCQ questions are written with a specific purpose in mind: to assess an end of week learning objective. Multiple choice questions are developed by content experts, usually those directly involved in teaching. Approximately one-third of the questions are designed to assess your knowledge, one-third are geared toward assessing your comprehension and the final third are aimed at assessing application. The process is known as a test blueprint and applies to other assessments as well.
The MCQs you will encounter on your assessments will provide all of the information you need to answer the question in the stem. There is one correct answer and three incorrect alternatives known as distractors.
Each question is reviewed by a question writing expert for format and then finalized for use by either the week lead or course director. After MCQs are used, each question is analyzed by reviewing the proportion of students who answered the question correctly. MCQs that are deemed to be too easy or too difficult may be eliminated. MCQs are also analyzed for their ability to discriminate between students who perform well overall and those that do not. Questions that lack this ability may also be eliminated. The question analysis information is retained with the question in the exam bank and is available in the future when the question is used again.
Once the mastery exercise results are finalized you will receive a report summarizing how you did overall, how you did on each type of question and which learning objectives you did not answer correctly. This is intended to guide you in which topics you may need to review further.
In order to make the assessments as appropriate as possible we invite students to add comments to the questions during or at the completion of their exercise. This is to ensure that the questions are relevant and clear which is an important aspect of what makes MCQ assessments useful for learning and feedback. The comments also help to highlight where there may be gaps in the curriculum. These comments will be collectively reviewed by the course director but will not be responded to individually.
MASTERY EXERCISE: When will I receive results of the mastery exercise?
Following the completion of each mastery exercise, the MD Program conducts an analysis of the assessment to ensure that all questions meet minimum standards for quality. This includes a review of item difficulty, reliability and clarity. Adjustments will be made to optimize the overall reliability of the assessment prior to the release of reports to students. This process will be completed by the course director and reports will be released to students approximately one week after students complete the mastery exercise.
MASTERY EXERCISE: Do I get the answers to the mastery exercises questions?
The MD Program test committee reviewed the literature and audited practices both nationally and internationally regarding the provision of answers to online tests. It was the recommendation of this committee that the best practice for the mastery exercises would be to provide students with feedback regarding those topics they did not answer correctly but not to provide actual questions or their correct answers.
The rationale for this is two-fold. First and foremost it is to support that the role of assessment as a tool for self-appraisal and not as an end unto itself. The results of the test are meant to help students identify whether they have mastered the material they covered. If they haven't, the expectation is that they return to the source material until they have a better grasp of the content. Secondly, we are aiming to create a bank of very high quality questions that are directed at assessing not just knowledge but comprehension and application. This requires the use of a secure and well mapped exam bank which precludes disseminating the questions and answers. Students are reminded that copying and disseminating questions is against university regulations.
MASTERY EXERCISE: How do I interpret the mastery exercise feedback report to support me in my learning?
The report you receive provides a breakdown by type of question and the specific learning objective assessed in the examination. You can use the report to determine how you did on certain types of questions e.g. knowledge as well as which learning objectives you should revisit to ensure you are understanding the material well enough. Each code corresponds to a learning objective which you can find on the course portal.
MASTERY EXERCISE: What happens if there is a university closure on a mastery exercise date?
When there is a university closure, for example for a snow day, the mastery exercise is deferred to the same week, potentially Wednesday, 9:00-10:00 AM (for year one). If one of the two campuses is not closed, that campus will carry out the mastery exercise.
MASTERY EXERCISES: Question Feedback
At the end of the mastery exercise (or during if preferred) students will be able to provide feedback on the mastery exercise questions using the NOTES/QUESTION FEEDBACK section at the bottom of any question they have a concern about. This will help us improve the quality of the questions, and also identify any questions that are not appropriate. Please note that this feedback is not intended to be a means of challenging answers or requesting adjustments. Course Directors will likely review comments for themes or for questions with a large number of comments when deciding whether to omit a question or adjust the answer key, but Course Directors will not be able to reply to each student’s concern.
PROGRESS TEST: What is a progress test?
Progress tests are one of the various assessment modalities used in the Foundations Curriculum to help you understand how you are progressing. A progress test is a comprehensive knowledge-based test that assesses your progress towards exit-level MD Program competencies – which are competencies you are expected to attain by completion of the MD Program. You don’t need to study for Progress Tests. In fact, the comprehensive nature of Progress Tests is intended to discourage students from preparing specifically for a test. The best preparation for the test is to engage in the curriculum and stay up to date throughout the program.
PROGRESS TEST: Why do we have progress testing?
Progress tests are designed to show your growth and competency acquisition over time. It a stable measure that shows you where you are, and where you need to be by the end of the program.
PROGRESS TEST: When will progress tests occur?
Progress tests will be given to students at regular intervals throughout the four years of the MD Program. First and second year students will have three progress tests. Schedule details can be found in MedSIS. Third and fourth year students have the opportunity to volunteer.
LEARNER CHART: Who has access to the learner chart?
The Foundations Director, Preclerkship Director, Student Assessment Director, Course Directors, Student Progress Coordinator, Student Progress Committee members have access to the learner chart. Also, the Academy Scholars that are assigned to the student will have access to the learner chart. Find out more about the Learner Chart here.
To access the learner chart, please follow this link: http://learnerchart.med.utoronto.ca/
ASSESSMENT SYSTEMS: What are the different assessment systems that are used in the Foundations?
COURSE REQUIREMENTS: What happens if I don’t complete a course requirement satisfactorily?
Please refer to course requirement documents on the learning portal. Course requirements can be found under each course folder (under: Assessments and evaluations/Assessment elements).
Also, please refer to the following policies on student progress for Foundations Curriculum students:
Standards for grading and promotion of undergraduate medical students in the Foundations Curriculum
Guidelines for assessment of undergraduate medical trainees in academic difficulty - Foundations Curriculum
All course requirement assessment elements must be completed satisfactorily. If this is not the case, students will have an opportunity to address this, and the Foundations Director or Course Director will be in contact with students regarding the next steps. Ultimately, ‘partial progress’ status for an unmet requirement must become ‘satisfactory progress’ in order be successful in the overall course.
ASSESSMENTS: I didn't do as well as I would have liked on an assessment, what should I do?
The transition to undergraduate medical education from other undergraduate, graduate, or professional programs isn’t always straightforward. Often, the learning strategies that worked for you in the past need to be adapted, or new strategies need to be developed and adopted, in order for you to thrive in this new academic environment.
The Academic e-Coach allows you to assess how you might adjust your current learning strategies and make the best use of resources that are available online, within the Faculty of Medicine, and within the broader University of Toronto community, to enhance your academic performance.
Consult the Academic e-Coach: http://www.md.utoronto.ca/academic-e-coach