CYCLE THREE - THE PERFORMANCE
Cycle three took place during the second semester and involved the faculty. I worked alongside the faculty on assessing and evaluating the junior class with their understanding of oral hygiene care attained in Preventive Dentistry. The students have an assessment called a process evaluation for oral hygiene instructions that they must pass when treating their patients. The process evaluation form assesses whether the students are able to transfer their knowledge attained during the first semester in Preventive Dentistry and applies it into practical application, the clinical setting that starts in the second semester.
The foundation of the evaluation is to determine individual student competency of the desired learning outcomes. All eleven students passed the Preventive Dentistry course. However, courses usually cover several topics as well as encompass a variety of skills. For example, Preventive Dentistry covered twenty-five major subject areas that required the students to use several different cognitive levels of learning in these areas, such as knowledge, comprehension, and applying that knowledge to assigned projects. The students’ final course grade would represent a composite of all these different aspects. A student may do well in several areas but poorly in one or more areas, and still achieve a high grade. Because of all these factors, overall course grades will not usually provide definitive information about how well students can master targeted learning outcomes. From a program perspective, individual course grades also do not provide a clear picture of how well students can retain, integrate, and apply topics across courses or two years of a curriculum, or to actual patient care or practice situations. For all these reasons, overall course grades are not an acceptable measure for assessment and evaluation purposes in the clinical setting.
Student learning is measured in variety of places within our program and by different methods. For example, there are certain skills that students need to possess early on, and our program assesses the extent of student mastery of these skills from the beginning of the program. There are also skills that students would be expected to develop with time and practice; and there are other skills that the students will develop early, but continue to demonstrate progress and sophistication with time.
In the second semester the students start seeing patients in clinic. This is the first time the students are able to apply what they learned in Preventive Dentistry into practical application or authentic context. At the conclusion of the second semester, the students are required to have completed treatment on eight patients. Multiple, two-hour clinic sessions, overseen by faculty, are needed to complete one patient for the first year student. During each of these appointments the students are required to pass several evaluations (assessments), which includes the oral hygiene instruction evaluation in that the faculty were surveyed on.
The process evaluation form used for this assessment contains 15 items that the students are to perform correctly. For a first year student a 90% score is needed to pass. As a second year student a 100% score is needed to pass. If a student does not pass the first time they can repeat the evaluation on another patient any time during the current semester. The skills the students are to perform correctly include the following:
The faculty assessment/evaluation enabled me to identify how well I have done at achieving my desired outcome of the students’ ability to implement oral hygiene concepts into patient care. It also allows me to develop, plan, and implement improvements in my curriculum when desired outcomes are not fully met.
Research Question
Will the faculty observe the students being able to apply what they learned in Preventive Dentistry by demonstrating their knowledge during oral hygiene instructions when treating their patients in clinic?
Evidence
Five faculty members will be asked to answer a five-question survey created to elicit their perceptions of the students’ ability to demonstrate the transference of knowledge attained in preventive dentistry to patient care. Faculty was encouraged to make additional comments.
Evaluations
Student/Faculty Discussion Board
The discussions on the student/faculty discussion board provided conflicting results. Over half, (6 students or 55%), of the junior class did well with oral hygiene instructions when educating their patients. These students also reflected often on how they could continually improve - demonstrated by this reflection, “I would like to give better OHI, it was my second time giving oral hygiene instructions during this semester and I would like to present the OHI more “smoothly”, I think that I present the information very choppy. I will try and practice giving OHI out loud to see if this helps me to improve my presentation.”
There are five students who demonstrated very little knowledge when presenting oral hygiene instructions. When asked to reflect on this as a learning opportunity on the discussion board they responded with textbook answers instead of using their own voice. This demonstrated their lack of understanding of the information presented in the preventive dentistry class or that they had memorized the information but had not transferred it.
Faculty Survey Results
Five faculty members, two full-time faculty members and three part-time faculty members, were sent a five-question survey through our faculty e-mail. The introduction included a brief description of my action research and what action research entails. I attached the survey questions that addressed the students’ ability to transfer what they learned about oral hygiene during the first semester in the Preventive Dentistry class and the students’ ability to apply it into clinic when seeing patients in the second semester.
Two surveys were returned (40% response rate), one by the director (a full-time faculty member) and one part-time faculty member. The following represents the results of the faculty survey on two of the five questions.
Question #1: Do you feel the Class of 2013 has been thorough in describing the formation and composition of dental biofilm (plaque), it’s relationship to oral disease (dental caries and periodontal infection), and specifically the relationship to the patient’s present condition?
The foundation of the evaluation is to determine individual student competency of the desired learning outcomes. All eleven students passed the Preventive Dentistry course. However, courses usually cover several topics as well as encompass a variety of skills. For example, Preventive Dentistry covered twenty-five major subject areas that required the students to use several different cognitive levels of learning in these areas, such as knowledge, comprehension, and applying that knowledge to assigned projects. The students’ final course grade would represent a composite of all these different aspects. A student may do well in several areas but poorly in one or more areas, and still achieve a high grade. Because of all these factors, overall course grades will not usually provide definitive information about how well students can master targeted learning outcomes. From a program perspective, individual course grades also do not provide a clear picture of how well students can retain, integrate, and apply topics across courses or two years of a curriculum, or to actual patient care or practice situations. For all these reasons, overall course grades are not an acceptable measure for assessment and evaluation purposes in the clinical setting.
Student learning is measured in variety of places within our program and by different methods. For example, there are certain skills that students need to possess early on, and our program assesses the extent of student mastery of these skills from the beginning of the program. There are also skills that students would be expected to develop with time and practice; and there are other skills that the students will develop early, but continue to demonstrate progress and sophistication with time.
In the second semester the students start seeing patients in clinic. This is the first time the students are able to apply what they learned in Preventive Dentistry into practical application or authentic context. At the conclusion of the second semester, the students are required to have completed treatment on eight patients. Multiple, two-hour clinic sessions, overseen by faculty, are needed to complete one patient for the first year student. During each of these appointments the students are required to pass several evaluations (assessments), which includes the oral hygiene instruction evaluation in that the faculty were surveyed on.
The process evaluation form used for this assessment contains 15 items that the students are to perform correctly. For a first year student a 90% score is needed to pass. As a second year student a 100% score is needed to pass. If a student does not pass the first time they can repeat the evaluation on another patient any time during the current semester. The skills the students are to perform correctly include the following:
- Has patient seated upright and comfortable in the dental chair.
- Establishes and maintains eye contact throughout process.
- Determines current knowledge level of patient.
- Personalizes assessment findings to patient’s oral hygiene needs.
- Builds on patient’s knowledge base to better understand his/her current dental health status.
- Ask patient to demonstrate oral hygiene techniques: toothbrushing, interdental needs and tongue scraping
- Evaluates technique and gives positive reinforcement.
- Demonstrates modifications to improve patient’s technique and relates it to the needs of the patient.
- Incorporates additional oral hygiene aids or strategies as needed.
- Has patient demonstrate new technique or discuss how he/she will implement strategy.
- Makes additional modifications as needed.
- Discusses ways to realistically incorporate new technique or strategy into life cycle
- Engages patient throughout process.
- Communicates at the appropriate level to meet patient’s needs.
- Provides patient with reinforcement aids.
The faculty assessment/evaluation enabled me to identify how well I have done at achieving my desired outcome of the students’ ability to implement oral hygiene concepts into patient care. It also allows me to develop, plan, and implement improvements in my curriculum when desired outcomes are not fully met.
Research Question
Will the faculty observe the students being able to apply what they learned in Preventive Dentistry by demonstrating their knowledge during oral hygiene instructions when treating their patients in clinic?
Evidence
Five faculty members will be asked to answer a five-question survey created to elicit their perceptions of the students’ ability to demonstrate the transference of knowledge attained in preventive dentistry to patient care. Faculty was encouraged to make additional comments.
Evaluations
Student/Faculty Discussion Board
The discussions on the student/faculty discussion board provided conflicting results. Over half, (6 students or 55%), of the junior class did well with oral hygiene instructions when educating their patients. These students also reflected often on how they could continually improve - demonstrated by this reflection, “I would like to give better OHI, it was my second time giving oral hygiene instructions during this semester and I would like to present the OHI more “smoothly”, I think that I present the information very choppy. I will try and practice giving OHI out loud to see if this helps me to improve my presentation.”
There are five students who demonstrated very little knowledge when presenting oral hygiene instructions. When asked to reflect on this as a learning opportunity on the discussion board they responded with textbook answers instead of using their own voice. This demonstrated their lack of understanding of the information presented in the preventive dentistry class or that they had memorized the information but had not transferred it.
Faculty Survey Results
Five faculty members, two full-time faculty members and three part-time faculty members, were sent a five-question survey through our faculty e-mail. The introduction included a brief description of my action research and what action research entails. I attached the survey questions that addressed the students’ ability to transfer what they learned about oral hygiene during the first semester in the Preventive Dentistry class and the students’ ability to apply it into clinic when seeing patients in the second semester.
Two surveys were returned (40% response rate), one by the director (a full-time faculty member) and one part-time faculty member. The following represents the results of the faculty survey on two of the five questions.
Question #1: Do you feel the Class of 2013 has been thorough in describing the formation and composition of dental biofilm (plaque), it’s relationship to oral disease (dental caries and periodontal infection), and specifically the relationship to the patient’s present condition?
Question #2: Do you feel the Class of 2013 has an increased understanding of oral hygiene and has been able to demonstrate their knowledge during OH instructions with their patient’s?
Additional comments made by the director in response to the survey:
Question #1: Do you feel the Class of 2013 has been thorough in describing the formation and composition of dental biofilm (plaque), it’s relationship to oral disease (dental caries and periodontal infection), and specifically the relationship to the patient’s present condition?
Question #2: Do you feel the Class of 2013 has been able to demonstrate their knowledge of the oral hygiene aids available and their indications for use?
Reflection
Cycle three involved the students applying what they learned from the first semester into clinic, when seeing patients. This cycle was quite a surprise and a great learning experience for me on several levels. The faculty surveys showed a portion of the students’ were able to display an increased level of understanding and there was a portion of the students who could not demonstrate the understanding they demonstrated during the course the previous semester. I also observed this of the students. I observed one student demonstrating how to floss using his patient’s fingers? The students are taught to demonstrate a flossing technique on a plastic mouth that represents a patient’s dentition. I also observed one of the students explaining oral hygiene to a patients’ parent and did an excellent job.
Traditionally, dental hygiene education has relied on a teacher-delivered, lecture-based educational approach along with a performance-based approach in the clinical setting. To increase the students understanding I implemented several blended learning activities in hopes of addressing this disconnect during the first semester in my Preventive Dentistry course. I am learning that the educational strategies I implemented result in discrete, isolated knowledge and performance in the absence of experience, while not implementing enough critical thinking and reasoning skills to the clinical environment, although there were some exceptions. Clinical thinking is characterized as a complex and often unconscious integration of clinical reasoning and data-collecting procedures aimed at patient care (Darby, 2003). The inherent skills required for clinical reasoning include the ability to sort information, recognize relevant patterns, and select key elements within that information. The comment made by the director was correct in that content-specific knowledge and the ability to collect meaningful data increases with experience.
Was I unrealistic in my expectation that the students would be able to demonstrate a high level of understanding when providing oral hygiene instructions? Yes, I did not consider the fact that the students had no real life experience, meaning the students have never been in a patient’s mouth before. Pictures, discussion boards, videos and case scenarios and presentations cannot provide the same experience although they can provide pattern recognition and organization of information that is necessary for creating accurate knowledge when treating patients. While the dental hygiene program does focus on teaching students a methodical process for approaching patient care, there still is a problem of transferring that knowledge into clinic. I also need to remember that these are first year dental hygiene students.
The differing results from the survey I received addressed another issue of faculty not being on the same level as displayed by my expectation of the first year students in my survey question #1. There are faculty members who pass students regularly with 100% on their assessments while other faculty members require more knowledge demonstration. The students are very aware of this and of course want to be evaluated by the easier faculty members. I believe that the faculty needs to better understand the expectations of what a process evaluation, assessment, and competency looks like for a first year student and a second year student. Additionally, faculty needs to better understand the premises underlying critical thinking and need guidance in methods for fostering good clinical judgment. A study by Behar-Horenstein showed that dental faculty rely primarily on teacher dominated instruction or coaching techniques such as hypothesis generation and questioning to develop critical thinking skills and do so only 2 percent of the time in the clinic setting.
During this cycle I have learned that it is necessary to incorporate more critical thinking and clinical reasoning into my curriculum. This will take a considerable amount of time and there is still some confusion on how to best teach and evaluate critical thinking and clinical reasoning skills. Other metacognitive measures that may have a utility in predicting student performance that I need to look at include motivation, maturity, age groups, and disposition.
I need to implement a backward design when I design my didactic courses by looking at the clinical process evaluations, assessments and competencies and overlap them into my learning outcomes where applicable. While this cycle did not provide the outcome I had hoped for it did provide me with the areas I need to change to produce different outcomes for the students. I do think that the students who demonstrated an increased understanding in the clinical setting where the students who were motivated, studied and processed the information they were presented with in Preventive Dentistry at a higher level.
Question #1: Do you feel the Class of 2013 has been thorough in describing the formation and composition of dental biofilm (plaque), it’s relationship to oral disease (dental caries and periodontal infection), and specifically the relationship to the patient’s present condition?
- This expectation is at a level that is beyond any first year DH student. This is a competency for the completion of the program. It’s not until students have treated 20-30 patients that they are able identify the relationship between the patient’s oral condition(s) and integrate plaque and oral health into the equation.
Question #2: Do you feel the Class of 2013 has been able to demonstrate their knowledge of the oral hygiene aids available and their indications for use?
- There are a few students who can do this but on the average, my answer is no. When not in clinic and in a didactic setting, more students are able to do this at the expected level. When we get into clinic, there seems to be a disconnect that occurs with most of the students. I’ve notice that this disconnect is more evident with certain members of the Class of 2013 than I’ve seen in many years.
- Overall all comment on this action research project: This project has enhanced the learning of many members of the Class of 2013 to a level that I’ve not seen before in a first year student. It has been a pleasure to see this level of understanding at such an early stage in the program. There is no doubt that this is a direct outcome from your Action Research. Unfortunately, the Class of 2013 has a large number of students who are struggling with understanding that short-term memorization is not the goal of the curriculum which is what they have done up to this point in their educational journey – memorize, take a test, regurgitate the memorize information, forget the information and move forward. These students aren’t participating at a level that will allow them to process information that they’ve memorized into a deeper understanding. Thank you for bringing constructivism into another course in the DH Program. It will provide future students the ability to truly learn the concepts covered in DEH-16 and incorporate this knowledge into patient care.
Reflection
Cycle three involved the students applying what they learned from the first semester into clinic, when seeing patients. This cycle was quite a surprise and a great learning experience for me on several levels. The faculty surveys showed a portion of the students’ were able to display an increased level of understanding and there was a portion of the students who could not demonstrate the understanding they demonstrated during the course the previous semester. I also observed this of the students. I observed one student demonstrating how to floss using his patient’s fingers? The students are taught to demonstrate a flossing technique on a plastic mouth that represents a patient’s dentition. I also observed one of the students explaining oral hygiene to a patients’ parent and did an excellent job.
Traditionally, dental hygiene education has relied on a teacher-delivered, lecture-based educational approach along with a performance-based approach in the clinical setting. To increase the students understanding I implemented several blended learning activities in hopes of addressing this disconnect during the first semester in my Preventive Dentistry course. I am learning that the educational strategies I implemented result in discrete, isolated knowledge and performance in the absence of experience, while not implementing enough critical thinking and reasoning skills to the clinical environment, although there were some exceptions. Clinical thinking is characterized as a complex and often unconscious integration of clinical reasoning and data-collecting procedures aimed at patient care (Darby, 2003). The inherent skills required for clinical reasoning include the ability to sort information, recognize relevant patterns, and select key elements within that information. The comment made by the director was correct in that content-specific knowledge and the ability to collect meaningful data increases with experience.
Was I unrealistic in my expectation that the students would be able to demonstrate a high level of understanding when providing oral hygiene instructions? Yes, I did not consider the fact that the students had no real life experience, meaning the students have never been in a patient’s mouth before. Pictures, discussion boards, videos and case scenarios and presentations cannot provide the same experience although they can provide pattern recognition and organization of information that is necessary for creating accurate knowledge when treating patients. While the dental hygiene program does focus on teaching students a methodical process for approaching patient care, there still is a problem of transferring that knowledge into clinic. I also need to remember that these are first year dental hygiene students.
The differing results from the survey I received addressed another issue of faculty not being on the same level as displayed by my expectation of the first year students in my survey question #1. There are faculty members who pass students regularly with 100% on their assessments while other faculty members require more knowledge demonstration. The students are very aware of this and of course want to be evaluated by the easier faculty members. I believe that the faculty needs to better understand the expectations of what a process evaluation, assessment, and competency looks like for a first year student and a second year student. Additionally, faculty needs to better understand the premises underlying critical thinking and need guidance in methods for fostering good clinical judgment. A study by Behar-Horenstein showed that dental faculty rely primarily on teacher dominated instruction or coaching techniques such as hypothesis generation and questioning to develop critical thinking skills and do so only 2 percent of the time in the clinic setting.
During this cycle I have learned that it is necessary to incorporate more critical thinking and clinical reasoning into my curriculum. This will take a considerable amount of time and there is still some confusion on how to best teach and evaluate critical thinking and clinical reasoning skills. Other metacognitive measures that may have a utility in predicting student performance that I need to look at include motivation, maturity, age groups, and disposition.
I need to implement a backward design when I design my didactic courses by looking at the clinical process evaluations, assessments and competencies and overlap them into my learning outcomes where applicable. While this cycle did not provide the outcome I had hoped for it did provide me with the areas I need to change to produce different outcomes for the students. I do think that the students who demonstrated an increased understanding in the clinical setting where the students who were motivated, studied and processed the information they were presented with in Preventive Dentistry at a higher level.