Accountability Measures 2021

Impact on P-12 Learning and Development

For the 2019-2020 cycle, the EPP developed new instruments, Completer Visit to the Classroom Assessment Instrument and Completers’ Interview on Teaching Effectiveness and P-12 Students Impact, to assess completers’ impact on P-12 students learning and development. In addition, the EPP did and action research to explore completer’s self-efficacy perception regarding teaching effectiveness and P-12 students’ impact.  All these measures evidenced that EPP completers have a positive impact on P-12 students learning and development.

The Completer visit to the Classroom Assessment Instrument is a rubric based on the InTASC Standards for effective teachers. It was developed to observe completers performance teaching P-12 students in a school scenario. The instrument is distributed in four stages of proficiencies and three in-between stages. The stages are: Distinguished (4); Proficient (3); Emerging (2); and Underdevelop (1). The in between stages describe that completers are close to achieving the prior stage but is still in progress. The rubric has a validity index of .89 and a Cronbach’s alpha index of .974 (See Exhibit 1.0 EPP Created Instruments). The Completers’ Interview on Teaching Effectiveness and P-12 Students Impact is an eleven open-ended questions interview. It was developed to assess completers’ perceptions on teaching effectiveness and P-12 students’ impact. The interview was validated for content by an expert panel (See Exhibit 1.0 EPP Created Instruments). The EPP did not have access to the State value-added measures on P-12 students’ growth and developed the action research to evidence completers teaching effectiveness and P-12 impact using multiple measures (qualitative and quantitative).

Completer Visit to the Classroom Assessment Instrument

Six (37%) of the 16 EPP completers for 2019-2020 were observed teaching a group of P-12 students in virtual classroom scenarios. The EPP used the Completer Visit Classroom Assessment Instrument to record completer’s compliance with InTASC categories: The Learner and Learning, Content Knowledge, Instructional Practices, and Professional Responsibilities.  Although proficient in all items scored, completers scores of (3.08, 3.16, 3.25 out of 4.0) were related to InTASC standards #3, The teacher understands how students differ in their approach to learning and creates instructional opportunities that are adapted to diverse learners; standard #9, The teacher is a reflective practitioner who continually evaluates the effects of his/her choices and actions on others and who seeks out opportunities to grow professionally; and standard #5, The teachers uses understanding of individual and group motivation and behavior to create a learning environment that encourages positive interaction, active engagement in learning and self-motivation. All completers are teaching in virtual platforms and are adapting teaching and learning resources to satisfy student needs. In addition, completers are inexperienced in the field of distance education and that could have been a critical factor creating engaging learning environments. Nevertheless, results show that EPP completers are performing at a level of proficiency of 3.08 and above in all the InTASC categories and standards (See Table 1.0 2019-2020 Completers Visit to the Classroom Assessment Instrument Scores Disaggregated Data by InTASC Standard and Specialization.

Completers’ Interview on Teaching Effectiveness and P-12 Students Impact

The EPP interviewed seven (44%) 2019-2020 completers using Completers’ Interview on Teaching Effectiveness and P-12 Students Impact. Data reflected that completers are using the assessment cycle to revise planning and to use a variety of teaching strategies to approach each learner needs. The interviews were coded and categorized for analysis. The themes that emerged from the categories were: Practical Courses, Real-Life Scenarios, Parent’s Disengagement, Individualized Education, Curriculum Adaptations, and Authentic Assessment. The findings show that completers are aware that P-12 student needs are diverse and that they have the responsibility and commitment to address them. In addition, they are using the assessment cycle to revise planning, reteach information, and to use a variety of learning strategies to ensure P-12 students learning success. All the interviewees reported P-12 students growth academically and demonstrating better dispositions to learn.

Action Research

The EPP’s 2019-2020 action research explored three (20%) completer’s self-efficacy perception regarding teaching effectiveness and P-12 students’ impact. The research questions were:

• What EPP preparatory experiences help develop teacher efficacy?
• How do completers perceive their self-efficacy?
• How do completers perceive their teaching effectiveness?

The EPP used various instrument to triangulate data: Completers Interviews on Teaching Effectiveness and P-12 Impact, Teachers’ Sense of Self-Efficacy Scale (a proprietary assessment), Principal’s Evaluations, Completer Visit to the Classroom Assessment Instrument, and Completers’ Artifacts (students’ assessments) (See Exhibit 1.1 Action Research). Completers provided evidence of classroom artifacts that have been used to evidence student’s growth. In general, all P-12 students were positively impacted by EPP completers by improving their academic performance. Completer 001 student’s performed at the expectancy level for Teaching Strategies Creative Curriculum. Completer 002 students’ had academic learning growths of 80% in English Oral Communication Skills, and Completer 003 students demonstrated gains in performance from 16% in the pretest to 42% in the posttest in the Kindergarten Performance Test. The EPP completers demonstrated a high sense of teacher efficacy that have served as a motivator to look for innovate ways of teaching and to help P-12 students attain the desired levels of performance. One of the recommendations of the study was to add P-12 student evaluations as another measure of impact for future studies and for EPP continuous improvement. In addition, the Puerto Rico Department of Education is working on resuming standardized testing for next year. This past two years (2018-2020) the tests have been cancelled due to the earthquakes and the COVID-19 pandemic. The standardized test will provide the EPP with a State value-added measure.

Indicators of Teaching Effectiveness

For 2019-2020, the EPP developed various instruments with the purpose of measuring completers teaching effectiveness. As stated in Measure 1, the Completer Visit to the Classroom Assessment Instrument is a rubric based in the InTASC Standards for effective teachers. The rubric was developed to observe completers performance as teachers with a group of P-12 students. The instrument is distributed in four stages of proficiencies and three in-between stages. The stages are: Distinguished (4); Proficient (3); Emerging (2); and Underdeveloped (1). The in-between levels are: Partial success at rating “4” (3.5); Partial Success at rating “3” (2.5); Partial Success at rating “2” (1.5).  The rubric has a validity index of .89 and a reliability index (Cronbach’s alpha) of .974 (See Exhibit 1.0 EPP Created Assessments). Six completers out of sixteen (37%) were observed in virtual classrooms while teaching a group of students. The same faculty member visited all completers minimizing differences on criteria and assuring reliability. Although proficient, completers scores of 3.08, 3.16, 3.25 out of 4.0 were related to InTASC standards #3, The teacher understands how students differ in their approach to learning and creates instructional opportunities that are adapted to diverse learners; standard #9, The teacher is a reflective practitioner who continually evaluates the effects of his/her choices and actions on others and who seeks out opportunities to grow professionally; and standard #5, The teachers uses understanding of individual and group motivation and behavior to create a learning environment that encourages positive interaction, active engagement in learning and self-motivation. As with other measures, the EPP have noted consistency in completer’s areas that are related with the transitioning from face-to-face teaching to virtual scenarios. Completers are teaching in digital platforms that are relatively new to everyone. They had no mentorship in this transitioning stage because it was an abrupt but necessary adoption for the school system due to the pandemic. Other than that, completers did perform at the distinguished and proficient stages. Three completers performed at partial success of a proficient rating (2.5) in some items and one performed, in one item, at the emerging stage (2) (See Table 1.0 Completers’ Visit to the Classroom Assessment Mean Scores Disaggregated by InTASC Standards and Specialization). Those completers that performed in some items below 3.0 belong to the Special Education, 4^th-6^th, and Preschool Education programs. Special Education and Preschool programs rely on hands-on instructional strategies that empower student’s construction of learning. The EPP analysis is that changes in teaching scenarios could account for these programs differences in scores comparing to other EPP programs. The EPP will present the information in the June Stakeholders data retreat for further analysis and to find other ways to integrate innovative ways to address learning for functionally diverse and preschool students.

In addition, the EPP conducted an action research to explore completer’s self-efficacy perception regarding teaching effectiveness and P-12 student impact (See Exhibit 1.1 Action Research). The EPP collected data from three completers (20%) using various instruments: Completers’ Interview on Teaching Effectiveness and P-12 Impact, Teachers’ Sense of Self-Efficacy Scale, Completers’ Visit to the Classroom, Principals Evaluations and Completers’ Artifacts. The research questions were:

• What EPP preparatory experiences help developed teacher self-efficacy?
• How do completers perceive their self-efficacy?
• How do completers perceive their teaching effectiveness?

The findings for research question 1 (RQ1) suggested that completers were satisfied with the preparation experiences received at the EPP. The findings suggested that having four practical experiences during preparation: Field Experiences I, Field Experiences II, Clinical Experiences I, and Clinical Experiences II were critical in their development of teacher self-efficacy. The EPP provide candidates with performance experiences in their core and specialization courses. In these courses they acquire content and pedagogical knowledge, teaching skills, and dispositions that give them self- confident in their capabilities. In addition, EPP school stakeholders provide opportunities for candidates to observe highly effective teachers as they interact with P-12 students. These observations are valuable examples that become part of the completers teaching repertoire. The findings are in accordance with what Maddux and Gosselin (2013) mentioned about self-efficacy development and performance and vicarious experiences.

The findings of RQ2 suggested that completers perceived themselves as having high self-efficacy.  Completers demonstrated that they have that confidence and self-control necessary to produce gains in students learning. In addition, individuals high in self-efficacy are known to be self-drive and to motivate others to engage in productive behaviors. Self-efficacy represents an individual’s belief in his/her ability to execute behaviors necessary to produce specific and productive outcomes (Bandura, 1977, 1997).  Tschannen-Moran and Woolfolk (1998) argued that competence, self-efficacy, and commitment are intertwined. Competence and commitment are also characteristics of effective teachers. Moreover, Lukácová et al. (2018) found that the higher the level of teachers’ self-efficacy, the better the teacher is in their tendency to adopt an active attitude to the teaching process. Teacher efficacy is also related to the positive use of teaching strategies and to be more students oriented.

The findings of RQ3 suggested that completers perceived themselves as effective teachers. Completer’s concept of effective teaching goes beyond grades as they seek for the social and emotional wellness of their students and their development of a sense of competence. In addition, effective teachers contribute to positive academic, attitudinal, and social outcomes for students. Completers also included parents and other professionals to individually planned and intervene for students success. These perceptions are in accordance with Goe et al. (2008) findings that effective teachers have high expectations for all students and help students learn. Goe et al. (2008) pointed that effective teachers collaborate with other teachers, parents, and professionals to ensure students success.

EPP completers demonstrated, through multiple sources of evidence, that they are effective teachers.

Satisfaction of Employers and Employment Milestones

For 2019-2020, the EPP used a new instrument Employers’ Satisfaction with Completer Survey to assess completer’s mastery of teaching and professional competencies as perceived by employers. The instrument has a content validity of .90 and a Cronbach’s alpha index of .802 (See Exhibit 1.0 EPP Created Assessments). The EPP sent the Employers’ Satisfaction with Completer Survey and had a 69% (9 out of 13 employers) return. Some employers used the Google Forms instrument, while others filled out a copy of the instrument and returned it by email. Three completers, out of sixteen, are not currently employed in teaching positions.

Five of those nine employers (55%) recruited candidates who did their Student Teaching at their schools. Employers’ answers fell between Strongly Agree and Somewhat Agree which demonstrated that employers perceived that completers are well prepared to perform as effective teachers. Completers scores (3.44 and 3.55 out of 4) corresponded to items #5 and #8. Item #5 is about creating learning environments, and item #8 is about the uses of formative and summative assessments. The EPP analysis reflected that completers received their education in traditional classrooms but had to transition to virtual classroom for their first experiences as teachers. These completers have been teaching virtually without prior experience in virtual teaching platforms. Changes in teaching modalities could account for the 3.4 scores particularly in students’ motivation, the creation of learning environments, and assessing learning. Nonetheless, all scores were excellent and demonstrated completer’s strengths and competencies as required by InTASC standards for teaching effectiveness. In addition, 33% (3) of employers gave perfect scores to completers. The EPP have started modifications in all courses to resolve areas for improvement related with the transition to virtual education scenarios. At this moment, two full-time faculties are being trained in the development of digital resources and educational materials appropriate for functionally diverse students. In addition, the university has been diligent to provide professional development related to virtual learning environments, rubric development, assessment strategies, Blackboard Collaborate, and reasonable accommodations, among others. It has been demonstrated that the EPP is using employer’s information to work toward continuous improvement. In addition, some employers are part of the EPP Steering Committee and will participate in the 2021 June data retreat for further analysis and discussion of these findings.

As far as employment milestones, five of the thirteen (38%) completers of 2019-2020 that are currently working as teachers, were employed in the same school that they did the student teaching. This datum reflected employer’s satisfaction with EPP completers’ performance. Retaining student teachers after graduation demonstrates that EPP completers are meeting employers’ expectations on teaching effectiveness and professional standards. Table 3.0 shows employers’ satisfaction with completer survey mean scores disaggregated by item number and employer response. Table 3.1 shows employers satisfaction with completer aggregated data.

In general, EPP completers are exceeding employers satisfaction with their performance as it is evidenced by the data provided and by retaining student teachers once they graduate and become completers.

Satisfaction of Completers

In 2019-2020, the EPP administered a satisfaction survey to 16 (100%) completers through telephone calls. The purpose of the survey is to learn completers’ satisfaction level with their preparation and to use that information for the continuous improvement of the Educator Preparation Program.  The instrument has been developed as part of EPP quality assurance process Satisfaction of Completers’ Survey (See Exhibit 1.0 EPP Created Assessments), which explains why the EPP reported one cycle of data for it. The instrument was validated for content .90 and its reliability was established by Cronbach’s alpha index of .910. The Completers’ Satisfaction Survey consists of 17-questions grouped by: Content Knowledge, Instructional /Pedagogical Knowledge and Skills, Diversity, Integration of Technology, and Research Skills that are aligned to InTASC, CAEP, and Puerto Rico Department of Education Standards. These areas represent the knowledge, skills, and dispositions that effective teachers exhibit in their classroom.  The survey has a Likert Scale of four where 4 means Well Prepared (very confident and very capable of doing the tasks), 3 means Sufficiently Prepared (Confident and capable of doing the tasks), 2 means Not Sufficiently Prepared (capable of doing the tasks with some effort or support), and 1 means Not At All Prepared (insecure of my ability for doing the task). The 2019-2020 completers obtained more than 3, Sufficiently Prepared, in all areas except item #16 where the mean score was 2.75 and is related to research skills. In general, the Special Education program obtained scores (means between 2.17 and 3.29) which represented that they are less confident in their knowledge and skills to do the tasks required to teachers. The Special Education Specialization is a K-12 program and is going through a systemic revision. The EPP is recommending the development of courses emphasizing elementary education, teaching methodologies, and instructional strategies for this specialization. It seems that courses are broad because there is too much content (k-12) and completers do not perceive that they have the mastery required to feel confident in the school scenario. Completers ’perceptions on their confidence and abilities provided critical information for the improvements that are needed regarding their preparation, specifically on Special Education. Table 4.0 shows completers’ satisfaction survey mean scores disaggregated by specialization.

In general, EPP completers are satisfied with their preparation as teachers and feel confident and capable of doing the tasks expected of an effective teacher.

Graduation Rates

The EPP selected  2014 cohort to establish graduation rates at six years or less. The cohort for undergraduate education students was defined by:

• First year students at education programs
• Regular type of admission
• Full time enrollment for their first term as undergraduate students (Fall).

After selecting the cohorts for each year of admission, students are followed during six years to determine how many could complete requisites and graduate. The quantity of graduates is divided by the base and multiply by a hundred. The accumulative graduation rate at six years reflect the percent of students that graduted in six or less years from the Teacher Preparation Programs.

The EPP, like other Educator Preparation Programs across Puerto Rico and the United States, has experimented a decrease in admissions, especially in Secondary Education programs. Nevertheless, the graduation rates of  EPP students have been consistant demonstrating the quality of the program.

Ability of Completers to Meet Licensing

In 2019-2020, the EPP integrated pedagogical situations in core courses to prepare candidates to solve educational problems using evidence-based research and content and pedagogical knowledge. These revisions sought to strengthen candidates’ skills and to improve the Puerto Rico Teacher Certification Test scores (PCMAS). Nevertheless, the 2020 PCMAS administration, scheduled for March 17, was cancelled due to the COVID-19 pandemic island-wide lockdown. The College Board Administration of Puerto Rico administered the test this past March2021 and results would not be available until June 2021. In the academic year 2018-2019, 15 completers took the PCMAS. Table 6.0 shows the single-assessment level pass-rate data for regular teacher preparation program for 2018-2019. Table 6.1 shows Aggregate Assessment Level Pass-Rate Data for EPP. There is no data for the 2010-202 Licensure Exams and the submission of it has been waived due to the pandemic.

*Test takers Rate: The ratio of aggregate number of students taking the assessment to the number of program completers for the institution and a specific academic year.

**Summary Pass Rate: The proportion of program completers who passed all tests they took for their areas of specialization among those who took one or more tests in their specialization areas.

Ability of Completers to be hired in Education Positions

One of the EPP goals is that all its completers get hire in education positions. For the 2019-2020 cycles, 12 out of 16 (75%) completers have been hired as teachers. Of those four (33%) are working in the public system of education and eight (66%) are working in the private educational system.  Two (12.5%) 2019-2020 completers are self-hired as teachers for homeschool students.  Table 7.0 shows completers hired in education positions disaggregated by specialization and school system.

The EPP recognize that completers are being successfully hired in positions related to education and that the educator preparation program have been strengthened through the application of data driven decisions.

Student Loan Default Rate

The EPP received in September 2020 the FY 2017 Cohort Default Rate form the United States Department of Education. The FY 2017 Cohort Default Rate was established at 3.7.  As established by the Higher Education Act of 1965 (HEA), as amended, the Higher Education Reconciliation Act of 2005 (HERA), Pub. L. 109-71 and the Department of Education’s regulations, the Inter American University, Metro Campus, is not subject to any sanctions based on the FY 2017 CDR. Schools with a cohort default rate of less than 15.0 percent for each of the most recent fiscal years may disburse loans in a single disburse. Table 8.0 shows Aggregated EPP Cohort Default Rate for years 2014 through 2017. Table 8.1 shows Disaggregated EPP Cohort Default Rate Information.

The EPP has maintained an excellent record through the years as shown in the information provided by the United States Department of Education.