Accountability Measures 2024

Completer Effectiveness

For the 2022-2023 cycle, the EPP used data from the Completers Visit to the Classroom Assessment Instrument, a Focus Group for a Completers Interview on Teaching Effectiveness and P-12 Students Impact, the Teachers’ Sense of Self Efficacy Short Scale results, and the Completers Satisfaction Survey.

The EPP has no access to state or private systems student growth data. The information is available through general school statistics that do not demonstrate completers’ contributions to P-12 learning and development. The EPP is using program information from multiple measures to demonstrate that completers are having a positive impact on P-12 students.

Completers Visit to the Classroom Assessment Instrument
The EPP coordinator visited six completers (26%) at their workplace and observed them for a teaching period. Table 1.0 shows completers demographics, areas of specialization, and workplace category (private/public).

The Completer Visit to the Classroom Assessment instrument consists of 21 items aligned to the InTASC Standards for Effective Teaching. The instrument scale is divided into four stages: Distinguished (4); Proficient (3); Emerging (2); and Underdeveloped (1). In addition, the instrument has three in-between stages: Partial success at rating Distinguished (3.5); Partial success at rating Proficient (2.5); and partial success at rating Emerging (1.5). Table 1.1 shows participants' disaggregated scores by item on their performance teaching their P-12 students observation instrument.

Completers’ Visit to The Classroom scores evidenced that 83% of completers performed in the proficient or higher category and 16% scored at partial success at rating proficient. The higher mean scores were obtained on InTASC standards 1 and 8. These InTASC standards are related to content and pedagogical knowledge and to the use of informal and formal assessment in appropriate ways. This group of completers had the opportunity to practice with the “post pandemic children” and learned, in their preparation as candidates, multiple ways of assessing prior and acquired knowledge. The lowest scores were obtained on InTASC standards 3 and 5. These completers are dealing with the offsets of all the changes that have influenced the educational system (pandemia, hurricanes, earthquakes). These standards are related to the use of instructional activities appropriate to students’ stages of development and to the relationship between the learning lessons and the students’ lives. It seems odd, that completer C3 is a preschool education specialist and did not master development stages when planning lessons. There are no definite trends between the previous years. The instrument was used for the first time for the 2019-2020 cycle of evidence and all the years that follows the EPP have seen different outcomes. The items of concern for all the other cycles are related to InTASC 3 standards. These standards deal with mastering developmental stages, learning styles, students’ strengths, and needs. The EPP is working on the program revision to strengthen specializations in preschool and primary education. Program revisions are systemic and the EPP is currently waiting for authorization to start the revised courses. Other specializations include knowledge from k-12 students, which might be too much information for each course, thus diluting it.

Completers Focus Groups on Teaching Effectiveness and P-12 Students Impact
The EPP invited all 2022-2023 completers through emails to a Focus Group Interview on February 9, 2024, at 4:00 pm at the university Multiple Use Room. Four (17.3%) out of 23 completers participated in the focus group interview. The structured interview consisted of eleven open-ended questions. The Focus Group was conducted by a trained counselor who is part of the EPP graduate adjunct staff. The EPP coded and categorized the qualitative data that derived in various themes that were used to analyze these data. Table 1.3 shows the central themes derived from each question of the interview.

Completers answers gave the EPP a broader perspective of the school scenarios, working conditions, and what is needed to be taught to prepare confident and knowledgeable teachers. The EPP understands that novice teachers struggled with certain administrative and teaching aspects like using the assessment cycle to re-teach, being confident with classroom management skills, differentiated instruction knowledge and strategies, and having few direct teaching experiences throughout their preparation. These answers were very similar to the 2020-2021 focus group interview. In this matter, the EPP is going through a system revision to improve candidates’ preparation to teach. Completers answers have been crucial in determining revisions. The EPP is proposing an increment in Field and Clinical Experiences hours in real school scenarios and roles. This is the second year that the EPP uses the Focus Group Interview. For the cycle 2019-2020, the EPP interviewed completers on Teaching Effectiveness and P-12 Students’ Impact by phone. Data reflected that completers were using the assessment cycle to revise planning and to use a variety of teaching strategies to approach each learner’s needs. These findings are completely different from the 2020-2021 and 2022-20223 focus groups. Trends between the 2020-2021 and the 2022-20223 focus groups shows that there are some gaps between knowledge and implementation of knowledge in completing the assessment cycle and on using teaching strategies to impact the universe of students in PK-12 classrooms. On the other hand, it seems that through the past three years (2018-2019; 2020-2021; 2022-2023) completers have been more aware of students socioemotional needs to promote dispositions to learning.

Teachers’ Sense of Self-Efficacy Scale

The Teachers’ Sense of Self-Efficacy Short Scale assesses teachers’ capability concerning instructional strategies, student engagement, and classroom management. The instrument uses 9 points rating scales: None (1) and (2), Very Little (3) and (4), Some degree (5) and (6), Quite a Bit (7) and (8), and A Great Deal (9). The EPP used it last year to support the Completers Case Study. For the 2020-2021 cycle, the scale was administered to twelve out of twenty-seven completers (44.44%). For the 2022-2023 cycle, the scale was administered to 5 (21.7%) completers. The scale distributes questions according to Efficacy in Student Engagement items 2, 3, 4, and 11; Efficacy in Instructional Strategies items 5, 9, 10, and 12; and Efficacy in Classroom Management items 1, 6, 7, and 8. Tables 1.4, 1.5. and 1.6 show the distribution of items and disaggregated scores. Table 1.7 shows The Teachers’ Sense of Self-Efficacy Short Scale aggregated mean scores and standard deviations.

For the 2022-2023 completers, results demonstrated that completers from different specializations perceived that they have the knowledge and skills to engage students, to use instructional strategies, and to manage the classroom. Mean scores were very close for the three areas that are measured in the Teachers’ Sense of Self-efficacy Instrument. Data from the 2020-2021 cycle Teachers’ Sense of Self-Efficacy Short Scale demonstrated that completers perceived the use of diverse and appropriate instructional strategies as an area of strength and that classroom management skills as a weaker point to their teaching. These results were consistent with the Completer Classroom Visit Assessment and the Focus Groups Interview findings. For this cycle, the triangulation between the Completer to the Classroom Visit Assessment and the Focus Group Interview showed that completers performances and perceptions differ. The Self-efficacy Instrument results suggested that classroom management is a strength, but on the focus group completers expressed some concerns about this aspect. For the 2019-2020 cycle, the EPP administered the instrument to three completers that were part of the case-study. Last-year findings (mean scores 7.53, 7.53, 6.90) were higher for student engagement items and for instructional strategies, but lower for classroom management items. This year (2022-2023) findings are very similar in all areas, and scores are lower than last year’s cycle. Nevertheless, there are not significant changes between cycles of data. The trends are that completers perceived that they have the confidence and self-control needed to produce gains in P-12 students learning.

Completers’ Satisfaction Survey

The Completer Satisfaction Survey was administered to the six (26%) out of 23 completers. The demographics and specializations are the same as portrayed in Table 1.0. The completer satisfaction instrument is a 17 questions instrument with a 4 criteria Likert-scale; Well-prepared (1); Sufficiently Prepared (2); Not Sufficiently Prepared (3); Not Prepared at All (4). The survey is intended to inquire how satisfied were completers with the EPP preparation. Completers’ satisfaction and sense of self-efficacy are often related and imply commitment to perform diligently. Table 1.8 shows Completers’ Satisfaction Survey disaggregated scores.

Overall, completers felt well-prepared or sufficiently prepared as teachers. Compared to last year’s cycle (2020-2021) that 48% of completers felts not sufficiently prepared probably because they did most of their preparation under the stress and uncertainty that were caused by hurricane Maria, the earthquakes that affected the island, and the Covid-19 pandemic, these cycle (2022-2023) completers felt well or sufficiently prepared to teach. Completers felt confident with the mastery of the subject, technology integration, and with the use and development of their teaching strategies. The areas where they felt less prepared were the ones pertaining to attention to diversity. Puerto Rico is experiencing an increase of diverse students in regular classrooms. To address this matter, the Puerto Rico Department of Education revised the Teacher Licensing Requirements and added a core course to teach inclusion strategies to candidates. The EPP started offering the course as a pilot and data will be available for CAEP recertification visit. There are no trends between last years’ data, which responded to the changes in virtual education due to the pandemia, and this year’s results.

Principal’s Teachers’ Evaluation Rubric
The Principal’s Teachers’ Evaluation Rubric is an instrument that the EPP piloted with the 2020-2021 completers. The instrument was developed to measure teachers’ competencies from a direct supervisor, other than the employer. In addition, the EPP wanted to use this information as an indirect measure of P-12 students’ growth. State statistics on public schools are limited to general information concerning core subjects and are not distributed by teachers. The same is true for the private educational system. This educational system uses a private company, and the information cannot be shared to particulars. The rubric is divided into the eight competencies that are expected for effective teachers: Communication and Interpersonal Skills; Organization and Planning; Classroom Management; Facilitation and number of; Assessment and Coaching; Collaboration and Teamwork; Caring and Inclusiveness; and Flexibility and Adaptability. It uses a three-point scale High Competency (3), Distinguished Proficient (2), and Emergent (1). This instrument was validated by ten judges. Validation results were 0.0884 medium validity, using Validity Index of Aiken and Cronbach’s Alpha of -.076. The instrument was completed at the EPP visit for Completers’ Classroom Observation Assessment by their direct supervisors. Table 1.9 shows direct supervisors scores for completers using the instrument.

Completers lower scores were related to Organizing and Planning, Classroom Management, and Assessment and Coaching. Completers strengths were demonstrated at Communication and Interpersonal Skill, Collaboration and Teamwork, Caring and Inclusiveness, and Flexibility and Adaptability. These findings are consistent with the ones from other instruments and established a trend as the 2020-2021 cycle had similar findings. The EPP is revising the program to add classroom strategies and assessment techniques to all courses content.

In conclusion, for Measure 1 the EPP used multiple instruments to demonstrate that 2022-2023 program completers effectively contributed to P-12 student growth learning and applied the professional knowledge, skills, and dispositions required for a highly competent teacher.

Satisfaction of Employer and Stakeholder

The EPP has been working on its Completers Case Study and all the information provided in the 2022-2023 Measures are from the same group of completers. The EPP wanted to have multiple sources of information from the same group of individuals to triangulate. In this matter, the EPP received six Employers’ Satisfaction with Completer Questionnaires. Employers used the Employers’ Satisfaction with Completer instrument. The instrument includes general information questions about the organization and 10 questions that are based on Interstate Teachers Assessment and Support Consortium (InTASC) standards for effective teachers. It has a five-criteria Likert Scale: Strongly Agree, Somewhat Agree, Somewhat Disagree, Strongly Disagree, and No Response. Table 2.0 shows Employers Satisfaction with Completers disaggregated scores by item scores for the 2022-2023 academic cycle.

Employers rated 78 % of completes as doing an exceptional job (Totally Agree). Only 21% of completers received Somewhat Agree in questions related to InTASC Standards 2, 3, 4 and 8. These standards are related to the use of assessment techniques, instructional strategies, attention to diverse learners, and planning to address development stages. Findings are consistent with the ones in other completers instruments. Nevertheless, scores are like what the EPP reported for the 2020-2021 cycle, but no trends can be established due to differences in areas that need improvement. Employers seem to be very satisfied with completers as novice teachers.

Pertaining stakeholders’ involvement, the EPP invited stakeholders for a Focus Group Interview regarding program strengths, areas for improvement, program revision, and Field and Clinical experiences. The meeting was on March 14, 2024, and the EPP used nine open-ended questions to start the interview. The purpose of the interview was to evaluate the effectiveness of the preparation of students who completed their bachelor’s degree in Teacher Education and it was intended to: identify the outcomes of the academic preparation and experiences to which they were exposed and their impact on the learning and development of P-12 students, to determine satisfaction of employers and stakeholders with the institution effectiveness preparing high quality completers, inquire into how completers contribute to P-12 growth, identify how completers apply content knowledge, skills, and dispositions acquired during their academic training. Table 2.1 shows the participants’ demographics. Table 2.2 shows emergent themes for each of the prompted questions at the Focus Group Interview.

The EPP has not met regularly with stakeholders because all Inter American University Teacher Preparation Programs are under systemic revision, and it must wait for the final resolutions to implement new changes. Nevertheless, several themes emerged from the Focus Group Interview that are being evaluated as we are getting ready to implement the program revision.

Candidate Competency at Completion

The EPP is using multiple measures to ensure that candidates are meeting program expectations at completion and are ready to meet licensure requirements. Among the measures used, the EPP collected data on Candidates Dispositions to the Profession. The dispositions instrument is used by the cooperative teacher during Clinical Experiences II to measure candidates’ disposition related to the profession and to their commitment with P-12 students’ growth and learning. The instrument has three components: Positive Commitment, Reflexive Learner, and Empathy. Table 3.0 and Table 3.1 show candidates scores in each of the Candidates’ Dispositions Instrument components disaggregated by specialization and academic term.

Findings from the Candidates’ Disposition Instrument completed by the cooperative teacher suggest that candidates are strongly committed to the profession and that they used reflection to improve their teaching and to grow professionally. In conclusion, for the August- December 2022 academic year, seven out of nine candidates (77%) obtained 90% or higher scores in the instrument. For the January-May 2023 academic term 10 out of 15 (67%) of completers obtained 90% or higher scores. The EPP noticed that the small percentage of candidates that had a lower score, especially during the second semester, were the same candidates that evaluated cooperative teachers and university supervisors poorly (Tables 3.2 and 3.3). Although the EPP has integrated reflection prompts in core courses and in the Clinical Experience course, it remains an area for improvement based on the Dispositions Instrument results of this cycle.

The EPP is using other measures to demonstrate that candidates are prepared to be competent teachers. Table 3.2 and Table 3.3 shows candidates scores in Clinical Experiences Portfolio, Teaching Unit, Action Research P-12 Impact Proposal, candidates’ evaluation for the cooperating teacher and university supervisor effectiveness, and candidates’ satisfaction with the program. These instruments have been previously submitted to CAEP and all have been validated and tested for inter rater reliability.

Overall, candidates from both terms performed above average with a few exceptions that performed average. The highest scores (91-100%), on both terms, were on the Portfolio and Teaching Unit and the lower scores (80-86%) were on the P-12 Impact Research Proposal. The EPP analyzed these results and integrated a Classroom Research Workshop for the Clinical Experience to strengthen research skills. Regarding candidates’ perceptions on cooperative teacher and university supervisor effectiveness. In conclusion, EPP candidates are demonstrating that they are competent and meet program expectations for effective teachers and educational leaders. Regarding candidates’ satisfaction, 87% of the August-December 2022 candidates rated 90% or above satisfaction with cooperative teacher effectiveness and 100% rated 100% satisfaction with university supervisors’ effectiveness. Overall, nine out of nine candidates felt 100% satisfaction with the program. The January-May candidates were less satisfied with cooperative teachers, 67% rated them with 90% or more, 60% rated university supervisor with 90% or more, and 87% (13 out of 15) were satisfied with the program. Variations from one semester to the next could be explained as follows: for the second semester, the preschool level cooperative teachers and university supervisor were new to the program. The same happened with the ESL university supervisor. Candidates felt that their needs were not attended appropriately. The EPP started a mentors training program to ensure that cooperative teachers and university supervisors comply with the responsibilities as assigned.

In addition, candidates took the Teacher Certification Exam offered by the State and administered by the College Board. On the 2021-2022 academic year, EPP candidates pass rate was above statewide. This measure evidenced the candidate’s competency at completion. For the 2022-2023 cycle, the EPP repeated the outstanding outcomes and perform above statewide. Table 3.4 shows 2022-2023 candidates aggregated pass rates on PCMAS (Prueba de Certificación de Maestros) Teachers Certification Exam.

The EPP candidates have demonstrated, by their performance in multiple measures, that are competent at completion and have approved licensure with a pass rate above statewide. The EPP is proud to acknowledge that candidates are well-prepared and competent in applying InTASC standards and complying with CAEP standards for quality in the teaching profession.

Ability of Completers to be Hired in Education Positions for which They have Prepared

The EPP had twenty-three (23) completers during the 2022-2023 academic year. Seven out of twenty-three (30%) are employed in the Puerto Rico public system of education. Seven out of twenty-three (30%) are employed in the Puerto Rico private system of education. Nine out of twenty-three (39%) are either working as teachers abroad, working in a non-educational related job, or are not working at all. Table 4.1 shows completers data disaggregated by gender and system of education where they are hired as teachers.

UIA completers are well distributed between the Puerto Rico public and private systems of education. In previous years, the EPP had more completers employed at the private system of education. Last year, the Puerto Rico Department of Education raised, significantly, teachers’ salaries and more completers are migrating to this system. The EPP is aware that males must be recruited in all specializations as women composed more than 87% of the completers. In addition, two out of nine (22%) of completers that are not working, are pursuing graduate degrees as full-time students. In conclusion, the EPP completers are well prepared and 60% that remained in Puerto Rico were hired in education positions for which they have been prepared. In addition, two out of twenty-three (8%) run their education business as Childcare Center owner and as a personal educational tutor.