Heidi L. Schnackenberg, Plattsburgh State University, heidi.schnackenberg@plattsburgh.edu
Elizabeth Zadoo, Narmer American College, nyzadoo@yahoo.com
Darcy Aubrey, Seton Catholic Central Schools, darcyaubrey@hotmail.com

Abstract
Accrediting bodies, such as the National Council for the Accreditation of Teacher Education (NCATE), strive to improve the quality of teacher preparation programs.  Through assessment-based analysis of institutional practices, NCATE accredits institutions of higher education that successfully address six Unit Standards.  Although all the NCATE Standards are important, this article is focused on aspects and issues surrounding Standard 2, Assessment and Unit Evaluation.  It is the intention of the authors that their experiences with the NCATE accreditation process will help other institutions develop their own system, with an understanding of what will make it a worthwhile endeavor.  What follows is a combination of guidelines for building the assessment system, lessons that we learned, and words of wisdom about the overall process of accreditation.
 

To better understand the NCATE accreditation process, it’s important to understand some terminology used by the agency.  As accreditation efforts help to ensure that the education programs within an institution of higher education meet the needs and expectations of the entire professional community, it’s important to define that community and its constituents.  According to the NCATE Glossary of Terms, the Professional Community is comprised of "full- and part-time faculty (including clinical faculty) in the professional education unit, faculty in other units of the college/university, P-12 practitioners, candidates, and others involved in professional education" (NCATE, 2006).  Essentially, the community includes all stakeholders that play any type of role in a teacher preparation program.

Within the Professional Community lies the Professional Education Unit (PEU).  The Unit is defined as "the institution, college, school, department, or other administrative body with the responsibility for managing or coordinating all programs offered for the initial and continuing preparation of teachers and other school personnel, regardless of where these programs are administratively housed" (NCATE, 2006a).  At most institutions, the PEU consists of faculty from a variety of areas of academic study.  These can include English Language Arts, Foreign Language, Mathematics, Sciences, Social Studies, Fine Arts, Business and Economics, and of course, programs within education.  In addition, school districts also play an important role in preparing teacher candidates, both as affiliates who support education programs and as consumers who rely on the institution as a source of quality educators, administrators, teacher resources, and professional expertise. The most essential members of the Professional Community include teacher education candidates, who interact with all of the other members of the Professional Community, influencing and being influenced by all of the previously mentioned groups.

It is in conjunction with, and with the cooperation of these aforementioned groups, that institutions of higher education attempt to address the six Standards outlined by NCATE.  Although all the NCATE Standards are very important, this article is focused on aspects and issues surrounding Standard 2, Assessment and Unit Evaluation.  It is the intention of the authors that their experiences with the NCATE accreditation process will help other institutions develop their own system, with an understanding of what will make it a worthwhile endeavor.  What follows below is a combination of areas in which we were successful, lessons that we learned, and words of wisdom along the lines of  “if we knew then what we know now.”

Purpose of Assessment System
Program evaluation via a clearly articulated and implemented assessment system provides data for program improvements and validates the benefits of current practices (Kimball, Harriman, & Hanley, 2002; NCATE, 2001; Pepper & Hare, 1999). The instruments used to gauge this data may range in purpose and/or form in order to include outcome-based and objective-based assessments, pre-/posttest comparisons, course evaluations, and student and alumni surveys, incorporating both quantitative and qualitative information (Gall, Gall, & Borg, 2003; Pepper & Hare, 1999). Knowing what to use in a comprehensive system and how to use it efficiently is unique to each environment. Noted limitations to the effectiveness of an assessment system include time constraints, allocation of resources, fear of negative outcomes based on findings, and a lack of training in research methods appropriate for evaluating school programs (Garcia & Floyd, 2002; Lusky & Hayes, 2001; Mitchell & Yamagishi, 2005).

In April 2002, Japan’s Ministry of Education announced that all primary and secondary schools must complete self-evaluations of their educational activities (this includes school management) and publicly report their findings (Nagao, Kuji-Shikatani, & Love, 2005). The initial reports led to the creation of a professional development project intended to equip “teacher-evaluation facilitators” with the skills and competencies to implement school evaluation systems. The curriculum of the 4-day seminar, developed by the Japan Evaluation Society in collaboration with the Canadian Evaluation Society, set forth six outcome objectives: evaluation theory; foundations in terminology, concepts, objectives, and processes; basic approaches, designs, data collection and analysis; reporting and utilization of results; implementation and organizational issues; and networking among school evaluators. Evaluation of the pilot test demonstrated a continuum, incorporating six forms of assessment: pre-post evaluation, daily workshop assessment by participants, assessment by instructors regarding participants and reflections of delivery, a six-month post-training mail survey, a six-month follow-up seminar, and continuing long-term follow-up assessments on the effectiveness of implementing evaluation programs after completing this professional development. Participants in this professional development activity responded positively to the information gained from the seminar experience, with 95% of their schools conducting the type of evaluations presented.

NCATE recommends that assessment systems for institutions of teacher educators base candidate performance on multiple assessment measures implemented across the duration of the program, from admission to completion (2006b). Because these measures directly relate to candidate success, the Professional Educational Unit is responsible for continuously conducting, evaluating, and revising procedures to eliminate bias, as well as to establish fairness, accuracy, and consistency of performance assessment procedures (Gollnick, 2006). Accountability of this process is achieved by sharing information about program effectiveness with all stakeholders.

Those committed to ongoing program evaluation are more likely to revise their practices to meet the ever-evolving needs of students in today’s society (Lusky & Hayes, 2001). Assessment systems are not unique to education programs.  Consider the CGR approach presented by engineering professor, Neelam Soundarajin (2004). Soundarajin developed Course-Group-Reporting (CGR) to connect the assessments already administered with decisions for program improvements. Each group of faculty becomes responsible to produce reports addressing various important issues from the data collected through a multitude of assessment instruments. This data may be used, not only for changes and improvements, but as evidence of success for current and future stakeholders in the educational process.

Working With Data
Collecting data with a clear purpose guides the selection of an evaluation model or approach (Gall, Gall, & Borg, 2003). The chosen model should fit the purpose of evaluation and the questions being asked, the methods for collecting data, and the relationship amongst all stakeholders involved or affected by the process. Another important consideration is what will be evaluated, taking into consideration program goals, resources and procedures, and program management. Once these have been decided, efforts should now focus on selecting the instruments that fit the design and timeline of the assessment system. Walter Kimball, Nancy Harriman, and Susie Hanley from the University of Southern Maine (USM) considered three attributes of assessment systems: authenticity, continuity, and fairness (2002). Guided by these attributes, their system follows candidates through an internship program with seven significant events strategically planned to review progress over time, beginning with orientation and culminating with a portfolio exhibition. Based on initial program evaluation data that revealed students perceptions of expected outcomes, the Extended Teacher Education Program of USM modified the curriculum to introduce and integrate outcomes earlier and more functionally throughout the program. Three years of subsequent evaluation data indicated improvement in this area, demonstrating beneficial use of their assessment system.

A vast range of information contributes to a comprehensive assessment system. Modified by the purpose and goals, this includes, but is not limited to: applicant qualifications and admissions data, candidate proficiencies, competency exams, licensure data, unit operations, program and course evaluations, alumni surveys, course assignments and rubrics, reflection journals, student-faculty conferences, portfolio assessment forms, focus groups, capstone projects, graduation rates, grade point averages, and field practicum evaluations (Garcia & Floyd, 2002; Ingram, Haynes, Davidson-Shivers, & Irvin, 2005; NCATE, 2006; Ward & McCotter, 2004; Weisenbach, 2002). To track these assessments, several methods have been employed, including a variety of matrices. Ward and McCotter (2004) documented rubric results to note patterns in candidate performance.  Garcia and Floyd (2002) surveyed 48 programs to describe what schools are doing to assess outcomes and how this data is used for program and curriculum development.  One program included in this study posed two guiding questions to analyze documentation: “To what extent does the curriculum reflect the mission, goals, and objectives of the program?” and, “To what extent have our graduates mastered the knowledge, values, and skills embodied in our mission, goals, and curriculum objectives?” (Garcia & Floyd, 2002, p.379). Those programs with a clearly defined purpose that used an organizational framework were more able to present a direct link matching specific objectives with corresponding measurement tools. Furthermore, for a program to effectively use the results to influence program planning, the gathered outcome data needs to have supporting process data to complete the loop. Citing theoretical background of effective methods for training teachers, Pepper and Hare (1999) modified Robert Stake’s Countenance Model of Educational Evaluation to address congruence between standards and data collection under the description matrix, and to demonstrate how judgments and recommendations influence program revision, elimination, and enhancements through research-based knowledge under the judgment matrix.  A curriculum matrix developed and implemented by the psychology program at Seton Hall University ensures that each department articulates and monitors their objectives across the spectrum of courses offered (Levy, Burton, Mickler, & Vigorito, 1999). Students as well as faculty completed data for this matrix, allowing comparison of impressions indicating how well the program achieved its objectives.

The 2002 research of Garcia and Floyd examined how a number of schools assess educational outcomes and then integrate the data into program and curriculum development. The most prominently implemented tools included alumni surveys, grades or GPAs, field practicum evaluations, and student course evaluations.  However, their discussion cautions against relying on tools focused on perceptions and attitudes, or measurements with questionable reliability and validity. Considering this advice, the following studies are encouraging.  Loadman, Freeman, Brookhart, Rahman, and McCague (1999) created the National Survey of Teacher Education Program Graduates to address validity and reliability through norm-referenced data of the instrument.  Alumni and student databases provide valuable and necessary information through surveys, enrollment, course evaluations, and cross-referenced data (Davidson-Shrivers, Inpornjivit, & Sellers, 2004; Ingram, Haynes, Davidson-Shrivers, & Irvin, 2005).  Reflections, though difficult to measure quantitatively, are a vital component of pre-service teacher growth and development that may be used as a benchmark assessment tool, as well as to guide planning and instruction (Kimball, Harriman, & Hanley, 2002; Ward & McCotter, 2004). Ward and McCotter (2004, p. 250) developed a rubric to determine the level of reflection (routine, technical, dialogic, and transformative) across dimensions of focus of concern, process of inquiry, and change in practice and perspective.  Teacher Work Samples are complex performance assessments that document teacher education candidates through teaching a set of lessons, covering all aspects of the instructional unit over a minimum of four weeks (Denner, Salzman, & Bangert, 2002). In linking assessment to performance, Denner, Salzman, and Bangert (2002) created two scoring rubrics, one holistic and one analytic, which they tested for generalizability and validity. Used in conjunction with a variety of assessment instruments, including both quantitative and qualitative measures, these tools may provide insight into patterns or discrepancies between the mission, goals, and objectives, and the actual outcomes of a teacher preparation program.

Collaborating With Colleagues and the Institution
Collaborating on an ongoing initiative such as program assessment means taking into consideration a number of factors regarding the experience and expertise each member brings to the endeavor, and the level of commitment required from various constituents. The notion of program assessment leads some faculty to fear program changes or cuts that may directly impact their employment (Garcia & Floyd, 2002; Lusky & Hayes, 2001; Sargent & Waters, 2002). The protection of tenure systems often allows more experienced faculty a buffer zone in which they may choose to pass on job requests in favor of other professional pursuits, creating more responsibilities for junior faculty (Bess & Goldman, 2001). At the same time, new faculty undergo a process of organizational socialization through which they embrace the university’s values, expected behaviors, and social knowledge required for recognition among the professional community (Schrodt, Cawyer, & Sanders, 2003). Untenured faculty are still receiving both formal and informal knowledge about these aspects of the institution. Results of research conducted by Olsen (1993) indicates that faculty satisfaction with the work environment declines in the first three years as they undergo high amounts of stress caused by the tenure process. In addition, the research conducted by Schrodt, Cawyer, and Sanders (2003) revealed comments indicating junior faculty desire for additional support from mentors and colleagues regarding issues of research, university politics, and career-related information. A recent study of adjunct faculty, who may be included in some manner in the assessment system, describes a significant and negative relationship between relative deprivation and job satisfaction,  professional commitment, and the performance of organizational citizenship behaviors, all of which influence the collaborative efforts required for an effective, sustainable assessment system (Feldman & Turnley, 2004). Efforts to reduce relative deprivation among members by recognizing their achievements and contributions to the professional community include fringe benefits, training opportunities, and competitive salaries.

Resources for evaluation research are “the personnel, equipment, space and other cost items needed to implement program procedures” (Gall, Gall, & Borg, 2003, pp. 547). Institutional support is an important contextual factor in the outcome of collaborative compilations and may come in the form of information technology staff, administrative staff, collegial supporters and mentors, and resource allocation (Sargent & Waters, 2004). A survey of deans and NCATE coordinators revealed costs in personnel, time, and technical resources were a primary concern in implementing and maintaining a properly functioning assessment system in compliance with Standard 2 (Mitchell & Yamagishi, 2005). Clarity and cohesiveness of an assessment system is directly related to the allocated resources (Garcia & Floyd, 2002). The design of the assessment system must take into consideration hidden costs in the form of supplies and resources that may be reallocated from other programs and budgetary expenses (Gall, Gall, & Borg, 2003).

Welsh and Metcalf (2003) explored four variables in a review of faculty support for institutional effectiveness activities relating to program assessment and accreditation proceedings. These variables include how the faculty and institution define quality, internal verses external motivational factors, depth and breadth of the project undertaking, and the level of involvement for various stakeholders. Participant responses demonstrate suspicion as to the purpose of accreditation endeavors, however increased institutional support appeared to diminish these sentiments. The discussion suggests three benchmarks of “best practices” for cultivating support from faculty. First, the primary purpose of implementing activities must convey an initiative to improve programs and services. Second, faculty need to develop a sense of ownership in the process in order to fully commit themselves. Third, an outcomes-oriented perspective allows stakeholders to see quantitative “proof” of progress and quality. These practices, along with open and honest communication facilitate in creating an atmosphere of collaboration and productivity.

Implications
Following from the discussion above, it is clearly important to have the administration at your institution fully supportive of the accreditation effort.  In general, this is not difficult to acquire as most administrations realize the criticality of accrediting their colleges, schools, and/or divisions of education.  (In some states, accreditation by a federally or state-recognized accrediting body is required!)  However, while verbal support from administration for accreditation is often forthcoming, it is important that money, release time, and most importantly, institutional cooperation also be given. 

Monetary support is essential for many of the initiatives that need to be undertaken to achieve accreditation.  It is likely that a dedicated “NCATE Coordinator” will need to be hired – or at least given release time – to organize this significant undertaking.  Many aspects need to be organized for the accreditation process, including data collection, subcommittee formation, and cooperation from other areas of the university such as administration, the arts and sciences, and media services.  As well, writing the final institutional report is an enormous job and clearly needs to be facilitated by one individual so that the document, likely put together by various individuals or groups, reads as if written in one voice.  These are merely a few of the vast responsibilities of an NCATE Coordinator and this individual needs to be compensated by the institution in some regard.

Another undertaking that requires monetary support is data collection for unit assessment purposes in order to provide data, or evidence, that students are attaining goals or objectives supported by the institution’s conceptual framework.  Often institutions do not consider the cost of mass mailings of instruments such as alumni or employer surveys, but for larger institutions, these could number in the thousands for any individual survey.  As well as mailing costs, photocopying, paper, envelopes, and return envelopes, also add to overall costs.  Further, once these instruments are returned, analysis of the data takes place and in one form or another, this also costs money.  If these instruments are electronic, then the development of web-based surveys could necessitate purchasing specialized software or paying a programmer/developer.  If survey instruments are hardcopy, then data analysis needs to be done by hand.  While a committee chair or professor could do this, it is advisable to hire a research assistant to complete this extensive, and often times tedious, work.  As this type of data collection is meant to be a continuous, sustainable endeavor it is important that a position or part of an individual’s workload be dedicated to this task and supported at the outset of undertaking accreditation.  However, it is strongly advised that an electronic data-gathering/data-analysis interface be adopted.  This is preferable not only because it is a more efficient way to work with necessary data, but it also facilitates the dissemination of gathered data to the involved parties, i.e. program chairs, directors of student teaching, deans, etc.  It is key that these various groups or individuals communicate easily and have access to candidate information as they progress through the various teacher preparation programs. 

Other monetary expenses may come in the form of course release or extra service pay for faculty or staff.  While often institutions give faculty the assurance that work on the accreditation effort will count as scholarship or service work, to the rest of the professional communities, that doesn’t count for much in terms of an individual’s professional profile.  As well, accreditation can be the whole of an individual’s service work and then some.  It is important that faculty working on the accreditation effort sacrifice neither their scholarly activities, nor the entirety of other service work, to accomplish this monumental task.  It is therefore important that they be given time or other compensation to complete work associated with NCATE accreditation. 

The writing of the institutional report is a difficult endeavor at best and involves the cooperation and collaboration of many diverse parties and individuals.  It takes an individual who has a certain amount of seniority within an academic institution, as well as one who has substantial experience with the NCATE process.  It’s best if the individual coordinating the writing of the report is the NCATE coordinator and/or senior faculty.  If this individual prefers to pass the writing or editing of the Institutional Report on to a faculty member who may be a better writer, that can work also, however the organizing and managing of the report needs to remain in the hands of the NCATE Coordinator.  The reason for this is that the information for many aspects of the report need to be culled from administration, institution-wide committees, personnel in charge of university resources and governance, faculty in the arts and sciences, as well as faculties and committees within education.  Within normal institutional operations it is quite difficult to coordinate all these sources of information, not to mention trying to do so under a deadline.  Only an individual with some clout in the university could feasibly do this with any success and expediency.  Not to mention that only a long-standing member at an institution would know where to go for needed information without having to spend extra time researching such resources.  Clearly, it’s unadvisable to leave accreditation solely to junior faculty, although certainly they can play important and necessary roles within such an effort.

To that end, it is important that the whole or the majority of accreditation responsibilities not fall to untenured faculty.  Unfortunately, more often than not, untenured faculty are the ones who are given various accreditation jobs to complete without having the proper knowledge or support to accomplish them.  It is senior (tenured) faculty who should be working on the various accreditation initiatives and reports, as they are the ones who are able to most efficiently and effectively complete this work.  Senior faculty has knowledge of the institution’s resources, human or otherwise, to enable them to implement necessary initiatives or acquire needed information quickly.  Specifically, facilitating the acquisition of assessment data, the discussion with the involved faculties of data-driven conclusions that may modify professional practice, and most importantly, the writing of the institutional report, should all be undertaken by dedicated, tenured faculty members. 

Recommendations
Before you enter into the accreditation process, it is important to remember not only the items discussed above, but also the following set of guidelines.  These guidelines will assist in creating an overall approach to accreditation with not only an eye on the beginning stages, but also the continuing processes, sustainable initiatives, and final outcomes.  They facilitate an overall view of accreditation in more or less a linear fashion, and assist in making it a much more effective and efficient process.  The recommended guidelines for NCATE accreditation are as follows: 

1. Create your conceptual framework with an eye heavily trained on performance-based outcomes and assessment.  Make it a collaborative, group effort, involving members from all areas of the professional unit (NCATE, 2006b).  Critical to the accreditation process when defining the framework is the involvement of faculty from the Arts and Sciences.  Input from all these groups and individuals will ensure a well-rounded and representative work including philosophies and ideologies from all professionals involved with teacher candidate preparation.
    
2. Define your objectives from your conceptual framework.  Make them clear and straightforward.  The group that created the conceptual framework is generally the best group to create the objectives.
    
3. Form a different committee to create assessment outcomes.  While the group that created the conceptual framework and resulting objectives can also create the assessment outcomes, it is better for a group with a fresh perspective to do this.  It will be easier for them to align the assessment with the objectives without the confounding nuances of the conversations that surrounded the creation of the conceptual framework and objectives.  Be sure to include representatives from all teacher certification programs, field placement and Arts and Sciences.  It is also crucial to the assessment effort to ask that a representative from Institutional Research or Alumni Affairs sit on the committee as well.  This is important because the assessment effort is larger than teacher education candidates attending university.  Data collection may also extend to alumni, employers, field supervisors, parents, P-12 students, and community members.  
    
4. Define your assessment outcomes from your conceptual framework objectives.  Make them performance-based and behavioral.  It’s important that they are written in terms that are measurable rather than general or vague.  This is essential when defining dispositional outcomes, which are often hard to identify.  It is helpful to thoroughly research this area in order to understand the difficulties of measuring dispositions quantitatively. 
    
5. Since unit assessment involves all teacher certification programs, it is important to bring the assessment outcomes that the committee created to all faculty members involved in the accreditation effort.  This can be accomplished either through assessment committee representatives bringing the outcomes to their home departments or having a large general session where all faculty could attend and the outcomes could be presented, discussed, modified, and hopefully, adopted.  It is important that this step occur because NCATE accreditation is not meant to be simply a “paper chase.” It is meant to be a process that promotes systemic change within teacher education and preparation.  In order for this to occur, all individuals concerned with teacher preparation and accreditation need to be involved in decision-making, either in a large group or small committee.  This allows faculty and members of the academic unit to take ownership of the accreditation process and any changes resulting from it.  As well, when all are involved in decision-making, each individual is able to discuss, and thereby promote, the goals, frameworks, outcomes, etc. that are adopted by the unit at large.  This promotes an institutional culture in which constituents are informed and invested in accreditation, making the process less of a set of hoops to jump through but rather, an opportunity to reflect upon and improve how teacher education candidates are prepared.
    
6. It is also important not to forget that unit assessment involves the assessment of unit operations as well as assessment of degree-granting programs.  Unit operations encompass activities such as the evaluation of advisement and student teaching placements.  It also involves the review and evaluation of the assessment system as a whole and an institution’s conceptual framework.  Ordinarily, a university or college has evaluation measures already in place for areas like academic advisement so it is only a matter of obtaining the necessary information from the appropriate office and documenting the process.  For endeavors such as assessing conceptual frameworks or assessment systems, new procedures may need to be developed, implemented and documented.
    
7. Once the academic unit has adopted the assessment outcomes, align them to the Special Professional Association (SPA) standards for each degree-granting program.  In this way, when the assessment outcomes are subsequently aligned to existing data collection instruments, the instruments measure both assessment outcomes and SPA standards.  Also, it is more than likely that modification of the instruments will be necessary to better measure the outcomes.  In addition, some of the outcomes may not be assessable with the current instruments. 
    
8. Create instruments, processes, or programs to measure the remainder of the assessment outcomes.  Be sure to have a mixture of both qualitative and quantitative measures for the overall collection of outcomes.  More than one assessment measure for any one outcome is desired, but not required.  Be sure that the assessment measures or processes that are put into place are sustainable, reasonable, and manageable in terms of an overall assessment program.  If unreasonable assessment procedures are established, it is likely that data will not be able to be collected over time and therefore candidate change, growth and improvement in attaining the knowledge, skills, and dispositions put forth in the conceptual framework will be impossible to track.
    
9. It is especially crucial to have input from members of the PEU from P-12 environments throughout the assessment process, but especially in steps 7 and 8.  It is with their help that manageable instruments will be created and adopted, and ultimately implemented.  If the P-12 constituents are a part of this endeavor from the outset, they will feel invested in the conceptual framework, assessment system, and the accreditation effort as a whole.  Without their cooperation, much needed evidence will not be gathered nor will the conceptual framework be promoted to all facets of education candidates’ experiences.
    
10. Align data collection instruments and evidence to the Four Transition Points outlined by NCATE.  It is fine to have more transitions, gateways, or checkpoints than those that are reviewed by the national accrediting body, but it is important that there are not less than four.  The four transition points NCATE examines are 1) entrance into education programs; 2) entry into student teaching, internship, practicum; 3) exit from student teaching, internship, practicum; and 4) exit from education programs.  Divide these four transition points into initial and advanced certification programs as both types of certification must be accounted for at each point.  Be clear what types of assessments are gathered and reviewed at each transition point.  Create rubrics or descriptors of acceptable performances/scores/etc. for each assessment at each point.  State with what body, office, or person, the transition points are tracked for each candidate and how successful transition from one point to the next is communicated to necessary individuals (i.e. program chairs, unit assessment chairs, deans, etc.). 
     
11. Investigate and adopt efficient data collection measures and procedures.  Create electronic databases and programming routines to extract standardized test data and other demographic data that may be available for assessment use.  Be sure answers to your surveys can be compiled and analyzed electronically.  Web-based instruments that export into programs such as Microsoft Access, Microsoft Excel, or SPSS are preferable.  As well, server space that is common to all stakeholders in the PEU should be attained to house these databases.  If that is not an option, create surveys that are answerable in Scantron format so that the hardcopies can simply be run through a machine and answers aggregated and data tabulated.  Using entirely hardcopy data collection instruments is not viable for sustained data collection.  A full-time research assistant would need to be hired to input and analyze data from numerous candidate surveys, employer surveys, and/or alumni surveys.  This is especially pertinent if you have large population of pre-service teacher candidates.  You may find, initially, that your institution has a multitude of data already, but collected in a variety of places and databases.  It is important to bring this information together in one place in order to make analyses more efficient and to facilitate the communication of important information.  The ability to link all of the data collection measures and databases will be an issue that should be addressed at the planning stages of your Assessment System. 
     
12. Prior to beginning any data collection, attain approval or exemption from the Use of Human Participants examining board at your institution.  This is a critical step in ethical assessment and data gathering processes.  To utilize all the data that you will likely need in your assessment, such as GPAs, curriculum codes, perhaps gender, ethnicity or age, you will need to attain this type of permission from your university.
     
13. Begin data collection.  Be specific about when you would like surveys or information returned from your various participant populations.  Include cover letters that explain the accreditation process and the importance of responses with surveys to potential employers or alumni.  Be sure that supervisors or professional development educators are aware of candidate deadlines for returning data collection instruments.  This will help to avoid updating “finalized” data.
     
14. Input and analyze incoming data as soon as it is returned.  Procrastinating on this effort merely leaves piles paper and many computer files lying around with the potential for getting lost or misplaced.  It is also critical to analyze data immediately so that timely conclusions and decisions about improving professional practice can be made and implemented. 
     
15. Be sure to aggregate and disaggregate incoming data.  Aggregated data is useful for assessment committee, unit, and administrative purposes.  Disaggregated data is more useful to individual degree-granting programs as well as departments in the arts and sciences that prepare teacher education candidates with specific academic concentrations.  Both aggregated and disaggregated data must be efficiently disseminated to the appropriate programs, governing bodies, and offices so that conclusions can be drawn about professional preparation of teacher education candidates.  These types of data can also be used as evidence during an NCATE site visit.  It is therefore, imperative that all data collection instruments are designed so as to identify a variety of anonymous, demographic information (such as degree program, arts and science concentration, year in university, etc.).  These demographic data will also assist in linking all gathered information together and analyzing it for trends across programs, concentrations, gender, etc.
     
16. Once data collection measures have been are in place, measures of fairness and consistency should be ensured.  If standardized tests are being used, reliability and validity information can generally be obtained from the testing companies.  When “homegrown” surveys are in use, true statistical validity and reliability testing may not be an option.  It is exceedingly difficult to validate self-made questionnaires so other measures of ensuring fairness and consistency should be implemented.  One example would be to assemble a small committee of individuals not involved in the original alignment of “homegrown” instruments to conceptual framework outcomes.  This ad hoc committee could then examine each of the instruments and document whether they do or do not align to certain outcomes based on specific criteria.  Granted, this is not a way to ensure validity or reliability in the statistical sense, but it is taking a step toward ensuring fairness and consistency and that is all that NCATE requires.  (In fact, when reviewing NCATE documentation, they do not use the terms “validity” and “reliability.”  Instead, their exact language is “fairness” and “consistency.”).
     
17. As an assessment committee, distribute the aggregated and disaggregated data to chairs of the education departments, chairs of the arts and sciences departments which participate in teacher education, chairs of the NCATE standards committees (if your institution has those), members of a unit governing body and other involved stakeholders.  This can take a vast amount of time to organize initially, but with the help of a graduate assistant or work-study student, the process can progress much more quickly.  It is also helpful to include a detailed, explanatory cover letter with your data, explaining the instrument(s) from which is was obtained, the purpose for obtaining the data, and how it was analyzed.  Once the data is properly organized and the cover letter created, it is simply a matter of making photocopies and doing (mostly) inter-office mailings.
     
18. Allow some time for departments and committees to review the aggregated and disaggregated data and to draw conclusions based on the data analysis.  What does it indicate about the professional practice within in the programs?  Do areas need to be modified?  Do certain aspects need to be recognized and praised for their success?  Questions such as these should be carefully considered and discussed. 
     
19. Once smaller groups have had a chance to review the data, each should decide upon recommendations to make to the overarching unit-governing group regarding modifications to teacher candidate preparation.  Since professional practice is being assessed as a “unit,” the recommendations made by one group or committee effect all the education programs and must be discussed, decided upon, and adopted as a whole.  If not, it is not “unit” assessment.
     
20. If the larger unit-governing group accepts recommendations for the modification of teacher preparation by program, or as a whole, then the revisions must follow the normal institution process, or “lines of authority,” for programmatic changes.  This usually involves the completion of standard institutional forms, acceptance by various curriculum committees, university governing bodies, and ultimately, the signature of the university or college president and/or provost.
     
21. Recommendations to modify professional teacher educator preparation need to be put into practice in a timely manner.  These recommendations can include omitting specific application procedures because they are redundant, raising or lowering required GPAs or standardized test scores, or emphasizing the use of scaffolding in teaching throughout the teacher preparation programs.  Modifications can be of a minor, more administrative nature, or more far-reaching into education courses and/or field experiences.  Whichever the case, it is important to discuss modifications to teacher preparation as a large group and come to mutually agreeable conclusions.  Otherwise, modifications or changes in practice will only be done by a few, rather than reaching across the unit.  As well, a timeline for larger changes to be implemented may be needed so that involved individuals can work on and institute initiatives in a manageable way.
     
22. Assess modifications using reliable and valid data gathering techniques (surveys, focus groups, pre-tests and posttests with a spectrum of stakeholders).  In some cases, be sure to allow enough time for a new initiative or program modification to take hold in the institution before assessing it.  Assessing a modification too early can result in skewed data and an inaccurate perception of the success or failure of the change.  Similarly, assessment of a modification too late can result in the propagation of an unsuccessful or even harmful change much too long.  The nature of the program modification, as well as input from initiative implementers and candidates, will likely dictate when to assess a change to the professional preparation of teacher education candidates.
     
23. Continue to communicate data-driven recommendations and improve the professional preparation of teacher education candidates.
     
24. At the time of this writing, for institutions having NCATE accreditation visits in the Fall of 2003 or later, at least three years of data or evidence of candidates attaining goals outlined in the conceptual framework is required.  It is also worth noting that institutions having their site visit at this time or later need to have their assessment systems in place and operational.  No longer is an “assessment plan” acceptable for accreditation.  Now any plans must be put into place and evidence of its successful operation must be exhibited.
     
25. Once an assessment system has been established and is in use, and other areas of the NCATE Standards are also progressing, schedule a “mock visit” with a consultant who is well versed in the process of NCATE accreditation.  The sooner before an actual on-site visit that this “mock visit” occurs, the better.  It is likely that, no matter how hard an institution has worked at the Standards, the consultant will find areas that need improvements and modifications.  The more time given an institution to make these required changes, the better for all concerned.  Making modifications to the Standards, especially the Assessment System, under a tight time frame is at best difficult and at others, not possible at all.  A “mock visit” scheduled at the earliest time an institution is ready is best. 
     

Conclusions
Although accreditation should be a positive process meant to reflect and improve upon the professional practice of teacher-education institutions, the process often evolves into something quite the opposite.  Rather than building collegiality and creating a collaborative, efficiently functioning unit, the process of accreditation can seem like “just one more thing” that is often pushed onto the shoulders of the unfortunate (or untenured!).  If you see emotions, tempers, and stress levels running high the closer it gets to your site visit, be assured that this happens at many institutions.  The important thing to remember is that, although accreditation is a serious process, it’s best not to take yourself too seriously as you go about getting ready for a site visit.  Many jobs and/or duties associated with accreditation are equally difficult and time consuming.  It’s important to remember that you’re not “the only one” and to be gentle with both yourself and your colleagues when it comes to deadlines, work to be done, and scouting about for necessary information.  As natural as it is to feel the stress of this process, take care to give compliments and support to your colleagues.  It will not only improve the overall atmosphere of your unit (which is quite critical during an accreditation visit), but it will also help to keep the process in perspective and, hopefully, to feel the value of what it is that you and your colleagues are doing. 

Finally, do not forget your candidates.  The reality of the situation is that this process is for the betterment of their professional preparation.  They too are stakeholders will be learning about the process.  It is important to keep open dialogue with these members of the educational community and when doing so, to keep a record of any discussions because they add to the assessment and the overall improvement of degree programs.

Bess, J. L., & Goldman, P. (2001). Leadership ambiguity in universities and K-12 schools and the limits of contemporary leadership theory. Leadership Quarterly, 12(4), 419-450.

Davidson-Shivers, G. V., Inpornjivit, K., & Sellers, K. (2004). Using alumni and student databases for program evaluation and planning. College Student Journal, 38(4), 510-520.

Denner, P. R., Salzman, S. A., & Bangert, A. W. (2001). Linking teacher assessment to student performance: A benchmarking, generalizability, and validity study of the use of teacher work samples. Journal of Personnel Evaluation in Education, 15(4), 287-307.

Feldman, D. C., & Turnley, W. H. (2004). Contingent employment in academic careers: Relative deprivation among adjunct faculty. Journal of Vocational Behavior, 64(2), 284-307.

Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Evaluation research. In A. E. Burvikovs, M. Kriener, & C. Tridente (Eds.) Educational research: An introduction (7^th ed.) (pp. 541-577). New York: Pearson Education, Inc.

Garcia, J. A., & Floyd, C. E. (2002). Addressing evaluative standards related to program assessment: How do we respond? Journal of Social Work Education, 38(3), 369-382.

Gollnick, D. (2006). Program assessment in the unit assessment system. Retrieved June 11, 2006, from http://www.ncate.org/documents/articles/UnitandProgramReview.pdf.

Ingram, K. W., Haynes, L. L., Davidson-Shivers, G. V., & Irvin, R. (2005). Building an alumni support community: Tracking alumni for program evaluation and added value. College Student Journal, 39(2), 203-217.

Kimball, W. H., Harriman, N., & Hanley, S. (2002). Assessment and standards for professional improvement. In A. R. Tom (Ed.), Rethinking standards through teacher preparation partnerships (pp. 37-52). Albany, NY: State University of New York Press.

Levy, J., Burton, G., Mickler, S., & Vigorito, M. (1999). A curriculum matrix for psychology program review. Teaching of Psychology, 26(4), 291-294.

Loadman, W. E., Freeman, D. J., Brookhart, S. M., Rahman, M. A., & McCague, G. J. (1999). Development of a national survey of teacher education program graduates. Journal of Educational Research, 93(2), 76-89.

Lusky, M. B., & Hayes, R. L. (2001). Collaborative consultation and program evaluation. Journal of Counseling & Development, 79(1), 26-38.

Mitchell, A., & Yamagishi, Y. (2005). The results are in: What deans and NCATE coordinators think about the NCATE unit standards. Retrieved June 11, 2006, from http://www.ncate.org/documents/articles/TheResultsAreIn.pdf.

Nagao, M., Kuji-Shikatani, K., & Love, A. J. (2005). Preparing school evaluators: Hiroshima pilot test of the Japan evaluation society's accreditation project. Canadian Journal of Program Evaluation, 20(2), 125-155.

National Council for Accreditation of Teacher Education (2001). Aligning assessments with standards: A synthesis of guidelines from current practice adapted for use in teacher education and NCATE accreditation. Retrieved June 11, 2006, from http://www.ncate.org/documents/aligning%20assessments%20and%20standards.pdf.

National Council for Accreditation of Teacher Education (2006a). Glossary of NCATE terms. In NCATE Unit Standards (2006 Edition) (pp. 52-58). Retrieved June 11, 2006, from:http://www.ncate.org/documents/standards/unit_stnds_2006.pdf.

National Council for Accreditation of Teacher Education (2006b).  NCATE Unit Standards (2006 Edition) (pp. 21-24).  Retrieved June 11, 2006, from: http://www.ncate.org/documents/standards/unit_stnds_2006.pdf.
 

Olsen, D. (1993). Work satisfaction and stress in the first and third year of academic appointment. Journal of Higher Education, 64(4), 453-471.
 

Pankratz, R. & Norman, A. (2004).  Lessons Learned – Teacher Education Program Assessment and Improvement. Paper

          presented at the annual meeting of the New York Association of Colleges for Teacher Education and the New York State

          Association of Teacher Educators; Saratoga Springs, NY.

Pepper, S. K., & Hare, R. D. (1999). Establishing research-based knowledge in teacher education. Journal of Research & Development in Education, 32(2), 104-112.

Sargent, L. D., & Waters, L. E. (2004). Careers and academic research collaborations: An inductive process framework for understanding successful collaborations. Journal of vocational behavior, 64(2), 308-319.

Schrodt, P., Cawyer, C. S., & Sanders, R. (2003). An examination of academic mentoring behaviors and new faculty members' satisfaction with socialization and tenure and promotion processes. Communication Education, 52(1), 17-29.

Soundarajan, N. (2004). Program assessment and program improvement: Closing the loop. Assessment & Evaluation in Higher Education, 29(5), 597-610.

Ward, J. R., & McCotter, S. S. (2004). Reflection as a visible outcome for preservice teachers. Teaching and Teacher Education, 20(3), 243-257.

Weisenbach, E. L. (2000). Making the case: Marshalling evidence about candidate proficiency. Retrieved June 11, 2006, from http://www.ncate.org/documents/articles/weisenbach%20making%20the%20case.pdf. 

Welsh, J. F., & Metcalf, J. (2003). Cultivating faculty support for institutional effectiveness activities: Benchmarking best practices. Assessment & Evaluation in Higher Education, 28(1), 33-45.