3.3 Institutional Effectiveness
3.3.1 Institutional Effectiveness
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:
Judgement of Compliance:
Compliance
Narrative:
Each academic program and support unit at the University of Tennessee at Chattanooga (UTC) annually identifies three to five expected outcomes which are consistent with the program’s mission, the UTC Strategic Plan, and priorities of the University’s governance bodies. These outcomes are documented in the TaskStream workspace assigned to each program and support unit. The extent to which expected outcomes are achieved is assessed by the development of an assessment plan which defines appropriate targets and means for measuring each outcome. The assessment plan is also recorded in the TaskStream work space. Evidence is then collected to determine if the outcomes have been attained. At the end of the academic year, assessment results are attached to the annual assessment plan, and attainment of outcomes is determined. In cases where evaluation by appropriate faculty and staff determine that outcomes were not fully attained, inappropriately assessed, or where the target was too high or too low, a plan for improvement is devised. The completed assessment plan is sent to the college dean or other supervisor for review. Upon completion of the assessment process, the plan is implemented for the next fiscal year. Information pertaining to UTC’s procedures for assessment and improvement may be found on the UTC Institutional Effectiveness web link on the Office of Planning, Evaluation, and Institutional Research (OPEIR) web page.
TaskStream Outcomes Management System
In 2008, UTC acquired and implemented the software application, AMS TaskStream©, for managing and documenting the outcomes assessment process. A full-time position of Outcomes Management Analyst was funded and filled to provide faculty and staff in each academic and support unit with essential training and support in the development of program outcomes and assessment plans and in the use of the AMS software. The Outcomes Management Analyst manages and maintains the TaskStream system and provides technical support to all campus users.
TaskStream allows all units to enter mission statements, outcomes, assessment plans, and assessment results. Curriculum mapping features allow the units to link outcomes to the University Strategic Plan, College or Division goals, accrediting standards, and other relevant standards and goals, both external and internal. All 2008-09 and 2009-10 assessment plans have been developed and filed in TaskStream. The implementation of TaskStream has enabled complete documentation of the Assessment-Improvement activities at UTC.
3.3.1.1 Educational Programs, to include student learning outcomes
Judgment of Compliance:
Compliance
Narrative
Identifies Expected Outcomes
Each degree-granting academic program at the University of Tennessee at Chattanooga annually identifies three to five expected outcomes. Each outcome is consistent with the program’s mission, the UTC Strategic Plan, and priorities of the colleges in which the program is housed. Relevant accrediting criteria and external standards are also linked to the outcomes. The outcomes and other relevant documents are recorded in the TaskStream workspace assigned to each academic program.
Each degree-granting academic program identifies annual outcomes, and develops an assessment plan at the beginning of the academic year – upon completion of the previous year’s assessment- improvement cycle. Results from the previous year’s assessment process are evaluated carefully to identify program improvements and changes in the assessment methodology on which to modify the plan.
Assesses Extent of Outcome Achievement
Each program uses an assessment plan to assess outcomes achievement. This plan defines appropriate targets and means for measuring each outcome and is recorded in the TaskStream work space. Evidence is then collected to determine the extent to which each outcome has been attained. At the end of the academic year, assessment results are attached to the annual assessment plan, and the extent to which each outcome has been attained is evaluated by the program faculty. In cases where faculty and staff determine outcomes were not fully attained, inappropriately assessed, or where the target was too high or too low, a plan for improvement is devised. The completed assessment plan is sent to the college dean or other supervisor for review.
Guidelines and examples for writing learning outcomes and assessment plans for academic programs are provided on the Institutional Effectiveness web link on the OPEIR web page.
Another means by which outcomes are assessed is through the Academic Program Review Process. UTC’s Academic Program Review process is a major summative aspect of the Academic Planning Process at UTC. It is set up on a five-year cycle and has been mandated for several decades by the Tennessee Higher Education Commission (THEC) as part of the Tennessee Performance Funding Program. A comprehensive review of each non-accreditable undergraduate and graduate program is required once every five years. The results of the review are used by THEC in determining additional revenue allocation to institutions that prove evidence of academic assessment and improvement.
At the institutional level, academic program review is intended to provide faculty and academic administrators with information to assess program strengths and weaknesses. This information should play a major role in helping faculty to identify annual outcomes and assessment procedures, improve quality, and justify needed resources. Program review is an essential component in academic planning and serves as a venue for evaluating the impact of the annual assessment-improvement process. At the heart of program review is the self-study process and the resulting self-study document. Program faculty conduct the self-study document and select an external reviewer from a peer institution to evaluate the program.
The Office of Planning, Evaluation and Institutional Research (OPEIR) works closely with academic deans and program faculty to coordinate the program review process on the UTC campus. OPEIR provides guidance and information during the self-study. All institutional data required in the self-study are provided by OPEIR. OPEIR also assists college deans in coordinating the external review and provides additional support after the completion of the self-study to help departments use program review results in the planning process.
During academic year 2009-10, five undergraduate programs and one graduate program underwent program review. The links below provide access to the each of these self-studies, the reviewer checklists, and the reviewers’ narrative evaluations:
BS and BA in Chemistry
BA in Foreign Languages
BS in Health and Human Performance
BS in Applied Mathematics and BA in Mathematics
PhD in Computational Engineering
Provides Evidence of Improvement
After annual assessment findings are compiled, summarized, and posted in the TaskStream workspace, faculty meet to discuss and interpret the findings by identifying appropriate programmatic changes to curriculum, methodology, technology, or learning environment. Assessment methodology, including measurements and targets, are also evaluated to determine if changes are indicated there. Improvement actions, other changes, and discussion of findings should be stated in the recommendations, notes, and reflections sections associated with each outcome in the findings workspace in TaskStream. An overall summary of action taken by the program faculty as a result of assessment should be stated in the Overall Recommendation of that workspace.
Completed Academic Program Assessment Plans – 2009-10 – Completed assessment plans for all academic programs are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative academic programs in each of UTC’s four colleges are provided here for academic years 2009 – 10 as documentation of the assessment and improvement process in those programs.
Completed Academic Program Assessment Plans – 2008-09 – Completed assessment plans for all educational support units are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative academic programs in each of UTC’s four colleges are provided here for academic years 2008-09 as documentation of the assessment and improvement process in those programs.
3.3.1.2 Administrative Support Services
Judgment of Compliance:
Compliance
Narrative:
Identifies Expected Outcomes
Each administrative support unit at the University of Tennessee at Chattanooga annually identifies three to five expected outcomes or objectives. Each outcome or objective is consistent with the mission of the administrative support unit, the UTC Strategic Plan, and priorities of the UTC division in which the unit is housed. Relevant external standards are also linked to the outcomes. The outcomes and other relevant documents are recorded in the TaskStream workspace assigned to each administrative support unit.
Assesses Extent of Outcome Achievement
Each unit uses an assessment plan to assess outcomes achievement. This plan defines appropriate targets and means for measuring each outcome and is recorded in the TaskStream work space. Evidence is then collected to determine the extent to which each outcome has been attained. At the end of the academic year, assessment results are added to the annual assessment plan (under results in the TaskStream workspace), and the extent to which outcomes were attained and goals were met is determined. In cases where evaluation by appropriate staff determine that outcomes were not fully attained, were inappropriately assessed, or where the target was too high or too low, a plan for improvement or a revised assessment plan is developed. The completed assessment plan is sent to the vice chancellor, associate vice chancellor, or other supervisor for review.
Guidelines and examples for writing learning outcomes and developing assessment plans for support units are provided on the Institutional Effectiveness web link on the OPEIR web page.
Provides Evidence of Improvement
Completed Administrative Support Services Assessment Plans – 2009-10 – Completed assessment plans for all administrative support units are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative administrative support units at UTC are provided here for academic years 2009-10 as documentation of the assessment and improvement process in those units.
Completed Administrative Support Services Assessment Plans – 2008-09 – Completed assessment plans for all administrative support units are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative administrative support units at UTC are provided here for academic years 2008-09 as documentation of the assessment and improvement process in those units.
3.3.1.3 Educational Support Services
Judgment of Compliance:
Compliance
Narrative:
Identifies Expected Outcomes
Each educational support unit at the University of Tennessee at Chattanooga annually identifies three to five expected outcomes or objectives. Each outcome or objective is consistent with the mission of the educational support unit, the UTC Strategic Plan, and mission, priorities, and goals of the UTC division in which the unit is housed. Relevant external standards are also linked to the outcomes. The outcomes and other relevant documents are recorded in the TaskStream workspace assigned to each educational support unit.
Assesses Extent of Outcome Achievement
Each unit uses an assessment plan to assess outcomes achievement. This plan defines appropriate targets and means for measuring each outcome and is recorded in the TaskStream work space. Evidence is then collected to determine the extent to which the outcomes have been attained. At the end of the academic year, assessment results are added to the annual assessment plan and the extent to which outcomes were attained and goals were met is determined. In cases where evaluation by appropriate faculty and staff determine that outcomes were not fully attained, inappropriately assessed, or where the target was too high or too low, a plan for improvement, or a revised assessment plan is developed. The completed assessment plan is sent to the vice chancellor, associate vice chancellor, or other supervisor for review.
Guidelines and examples for writing learning outcomes and developing assessment plans for support units are provided on the Institutional Effectiveness web link on the OPEIR web page.
Provides Evidence of Improvement
Completed Educational Support Assessment Plans – 2009-10 – Completed assessment plans for all educational support units are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative educational support units at UTC are provided here for academic years 2009-10 as documentation of the assessment and improvement process in those units.
Completed Educational Support Assessment Plans – 2008-09 – Completed assessment plans for all educational support units are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from representative educational support units at UTC are provided here for academic years 2008-09 as documentation of the assessment and improvement process in those units.
3.3.1.4 Research within its educational mission
Judgment of Compliance:
Compliance
Narrative:
Identifies Expected Outcomes
The University of Tennessee’s mission statement clearly addresses its commitment to research excellence. The University’s Strategic Plan includes four major research goals (or action steps) in two of the four University-wide Strategic Initiatives to be implemented during the period 2008-2013. Strategic Initiative II, Partnerships for Education and Research, contains three research goals, and Strategic Initiative IV, Enabling Partnerships, contains one. Multiple measurable outcomes have been identified for each of the goals by the University Planning Council and Executive Team, and responsibility for implementing and evaluating each goal was assigned to one or more UTC divisions, departments, offices, or governance bodies. The University Planning Coordinator has overall responsible for overseeing assessment of the Strategic Plan.
Assesses Extent of Outcome Achievement
Assessment of all components of the Strategic Plan is addressed in Action Step 4.3: Implement a formal process (report card) for evaluating all components of the Strategic Plan (strategic direction, initiatives, action steps, and measurable outcomes) on a yearly basis. The Plan assigns responsibility for this evaluation to the Chancellor and the Executive Team, who in turn have assigned specific responsibility for monitoring and implementing all aspects of the Strategic Plan, including the evaluation process, to the University Planning Coordinator who oversees the following process: A committee formed by the Executive Team meets annually to evaluate progress toward each goal and determine the most appropriate format for disseminating the information to the University community and the general public. In addition to targeting paper reports to selected groups and individuals, all information is posted to the UTC Strategic Planning web page.
The committee meets each spring with the University Planning and Resource Advisory Council (UPRAC) and the Executive Team to discuss progress and allow opportunities for feedback on the draft report and its tables. After the UPRAC and Executive Team approve the report the University Planning Coordinator incorporates the data, recommendations, and observations into the Annual Report of the Strategic Plan.
Provides Evidence of Improvement
Assessment Matrix for Strategic Plan Goals and Outcomes - The University Planning Coordinator developed a matrix to track progress of the 28 action committees responsible for implementing the various goals and outcomes o the UTC Strategic Plan. The matrix identifies the means of assessment, responsible parties, achievements for each of the first two years of implementation, and mid-project recommendations and comments associated with each goal. The lists of achievements are based on individual, semester reports submitted by the chairs of the action committees. The four major strategic plan goals that relate to UTC’s research function are highlighted in the Assessment Matrix.
3.3.1.5 Community/public service within its educational mission
Judgment of Compliance:
Compliance
Narrative:
Identifies Expected Outcomes
The University of Tennessee’s mission statement affirms its commitment to excellence in service and its dedication to meeting regional needs through partnerships and community involvement. The University’s Strategic Plan includes five major community/public service goals (or action steps) in three of the four University-wide Strategic Initiatives to be implemented during the period 2008-2013. Strategic Initiative II, Partnerships for Education and Research, contains one research goal, Strategic Initiative III, Partnerships for Diversity contains one, and Strategic Initiative IV, Enabling Partnerships, contains three. Multiple measurable outcomes have been identified for each of the goals by the University Planning Council and Executive Team. Responsibility for implementing and evaluating each goal was assigned to one or more UTC divisions, departments, offices, or governance bodies, while the University Planning Coordinator has overall responsible for overseeing assessment of the Strategic Plan.
Several units such as Continuing Education, Center for Community and Career Education, Cadek Conservatory, Challenger Center, Center for Applied Social Research, Children’s Center, Educational Opportunity Center, and the Southeast Center for Education in Arts have missions which directly support the outreach/public service aspect of UTC’s mission and annually identify outcomes for outreach efforts they direct which serve the community and region.
Assesses Extent of Outcome Achievement
Assessment of all components of the Strategic Plan is addressed in Action Step 4.3: Implement a formal process (report card) for evaluating all components of the Strategic Plan (strategic direction, initiatives, action steps, and measurable outcomes) on a yearly basis. The Plan assigns responsibility for this evaluation to the Chancellor and the Executive Team, who in turn have assigned specific responsibility for monitoring and implementing all aspects of the Strategic Plan, including the evaluation process, to the University Planning Coordinator who oversees the following process: A committee formed by the Executive Team meets annually to evaluate progress toward each goal and determine the most appropriate format for disseminating the information to the University community and the general public. In addition to targeting paper reports to selected groups and individuals, all information is posted to the UTC Strategic Planning web page.
The committee meets each spring with the University Planning and Resource Advisory Council (UPRAC) and the Executive Team to discuss progress and allow opportunities for feedback on the draft report and its tables. After the UPRAC and Executive Team approve the report the University Planning Coordinator incorporates the data, recommendations, and observations into the Annual Report of the Strategic Plan. Assessment of outcomes associated with the five outreach/public service goals is a result of this process.
Support units identified above that directly support the outreach/public service function of the University assess annual outcomes in the manner described for administrative support services in section 3.3.1.5 above.
Provides Evidence of Improvement
Assessment Matrix for Strategic Plan Goals and Outcomes - The University Planning Coordinator developed a matrix to track progress of the 28 action committees responsible for implementing the various goals and outcomes o the UTC Strategic Plan. The matrix identifies the means of assessment, responsible parties, achievements for each of the first two years of implementation, and mid-project recommendations and comments associated with each goal. The lists of achievements are based on individual, semester reports submitted by the chairs of the action committees. The fiver major strategic plan goals that relate to UTC’s outreach/public service function are highlighted in the Assessment Matrix.
Completed Outreach/Public Service Unit Assessment Plans – 2009-10 – Completed assessment plans for units that support UTC’s outreach/public service function are maintained in the program’s assigned TaskStream workspace. Finalized plans are found in TaskStream on the links “Findings per Measure” and “Overall Recommendations” in the work space for each fiscal (or academic) year. Completed assessment plans from outreach units at UTC are provided here for academic years 2009-10 as documentation of the assessment and improvement process in those units.
Documentation:
- OPEIR Website
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/ - Institutional Effectiveness Link
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/UTCPlanningEvaluationInstitutionalResearch.php - AMS TaskStream
https://www.taskstream.com/pub/ - THEC Academic Program Review
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/ProgramReview.php - Chemistry Program Review Self-Study
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Self-StudyChemistry.pdf - Chemistry Program Review Checklist
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Checklist.pdf - Chemistry Program Review Reviewers' Narrative
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ReviewersNarrativeReport.pdf - Foreign Languages Program Review Self-Study
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Self-StudyForeignLangandLit.pdf - Foreign Languages Program Review Checklist
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Checklist_001.pdf - Foreign Languages Program Review Reviewers' Narrative
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ExternalReviewersNarrativeReport_000.pdf - Health and Human Performance Program Review Self-Study
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Self-StudyHHP.pdf - Health and Human Performance Program Review Checklist
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ChecklistHHP.pdf - Health and Human Performance Program Review Reviewers' Narrative
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ExternalReviewersNarrativeReport_001.pdf - Applied Mathematics Program Review Self-Study
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Self-StudyMathandAppMath.pdf - Applied Mathematics Program Review Checklist
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Checklist_002.pdf - Applied Mathematics Program Review Reviewers' Narrative
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ExternalReviewersNarrativeReport_002.pdf - Computational Engineering Program Review Self-Study
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Self-StudyCompEng.pdf - Computational Engineering Program Review Checklist
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/Checklist_000.pdf - Computational Engineering Program Review Reviewers' Narrative
http://www.utc.edu/Administration/PlanningEvaluationAndInstitutionalResearch/documents/ExternalReviewersNarrativeReport.pdf - Completed Academic Program Assessment Plans - 2009-2010
- Completed Academic Program Assessment Plans - 2008-2009
- Completed Administrative Support Services Assessment Plans - 2009-2010
- Completed Administrative Support Services Assessment Plans - 2008-2009
- Completed Educational Support Assessment Plans - 2009-2010
- Completed Educational Support Assessment Plans - 2008-2009
- Research Goals
- Completed Outreach Assessment Plans 2009-2010
- Completed Outreach Assessment Plans 2008-2009
- Assessment matrix
- Outreach Goals
