ASSESSMENT OF LIBRARY INSTRUCTION @ Randall Library
What is Assessment? Assessment is an ongoing process aimed at understanding and improving student learning. 1 Main purposes: 2 Increase student learning Strengthen instructional program Answer calls for accountability http://www.acrl.org/ala/mgrps/divs/acrl/issues/infolit/resources/assess/issues.cfm http://www.acrl.org/ala/mgrps/divs/acrl/issues/infolit/resources/assess/issues.cfm (1) Oakleaf & Kaske, 2009 (2)
Why Assess Library Instruction? To increase student learning To improve instruction To inform future assessment efforts To provide data for accountability measures To collaborate with faculty To collaborate with other librarians To argue for increased resources http://libraryassessment.org/bm~doc/oakleaf_assessment_cycle.pps Forward to Slide 9
Why Assess Library Instruction? Well done assessment is essential to documenting the effects of information literacy programs and the degree to which they are meeting their objectives. The assessment process can also assist anyone involved in information literacy instruction by getting them actively involved in the process and engaged in classroom activities, by giving them feedback about students thinking and learning, and by helping them focus on goals and objectives Assessing Student Learning Outcomes for Information Literacy Instruction in Academic Institutions
Resistance and Misperceptions _____ is making us do this. _____ is trying to tell us how to teach. We have to get this over with. (Once and done) I dont have time for this. I have to develop a pre-test and a post-test. Assessment is a fad.
What Needs to be Assessed? Macro to micro UNCW curriculum as a whole All UNCW programs / majors All UNCW courses Current Basic Studies program / New University Studies program (General Education) Information Literacy (IL) component of University Studies curriculum (as a whole) Information Literacy courses within IL component Library instruction program Individual library instruction sessions
Past Library Instruction Assessment Library Instruction Sessions Primary focus on teaching evaluation Online student evaluation forms Peer (librarian) observations Faculty evaluations of teaching (Attending instructor) Focus has been on us Perceptions of teaching In general, little measure of what students are actually learning / can do Some measure of what they feel they are learning Indirect assessment
Moving Forward Comprehensive library instruction assessment plan Instruction program (holistically) Student learning outcomes for individual instruction sessions Ongoing, continuous process
Strategic Plan for Assessment Determine purpose(s) of assessment Examine mission of the university, library, and instruction program Examine structure of library instruction program Examine and redefine program goals Collect and examine relevant data / statistics related to program goals (e.g. How many students take ENG 201?) Examine program resources Examine / establish collaborations with other faculty Establish aggressive yet attainable timeline What will we assess? How will we assess? When will we assess? What will we change because of assessment data?
Purpose(s) Why are we engaging in assessment? What do we hope to gain from assessment efforts? Main purposes: Increase student learning Strengthen instructional program Answer calls for accountability See slide #3
Other Purposes Connection between library instruction assessment and UNCW assessment Only area offering courses not represented in UNCW assessment efforts (?) No representation on Learning Assessment Council Instruction sessions a component of UNCW courses Promotion of library instruction program and information literacy Internally-focused purposes: Improve workload, workflow, organizational structures in library, etc.
Driving Forces (Randall Library) Formation of Information Literacy Strategic Planning Team (Fall 2008) Instructional Services Coordinator training: ACRL Immersion Program (Summer 2008) ACRL Immersion Program Master of Science in Instructional Technology (MIT) MIT 531 (Assessment of Learning Outcomes) (Summer 2009) LOEX Conference (May 2010) NCLA Bibliographic Instruction Group: Assessment & Student Learning Outcomes Workshop (May 2010) Georgia Conference on Information Literacy (October 2010) Library Assessment Conference (October 2010)
ILSPT Mission Statement: The Information Literacy Strategic Planning Team will lead long-term, strategic planning to help transform the current set of instructional services activities into a well conceived, planned program that is integrated into both the basic studies curriculum and into each major. An information literacy program is one that utilizes well placed instruction sessions and tutorials/online modules to integrate information literacy skills into the curriculum of courses and builds skills progressively through a students college career. Add: Led by the Coordinator of Instructional Services, ILSPT will plan for and oversee the assessment of library instruction
Driving Forces (RL & UNCW) UNCW SACS accreditation New/revised University Studies program which includes Information Literacy Component Need for library to demonstrate importance, effectiveness, need for resources, etc.
Driving Forces (UNCW) Highest priority: Student learning UNCW shift to culture of assessment 2005 campus assessment workshops (Susan Hatfield, consultant)assessment workshops Learning Assessment Council Learning Assessment Council Institutional Research & Assessment Assessment directors Assessment directors Assessment of Basic Studies National Survey of Student Engagement (NSSE)NSSE Collegiate Learning Assessment (CLA)CLA General Education Assessment Committee (Report)Report
Assessment so Far (UNCW) UNCW Learning Goals General Education Report (March 2009) On Information Literacy No librarians were consulted in the construction of this process / report Instructional Services Coordinator contacted the Director of University Planning and sent the Information Literacy Competency Standards to Director through campus mail Communication was not returned http://uncw.edu/assessment/Documents/General%20Education/GenEdAssessmentCommitteeReportMarch2009.pdf
General Education Report All UNCW freshman seminar participants undergo instruction by the Randall Library in locating and using information, although there is no current assessment of students information literacy proficiency either before or after that instruction.
General Education Report Recent data indicate that college students have difficulty developing a topic question and are often not aware of informational resources available to them. Even when aware of appropriate information resources, they may lack the ability to adequately evaluate and synthesize information. This can result in classroom work based on haphazard discussion groups or personal opinion. Albitz, R.S. (2007) The what and who of information literacy and critical thinking in higher education. Libraries and the Academy. 7, 1, 97 109
General Education Report Of 37 common learning outcomes established for the Basic Studies categories, only two (CMP3 and IP1A) fit into the category of information literacy. Perhaps there is not sufficient emphasis in the spectrum of Basic Studies courses on information literacy in a broad range of disciplines, or perhaps additional common learning outcomes should be developed to encompass efforts that are currently being made. CMP3: Employ a writing process that includes finding, evaluating, analyzing, and synthesizing appropriate primary and secondary sources, and successfully integrating personal knowledge with source material. IP 1A: Access and analyze information about topics or issues that are interdisciplinary or transdisciplinary.
National Survey of Student Engagement First-year students were asked in NSSE about the frequency with which they worked on a paper or project that required integrating ideas or information from various sources, a component of information literacy. UNCW first-year students reported a frequency that was statistically significantly below that reported by our selected peers, significantly below that reported by national masters universities, and significantly below that reported by all NSSE 2007 institutional participants. This information led to the development of a rubric-based assessment plan for information literacy to be implemented with the comprehensive assessment of Basic Studies beginning Fall 2009. Assessment tool selected: The American Association of Colleges and Universities (AAC&U) Information Literacy Metarubric The American Association of Colleges and Universities (AAC&U) Information Literacy Metarubric http://uncw.edu/assessment/BasicStudiesAssessment.html http://uncw.edu/assessment/Documents/General%20Education/2004-2005NSSE-BasicStudies.pdf
On the Horizon University Studies (Fall 2011) Students who complete the University Studies program will: Goal 3. Locate, evaluate, and effectively use information by applying a variety of academic and technological skills (Information Literacy) http://www.uncw.edu/universitystudies/documents/Univ.StudiesCurriculumReport.pdf
Information Literacy Curriculum Component of University Studies curriculum Designed to foster the development of students information literacy skills by requiring three information literacy intensive courses: The First Year Experience and two additional information literacy intensive courses, with at least one in the major field of study.
Information Literacy Curriculum Based on the Information Literacy Competency Standards for Higher EducationInformation Literacy Competency Standards for Higher Education The information literate student 1. Determines the nature and extent of the information needed. 2. Accesses needed information effectively and efficiently. 3. Evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system. 4. Individually or as a member of a group, uses information effectively to accomplish a specific purpose. 5. Understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.
Information Literacy Curriculum University Studies Information Literacy Student Learning Outcomes (adopted by Faculty Senate November 2009) The student will: IL 1. Be able to determine the nature and extent of information needed to solve a problem. IL 2. Access information effectively and efficiently from a variety of sources. IL 3. Evaluate information critically and incorporate appropriate information into his or her knowledge base. IL 4. Individually, or as a member of a group, use information effectively to accomplish a specific purpose. IL 5. Understand many of the economic, legal, and social issues surrounding the use of information and access and use information ethically and legally.
Information Literacy Curriculum 9 hours of courses (Double dipping from other components) Content reinforced throughout curriculum Course in the Major (Research Methods or other)(3 hours) Library instruction sessions focus on IL Comp Standards 2 (Accessing info through discipline specific resources) and 5 (Ethical use: Discipline specific citations and writing) English Composition or other (e.g. LIB 103) (3 hours) Library instruction sessions focus on IL Comp Standards 1 (Determining info needed), 2 (Accessing info), and 4 (Using info effectively) Freshman Seminar (3 hours) Library instruction sessions focus on IL Comp Standards 3 (Evaluating information) and 5 (Ethical use: Plagiarism)
Whats Different for Us? Every major must have an information literacy intensive course Faculty must supply information literacy student learning outcomes for that course Information literacy on the radar in a new way Just send em to the library does not apply Potential for collaborations that did not exist before Information Literacy is a goal of the entire University Studies program The university will assess this
Library Instruction Program Shift From bibliographic instruction sessions to information literacy Moving from scattered one shot instruction sessions to integration of information literacy skills instruction into UNCW curriculum Defining the Instruction Program Information literacy curriculum (University Studies) Freshman Seminar instruction English composition instruction (ENG 101, 103, and 201) Instruction in each major Workshops & tours Online tutorials LIB Courses Regional engagement / instruction (high school student instruction, Pathways workshops, etc.)
Current Library Instruction AY 2008-2009 Course-related instruction sessions, workshops, and tours 587 sessions 678 contact hours Teaching 10,750 users PLUS Thirty four hours of credit courses offered LIB 101 (4 sections, 1credit hour, 19 students in each) LIB 103 (9 sections, 3 credit hours, 19 students in each) LIB 104 (1 section, 3 credit hours, 19 students in each) https://share.uncw.edu/sites/aa/library/pubsvcs/Shared%20Documents/Annual%20Reports/annrept0809.docx
Moving to Assessment: A System Part 1: Expected Outcomes (Part of Assessment Plan) Part 1: Expected Outcomes Part 2: Assessment Methods and Tools (Part of Assessment Plan) Part 2: Assessment Methods and Tools Part 3: Implementation (Part of Assessment Plan) Part 3: Implementation Part 4: Summary of Findings (Part of Assessment Report) Part 4: Summary of Findings Part 5: Actions Taken (Part of Assessment Report) Part 5: Actions Taken http://www.uncw.edu/cas/assessmentplanandreport.html
Steps of Assessment 1. Determine what we need to assess by creating Student Learning Outcomes for: Freshman Seminar library instruction sessions English Composition (ENG 101, 103, and 201) library instruction sessions Instruction sessions in each major (as applicable) LIB Courses ILSPT has reviewed ACRL Standards SLOs will map to ACRL Standards 2. Determine how we are going to assess Means of Assessment (what method or tool we will use) My Research Strategy Worksheet (scored using a rubric), sample of student products, online quiz, 5 minute paper, tests, etc. 3. Determine what and when we will assess 4. Review assessment information 5. Make changes based on assessment information Repeat all steps continuously
Assessment Cycle Implement Changes Formulate Outcomes Develop Means of Assessment Gather & Analyze Data Interpret Results Propose Improvements to Program Library Instruction Program
Assessment Checklist 1. What are our research questions? (What are we trying to discover about student skills, knowledge, abilities, etc.; and what evidence do we have already?) 2. What is the expected level of performance? 3. When in the students career do we assess this outcome? (entry, end of sophomore year, senior, etc.) 4. In which course(s) or venue? 5. What student work/artifacts are collected? 6. How is the student work evaluated? (criteria/rubric) 7. Who evaluates the student work? 8. Who analyzes the results? 9. Where do recommendations for action go? 10. Who takes action? (And how do we ensure changes are evidence based and data driven?) 11. How is the process documented? 12. Where is the documentation kept? 13. What is the timetable/schedule for determining which outcomes are assessed when? Developed by the General Education Assessment Committee for designing assessment of a learning outcome.
Who Is Going to Do This? All Instruction Librarians Comprehensive plan coordinated by Instructional Services Coordinator Supported by Information Literacy Strategic Planning Team Assessment at micro level (instruction sessions) Done by individual librarians with assistance from Instructional Services Coordinator Will seek guidance from campus experts (e.g. Dr. Linda Seifert, Director of Assessment, College of Arts and Sciences)
First Steps Completion of assessment structure and plan outline (Summer 2010) Creation of assessment website (Summer/Fall 2010) LIB 103 assessment planning (Begin Fall 2010) Freshman Seminar assessment planning SLOs and means of assessment (Begin Fall 2010) English Composition (ENG 101, 103, and 201) assessment planning Draft of SLOs (Fall 2010) Assessment of SLOs (Spring 2011) Assessment planning for the majors SLOs and means of assessment (Begin Fall 2010)
Assessment & Evaluation SWOT/SOAR of library instruction program (Fall 2010) Evaluation of library instruction program coordinator based on Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators (Fall 2010) Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators Self analysis of librarians based on Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators (Fall 2010)Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators Continuous use of online student evaluation forms Continued annual peer assessment of teaching Continued use of SPOTS for LIB courses Future implementation of faculty surveys and evaluations Review benchmarking data (Collected in fall 2008 and spring 2009) Review of LibQual+ Survey results (Administered in 2008)LibQual+ UNCW efforts: NSSE, CLA, General Education Assessment, etc.
To Explore in the Future Large Scale Information Literacy Assessment Tools for UNCW: Project SAILS (Standardized Assessment of Information Literacy Skills) Project SAILS Sample Sample James Madison Information Literacy Test (ILT) James Madison Information Literacy Test (ILT) iCritical Thinking Certification (ETS) iCritical Thinking Certification Replaced the ETS iSkills Test Graduate Courses
Helpful Resources ACRL Information Literacy website (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm ACRL Institute for Information Literacy (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm ARL Library Assessment Conference (http://libraryassessment.org)http://libraryassessment.org A Twenty Year Path: Learning about Assessment; Learning from Assessment (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER1/97)http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER1/97 Information Literacy in the Disciplines (http://wikis.ala.org/acrl/index.php/Information_literacy_in_the_disciplines)http://wikis.ala.org/acrl/index.php/Information_literacy_in_the_disciplines
Helpful Resources Internet Resources for Higher Education Outcomes Assessment (http://www2.acs.ncsu.edu/UPA/assmt/resource.htm)http://www2.acs.ncsu.edu/UPA/assmt/resource.htm Megan Oakleafs Presentations and Publications (http://meganoakleaf.info/presentations.html)http://meganoakleaf.info/presentations.html National Institute for Learning Outcomes Assessment (http://learningoutcomesassessment.org/)http://learningoutcomesassessment.org/ Task Force on Academic Library Outcomes Assessment Report (http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/taskforceacademic.cfm)http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/taskforceacademic.cfm Utah State University Library Instruction Assessment (http://library.usu.edu/instruct/assessment/index.php)http://library.usu.edu/instruct/assessment/index.php Writing Information Literacy Assessment Plans: A Guide to Best Practice (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER2/98)http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER2/98
Library Instruction Assessment Website Anne Pemberton Instructional Services Coordinator Chair, Information Literacy Strategic Planning Team firstname.lastname@example.org http://library.uncw.edu/staff/anne_pemberton