2009 Pre-conference Workshops

A. Ethical and Inclusive Excellence Imperatives in a Globalizing World: An Integral Evaluator Model by Hazel Symonette

9:00am - 4:30pm (lunch not provided; lunch break 12:00-1:30pm)

$160.00 (non-member); $100.00 (member)

For inclusive excellence and ethical practice, evaluators need to commit to staying open and light on their feet while maintaining a boundary-spanning stance of ravenous lifelong learner. Embracing the de-centering realities and complexities of difference and salient diversity-divides is foundational. What evidence do we have that we are clear, unobstructed channels for hearing and heeding the full spectrum of voices, that we are attuned to what a particular situational/relational context is calling for from us? How do we know and who says so? How well prepared are we for honoring the sacred trust of our roles as privileged authorities, as impactful judgment-makers?

Dr. Symonette's evolving quadrant model offers the beginnings of a holistic evaluator framework for calibrating and cultivating self as responsive instrument. Crafted from the vantage point of an individual evaluator, the model builds upon Ken Wilber's Integral Quadrant Model. Dr. Symonette focuses on the most underdeveloped and untended dimensions of an integral model: notably, the interconnections among interior environments, both the individual and the collective. Moving beyond the typical "flatland" frameworks, she has mapped her Self-As-Instrument concepts across the 4 quadrants, e.g., unilateral self-awareness in the upper left quadrant and multilateral self-awareness, in the lower left. Her model offers a framework of sensitizing concepts and questions for mindfully scanning, tracking and monitoring who factors--notably, the human systems dynamics. These items speak to the multiple dimensions of diversity that live in the interpersonal interface among human beings: e.g., the evaluator and those who are evaluated; the information-providers and the information-seekers.

Attendees will be encouraged to regularly examine the ways in which involving various stakeholder groups calls for them to look through windows versus looking in a mirror at processes and practices that are responsive to and reflective of their lived experiences. To what extent do the persons that evaluators engage discern and experience evaluative processes, protocols, practices and products as representative of, congruent with, responsive to and accurately reflecting their lived realities--internal sociocultural structures and rhythms (experiential validity)?

Many of the issues explored in this workshop will proactively facilitate responsiveness to the ethical and inclusive excellence imperatives of the newly revised Program Evaluation Standards expected to be published in 2010. Principles and expectations related to cultural and contextual responsiveness are woven throughout. Of course, these considerations are also directly relevant for the AEA Guiding Principles for Evaluators.

B. Beyond Survey Monkey: Technology in Evaluation by Kavita Rao, Charles Giuli, Donnel Nunes, Morris Lai

9:00am - 12:00pm

$80.00 (non-member); $50.00 (member)

Ever wonder what evaluation might look like in ten years or how technology has already changed how data are collected or findings are reported? If you want to learn innovative ways to use technology in evaluation, this is the H-PEA workshop to go to. Multiple presenters will discuss and demonstrate how cutting-edge evaluations use technology.

Virtually There by Kavita Rao and Charles Giuli

To connect with our clients in our geographically-dispersed service region of Micronesia, American Samoa, and the Hawaiian islands, Pacific Resources for Education and Learning (PREL) has been increasingly using internet-based telecommunication technologies. Web-conferencing and internet-based telephony tools on the computer provide ways in which to connect with others "synchronously" and at low cost. In our portion of the workshop, we will discuss how we have used web-conferencing software (Elluminate Live!) and internet telephony software (Skype) to implement virtual meetings, create virtual classrooms, and conduct evaluation efforts in Micronesia, American Samoa and Hawai‘i.

Anonymous Feedback Systems by Donnel Nunes

Looking to strategies from the business world as a guide, many classrooms are turning to the use of anonymous feedback systems to increase student participation, engagement, and authenticity of responses, and to collect data. Within the last year, web-based systems have emerged that reduce cost, simplify use, and increase mobility and accessibility. In this session we will discuss how this technology is being used to support behavior health services at a local intermediate school and provide audience members an opportunity to experience how this can be used for group evaluation purposes. Participants are encouraged to bring their cell phones.

Video Reporting by Morris K. Lai

In 2003, Māori and Kanaka Maoli (Hawaiian) evaluators formed the Evaluation Hui, which collectively developed an indigenous framework for evaluation. One of the major differences between indigenous and mainstream evaluation approaches is the former's emphasis on mo‘olelo (stories) for collecting as well as for reporting evaluation data. In my portion of the workshop I will use video and other methods to provide examples of such an approach. I will discuss why an indigenous approach to evaluation and using video are more appropriate for Hawaiians and can also improve mainstream evaluation.

Presenters

Kavita Rao is an Assistant Professor in the Special Education Department, College of Education, University of Hawai‘i at Mānoa. For the past decade, Kavita has worked with teachers and students in Hawai‘i, Guam, American Samoa, CNMI, Palau, RMI, and the Federated States of Micronesia. Her research includes designing and developing culturally-responsive distance learning programs for teacher training in the Pacific. She is also interested in the use of multimedia technologies to support culturally and linguistically diverse learners and students receiving special education services.

Charles Giuli is the director for External Evaluation for the Pacific Resources for Education and Learning (PREL), where he manages evaluation contracts in Hawai‘i and the Pacific region. He received his doctorate in educational psychology from the University of Hawai‘i. Before joining PREL, Giuli worked for Curriculum Research & Development Group and Kamehameha Schools. Since joining the center for external evaluation at PREL in 1999, he has conducted over 130 evaluations. Dr. Giuli's experience includes planning, project management, fiscal management, budgeting, needs assessment, program monitoring, testing, measurement, questionnaire design, and research design. In this session, Dr. Giuli will bring to bear his experience collecting data from a distance.

Donnel Nunes is a behavioral health specialist at Kailua Intermediate School on O‘ahu. Prior to this, he collaborated on research in the area of performance psychology and worked in a residential psychiatric facility. His current focus is exploring ways to incorporate technology and media into mental health practice. He regularly uses film, music, and other creative software for the purpose of fostering engagement, increasing disclosure, and collecting data. Recently, he was lead author on a paper titled: "Technology and the Adolescent: Pairing Modern Media and Technology with Mental Health Practice" that is being disseminated by the Center for School Mental Health at the University of Maryland School of Medicine. In Fall 2009, he will pursue further study towards a PhD in Educational Psychology at the University of Hawai‘i.

Morris K. Lai earned degrees in mathematics from Stanford University and the University of Hawai‘i (UH) and a PhD in curriculum and educational statistics from the University of California at Berkeley. Shortly after completing his doctoral work, he returned to Hawai‘i, where he has worked at UH's Curriculum Research and Development Group since 1974. He has designed and conducted evaluations, ranging from free to $90,000, and has been the principal investigator of numerous externally funded grants, which most recently have been focused on the education of Hawaiian students. He is a longtime member of the graduate faculty of UH's Department of Educational Psychology and a member of the Evaluation Hui (Māori and Kanaka Maoli [Hawaiian] evaluators) since 2003.

C. Navigating the Logic Model for Improved Planning and Results by Lily Bloom Domingo and Jennifer Cornish Creed

1:30pm - 4:30pm

$80.00 (non-member); $50.00 (member)

A logic model can improve program planning and results. In this interactive workshop, participants will practice creating a logic model for a program of their choice and receive feedback from the facilitators. They will learn key concepts such as the relationship between a program's Mission, Planning, and Outcomes. The facilitators will explain the components of the logic model and a tried-and-true method for creating a useful program logic model. During discussion, participants will address some of the larger questions and observations about the logic model and facilitators will talk about how the logic model supports quality evaluation.

Participants are encouraged to attend with others from their organization and work together on a program logic model.

Facilitators

Lily Bloom Domingo is chief operating officer and program director for Training and Consulting Services at the Hawai‘i Alliance of Nonprofit Organizations (formerly Hawai‘i Community Services Council). Lily has worked in the nonprofit sector for over 20 years and at HANO since 2000. She specializes in nonprofit leadership and management, particularly strategic planning, board governance, and outcomes design and implementation, and in dialogue facilitation. She has provided consulting services to a wide range of nonprofit, government, and for-profit organizations. In 2003, Lily served as the lead facilitator for the Lieutenant Governor's Drug Control Strategy Summit. She designed a consensus building process, utilizing a team of 41 professional facilitators that engaged over 350 people from diverse disciplines and resulted in eleven recommendations for future action. In 2008 Lily received Envision Hawai‘i’s Award for Excellence in Mentorship, recognizing her supportive, informal mentoring style that has touched the lives of many nonprofit professionals and social entrepreneurs. Lily has collaborated with Dr. Donna Ching, Professor and Extension Specialist in Agricultural Leadership, to design and offer training in Advanced Facilitation. She also developed and taught a course in Community Needs and Resources at the University of Hawai‘i’s College of Tropical Agriculture and Human Resources. Previously, Lily worked at Aloha United Way in Honolulu, Hawai‘i and the United Way of the Central Coast in San Luis Obispo, California where she developed training materials and provided training and technical assistance to numerous organizations. Lily has a M.S. in Community Development from the University of California at Davis. She received her B.S. in Human Development from the University of Hawai‘i at Mānoa. In addition, she has specialized training in Collaborative Leadership, Facilitating Strategic Thinking and Planning, Appreciative Inquiry, Outcomes Training, and Qualitative Research Methods. A resident of Hawai‘i for over 35 years, she currently serves as president of the board of the Susannah Wesley Community Center.

Jennifer Cornish Creed has been a training specialist with the Training and Consulting Services program of Hawai‘i Alliance of Nonprofit Organizations (formerly Hawai‘i Community Services Council) since 2000 and in that capacity has helped design and deliver training, consulting and technical assistance in Strategic Planning, Board Governance, Outcomes Design and Implementation and other Nonprofit Leadership and Management topics to non-profit organizations statewide. Jennifer has worked in the nonprofit community for 17 years. She has an extensive background in workshop development and the design of related training materials (print and audio/visual). In her work at HANO, she has primarily been involved in planning, implementing and evaluating training services. She has also spearheaded the planning and delivery of HANO's conferences, and is actively engaged in HANO's communications services. Jennifer is a trained facilitator and has led numerous planning and meeting facilitation on behalf of HANO. Prior to joining HANO, Jennifer worked with the Hawai‘i Community Loan Fund and the Hawai‘i Institute for Continuing Legal Education. Jennifer grew up on the Windward side of O‘ahu. She received her bachelor's degree in Art History from the University of Hawai‘i at Mānoa. She did graduate work at Tufts University in Museum Administration and Museum Education. Jennifer has additional training in Collaborative Leadership and Facilitating Strategic Thinking and Planning.