An Evaluation Needs Assessment is a process through which we can identify the capacities and areas for development in Monitoring, Evaluation and Learning within a team or organisation.
It involves looking at the ways that data collection, analysis, reporting, reflection and learning currently occur, and how they could be strengthened. It also considers the types of MEL that would be most useful for achieving organisational goals. This can lead to the development of an action plan to strengthen MEL capacity.
Following completion of the project, Djirra’s Monitoring, Evaluation & Learning Manager, Kathryn Robb, kindly agreed to be interviewed by Fran Demetriou from Lirata so that we could share some of the learnings from the project from Djirra’s perspective.
Interview with Kathryn Robb
FD: Can you tell us a bit about Djirra, AFVLS and where your role fits in?
KR: Djirra is a specialist Aboriginal Community Controlled Family Violence organisation which provides culturally safe legal and non-legal support, delivers community based early intervention and prevention programs and undertakes policy and law reform work.
Since September 2017, Djirra has been engaged in an internal evaluation capacity building program with their Community Engagement Unit, developing ground-up, culturally informed theories of change and MEL frameworks. My role was created to support the ongoing roll-out in Community Engagement and to expand the evaluation capacity building program to other areas of Djirra.
FD: What needs to be considered when undertaking monitoring, evaluation and learning in this space?
KR: I think any evaluation process that is going to take place in an Aboriginal Community Controlled Organisation needs to recognise that there has been a long legacy of top down, disempowering and extractive evaluation processes. Unpacking what evaluation represents and the value it can bring to ACCOs is key.
Across the organisation there is a real commitment to ground up, culturally informed MEL.
FD: What were the drivers for building evaluation capacity at Djirra?
KR: There were a number of reasons. On one hand, it was due to pressures from the external funding environment – as Djirra has expanded and grown, so too has the number and complexity of funding reporting and evaluation requirements. The bigger driver was the desire to strengthen how we design and use MEL processes to benefit the organisation and to benefit the Aboriginal communities we are working with. Across the organisation there is a real commitment to ground up, culturally informed MEL, of building capacity to design MEL for internal program and service improvement, learning and downward accountability to community.
FD: Why did you start with an Evaluation Needs Assessment?
KR: We wanted to be able to map out the current state of things and the different dimensions of capacity in order to really understand strengths, challenges, barriers, and the causes underlying some of the issues. We knew from our experience with the Community Engagement Unit, unless you spend time really understanding what is going on, you can’t come up with a capacity building program that fits the context, and the team – what they need and how they work.
FD: What was involved in the project? What parts of the process were the most useful?
KR: Lirata developed a really useful capacity framework at the outset. Breaking down capacity into domains across individual, organisational and MEL Stages allowed for a much more comprehensive insight into what was going on. The domains related to staff attitudes and perceptions of evaluation and integration and use were so helpful to understand our staff’s lived experience of MEL.
Another really useful part was mapping all the external funding data requirements. We knew it was big and complex but the map helps us identify at which point issues are arising.
A working group from across the organisation were involved which was a great way to share ownership and accountability of the process, findings and next steps.
Undergoing a needs assessment is an essential first step if you want a tailored program that will fit the needs and context of your organisation.
FD: What have been the benefits of undertaking this project?
KR: We now have a comprehensive map of gaps and strengths, and we know where to start as needs have been ordered by priority. It has also provided us with a baseline to measure our progress when we start implementing an evaluation capacity building program.
FD: And what are the biggest challenges on your radar?
KR: I think one of the biggest challenges will be trying to balance the focus on external reporting requirements and the use of an existing data management system with a MEL design which speaks to ground up, culturally informed articulations of outcomes and culturally safe ways to measure these outcomes.
FD: What advice would you give to others who are starting their evaluation capacity building journey?
KR: I think undergoing a needs assessment is an essential first step if you want a tailored program that will fit the needs and context of your organisation. It unpacks for everyone what is even meant by evaluation capacity and all the different intersecting and interdependent factors which need to be considered if you want to build a meaningful and sustainable evaluative culture and practice in an organisation.
This article is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Demetriou, F. & M. Planigale. 2021. Evaluation Needs Assessment in an Aboriginal Community Controlled context: An interview with Kathryn Robb. Melbourne: Lirata Ltd.
About the authors
Assistance with evaluation capacity building
Lirata assists service providers, advocacy organisations and funders to build their MEL capacity and to undertake effective monitoring and evaluation. We assist organisations to undertake Evaluation Needs Assessments, and provide training and mentoring in program logic, Theory of Change and MEL Frameworks.
For further information or assistance, please contact the Lirata team.