Evaluation5's Blog

April 18, 2010

Top 10 – How not to………..

Filed under: behaviour, evaluation5.0 — Tags: , — Rosien Herweijer @ 12:06 pm

Somewhere on Internet a list is circulating with 26 hints for programme managers on how  to respond to evaluation findings.

The list oozes frustration. Below some of the most charming tips:

  • attack the evaluation’s methodology;
  • do not participate in the evaluation, but argue that the findings lack an adequate contextual background;
  • indicate the findings are reasonable, but unable to be implemented due to a lack of resources, political opposition, the staff need training etc;
  • state that the program’s environment has changed, and the findings are no longer relevant;
  • argue that while the program has not achieve it’s goals, it does achieve other important things that are too subtle to be easily measured;

And my personal favorite,  number 23:

  • agree with the findings and indicate that you have known about this for some time, and you started making changes months ago;

To find the original article you have to subscribe (and pay!) but you will find the complete list at this blog

I have done evaluations, have been evaluated  and have contracted evaluators. I could easily compile a list of my own negative behaviours in evaluation situations.

Question:  What do you do when an organization you have been funding for a number of years wants to postpone the evaluation you planned to do this year because they say it would be the third in a row they would have to do since January this year:

  1. Tell them you will check with your boss, wait a few weeks  and try any other item on this list
  2. Do not respond and simply send the ToRs:  you are fed-up with all these people who do not understand that taxpayers need information on what is being achieved or not with government money.
  3. Send an e-mail that you feel sorry for them but that your boss is really strict with the rules
  4. Call them to say that unfortunately the other evaluations cannot replace this on because those where not really objective and only intended for internal learning.  If they object go to 5 or 6.
  5. Explain that in your country the majority MPs wants to cut development budgets and that they look for every argument they can get,  so you really cannot skip any mandatory evaluations
  6. Call them again and explain them that this evaluation will provide them with valuable lessons learned

Another list could be on…….what to you do when you are an evaluator and the person who hired you tells you (s)he thinks your evaluation report is subjective .….

  1. Suggest that true objectivity does not exist but that your findings are based on real evidence
  2. With more time and a serious budget for the evaluation,  you could certainly have ensured that your findings would have been based on more empirical research and a broader variety of sources……
  3. You are very sorry, indeed, and yes, you did feel that the organization evaluated was keeping information from you.
  4. You hurry to offer to edit the report and ask what kind of arguments (s)he would like to see included.
  5. You say they can add their own Executive Summary if they wish, nobody will read the main report anyway.
  6. You insist you have high professional standards and that you will seriously consider the critique and get back to her/him.  When responding pick one of the above
  7. Ask to speak with her/his team leader:  the team leader will be the one who decides whether your report will be accepted or not anyway
  8. Smile, raise you eyebrows and say you have delivered everything agreed in the ToR

Frustration breeds frustration,  and that frustration with evaluation can come from any perspective.  The question is how you can get a more positive energy in this (Bermuda?) triangle of evaluator – evaluate – finanancier?

Psychologist will explain that – conscious or not – people respond to behaviour with very specific ‘counter’ behavior. They may argue that if you behave paternalistic (or maternalistic for that matter) towards someone, this person will return the favour behaving like a child, or surprise you with an aggressive effort to diminish you, trying to cast you in the role of child.  Parent-child relations are quite unsuitable for a constructive evaluation process. At the same time,  according to the theory of Transactional Analysis they are very hard to escape from: you will have to make a real and conscious effort to pull each other out of such roles, back into a relation between professionals. How do you recognize such a process? How do you get out such negative dynamics?  If frustration breeds frustration, maybe trust breeds trust? Sometimes it feels that we spend a lot of time designing perfect monitoring processes and developing superior evaluation methodologies and tools, but to what extent are we really ready to discuss our behaviours?

If you are interested in discussing this further, go to www.evaluators5-0.net


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: