EVALUATIONGENERAL ISSUES.docx

EVALUATIONGENERAL ISSUES.docx

《EVALUATIONGENERAL ISSUES.docx》由会员分享，可在线阅读，更多相关《EVALUATIONGENERAL ISSUES.docx（38页珍藏版）》请在冰豆网上搜索。

EVALUATIONGENERALISSUES

EVALUATION-GENERALISSUES

Definitionofevaluationanditstypes1

Basicmethodsofevaluationsurveys7

Datacollectionmethods7

Documentanalysis7

Individualinterviews7

Questionnairesurvey8

Focusgroups8

Observationtechniques8

Grouptechniques9

Dataanalysismethods9

Statisticalanalysis9

Qualitativeanalysis9

Assessmentmethods9

Experts'panel10

Benchmarking10

SWOTanalysis10

Cost-benefitanalysis10

Cost-effectivenessanalysis10

Econometricmodels11

Planningevaluation12

Planningevaluation12

Orderingevaluation18

Procedureforawardingthecontract（Article36paragraph1item2）19

Descriptionoftheobjectofcontract（Article36paragraph1item3）20

Contractexecutiondate（Article36paragraph1item7）22

Descriptionofhowtopreparetenders（Article36paragraph1item14）22

Executionofpartofthecontractbysub-contractor（Article36paragraphs3and4）22

Selectioncriteriaforexternalevaluators24

Managingevaluation27

Preservingtheindependenceofevaluation27

Partnersintheevaluationprocess29

TheSteeringGroup,itsroleandcomposition30

Definitionofevaluationanditstypes

Evaluationisdefinedvariously,dependingonthesubjectmatter,appliedmethodologyortheapplicationofitsresults.Ingeneral,thedefinitionofevaluationcanbestatedas„judgementonthevalueofapublicinterventionwithreferencetodefinedcriteriaofthisjudgement.Themostfrequentlyusedcriteriaare:

itsconformitywiththeneeds（ofthesector,beneficiaries）,relevance,efficiency,impactandsustainabilityofitseffects.Moregeneralmeaningoftheterm„evaluation”,emphasisingitsutilitariancharacter,isgivenbyKorporowicz,definingevaluationasasystematicsurveyofvaluesorfeaturesofagivenprogramme,activityoranobject,takingintoconsiderationtheadoptedcriteriatoenhance,improveorunderstandthembetter.Evaluationalwaysisthestudywithanobjective.Thelackofthepreciselydefinedevaluationobjectivecallsitsreasonablenessintoquestion.Oneofthemostcrucialevaluationobjectivesistoprovidethepeopleorderingtheevaluationwithreliableandproperlysubstantiateddatathatwillenablethemtomakedecisions.

IntheCouncilRegulation（EC）No1260/1999of21June1999layingdowngeneralprovisionsontheStructuralFunds,thereisanindicationthatevaluationiscarriedouttoestablishtheefficiencyofstructuralassistancefromCommunityaswellastoestimateitsimpactwithreferencetoaims,andalsotheanalysisofimpactonspecificstructuralproblems.Thisstatementtellsustotreatevaluationasatoolforplanningandmanagingstructuralassistanceaswellasanelementofthesystemsupportinganeffectiveuseofthosefunds.

Tounderstandproperlytheessenceofevaluationstudies,itisnecessarytoemphasize,thatevaluationissometimesassociatedand/ortakenforcontrol,auditormonitoring.Thesetermsshouldnotbeidentifiedwithevaluation,althoughtheycanbe（inspecificcases）atoolforupdatingthedatacollectedduringevaluationaswellasfortheneedsofanalysescarriedoutduringevaluation.

Theexplanationsofterms:

control,audit,monitoringarepresentedbelow.

-Auditistheverificationofcomplianceoftheuseofresources（mostlyfinancial）withthebindinglegalregulationsandspecificstandardse.g.therulesgoverningtheuseofassistance.Informationobtainedfromtheauditcanbeusedinevaluationforestimatingefficiencyofaninterventionandalsoasthecomparativedataforothersimilarenterprises.

-Control,likeaudit,canrefertothefinancialandlegalaspectsofagivenprojectimplementation.Moreoveritcanalsoapplytoastudyoforganisationalandmanagingstructures.Asopposedtoaudit,whichasarulehasanoverallandcomprehensivecharacter,controlcanhaveafragmentarycharacterandapplytooneaspectoftheinstitution'soperation,e.g.theproceduresofimplementinginnovationsorthequalitysystem.

-Monitoring-"regulargatheringandexaminationofquantitativeandqualitativedataonprojectsandwholeprogrammeimplementationwithregardtofinancialandphysicalissues,theobjectiveofwhichistoensurethecomplianceoftheprojectandprogrammeimplementationwiththeguidelinesandobjectivesapprovedbeforehand".Monitoringisusuallyconductedsimultaneouslywiththeimplementedinterventionandisdesignedforverifyingthisprocess,particularlytheachievementofassumedoutputsandresultsoftheundertakenmeasuresaswellasinputsmobilisedfortheirimplementation.

Asithasalreadybeensaid,audit,controlaswellasmonitoringcanbeusedasthesourceofinformation,evaluationhoweveremploysalsoitsownmethodology.Beforethesuitablemethodsforevaluationarepresented,itisworthtodistinguishevaluationtypes.

Evaluationtypescanbeclassifiedapplyingvariouscriteria.Oneofthemistimewhenevaluationiscarriedoutwithrespecttotheimplementationofaprogramme（ex-anteevaluation,mid-termevaluationandex-postevaluation）.Anothercriterionisthe"location"ofthosewhoconductevaluationandtheirdependenceontheprogrammeexecutors.Ifevaluationisconductedbyanindependentcontractor,thenwetalkabouttheexternalevaluation.Thisevaluationisassumedtoguaranteeindependenceofjudgementsandopinions.Itsadvantageisthefactthatitiscarriedoutbycompaniesspecialisinginthiskindofactivity,andthatensuresprofessionalismoftheservicesprovided.Thisevaluationcanbesubjecttoriskofinappropriatelyformulatedconclusionsandrecommendationsmadebyevaluators,resultingfromthelackofanin-depthknowledgeofinstitutionsinvolvedintheimplementationofanevaluatedproject.

Incaseofevaluationconductedbypeopledirectlyorindirectlyconnectedwiththeadministrationresponsibleforaproject,wetalkabouttheinternalevaluation.Owingtothisrelationknowingthespecificityofagiveninstitutioncanbeusedduringevaluationandbecauseofthatformulatedrecommendationscanbemoreuseful.Ontheotherhandhowever,themainaccusationofthistypeofevaluationisitslackofappropriateobjectivityatanalysesanddatainterpretationaswellasthelackofatrainedpersonnel,whobesidetheireverydaydutiesrelatedtoworkinaninstitution,couldengageinevaluationtasks.AsfarasevaluationoftheStructuralFundsisconcerned,wefrequentlydealwiththeexternalevaluation（mentionedintheframeworkregulation）.Onlytheon-goingevaluationcanbeconductedastheinternalone.

TheCouncilRegulation（EC）No1260/1999of21June1999layingdowngeneralprovisionsontheStructuralFundsenumeratesthreetypesofevaluationinChapter3:

ex-ante（Article41）,mid-term（Article42）andex-post（Article43）.Besides,thereisalsotheso-calledon-goingevaluation,theconductingofwhichisnotrequiredintheframeworkregulationaswellasso-calledthematicevaluation.AndtheActof20April2004ontheNationalDevelopmentPlan（Articles57-61;OJof24May2004）,indicatesthreetypesofevaluation:

（1）ex-anteevaluation-conductedbeforethebeginningofimplementingaProgramme;

（2）mid-termevaluation;（3）ex-postevaluation-aftertheendofimplementingaProgramme.

BriefdescriptionsofspecificevaluationtypesinaccordancewiththeterminologyproposedbytheCouncil（EC）,willbepresentedbellow.

-Ex-anteevaluation-isperformedbeforeprogrammeimplementationanditsobjectiveistoassesswhethertheplannedinterventionisaccuratewithregardtoneeds（ofasectororbeneficiaries）aswellascoherentwithreferencetoplannedaimsandhowtheywillbeimplemented.Itcanalsobetheassessmentofacontext,theidentificationofpotentialdifficultiesaswellasthediagnosisoftargetgroupneedsandexpectations.

IntheworkingpaperissuedbytheEuropeanCommissiontheex-anteevaluationisdefinedasaninteractiveprocessprovidingjudgementandrecommendationsbyexperts,separatelyfromtheplanners,onpolicyorprogrammeissues.Theobjectiveoftheex-anteevaluationistoimproveandstrengthenthefinalqualityofaPlanorProgrammeunderpreparation.Inthisregard,thisevaluationworkhastofacilitateaconstructivedialoguebetweenpeopleresponsibleforaPlanorprogrammeandtheexperts.Theex-anteevaluationalsoconstitutesakeyelementenablingtounderstandthestrategyandallocatefinancialresources,indicatingclearlytherationaleandthescopeofchoicesmade.Intheframeworkregulation（Articles40and41）sixmainelementsoftheprogramme,whichshouldbecoveredbytheex-anteevaluation,areenumerated.Theseare:

（1）theanalysisoftheso-farexperiences;

（2）thediagnosisofthesocio-economiccontextofassistance;（3）theassessmentofthelegitimacyofchoicesmadeandprioritiesofmeasuresacceptedaswellastheassessmentoftheirinternalandexternalcoherence;（4）theassessmentofthequantificationofobjectives:

（5）theassessmentoftheanticipatedsocio-economicinfluenceaswellasresourceallocation;（6）theassessmentoftheacceptedprogrammeimplementationarrangements.

-Mid-termevaluationistheevaluationperformedtowardsthemiddleoftheimplementationofanintervention.Thisevaluationcriticallyconsidersthefirstoutputsandresults,whichenableassessingthequalityofprogrammeimplementation.Itisessentialfortheassessmentoftheassumptionsmadeduringthepreparationstage,particularlyobjectivesandagreedindicatorsaswellasthecurrentcontextoftheimplementation.Thisisespeciallycrucial,asachangeinsocio-economicconditionscanmaketheinitialdiagnosisthatwasthestartingpointfortheimplementedintervention,outdate