2001 Annual Review of Development Effectiveness: Making Choices

  • Published on

  • View

  • Download


2001 Annual Review ofDevelopmentEffectiveness2001 Annual Review ofDevelopmentEffectivenessMaking ChoicesMaking Choices1818 H Street, N.W.Washington, D.C. 20433, U.S.A.Telephone: 202-473-1000Facsimile: 202-477-6391Telex: MCI 64145 WORLDBANKMCI 248423 WORLDBANKInternet: www.worldbank.orgOperations Evaluation DepartmentPartnerships & Knowledge Programs (OEDPK)E-mail: ecampbellpage@worldbank.orgE-mail: eline@worldbank.orgTelephone: 202-458-4497Facsimilie: 202-522-3125World Bank InfoShopE-mail: pic@worldbank.orgTelephone: 202-458-5454Facsimilie: 202-522-15002001 Annual Review of Development Effectiveness Making Choices2001 Annual Review of Development Effectiveness Making ChoicesTHE WORLD BANK THE WORLD BANKTHE WORLD BANKWORLD BANK OPERATIONS EVALUATION DEPARTMENTISBN 0-8213-5139-71 5 1 3 99 780821 351390ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATIONThe Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directlyto the Banks Board of Executive Directors. OED assesses what works, and what does not; how a borrower plansto run and maintain a project; and the lasting contribution of the Bank to a countrys overall development. Thegoals of evaluation are to learn from experience, to provide an objective basis for assessing the results of theBanks work, and to provide accountability in the achievement of its objectives. It also improves Bank work byidentifying and disseminating the lessons learned from experience and by framing recommendations drawn fromevaluation findings.OPERATIONS EVALUATION DEPARTMENT2001 AnnualReview ofDevelopmentEffectivenessMaking ChoicesWilliam Battaile2002THE WORLD BANKWashington, D.C.W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N Thttp://www.worldbank.org/oed 2002 The International Bank for Reconstruction and Development / The World Bank1818 H Street, NWWashington, DC 20433All rights reserved.Manufactured in the United States of AmericaFirst Printing May 20021 2 3 4 03 02 1The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect theviews of the Board of Executive Directors of the World Bank or the governments they represent.The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denomina-tions, and other information shown on any map in this work do not imply on the part of the World Bank any judgmentof the legal status of any territory or the endorsement or acceptance of such boundaries.Rights and PermissionsThe material in this work is copyrighted. No part of this work may be reproduced or transmitted in any form or by anymeans, electronic or mechanical, including photocopying, recording, or inclusion in any information storage and retrievalsystem, without the prior written permission of the World Bank. The World Bank encourages dissemination of its workand will normally grant permission promptly.For permission to photocopy or reprint, please send a request with complete information to the Copyright ClearanceCenter, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com.All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher,World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org.ISBN 0-8213-5139-7Library of Congress Cataloging-in-Publication Data has been applied for.Printed on Recycled Paperi i iContentsvii Acknowledgmentsix Foreword, Prefacio, Avant-Proposxiii Executive Summary, Resumen, Rsum Analytiquexxix Abbreviations and Acronyms1 1. Making Development Choices5 2. Instruments of Bank Assistance5 Financial Services5 Investment, Adjustment, and Other Instruments7 Recent Lending Choices9 Nonfinancial Services9 ESW Reflecting Corporate Priorities10 Clear Instrument Guidance Is Key13 3. Performance at the Instrument Level13 Performance of Lending Assistance14 Performance Trends14 Outcome Trends15 Institutional Development Impact15 Sustainability15 Aggregate Project Performance Index (APPI)17 Performance by Region17 Performance by Sector19 Performance by Lending Instrument22 Determinants of Success22 Country Conditions Matter24 Project-Level Performance Also Matters25 Performance of Nonlending Assistance29 4. Country Strategies, Instruments, and Outcomes29 Corporate Selectivity and Country Strategies32 Country Strategies and Instrument Choice32 Defining Comparative Advantage Depends on Partners33 Logical Framework Application in the CAS34 Country Strategies Are Dominated by Lending35 Country Strategies in the Absence of a Lending Program37 Risk Management Through Gradual Engagement/Disengagement37 How Instruments Contribute to Country Outcomes37 Country Capacity38 Borrower Commitment and Policy Environment39 Directions for Effective Instrument Use in Country Strategies41 5. Sector and Thematic Strategies, Instruments, and Outcomes41 Selectivity Across Sectors43 Instrument Choice and Performance43 Instrument Choice Among Sectors44 Sector-Specific Issues in Instrument Choice46 Combining and Sequencing Instruments48 Innovative Approaches to Sector Assistance: SWAPs and Social Funds51 Policies and Actions in Support of Thematic Strategies53 High-Potential Approaches for Implementation57 6. Findings and Implications57 Continued Gains in Performance57 Implications for Policy57 Corporate Selectivity58 Enhancing Country Strategies59 Instrument Selection in Support of Sector Strategies59 Implications for EvaluationAnnexes61 Annex A: The Banks Lending Instruments63 Annex B: A List of Disclosed OED Country Assistance Evaluations65 Annex C: Statistical Tables65 Table C.1. Outcome, Sustainability, Institutional Development (ID)Impact and Aggregate by Various Dimensions, Weightedby Projects, FY9699 and FY0001 Exits; Projects at Risk,Similarly Disaggregated, for the Active Portfolio66 Table C.2. Outcome, Sustainability, Institutional Development (ID)Impact and Aggregate by Various Dimensions, Weighted byDisbursements, FY9699 and FY0001 Exits; Disbursementsat Risk, Similarly Disaggregated, for the Active Portfolio67 Annex D: OED Evaluation Methodology71 Annex E: Managing Development Effectiveness: An Overview from theCODE Chairperson73 Endnotes77 Bibliography2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s si vBoxes20 Box 3.1. Which Objectives Have Bank Projects Been Most Effective inAchieving?27 Box 3.2. Bank-Managed Special Programs28 Box 3.3. Quality of the Banks Poverty Assessments35 Box 4.1. Synergy Between Sector Work and Lending36 Box 4.2. Kenya: Enhancing Lending Strategy in a Poor PolicyEnvironment38 Box 4.3. Getting Better Instrument Results in Poorly PerformingCountries48 Box 5.1. Combining Investment and Adjustment Lending forInstitutional ReformPvTables9 Table 2.1. Evolution of Diagnostic ESW10 Table 2.2. Country Coverage of Fiduciary Diagnostic ESW (FY9701)25 Table 3.1. Reasons for Unsatisfactory Outcomes34 Table 4.1. Country Program and Portfolio Performance39 Table 4.2. Quality of ESW by Country Policy and InstitutionalEnvironment50 Table 5.1. Core Features of SWAPs and Social Funds51 Table 5.2. Tools to Support Thematic StrategiesFigures2 Figure 1.1. Three Dimensions of Selectivity6 Figure 2.1. The Instrument Toolkit6 Figure 2.2. Lending Instrument Choice (FY9001)8 Figure 2.3. Lending Risks, by CPIA and Credit Ratings (FY9601Approvals)8 Figure 2.4. Costs Differ by Lending Instrument (FY9601 Exits)14 Figure 3.1. Approval Years of Recently Evaluated Projects15 Figure 3.2. Trends in Outcomes16 Figure 3.3. Trends in Institutional Development Impact16 Figure 3.4. Trends in Sustainability17 Figure 3.5. Trends in Outcomes for Africa and Other Regions18 Figure 3.6. Trends in Outcomes by Region18 Figure 3.7. Trends in Outcomes by Sector19 Figure 3.8. Trends in Outcomes of Adjustment Operations21 Figure 3.9. Trends in Outcomes of Investment Operations22 Figure 3.10. Risks and Rewards of Lending Instruments23 Figure 3.11. The Cost Dimension, by CPIA Groups (FY90FY01 Exits)23 Figure 3.12. Instrument Performance Is Sensitive to Country Conditions(FY9601 Exits)24 Figure 3.13. Adjustment Outcomes in Africa Related to the CFADevaluation26 Figure 3.14. Improved Bank Performance as Evaluated by OED and QAG31 Figure 4.1. Better Lending Outcomes Associated with Higher Policyand Institutional Quality (FY9601 Exits)31 Figure 4.2. More Lending to Countries with Better Project Outcomes(FY9601 Exits)C o n t e n t sv42 Figure 5.1. Major Shifts in Sectoral Commitments (FY9092 to FY9901Approvals)43 Figure 5.2. Different Sectors Rely on Different Instrument Groups(FY9601 Approvals)44 Figure 5.3. The Move to Adjustment Lending in Finance (FY9001Approvals)47 Figure 5.4. Costs and Outcomes Are Influenced by Quality ofOperating Environment2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sv iv i iAcknowledgmentsThis report was prepared by a team ledby William Battaile. Carlos Reyes wasresponsible for Chapter 3 on instrumentperformance trends. Other team members wereZamir Islamshah, Gillian Perkins, and DeepaChakrapani. The team received excellent supportfrom Parveen Moses, Annisa Cline-Thomas, andBill Hurlbut. The report benefited from exten-sive contributions from OED staff, and drew ona wide range of recent OED country, sector, andthematic evaluations. Background papers andsupplemental analyses were provided by ManuelPenalver-Quesada, Laurie Effron, Nalini Kumar,Jed Shilling, Catherine Gwin, Diana Qualls,Anara Omarova, and Oliver Rajakaruna. The comments and guidance from peerreviewers are gratefully acknowledged, includ-ing Susan Stout, Irene Xenakis, Marisela MontoliuMuoz, David Hughart, Shawki Barghouti, JaimeBiderman, Aloysius Ordu, and Tim Johnston. TheReview was completed under the guidance ofVictoria Elliott, Manager of OEDCM.This study was published in the Partnershipsand Knowledge Group (OEDPK) by the Out-reach and Dissemination Unit. The task teamincludes Elizabeth Campbell-Pag (task teamleader), Caroline McEuen (editor), and JuicyQureishi-Huq (administrative assistant).Director-General, Operations Evaluation: Robert PicciottoDirector, Operations Evaluation Department: Gregory IngramManager, Corporate Evaluations and Methods: Victoria ElliottTask Manager: William BattaileThis Page Intentionally Left Blanki xFOREWORDIn the face of a rapidlychanging developmentagenda, the World Bank has provenremarkably flexible in adapting andexpanding its operational toolkit.New lending instruments, analyticaltools, and partnership arrangementshave emerged to respond to the myr-iad needs and preferences of bor-rowers and to implement corporatestrategies. These innovations havecontributed to improvements in pro-ject and program performance. The broad endorsement of theMillennium Development Goalswithin the development commu-nity has put the spotlight on theresults of development activities.Accordingly, attainment of tan-gible, measurable, and sustainableimprovements in the health, edu-cation, and standard of living ofthe worlds poor is the overarch-ing challenge facing the Bank andits development partners. This is the fifth Annual Reviewof Development Effectiveness(ARDE). The 1997 Review con-centrated on the linkages betweenaid and development. The 1998Review reflected the lessons ofthe East Asia financial crisis. The1999 Review looked at the chal-lenges of implementing the Com-prehensive DevelopmentFramework, and identified prom-ising practices for dealing withthem, predicated on strong coun-try commitment to poverty reduc-tion and sustainable growth. The2000 Review concluded that theBank could implement its strate-gies more effectively by judiciousPREFACIOEn el actual contexto de rp-ida transformacin de lasactividades de desarrollo, el BancoMundial ha demostrado una notableflexibilidad para adaptarse yampliar sus instrumentos operacio-nales. Han aparecido nuevos dispo-sitivos crediticios, herramientasanalticas y mecanismos de asocia-cin en respuesta a las innumera-bles necesidades y preferencias delos prestatarios y para aplicar lasestrategias institucionales. Estasinnovaciones han significado mejo-ras en los resultados de los proyec-tos y los programas.La amplia aceptacin de losobjetivos de desarrollo del mile-nio entre las instituciones intere-sadas en el desarrollo ha puestoen primer plano los resultados delas actividades de desarrollo. Enconsecuencia, el logro de mejorastangibles, cuantificables y sosteni-bles en los terrenos de la salud,educacin y nivel de vida de lospobres de todo el mundo es eldesafo general con que seenfrentan el Banco y sus asocia-dos en el desarrollo.Este es el quinto Examenanual de la eficacia en trminosde desarrollo. El Examen de 1997se centr en las vinculacionesexistentes entre la ayuda y el des-arrollo. En el de 1998 se conside-raron las enseanzas extradas dela crisis financiera de Asia orien-tal. En el de 1999 se analizaronlos desafos que representaba laaplicacin del Marco Integral deDesarrollo, y se identificaron lasprcticas ms prometedoras paraAVANT-PROPOSConfronte lvolutionrapide de lordre du jour dudveloppement, la Banque mondialea russi adapter et largir lagamme de ses instruments opra-tionnels avec une souplesse remar-quable. Elle a su rpondre auxbesoins multiples des emprunteurset satisfaire leurs prfrences encrant de nouveaux instruments deprt et de nouveaux outils danalyse,et en forgeant des accords de parte-nariat. Ces innovations ont contribu amliorer les rsultats des projetset des programmes. Ladoption des Objectifs dedveloppement pour le mill-naire par lensemble de la com-munaut internationale a mis envidence limportance des rsul-tats des activits de dveloppe-ment. La mission primordiale quese sont fixe la Banque et de sespartenaires pour le dveloppe-ment, damliorer de faon tangi-ble, mesurable et durable la sant,lducation et le niveau de vie despauvres du monde entier, estdonc plus que jamais dactualit. Nous publions cette anne lacinquime dition de lExamenannuel de lefficacit du dvelop-pement (ARDE). Lexamende 1997 dcrivait les liens entrelaide et le dveloppement etcelui de 1998 tirait les leons dela crise financire dAsie de lEst.Lexamen de 1999 se penchait surles problmes de mise en uvredu Cadre de dveloppement int-gr et dgageait des moyens pro-metteurs pour les rsoudre,reposant sur la ferme volont desFRANAISESPANOLENGLISHadaptation to diverse insti-tutional and social environ-ments, as well as byacknowledging and manag-ing differences betweenclient and Bank priorities. This years Review high-lights the choice of lendingand nonlending instruments andactivities to achieve developmentobjectives. It complements theQuality Assurance Groups annualassessment of the active lendingportfolio and of recent analyticaland advisory services. As in prioryears, the Review concentrates onlong-term development effective-ness trends. It finds that selectingthe right combination andsequence of activities for a partic-ular set of objectives can makethe difference between successand failure.This conclusion holds at alllevelsthe individual project, thecountry assistance program, andthe Banks global, sectoral, andthematic priorities. At each levelthere is unexploited potential tomake judicious use of lendingand nonlending activities, tochoose instruments that areadapted to the problems faced,and to leverage work done bypartners. Such strategic selectivityis especially important in coun-tries with a poor policy frame-work, small or newly reactivatedlending programs, or acute institu-tional development needs. The findings of the 2001 ARDEdemonstrate sustained progress inportfolio performance and suggestseveral directions for future Bankoperations. First, the ongoingupdating of the policy frameworkfor investment and adjustmentlending offers a good opportunityto offer operational guidance andsuperarlos, basadas en undecidido compromiso delos pases con la reduccinde la pobreza y el creci-miento sostenible. En elExamen de 2000 se con-cluy que el Banco podraaplicar sus estrategias conmayor eficacia si se adaptaba pru-dentemente a los distintos entor-nos institucionales y sociales, yreconoca y resolva las diferen-cias entre las prioridades de losclientes y del Banco.En el Examen del presente aose hace hincapi en la gama deinstrumentos y actividades definanciamiento y de otro tipo dis-ponibles para el logro de losobjetivos del desarrollo. Es uncomplemento de la evaluacinanual del Grupo de garanta decalidad sobre la cartera de prsta-mos activos y sobre los recientesservicios de anlisis y asesora.Como en aos anteriores, el Exa-men se concentra en las tenden-cias a largo plazo de la eficacia entrminos de desarrollo. Se com-prueba que la seleccin de lacombinacin y secuencia adecua-das de actividades para un con-junto determinado de objetivospuede representar la diferenciaentre el xito y el fracaso.Esta conclusin es vlida entodos los niveles: proyectos indi-viduales, programas de asistenciaa los pases y prioridades mundia-les, sectoriales y temticas delBanco. En cada nivel, hay margenpara utilizar sabiamente las activi-dades de financiamiento y otrotipo, elegir instrumentos queestn en consonancia con los pro-blemas existentes, y conseguir unefecto multiplicador con las activi-dades realizadas por las institucio-nes asociadas. Esta selectividadpays de faire reculer lapauvret et de promouvoirune croissance durable.Lexamen de 2000 notait enconclusion que la Banquepourrait mettre en uvreses stratgies plus efficace-ment en sadaptant judi-cieusement au contexteinstitutionnel et social, tout enprenant en compte et en grantles diffrences entre les prioritsdes clients et celles de la Banque. Lexamen de cette anne meten vidence la gamme des instru-ments de prt et des services horsprt qui permettent datteindre lesobjectifs de dveloppement. Ilcomplte lvaluation annuelle duportefeuille actif et des servicesrcents danalyses et de conseileffectue par le Groupe dassu-rance de la qualit. Comme lesannes prcdentes, lexamen seconcentre sur les tendances delefficacit du dveloppement long terme. Il aboutit la conclu-sion que la diffrence entre lesuccs et lchec rside dans ledosage et lchelonnement dacti-vits adaptes une srie particu-lire dobjectifs.Cette conclusion est valable tous les niveaux les projets indi-viduels et les programmes daideaux pays comme les priorits glo-bales, sectorielles et thmatiquesde la Banque. Il existe chaqueniveau un potentiel que lonpourrait exploiter en utilisant judi-cieusement des services de prt ethors prt, en choisissant desinstruments adapts aux probl-mes qui se posent et en tirantparti des travaux accomplis parles partenaires. Il est particulire-ment important de faire preuvede cette slectivit stratgiquedans les pays o le cadre de2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sxENGLISHFRANAISESPANOLF o r e w o r dx iimprove instrument choice.Second, in poorly perform-ing low-income countries,simple operations, pilotprojects, and nonfinancialactivities have particularpotential to deliver results.Third, for adjustment oper-ationsa growing share of Banklendingsuccess is more likelywhen the domestic consensus forreform is strong and other Bankinstruments are brought to bearboth upstream and downstreamof the adjustment process. estratgica es especial-mente importante en lospases con un marco depolticas poco adecuado,programas de financia-miento de pequea escalao recientemente reactiva-dos o con necesidadesagudas de desarrollo institucional.Las conclusiones de este Exa-men anual de la eficacia en trmi-nos de desarrollo revelan uncontinuado progreso en los resul-tados de la cartera y parecen indi-car varias orientaciones para lasoperaciones futuras de Banco. Enprimer lugar, la actualizacinconstante del marco de polticaspara la inversin y los prstamosde ajuste representa una buenaoportunidad de ofrecer asesora-miento operacional y mejorar lagama de instrumentos disponi-bles. En segundo lugar, en lospases de ingreso bajo con resul-tados inadecuados, pueden resul-tar especialmente eficaces lasoperaciones de pequea enverga-dura, los proyectos piloto y lasactividades no financieras. En ter-cer lugar, por lo que se refiere alas operaciones de ajuste unaparte creciente de los prstamosdel Banco es ms probable quese consigan xitos cuando el con-senso interno en favor de lareforma es firme y se cuenta conotros instrumentos del Bancotanto en las fases iniciales comofinales del proceso de ajuste.laction des pouvoirspublics est dfaillant, dontles programmes de prtssont limits ou ont tractivs depuis peu oudont les besoins de dve-loppement institutionnelsont aigus. LExamen annuel de lefficacitdu dveloppement 2001 rvleque des progrs durables ont traliss en matire de perfor-mance du portefeuille, et proposeplusieurs orientations pour lesoprations futures de la Banque.Premirement, une mise jourrgulire du cadre dans lequelsinscrivent les prts dinvestisse-ment et les prts lajustement estun bon moyen dimprimer desorientations oprationnelles etdamliorer le choix des instru-ments. Deuximement, dans lespays faible revenu dont lesrsultats sont mdiocres, des op-rations simples, des projets piloteset des activits autres que finan-cires sont les plus aptes don-ner des rsultats. Troisimement,les oprations dajustement quireprsentent une part croissantedes prts de la Banque ontdavantage de chances de russirlorsque le consensus national enfaveur des rformes est solide, etque dautres instruments de laBanque interviennent tant enamont quen aval du processusdajustement. Robert PicciottoDirector-General, Operations EvaluationENGLISHESPANOLFRANAISThis Page Intentionally Left Blankx i i iEXECUTIVESUMMARYThe challenge of develop-ment, never straightforward,has become even more complex asthe number of actors has grown andthe desire for demonstrable resultshas intensified. In seeking to imple-ment its poverty reduction mission,the World Bank has to match over100 client countries tackling innu-merable development issues with afinancial product line ranging fromsmall grants to large loans, adiverse nonfinancial product line,and a growing cast of public, pri-vate, and civil society partners. Ifthe Bank is to continue improving itsdevelopment effectiveness, it willhave to make the right choicesabout how, when, and with whom toengage (and disengage). Making theright choices with client countriesand other development partnerswhat is often called selectivityisthe theme of this Review. The Bank exercises selectivitythrough the choices it makes inthree dimensions: its corporategoals, strategies for countries, andspecific activities or instruments.Decisions on each of thesedimensions help determine theinstitutions development effec-tiveness. The overarching corpo-rate missionto reduce povertythrough a focus on resultshasbeen translated into corporategoals and institutional initiativesin support of them. Individualcountry strategies are expected toreflect the Banks corporate goals,while at the same time matchingclient needs and the comparativeadvantages of partners in particu-RESUMENEl desafo del desarrollo,siempre complicado, haadquirido mayor complejidad toda-va a medida que se ha multiplicadoel nmero de agentes y se ha inten-sificado el deseo de resultadosdemostrables. El Banco Mundial,empeado en cumplir su misin dereduccin de la pobreza, tiene queintegrar la realidad de sus ms de100 pases clientes con innumera-bles problemas de desarrollo conuna lnea de productos financierosque van desde pequeas donacionesa grandes prstamos, una amplialnea de productos no financieros yun grupo cada vez ms numeroso deinterlocutores del sector pblico yprivado y de la sociedad civil. Paraque el Banco contine mejorando sueficacia en trminos de desarrollo,tendr que tomar las decisiones ade-cuadas sobre cmo, cundo y conquin va a mantener (o dejar detener) relaciones. El tema del pre-sente Examen es precisamentecmo tomar las decisiones adecua-das con los pases clientes y otrasinstituciones interesadas en el des-arrollo: lo que suele conocerse conel nombre de selectividad.El Banco ejerce la selectividada travs de las decisiones queadopta en tres dimensiones: susmetas institucionales, las estrate-gias para los pases y las activida-des o instrumentos especficos.Las decisiones sobre cada una deestas dimensiones ayudan a deter-minar la eficacia en trminos dedesarrollo de la institucin. Lamisin institucional general reducir la pobreza mediante unaRSUM ANALYTIQUELe dfi du dveloppement,qui na jamais t simple,devient encore plus complexe mesure que davantage dacteursinterviennent et que la volontdobtenir des rsultats tangiblessintensifie. Pour accomplir sa mis-sion de faire reculer la pauvret, laBanque mondiale doit offrir plusdune centaine de pays clientsconfronts dinnombrables probl-mes de dveloppement laccs ceux de sa ligne de produits finan-ciers, allant de dons modestes desprts importants, et ceux de savaste gamme de produits non finan-ciers dont ils ont prcismentbesoin chacun, et les mettre en rap-port avec les partenaires de plus enplus nombreux des secteurs publicet priv et de la socit civile quiconviennent. Si la Banque veutcontinuer amliorer lefficacit dudveloppement, elle devra choisirjudicieusement dans quelles condi-tions, quand et avec qui prendre desengagements (ou se dsengager).Faire les choix qui simposent avecles pays clients et les autres parte-naires du dveloppement, ce quonappelle souvent la slectivit, serale thme du prsent Examen.La Banque fait des choix slec-tifs dans trois domaines : sesobjectifs institutionnels, les strat-gies pour les pays et les activitsou les instruments particuliers. Lesdcisions prises dans chacun deces domaines contribuent dter-miner lefficacit de linstitutionen matire de dveloppement. Lamission institutionnelle primor-diale, savoir de faire reculer laENGLISHESPANOLFRANAISlar settings. Even with theright strategies in place,results on the grounddepend on choosing theright instrumentsthefocus of this report.The ARDEs review ofselectivity identifies threethemes common to the project,sector, and country levels ofanalysis: Good diagnosis, provided byhigh-quality economic and sec-tor work, is critical for estab-lishing realistic developmentobjectives. Country assistancestrategies, sector strategies, andoperational guidance all informthe matching of instrumentswith objectives. The choice of instrumentshould reflect not only theobjectives of the individualoperation, but also past perfor-mance in the country and sec-tor context. Appropriatesequencing and tapping ofcomplementarities amonginstruments help improveoutcomes. A poor policy and institutionalenvironment compromises theeffectiveness of both lendingand nonlending interventionsand calls for a nuanced selec-tion of instruments. In weakcountry environments, stand-alone technical assistance oper-ations have performed best,while structural adjustmentlending tends to be the riskiestinstrument. Simple projectdesignsor a series of simpleinterventionsprovide betterresults than complex and multi-faceted undertakings. Even iflending is constrained by poorperformance, carefully selectednonlending activities can be2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx i vENGLISHmayor atencin a los resul-tados se ha traducido enmetas e iniciativas institu-cionales en apoyo de stos.Se espera que las estrate-gias de los distintos pasesestn en consonancia conlas metas institucionales delBanco, y que al mismo tiempocorrespondan a las necesidadesdel cliente y a las ventajas compa-rativas de los socios en los dife-rentes contextos. Aun cuando seapliquen las estrategias pertinen-tes, los resultados sobre el terrenodependen de la eleccin de losinstrumentos adecuados: ese esprecisamente el tema del presenteinforme.El examen sobre la selectividadrevela la existencia de tres temascomunes en los tres niveles deanlisis (proyectos, sectores ypases): Un diagnstico correcto, conse-guido a travs de estudios eco-nmicos y sectoriales de altacalidad, es condicin decisivapara establecer objetivos dedesarrollo realistas. Las estrate-gias de asistencia a los pases,las estrategias sectoriales y laorientacin operacional contri-buyen a emparejar los instru-mentos con los objetivos. La eleccin de instrumentodebe corresponder no slo alos objetivos de cada operacinsino tambin al rendimientoanterior del pas y el contextosectorial. Los resultados pue-den mejorar gracias a una ade-cuada seleccin del orden delos instrumentos y al aprove-chamiento de las complemen-tariedades entre distintosinstrumentos. Un entorno normativo e insti-tucional desacertado pone enpauvret en se concentrantsur les rsultats, se concr-tise par des buts et desinitiatives institutionnelsadapts. Les stratgiesadoptes pour chaque paysdoivent reflter les butsinstitutionnels de la Ban-que, tout en faisant concider lesbesoins des clients et les avanta-ges comparatifs des partenaires enfonction des circonstances parti-culires. Mme si des stratgiesadquates sont mises en place, lesrsultats sur le terrain ne peuventtre obtenus quen choisissant lesinstruments adquats. Cest lobjetde ce rapport.Lexamen de la slectivit, dansle cadre de lARDE, permet dedfinir trois thmes communs auxtrois niveaux de lanalyse (projet,secteur et pays) : il est essentiel, pour fixer desobjectifs de dveloppementralistes, de raliser un bondiagnostic lissue dtudesconomiques et sectorielles dequalit. Les stratgies daide-pays, les stratgies sectorielleset les orientations oprationnel-les sont autant de moyens defaire concider les instrumentset les objectifs ; le choix dun instrument doitcadrer non seulement avec lesobjectifs dune opration parti-culire, mais galement avecles antcdents du pays et dusecteur. En choisissant lemoyen appropri dchelonnerles instruments et dexploiterleurs complmentarits, onpeut amliorer les rsultats ; la mdiocrit des politiques etdes institutions comprometlefficacit des interventions deprt et hors prt et exige unchoix nuanc des instruments.ESPANOLFRANAISuseful, especiallyto keep the Bank pre-pared for possible re-engagement. An Evolving ToolkitDevelopment effectivenessdepends on selecting theright instruments and deployingthem in appropriate sequence andcombination, in light of countryand sector characteristics. Theframework for the Banks activ-ities, as set out in the Articles ofAgreement, is sufficiently broad topermit a wide range of develop-ment assistance activities as wellas innovation over time as cir-cumstances have changed andunderstanding of the develop-ment process has improved. Manyinnovations to the Banks financialand nonfinancial toolkit wereintroduced over the 1990s. Forexample, the new adaptable lend-ing instruments promise toincrease the Banks flexibility,reduce risks to borrowers and theBank, and facilitate exit fromfloundering operations. Similarly,nonfinancial activities becamemore diversified and participatorywith greater emphasis on nurtur-ing reform and enhancing thequality of partnerships.Distinctions among lendinginstruments reflect their ability toaddress differing developmentobjectives and their costs, syner-gies, and complementarities.Sharper operational guidancewould help country assistancestrategies choose instrumentsappropriate to specific objectivesand to sector and country condi-tions. In particular, now that con-siderable effort has been devotedto establishing the standards andscope of diagnostic instruments,E x e c u t i v e S u m m a r yx vENGLISHpeligro la eficacia de lasintervenciones crediti-cias y no crediticias yrequiere una seleccinmatizada de instrumen-tos. En los pases dondelas circunstancias sonpoco favorables, las operaciones de asistencia tc-nica independientes han dadolos mejores resultados, mien-tras que el prstamo para finesde ajuste estructural suele serel instrumento de mayorriesgo. La sencillez del diseode los proyectos o de unaserie de intervenciones per-mite obtener mejor resultadoque iniciativas complejas y endistintos frentes. Aun cuandolos crditos se vean limitadospor un mal desempeo, unaseleccin esmerada de activida-des no crediticias puede resul-tar til, sobre todo para que elBanco est preparado para unposible nuevo compromiso. Instrumentos en evolucinLa eficacia en trminos de des-arrollo depende de la seleccinde los instrumentos adecuados yde su utilizacin en un orden ycon una combinacin adecuados,teniendo en cuenta las caracters-ticas de cada pas y sector. Elmarco para las actividades delBanco, establecido en el Conve-nio Constitutivo, es lo suficiente-mente amplio como para permitiruna gran variedad de actividadesde asistencia para el desarrollo ascomo para introducir innovacio-nes a lo largo del tiempo amedida que vayan cambiando lascircunstancias y se llegue a unamejor comprensin del procesode desarrollo. Durante el deceniode 1990 se introdujeron muchasDans les pays o lecadre daction est faible,les oprations de pureassistance techniquedonnent gnralementles meilleurs rsultats,tandis que les prts lajustement structurel sont les instruments les plusrisqus. La formulation de pro-jets simples, ou une sriedinterventions simples, don-nent de meilleurs rsultats quedes oprations complexes etmultidimensionnelles. Mmedans les cas o les prts sontlimits par une performancemdiocre, des activits horsprt choisies avec soin peuventse rvler utiles, en particulierpour prparer la Banque unventuel rengagement. Un ventail dinstruments envolutionLe dveloppement ne peut treefficace que si lon choisit lesinstruments adquats et si on lesdploie en les chelonnant et enles combinant de faon appro-prie, en fonction des caractristi-ques du pays et du secteur. Lecadre des activits de la Banque,tel quil est dfini dans les Statuts,est suffisamment large pour per-mettre non seulement de menerune vaste gamme dactivitsdaide au dveloppement, maisaussi dinnover mesure que lasituation volue et que le proces-sus de dveloppement est mieuxcompris. Dans les annes 90, laBanque a introduit de nombreuxinstruments financiers et nonfinanciers novateurs. Ainsi, lesnouveaux prts volutifs promet-tent daugmenter la souplesse dela Banque, de rduire les risquesauxquels sexposent les emprun-ESPANOLFRANAISthey should be deployed toensure priority coverage ofcountries with highvulnerability.Improved InstrumentPerformanceAs forecast in last yearsReview, the Strategic Compact tar-get of 75 percent satisfactory out-comes for lending has been met.The latest project evaluation dataconfirm significant improvementin the Banks lending perfor-mance, especially for FY00 exitingprojects. The upward trend contin-ues into FY01. The first half ofFY01 exiting projects, now evalu-ated by OED, has 82 percent satis-factory outcome ratings. Solidimprovements in sustainability andinstitutional development impactare also evident. Seventy-one per-cent of projects exiting during theFY0001 period are rated to havelikely or highly likely resilience tofuture risks; one of every two pro-jects evaluated during the sameperiod also shows substantial orbetter institutional developmentimpact ratings.Recent evaluations showimproved performance from pro-jects in the Africa Region follow-ing its portfolio improvementdrive. Looking at lending instru-ment groups, the outcomes ofadjustment operations declinedslightly for the FY0001 exitperiod, while investment projectoutcomes improved markedly.The quality of project-level inputsremains a key determinant ofproject success. The outcomes ofprojects are also closely correlatedwith the Banks Country Policyand Institutional Assessment(CPIA) ratings. Moreover, invest-ment interventions cost signifi-2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx v iENGLISHinnovaciones en el instru-mental financiero y nofinanciero del Banco. Porejemplo, los nuevos instru-mentos crediticios adapta-bles ofrecen posibilidadesde aumentar la flexibilidaddel Banco, reducir los ries-gos para los prestatarios y elBanco y facilitar el abandono deoperaciones condenadas al fra-caso. De la misma manera, lasactividades no financieras sonahora ms diversificadas y basa-das en la participacin, y hacenmayor hincapi en fomentar lareforma y mejorar la calidad delas asociaciones.Las distinciones entre los ins-trumentos crediticios responden asu capacidad de abordar los dife-rentes objetivos del desarrollo ysus costos, sinergias y comple-mentariedades. Una orientacinoperacional ms acertada ayuda-ra a las estrategias de asistencialos pases a elegir instrumentosadaptados a objetivos especficosy a la situacin de cada sector ypas. En particular, ahora que sehan desplegado notables esfuer-zos por establecer las normas yalcance de los instrumentos dediagnstico, deberan utilizarsestos para garantizar una cober-tura prioritaria de los pases msvulnerables.Mejor desempeo de losinstrumentosComo se prevea en el Examendel ao pasado, se ha alcanzadoel objetivo del Pacto Estratgicode un 75% de resultados satisfac-torios para las actividades crediti-cias. Los datos ms recientes delas evaluaciones de proyectosconfirman una significativa mejoraen el desempeo de los prsta-teurs et la Banque et defaciliter le retrait en casdchec. De mme, les acti-vits non financires sontdevenues plus diversifieset participatives et privil-gient davantage les rfor-mes et lamlioration de laqualit des partenariats.Ce qui distingue les instrumentsde prt, cest leur capacit viserdes objectifs de dveloppementdiffrents ainsi que leurs cots, lessynergies quils permettent et leurscomplmentarits. Des directivesoprationnelles plus prcises aide-raient choisir les instruments quiconviennent des objectifs parti-culiers et qui sont adapts lasituation dun secteur et dun paysdonns dans le cadre des Strat-gies daide aux pays. Alors quedes efforts considrables sontaujourdhui dploys pour dfinirles normes et la porte des outilsde diagnostic, il est particulire-ment important dappliquer cesoutils en priorit aux pays les plusvulnrables.Des instruments plusperformantsComme le prvoyait lexamen delan dernier, lobjectif du Pactestratgique, qui fixait 75 % laproportion des prts devant obte-nir des rsultats satisfaisants, a tatteint. Les dernires donnes surlvaluation des projets confirmentque les rsultats des prts de laBanque se sont sensiblementamliors, en particulier pour lesprojets sortis du portefeuilledurant lexercice 00. Cette ten-dance sest poursuivie durantlexercice 01. La premire moitides projets sortis du portefeuilledurant lexercice 01 (qui ontaujourdhui t valus parESPANOLFRANAIScantly more, per amountdisbursed, when under-taken in low CPIAcountries. The Strategic Compactgoalof 85 percent satis-factoryhas also beenachieved in nonlendingservices. Self-evaluation resultsshow a broad improvement in thequality of economic and sectorwork (ESW) as it becomes moreparticipatory, client-oriented, andresult-focused. There remainsroom for improvement in thepoverty focus of these instru-ments, as well as in the qualityand impact of ESW in poorlyperforming countries, especiallyfiduciary reviews.Instrument Choice AffectsCountry-Level ResultsThe outcome of country pro-grams, as measured in the morethan 50 Country Assistance Evalu-ations (CAEs) conducted thus farby OED, depends in part on howwell selectivity has been exercisedat the corporate, country, andinstrument levels. Corporate prior-ities are conveyed to individualcountry programs through theBanks support for the Compre-hensive Development Frameworkand the Poverty Reduction Strat-egy Initiative, inter-country lend-ing allocation decisions, and thedistribution of administrativeresources among Regions andNetworks. OEDs recent IDAReview finds that theperformance-based lending allo-cation system has evolved overthe past decade to reflect newdevelopment knowledge andevolving corporate priorities.These changes have strengthenedthe link between countries policyE x e c u t i v e S u m m a r yx v i iENGLISHmos del Banco, sobre todoen lo que respecta a losproyectos vigentes en elejercicio de 2000. La ten-dencia ascendente se pro-longa en el ejercicio de2001. La primera mitad delos proyectos finalizadosen este ltimo ejercicio, evaluadosahora por el Departamento deEvaluacin de Operaciones(DEO), ha merecido un 82% decalificaciones satisfactorias. Seobservan tambin mejoras consis-tentes en la sostenibilidad y enlos efectos del desarrollo institu-cional. Se considera que el 71%de los proyectos existentesdurante los ejercicios 200001resistirn probablemente o muyprobablemente a los riesgosfuturos; uno de cada dos proyec-tos evaluados durante el mismoperodo tendra efectos cuandomenos considerables en el des-arrollo institucional.Evaluaciones recientes revelanuna mejora en los resultados delos proyectos en la regin defrica tras la campaa empren-dida para mejorar su cartera. Si seconsideran los grupos de instru-mentos crediticios, los resultadosde las operaciones de ajuste dis-minuyeron ligeramente en el per-odo de salida correspondiente alos ejercicios de 200001, mien-tras que los resultados de los pro-yectos de inversin mejoraronnotablemente. La calidad de losinsumos de los proyectos conti-na siendo un determinante clavedel xito de los mismos. Losresultados estn tambin estrecha-mente correlacionados con lascalificaciones de la evaluacin delas polticas e instituciones nacio-nales (EPIN), del Banco Mundial.Adems, las intervenciones enlOED) ont obtenu desrsultats satisfaisants pour82 % dentre eux. De nettesamliorations en matirede viabilit et dimpact surle dveloppement institu-tionnel sont galement vi-dentes. Soixante et onzepour cent des projets sortis duportefeuille durant les exercices0001 ont t jugs susceptiblesou trs susceptibles de rsister des risques futurs. Parmi les pro-jets valus durant cette mmepriode, un sur deux est consi-dr comme ayant au moins unimpact apprciable sur le dve-loppement institutionnel.Les valuations rcentes indi-quent que les rsultats des projetsde la rgion Afrique se sont am-liors grce au lancement du pro-gramme damlioration duportefeuille. En matire dinstru-ments de prt, le rsultat des op-rations dajustement a lgrementdclin pour les projets sortis duportefeuille durant les exercices0001, alors que le rsultat desprojets dinvestissement sest nota-blement amlior. La qualit desapports aux projets reste un l-ment dterminant du succs desprojets. Le rsultat des projets estgalement troitement li lanote qui leur est attribue dans lecadre de lvaluation de la politi-que et des institutions nationales(EPIN). En outre, les oprationsdinvestissement sont beaucoupplus coteuses, par montantdcaiss, lorsquelles sont entre-prises dans des pays dont la per-formance est juge faible dans lecadre de lEPIN.Lobjectif du Pacte stratgique,dobtenir 85 % de projets satisfai-sants, a galement t atteint pourles services hors prt. Les rsultatsESPANOLFRANAISand institutional perfor-mance and lending levels.In its allocation decisionsfor IBRD (InternationalBank for Reconstructionand Development; alsoWorld Bank) borrowers,the Bank has also recog-nized that better developmentprospects result from lendingunder good country policies andinstitutions. There is untappedpotential to select and sequenceinstruments more strategically inindividual country programs, link-ing them more explicitly to objec-tives. When country circum-stances require unanticipatedchanges in the assistance pro-gram, it is important to use timelyCAS updates and progress reportsto provide an agreed-upon strate-gic context for Bank activities.A special challenge is gettingbetter results in poor policy andinstitutional environments.Reviews of country and projectevaluations show that the perfor-mance of both lending and non-lending instruments is under-mined by weak policies andinstitutions. In such adverse envi-ronments, stand-alone technicalassistance fares better than otherforms of lending, suggesting thatthe Bank should continue focus-ing on capacity and institutionbuilding in such countries. Partic-ular care is needed for adjustmentoperations, which tend to have alower likelihood of success in lowCPIA countries. For all lending,recent CAEs show that a series ofsimple interventions provides bet-ter results than complex undertak-ings. Pilot programs also providea valuable instrument to testuncertain environments and tobuild capacity. The success of2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx v i i iENGLISHforma de inversin tienenun costo significativamentemayor, por cantidad des-embolsada, cuando se rea-lizan en pases donde laEPIN ha ofrecido califica-ciones bajas.Se ha conseguido tam-bin la meta de Pacto Estratgico85% de calificaciones satisfacto-rias en los servicios no crediti-cios. Los resultados de laautoevaluacin revelan unaamplia mejora de la calidad de losestudios econmicos y sectoriales,que ahora estn ms basados enla participacin, ms orientados alos clientes y ms centrados enlos resultados. Sigue habiendomargen de mejora en la orienta-cin de estos instrumentos haciala pobreza, as como en la calidady efectos de los estudios econ-micos y sectoriales en los pasescon resultados deficientes, sobretodo en los exmenes fiduciarios. La eleccin del instrumentoinfluye en los resultados obtenidos en los pasesLos resultados de los programaspor pases, cuantificados en msde 50 evaluaciones de asistencia alos pases realizadas hasta ahorapor el DEO, dependen en partedel acierto con que se ha practi-cado la selectividad en cuanto ala institucin, el pas y el instru-mento. Las prioridades institucio-nales se transmiten a losprogramas de cada pas a travsdel apoyo del Banco al MarcoIntegrado de Desarrollo y la ini-ciativa de lucha contra la pobreza,las decisiones de asignacionescrediticias entre los distintos pa-ses y la distribucin de los recur-sos administrativos entre regionesy redes. En el reciente examen dedes auto-valuations rv-lent une nette amliorationde la qualit des tudesconomiques et sectorielles(ESW), qui font davantageappel la participation etsont plus axes sur lesbesoins des clients et lesrsultats. Ces instruments doiventencore accorder davantagedimportance au problme de lapauvret, et il y a galementnature amliorer la qualit etlimpact des ESW, en particulierles examens fiduciaires, dans lespays dont la performance estjuge faible.Le choix des instrumentsconditionne les rsultats auniveau des paysLes rsultats des programmes parpays, mesurs daprs plus de cin-quante valuations de laide auxpays (CAE) effectues jusqu pr-sent par lOED, dpendent en par-tie de la qualit de la slectivit auniveau de linstitution, du pays etde linstrument. La Banque faitpasser ses priorits dans les pro-grammes-pays en apportant sonsoutien au Cadre de dveloppe-ment intgr et au Cadre stratgi-que de rduction de la pauvret,en dcidant de laffectation desprts entre les diffrents pays eten rpartissant les ressourcesadministratives entre les Rgionset les Rseaux. Lexamen de lIDArcemment ralis par lOEDrvle que le systme doctroi descrdits en fonction des rsultats avolu en dix ans, et reflte lesnouvelles connaissances acquisesen matire de dveloppement,ainsi que lvolution des prioritsinstitutionnelles. Ces changementsrenforcent le lien entre lefficacitdes politiques et des institutionsESPANOLFRANAISpilot initiatives suggestsscope for their expandeduse in poorly performingcountrieswith a gradualshift to up-scaling as imple-mentation conditionsimprove. Diagnostic studiesare critical in these coun-tries to ensure accurate assess-ment of borrower ownership andprogram risks.Across the spectrum of countryenvironments, recent evaluativeevidence confirms country capac-ity and borrower commitment askey drivers of effectiveness. Theevidence suggests that the Bankcan do more to take these factorsinto accountfor example,through better assessments ofimplementation capacity andgreater reliance on local knowl-edge. Since many borrowers areunfamiliar with the full menu ofBank instruments, the Bank needsto keep them abreast of thechoices offered by the Banksexpanding toolkit so as to securetheir full ownership. Implementing Sector and Thematic StrategiesThe evolution of the Banks cor-porate priorities is reflected in therecent expansion of lending forsocial protection, economic pol-icy, and finance, and in a strongerfocus on public sector manage-ment and institutional reform.Declines in the share of lendingfor rural development and educa-tion, by contrast, seem inconsis-tent with defined priorities andhighlight the need for updatedoperational strategies in these sec-tors. Specific investment loanscontinue to be the predominantlending instrument, and the use ofnew programmatic lending instru-E x e c u t i v e S u m m a r yx i xENGLISHla AIF realizado por elDEO se observa que el sis-tema de asignacin de cr-ditos en funcin delresultado ha evolucionadodurante el pasado deceniode acuerdo con la nuevaconcepcin del desarrolloy las nuevas prioridades institu-cionales. Estos cambios han inten-sificado la relacin entre lapoltica y resultados instituciona-les de los pases y el volumen definanciamiento. En sus decisionesde asignacin para los prestatariosdel Banco Internacional deReconstruccin y Fomento (BIRF),el Banco ha reconocido tambinque los prstamos en el marco depolticas e instituciones nacionalesadecuadas se manifiestan enmejores perspectivas para el des-arrollo. Sigue habiendo posibilida-des de seleccionar y ordenardebidamente los instrumentos enforma ms estratgica en los pro-gramas de los distintos pases,vinculndolos ms expresamentecon los objetivos. Cuando las cir-cunstancias de un pas requierencambios imprevistos en el pro-grama de asistencia, es importanteutilizar actualizaciones oportunasde las estrategias de asistencia alos pases e informes de situacinpara ofrecer un contexto estrat-gico convenido para las activida-des del Banco.Un desafo especial es conseguirmejores resultados cuando las con-diciones normativas e instituciona-les no son las mejores. Del examende las evaluaciones sobre pases yproyectos se desprende que el des-empeo de los instrumentos crediti-cios y no crediticios se ve afectadocuando las polticas e institucionesson dbiles. En estas condicionesadversas, la asistencia tcnica por snationales et les niveaux definancement. Dans ses dci-sions relatives laffectationaux emprunteurs des res-sources de la Banque inter-nationale pour lareconstruction et le dve-loppement (BIRD, ou Ban-que mondiale) la Banque tientcompte aussi du fait que les per-spectives de dveloppement sontmeilleures lorsque les prts sontaccords dans le cadre de bonnespolitiques nationales et dinstitu-tions solides. Il est possible, dansle cadre de chaque programme-pays, de choisir les instruments etdchelonner leur utilisation demanire plus stratgique, en lesliant plus prcisment aux objec-tifs. Lorsque les circonstances par-ticulires dun pays obligent apporter des modifications impr-vues au programme daide, il estimportant dutiliser les actualisa-tions de la Stratgie daide au payset les rapports tablis sur lavance-ment de sa mise en oeuvre pourconvenir du contexte stratgiquedes activits de la Banque.Il est particulirement difficiledamliorer les rsultats dans lespays o le cadre directif et institu-tionnel est dfavorable. Lexamendes valuations nationales et desvaluations des projets rvle quela performance des instrumentsde prt et services hors prt estcompromise lorsque les politiqueset les institutions laissent dsi-rer. Dans ces conditions dfavora-bles, les oprations dassistancetechnique pure donnent alors demeilleurs rsultats que les prts,ce qui tend indiquer que laBanque devrait continuer seconcentrer sur le renforcementdes capacits et des institutionsdans ces pays. Il faut tre particu-ESPANOLFRANAISments is growing. Overall,the sequencing of opera-tions and the synergiesbetween them are particu-larly important when objec-tives include complex insti-tutional reforms. Inparticular, success is morelikely where lending is under-pinned by upstream analyticalwork together with parallel sup-port for piloting and flexibledownstream implementation, andby continuity in policy dialoguewhere relevant. The Banks crosscutting the-matic objectives, such as environ-mental sustainability and gender,can best be achieved throughcomplementary use of both lend-ing and nonlending tools. Of thestrategies recently reviewed byOED, environmental sustainabilityhas exploited the widest range ofinstruments, leveraging effectiveimplementation of safeguards,along with direct environmentallending and substantial relianceon regional and global partner-ships. Successful mainstreamingof thematic initiatives requires thestrategic use of instruments aswell as clearer Regional and Net-work accountability. For the successful implementa-tion of both sector and thematicstrategies, several broad lessonsemerge. Making the right choiceof instrument when undertakinginstitutional reform requires clarityabout the operations objectivesand a good understanding ofcountry and sector conditions. Forexample, programmatic invest-ment lending may be appropriatewhere the emphasis is on a sus-tained medium- to long-term pro-gram of phased policy andinstitution-building reformssola da mejores frutos queotras formas de financia-miento, lo que parece indi-car que el Banco deberacontinuar concentrndoseen el fortalecimiento de lacapacidad de las institucio-nes de esos pases. Debeprestarse especial atencin a lasoperaciones de ajuste, que suelenofrecer menor probabilidad dexito en los pases donde la EPINno ha sido satisfactoria. En relacincon el conjunto de todos los prsta-mos, las recientes evaluacionessobre la asistencia a los pases reve-lan que una serie de intervencionessencillas permite mejores resultadosque iniciativas ms complejas. Losprogramas piloto son tambin uninstrumento valioso para compro-bar entornos inciertos y para el des-arrollo de la capacidad. El xito delas iniciativas piloto indica que haymargen para una mayor utilizacinen los pases con resultados bajos,donde se podra avanzar gradual-mente hacia iniciativas de msenvergadura a medida que mejora-ran las condiciones de aplicacin.Los estudios de diagnstico sonfundamentales en estos pases paragarantizar una evaluacin adecuadade la identificacin de los prestata-rios y de los riesgos de losprogramas.Evaluaciones recientes confir-man que, cualquiera que sea lasituacin nacional, la capacidaddel pas y el compromiso delprestatario son factores decisivosde la eficacia. Los testimoniosparecen indicar que el Bancopuede hacer ms para tener encuenta estos factores por ejem-plo, mejorando las evaluacionesde la capacidad de aplicacin yrecurriendo ms al conocimientolocal. Como muchos prestatarios2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx xENGLISHESPANOLlirement vigilant pour lesoprations dajustement,qui ont en gnral moinsde chances de russir dansles pays dont la perfor-mance est juge faible dansle cadre des EPIN. Les va-luations de laide aux paysralises rcemment montrent quedans toutes les oprations de prt,une srie dinterventions simplesdonne de meilleurs rsultats quedes oprations complexes. Lesprogrammes pilotes sont gale-ment un moyen prcieux dva-luer des situations mal connues etde renforcer les capacits. Le suc-cs des programmes pilotes indi-que aussi quil y a matire lesutiliser davantage dans des paysdont la performance est mdiocre,en largissant progressivementleur champ dapplication mesure que les conditions demise en uvre samliorent. Lestudes de diagnostic sont essen-tielles dans ces pays, afin dva-luer prcisment ladhsion desemprunteurs et les risques duprogramme.Quel que soit le contexte quiprvaut dans les diffrents pays,les valuations rcentes confir-ment que les capacits des payset le degr dengagement delemprunteur sont les facteursdterminants de lefficacit. Toutporte croire que la Banque peutfaire plus pour tenir compte deces facteurs, par exemple en va-luant mieux la capacit de miseen uvre et en faisant davantageappel au savoir local. tant donnque de nombreux emprunteurs neconnaissent pas toute la gammedes instruments de la Banque,celle-ci doit les tenir informs desoptions que leur offre lventailtoujours plus grand dinstrumentsFRANAISespecially when the rangeof stakeholders and institu-tions involved is large, andwhen there are substantialchallenges for consultation,consensus building, pilot-ing, monitoring, and evalu-ation. The Banks system-atic interactions with internationaland national stakeholders haveimproved Bank policies andhelped build a broad consensusaround some of its thematic prior-ities, but consultative processesneed to become more expeditiousand businesslike. At the countrylevel, partnerships have a crucialrole in setting the pace and direc-tion of reform, particularly inlarge countries where selectivity isessential to leveraging the Banksrelatively small share in externalassistance. The transaction costscan be high in some cases, andmore strategic attention is neededto improve cost effectiveness andselectivity in the use of partner-ships. Finally, the effectiveness oflending operations for sector andthematic objectives is stronglyinfluenced by the extent andquality of stakeholderparticipation. Corporate ImplicationsIn the continuing drive toenhance the impact of develop-ment assistance on poverty andgrowth, this Review has high-lighted areas in which the Bankcan further increase its develop-ment effectivenessby makingthe right choices in line withcountry performance and poten-tial, corporate priorities, and com-parative advantage.At the instrument level, a uni-form treatment covering eachinstruments role, design, and les-no estn familiarizados contodos los instrumentos delBanco, ste debe ayudarlesa actualizar sus conoci-mientos sobre las opcionescada vez ms amplias ofre-cidas por el Banco, con elfin de conseguir su totalidentificacin.Aplicacin de estrategiassectoriales y temticasLa evolucin de las prioridadesinstitucionales del Banco se ponede manifiesto en la recienteexpansin del crdito para finesde proteccin social, las polticaseconmicas y el financiamiento, yla mayor atencin a la gestin delsector pblico y la reforma institu-cional. La disminucin de la partede los prstamos para desarrollorural y educacin, por el contra-rio, parece estar en contradiccincon las prioridades establecidas ysubraya la necesidad de estrate-gias operacionales actualizadas enesos sectores. Los prstamosorientados expresamente a lainversin continan siendo el ins-trumento crediticio predominante,y est aumentando la utilizacinde nuevos instrumentos de finan-ciamiento programtico. En trmi-nos generales, la secuencia de lasoperaciones y las sinergias entreellas adquieren especial importan-cia cuando entre los objetivos seincluyen complejas reformas insti-tucionales. En particular, es msprobable que se consiga el xitocuando los prstamos estn sus-tentados en estudios analticosprevios junto con un apoyoparalelo a la experimentacin yuna aplicacin ms flexible enetapas sucesivas y en un conti-nuado dilogo sobre polticas,cuando convenga. leur disposition, afin desassurer de leur pleineadhsion. Mise en uvre de stratgies sectorielleset thmatiquesLvolution des prioritsinstitutionnelles de la Banque setraduit par laccroissement rcentdes prts destins la protectionsociale, des oprations portant surla politique conomique et lesfinances, et par une attentionaccrue la gestion du secteurpublic et aux rformes institution-nelles. Par comparaison, la baissede la proportion des prts enfaveur du dveloppement rural etde lducation semble aller lencontre des priorits dfinies, etsouligne la ncessit dactualiserles stratgies oprationnelles dansces secteurs. Les prts dinvestis-sement spcifiques continuentdtre linstrument prdominant,et les nouveaux instruments deprt programmatiques sont deplus en plus utiliss. Danslensemble, lchelonnement desoprations et les synergies sontparticulirement importants lors-que les objectifs comprennent desrformes institutionnelles com-plexes. En particulier, les chancesde succs sont accrues lorsque lesprts sont tays par des travauxanalytiques en amont, parallle-ment un appui des exprien-ces pilotes et une mise enuvre souple en aval, et undialogue permanent sur les mesu-res prendre, le cas chant.Le meilleur moyen pour laBanque datteindre ses objectifsthmatiques pluridisciplinaires,tels que la prservation longterme de lenvironnement et laparit des sexes, est dutiliser deE x e c u t i v e S u m m a r yx x iENGLISHFRANAISESPANOLsons from past perfor-mance could constructivelyinform instrument choicefor Bank managers andborrowers. The ongoingupdate and conversion ofthe Operational Policy onadjustment lending, to befollowed by a similar process forinvestment lending, will offer agood opportunity to incorporatethese improvements.The Bank is devoting renewedattention to achieving betterresults in countries with poor pol-icy and institutional environments,with a current focus on nonlend-ing support. It has established ahigh-level task force devoted toimproving Bank support to low-income, poorly performing coun-tries. This Review presents evalua-tion findings that are germane tothis difficult task. Recognizing thesignificant performance differen-tial across instruments is animportant first step. In addition,over-complexity of projects whencountry capacity is limited leadsto excessive risks. More conscioustailoring of instruments and part-nerships to country conditionsshould bring precious develop-ment effectiveness gains in diffi-cult operating environments.Experimentation with outcome-based operations and innovativepartnerships with private and vol-untary sector organizations shouldbe encouraged. Lastly, the role ofnonfinancial activities in poorlyperforming countries deservesspecial attention, even when cli-ents are not actively borrowing.Implementation of sector strat-egies within countries requiresclarity about the countrys devel-opment objectives and a goodunderstanding of country and sec-Los objetivos temticostransversales del Banco,como la sostenibilidadambiental y el gnero, pue-den lograrse ms fcil-mente mediante lautilizacin complementariade instrumentos crediticiosy no crediticios. De las estrategiasrecientemente examinadas por elDEO, la sostenibilidad ambientales la que ha utilizado la mayorvariedad de instrumentos, habi-endo multiplicado la aplicacineficaz de salvaguardias junto conel financiamiento ambientaldirecto y una considerable utiliza-cin de asociaciones regionales ymundiales. La incorporacin efi-caz de las iniciativas temticasrequiere la utilizacin estratgicade instrumentos as como unadelimitacin ms clara de respon-sabilidades entre regiones y redes.En lo que se refiere a la aplica-cin eficaz de las estrategias secto-riales y temticas, puedenextraerse varias enseanzas gene-rales. Para elegir adecuadamenteel instrumento cuando seemprende la reforma institucionalse requiere claridad acerca de losobjetivos de la operacin y unacomprensin adecuada de lascondiciones del pas y el sector.Por ejemplo, los prstamos parainversiones programticas puedenresultar indicados cuando lo queinteresa es un programa conti-nuado a mediano o largo plazo dereformas escalonadas normativas yde fortalecimiento institucional sobre todo cuando la variedad departes interesadas e institucioneses considerable, y cuando existendificultades notables para entablarconsultas o lograr el consenso ypara las actividades de experimen-tacin, seguimiento y evaluacin.faon complmentaire lesinstruments de prt et lesservices hors prt. Parmiles stratgies rcemmentexamines par lOED, cestcelle ayant trait au dve-loppement cologiquementviable qui a exploit laplus vaste gamme dinstruments,en tirant parti des mcanismes deprotection efficaces mis en placeet de prts directs en faveur delenvironnement, et en forgeantde nombreux partenariats rgio-naux et mondiaux. Lintgrationrussie des initiatives thmatiquespasse par une utilisation stratgi-que des instruments, et par unedfinition plus claire des respon-sabilits des Rgions et desRseaux. On peut tirer plusieurs leonsgnrales en vue de la bonnemise en uvre de stratgies sec-torielles et thmatiques. Pourchoisir judicieusement linstru-ment adquat lorsquon veut rali-ser des rformes institutionnelles,il faut une bonne connaissancedes objectifs de lopration, ainsique de la situation du pays et dusecteur. Ainsi, des prts program-matiques peuvent tre indiquslorsque laccent est mis sur unprogramme continu moyen/longterme de rformes progressivesdes politiques et des institutions en particulier lorsque les partiesprenantes et les institutionsconcernes sont nombreuses etquil existe des besoins importantsde consultation, de recherche deconsensus, dinterventions pilotes,de suivi et dvaluation. Les con-tacts troits que la Banque entre-tient systmatiquement avec lesparties prenantes au niveau natio-nal et international ont amliorles politiques de la Banque et2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx x i iENGLISHFRANAISESPANOLtor conditions. Adjustmentlending, supported as nec-essary by capacity-buildingassistance, is more effectivewhen national ownershipand consensus on sectorreforms are strong andwhen monitoring and eval-uation arrangements are in place.In sectors where these prerequi-sites are lackingfor example,education and health in somecountriesprogrammatic invest-ment lending may be the pre-ferred approach, supported whererelevant by Learning and Innova-tion Loans (LILs) and targetedcapacity building. Support of theBanks crosscutting, thematicobjectives depends less on theuse of specific lending instru-ments than on genuine main-streaming of these operationalemphases across instruments andon strategic application of safe-guards and partnerships.The findings in this Reviewhave implications for evaluation: Nonfinancial activities continueto grow in importance. Astrong evaluative framework isneeded to ensure their contin-ued effectiveness, building onthe Quality Assurance Groupscurrent assessments of theirquality at entryextendingquality assurance to all servicesof the Knowledge Bank. The quality of Poverty Reduc-tion Strategy Papers (PRSPs) iscritical to the Banks develop-ment effectiveness becausePRSPs provide the frameworkfor a comprehensive approachto ownership, partnership, andresults orientation. PRSPsalready emphasize monitoringprogress on selected indicators;this now should be comple-Las interrelaciones sistem-ticas del Banco con laspartes interesadas interna-cionales y nacionales hanmejorado las polticas deste y ayudado a conseguirun amplio consenso entorno a algunas de las prio-ridades temticas, pero los proce-sos consultivos deben ser msacelerados y sistemticos. Dentrode los pases, las asociacionespueden desempear un papeldecisivo para establecer el ritmo yorientacin de la reforma, sobretodo en los grandes pases dondela selectividad es fundamentalpara multiplicar los efectos de laparte relativamente pequea delBanco en la asistencia externa.Los costos de transaccin puedenser elevados en algunos casos, yse requiere ms atencin estrat-gica para mejorar la eficacia enfuncin de los costos y la selecti-vidad en el uso de asociaciones.Finalmente, la eficacia de las ope-raciones crediticias con respecto alos objetivos sectoriales y temti-cos depende fuertemente delalcance y calidad de la participa-cin de las partes interesadas.Repercusiones institucionalesEn el constante esfuerzo poraumentar los efectos de la asisten-cia para el desarrollo en lapobreza y el crecimiento, en elpresente Examen se han desta-cado las reas en que el Bancopuede aumentar todava su efica-cia en trminos de desarrollo,adoptando las decisiones adecua-das de conformidad con el des-empeo y potencial de un pas,las prioridades institucionales ylas ventajas comparativas.En lo que se refiere a los ins-trumentos, un tratamiento uni-contribu forger un vasteconsensus autour de certai-nes de ses priorits thma-tiques. Le processus deconsultation doit cependantdevenir plus rapide et pro-fessionnel. Au niveau despays, les partenariats ontun rle crucial jouer en fixant lerythme et lorientation des rfor-mes, en particulier dans lesgrands pays o la slectivit estessentielle pour assurer un maxi-mum defficacit la part relative-ment faible de la Banque danslaide extrieure. Les cots detransaction peuvent tre levsdans certains cas, et il faut satta-cher davantage, sur un plan stra-tgique, recourir auxpartenariats de faon plus cono-mique et slective. Enfin, leffica-cit des oprations de prt dupoint de vue des objectifs secto-riels et thmatiques est fortementinfluence par le degr et la qua-lit de la participation des partiesprenantes. Implications pour linstitutionSinscrivant dans la ligne de lac-tion mene pour accrotre limpactde laide au dveloppement sur lapauvret et la croissance, le pr-sent Examen appelle lattentionsur les domaines dans lesquels laBanque peut encore amliorerson efficacit sur le dveloppe-ment, en oprant les choix quisimposent compte tenu de la per-formance et du potentiel du pays,de ses priorits institutionnelles,et de son avantage comparatif.En ce qui concerne les instru-ments, une tude analysant de lamme manire le rle de chaqueinstrument, leur conception et lesleons tirer des rsultats obte-nus par chacun pourrait clairerE x e c u t i v e S u m m a r yx x i i iENGLISHFRANAISESPANOLmented by comparableattention to capacitybuilding for systematicprogram evaluation ofinterventions and theirresults. Country and sector strat-egies can be strength-ened through a more transpar-ent and objective record ofpast performance. A strongerindependent and self-evaluation focus would befacilitated by a closer alignmentof inputs to results, using a log-ical results chain and verifiableperformance indicatorsCountry Assistance Strategiesand Sector Strategy Papers. Finally, since evaluation find-ings testify to the importanceof instrument selection andsequencing for developmenteffectiveness, country, sector,and thematic evaluationsshould address more systemati-cally whether the right instru-ments were used for the devel-opment goals selected, andwhether the complementarityof instruments was exploitedjudiciously. Similarly, project-level evaluation should capturesynergies from complementaryor sequenced instruments (forexample, adjustment loansand stand-alone technicalassistance).forme en que se considerela funcin, diseo y ense-anzas del pasado de cadauno de ellos podra orien-tar a los administradoresdel Banco y a los prestata-rios a tomar decisiones conconocimiento de causa. Elactual proceso de actualizacin yconversin de la Poltica Opera-cional sobre financiamiento parafines de ajuste, que deber irseguida de un proceso semejantecon los prstamos para proyectosde inversin, ofrecer una buenaoportunidad de incorporar esasmejoras.El Banco est dedicando reno-vada atencin a lograr mejoresresultados en los pases conentornos normativos e institucio-nales poco adecuados, y actual-mente insiste en los instrumentosno crediticios. Ha establecido ungrupo de trabajo de alto nivelcuya finalidad es mejorar elapoyo del Banco a los pases deingreso bajo y con resultadospoco satisfactorios. En este Exa-men se presentan las conclusio-nes de la evaluacin relacionadascon esta difcil tarea. El reconoci-miento de la importante diferen-cia de resultados entre losdistintos instrumentos es un pri-mer paso importante. Adems,unos proyectos demasiado com-plejos cuando la capacidad delpas es limitada generan excesivosriesgos. Una adaptacin msconsciente de los instrumentos yasociaciones a la situacin delpas debera permitir valiosasmejoras de la eficacia en trminosde desarrollo en condiciones rea-les difciles. Deberan alentarse laexperimentacin con las opera-ciones basadas en los resultados ylas asociaciones renovadoras conutilement les choix des res-ponsables de la Banque etdes emprunteurs. Lactuali-sation et ladaptation encours de la Politique opra-tionnelle sur lajustementstructurel, ainsi que ladmarche similaire prvuepour la politique relative auxprts dinvestissement, offrent unebonne occasion dintroduire cesamliorations.La Banque semploie plus quejamais obtenir de meilleursrsultats dans les pays o les poli-tiques et les institutions laissent dsirer, en privilgiant une assis-tance hors-prt. Elle a mis enplace un groupe de travail dehaut niveau ayant pour mandatdamliorer la qualit de sonaction en faveur des pays faiblerevenu peu performants. LExa-men de cette anne prsente lesconclusions de lvaluation serapportant cette tche difficile. Ilest important, dans un premiertemps, de constater les cartssignificatifs qui existent entre laperformance des diffrents instru-ments. Par ailleurs, opter pour desprojets trop complexes lorsque lepays a des capacits limitesintroduit des risques excessifs. Unplus grand souci dadaptation desinstruments utiliss et des parte-nariats la situation du paysdevrait conduire des amliora-tions fructueuses de lefficacit dudveloppement dans les contextesoprationnels difficiles. Il fautencourager des oprations dont lefinancement est subordonn lobtention de rsultats, ainsi quedes partenariats novateurs avec lesecteur priv et les organismes dusecteur associatif. Enfin, dans lespays peu performants, il fautconsacrer des efforts particuliers 2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx x i vENGLISHFRANAISESPANOLorganizaciones del sectorprivado y el voluntariado.Finalmente, merece espe-cial atencin la funcin delas actividades no financie-ras en los pases con des-empeo poco satisfactorio,aun cuando los clientes nosean prestatarios activos.La aplicacin de estrategiassectoriales en todos los pasesrequiere claridad acerca de losobjetivos de desarrollo del pas yuna comprensin adecuada de lascondiciones nacionales y sectoria-les. El prstamo para fines deajuste, respaldado en la medidanecesaria con ayuda para el forta-lecimiento de la capacidad, esms eficaz cuando el protago-nismo nacional y el consensosobre las reformas sectoriales sonslidos y cuando se han instau-rado mecanismos de seguimientoy evaluacin. En los sectores enlos que no se dan esos requisitospor ejemplo, la educacin y lasalud, en algunos pases quizssea ms adecuado el prstamopara inversiones programticas,respaldado cuando convenga conprstamos para el aprendizaje y lainnovacin y actividades selecti-vas de fortalecimiento de la capa-cidad. El apoyo a los objetivostransversales y temticos delBanco no depende tanto de la uti-lizacin de instrumentos especfi-cos de financiamiento cuanto deuna autntica incorporacin deesos enfoques operacionales enlos distintos instrumentos y de laaplicacin estratgica de salva-guardias y asociaciones.Las conclusiones de este Exa-men tienen repercusiones para laevaluacin: Las actividades no financierasson cada vez ms importantes. des activits autres quefinancires qui peuventjouer un rle important,mme lorsque les paysclients ne sont pas desemprunteurs actifs.La mise en uvre destratgies sectorielles natio-nales exige que les objectifs dedveloppement du pays soientclairs, et la situation du pays et dusecteur bien connue. Les prts lajustement, appuys en tant quede besoin par une aide au renfor-cement des capacits, sont plusefficaces lorsque le pays est vri-tablement aux commandes et queles rformes sectorielles recueil-lent ladhsion de la population,en mme temps que des disposi-tions de suivi et dvaluation ontt mises en place. Dans les sec-teurs o ces conditions pralablesne sont pas runies, par exemple,ceux de lducation et de la santdans certains pays, il peut treprfrable de recourir des prtsdinvestissement programmati-ques, conforts le cas chant pardes Prts au dveloppement desconnaissances et linnovation(LIL) et par un renforcement cibldes capacits. Lengagement enfaveur des objectifs thmatiquespluridisciplinaires qua adopts laBanque est davantage fonctiondune vritable prise en comptede ces priorits oprationnellesdans le cadre de lutilisation detous les instruments et de lappli-cation stratgique de politiques desauvegarde et de la formation departenariats que de lutilisation detel ou tel instrument de prt.Les conclusions du prsent Exa-men ont des consquences pourlvaluation des oprations : Les activits autres que financi-res continuent de se dvelop-E x e c u t i v e S u m m a r yx x vFRANAISESPANOLUn slido marco deevaluacin es condicinnecesaria para conseguiruna eficacia continuada,basada en evaluacionesactuales del Grupo degaranta de calidadacerca del nivel de calidad inicial que ampla lagaranta de calidad a todos losservicios del Banco deconocimientos. La calidad de los documentosde estrategia de lucha contra lapobreza (DELP) es fundamen-tal para la eficacia del Bancoen trminos de desarrollo, yaque ofrecen el marco para unplanteamiento integrado de laidentificacin con las activida-des, las relaciones de asocia-cin y la orientacin hacia losresultados. Los DELP hacen yahincapi en la supervisin delos progresos con respecto aalgunos indicadores selecciona-dos; ello debera completarseahora con una atencin com-parable al fortalecimiento de lacapacidad para la evaluacinprogramtica y sistemtica delas intervenciones y de susresultados. Las estrategias relativas a lospases y sectores se puedenreforzar mediante un registroms transparente y objetivo delrendimiento anterior. Se puedeconseguir una atencin mayor,independiente y basada en laautoevaluacin mediante unaarmonizacin ms estrechaentre los insumos y los resulta-dos, utilizando una cadena deresultados lgicos e indicado-res de desempeo verificables:estrategias de asistencia a lospases y documentos de estra-tegia sectorial.per. Il faut les doter duncadre dvaluation solidepour prserver leur effi-cacit, en sappuyant surles valuations de leurqualit lentrequeffectue actuellementle Groupe dassurance de la qualit il sagit dlargirlassurance de la qualit tousles services assurs par la Ban-que du Savoir. Lefficacit de laction de laBanque en faveur du dvelop-pement repose sur la qualitdes Documents de Stratgiepour la Rduction de la Pau-vret (DSRP), car ceux-ci per-mettent daborder lappropria-tion par le pays de sesobjectifs de dveloppement,les partenariats et la mise enavant de lobtention de rsul-tats dans une perspective glo-bale. Les DRSP mettent doreset dj laccent sur le suivi decertains indicateurs ; il fautmaintenant porter la mmeattention aux renforcement descapacits en vue dune valua-tion de programme systmati-que des interventions et deleurs rsultats. Dresser un bilan plus objectifet transparent des rsultatsantrieurs contribue renforcerles stratgies nationales et sec-torielles. Une mise en corres-pondance plus troite desapports avec les rsultats, laide dun enchanement logi-que des rsultats et dindica-teurs de performance vrifia-bles (fournis par les Stratgiesdaide-pays et les Documentsde stratgie sectorielle) facilite-rait un renforcement de lauto-valuation et de lvaluationindpendante.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx x v iFRANAISESPANOL Finalmente, como losresultados de la evalua-cin acreditan la impor-tancia de la seleccin yordenacin cronolgicade los instrumentos parala eficacia en trminosde desarrollo, las eva-luaciones nacionales, sectoria-les y temticas deberan consi-derar de forma ms sistemticasi se utilizaron los instrumentosadecuados para los objetivosde desarrollo seleccionados, ysi se aprovech sensatamentela complementariedad de losinstrumentos. De la mismamanera, la evaluacin de losproyectos debera aprovecharlas sinergias entre instrumentoscomplementarios o debida-mente escalonados (por ejem-plo, prstamos para fines deajuste y asistencia tcnicaindependiente). Enfin, puisque lesconclusions de lvalua-tion attestent de limpor-tance du choix desinstruments et de lche-lonnement des interven-tions pour lefficacit dudveloppement, les va-luations nationales, sectorielleset thmatiques devraient tu-dier plus systmatiquement silon a utilis les instrumentsadapts aux objectifs retenus etsi lon a exploit comme ilconvenait la complmentaritdes instruments. De mme,lvaluation dun projet devraitanalyser les synergies qui nais-sent de lutilisation chelonneou complmentaire de diversinstruments (par ex., prts lajustement et assistance tech-nique autonome).E x e c u t i v e S u m m a r yx x v i iFRANAISESPANOLThis Page Intentionally Left Blankx x i xAFR Africa RegionAPL Adaptable Program LoanAPPI Aggregate Project Performance IndexARDE Annual Review of Development EffectivenessAROE Annual Report on Operations EvaluationBP Bank proceduresCAE Country Assistance EvaluationCAS Country Assistance StrategyCDF Comprehensive Development FrameworkCEM Country Economic MemorandumCFA Communaute Financiere AfricaineCFAA Country Financial Accountability AssessmentCODE Committee on Development EffectivenessCPAR Country Procurement Assessment ReportCPFA Country Profile of Financial AccountabilityCPIA Country Policy and Institutional AssessmentDDO Deferred Drawdown OptionDRL Debt Reduction LoanEAP East Asia and Pacific RegionECA Europe and Central Asia RegionERL Emergency Recovery LoanESW Economic and sector workFIL Financial Intermediary LoanFRM Resource Mobilization DepartmentFSAP Financial Sector Assessment ProgramGEF Global Environmental FacilityHNP Health, nutrition, and population IBRD International Bank for Reconstruction and Development (World Bank)ICR Implementation Completion ReportID Institutional developmentIDA International Development AssociationIFC International Finance CorporationIGR Institutional Governance ReviewIMF International Monetary FundLCR Latin America and the Caribbean RegionLIL Learning and Innovation LoanMIGA Multilateral Investment Guarantee AgencyMNA Middle East and North Africa RegionMT Montreal Protocol FundNGO Nongovernmental organizationNSSI National Social Security InstituteOED Operations Evaluation DepartmentOP Operational PolicyOPCS Operations Policy and Country Services PA Poverty AssessmentPBA Performance-based allocationABBREVIATIONS AND ACRONYMS2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s sx x xPER Public Expenditure ReviewPPAR Project Performance Assessment ReportPRSC Poverty Reduction Support CreditPRSI Poverty Reduction Strategy InitiativePRSP Poverty Reduction Strategy PaperPSAL Programmatic Structural Adjustment LoanPSM Public sector managementQAE Quality at entryQAG Quality Assurance GroupRIL Rehabilitation Import LoanSAL Structural Adjustment LoanSAR South Asia RegionSECAL Sector Adjustment LoanSF Special Financing GrantsSIL Specific Investment LoanSIM Sector Investment and Maintenance LoanSNAL Sub-National Adjustment LoanSSAL Special Structural Adjustment LoanSSP Sector Strategy PaperSSR Social and Structural Review SWAP Sectorwide approachTAL Technical Assistance LoanUNDP United Nations Development ProgramWID Women in Development1Making DevelopmentChoicesThe challenge of development, never straightforward, has becomeeven more complex as the number of actors has grown and the de-sire for demonstrable results has intensified. In seeking to implementits poverty reduction mission, the World Bank has to match over 100 clientcountries tackling innumerable development issues with a financial productline ranging from small grants to large loans, a diverse line of nonfinancialproducts, and a growing cast of public, private, and civil society partners. Ifthe Bank is to continue improving its development effectiveness, it will haveto make the right choices about how, when, and with whom to engage (anddisengage). Making the right choices with client countries and other devel-opment partnerswhat is often called selectivityis the theme of this Review. Some of the emerging challenges emanatefrom the development community. Donors areunder rising pressure to satisfy taxpayers that aidachieves results. Donors and recipients alikeare impatient with the high transaction cost ofaid and seek to eliminate overlap and aid bom-bardment, especially in small countries. TheMillennium Development Goalswith their spot-light on measurable outcomesset dauntingbenchmarks. Progress (or lack of progress) to-ward the goals will be plain for all to see. Mean-while, the operating environment has becomesubject to new threatsthe AIDS pandemic, il-legal drugs, terrorism, armed conflicts, and failedstatesthat intensify the underlying structuralcauses of poverty. The attacks of September 11,2001, have exacerbated a global economic slow-down, pushing hundreds of thousands of peo-ple below the poverty line. Concern is rising thatdemands for urgent liquidity and humanitarianassistance to directly affected countries maysideline or supplant longer-term efforts towardsocial and structural development in these andother low-income countries.Against these challenges, the World Bank haslearned from experience, amassed a store ofknowledge, and diversified its toolkit. These as-sets have been enhanced in three significantways. First, research has demonstrated that poli-cies, institutions, and governance are critical todevelopment success. These factors are nowsquarely on the table in the allocation of aid11resources. Second, attention to quality assur-ance and evaluationas noted by OEDs20002001 Annual Report on Operations Eval-uation1has helped lending and nonlendingproducts to deliver improvements in perform-ance. Third, new development paradigms andaid processes (such as the Comprehensive De-velopment Framework [CDF], the Poverty Re-duction Strategy process, and sectorwideapproaches) have been created to address prob-lems collaborativelyat the country, regional,and global levels. OEDs Annual Review of Development Effec-tiveness (ARDE) tracks the Banks imprint on thiscomplex landscape. Over the past few years,evaluations at the project, country, sector, andglobal levelsas synthesized in successiveARDEshave pointed to selectivity as a vitaltool for achieving development results, and asa continuing area of challenge for the Bank. Mostrecently, the 1999 ARDE, Toward a Compre-hensive Development Strategy, showed that theCDF means that a one-size-fits-all mentalityneeds to be replaced by a customization mind-set and that the CDF principle of partnership im-plies that the Bank should let go of areaswhere it does not have a comparative advantage.The ARDE for 2000, From Strategy to Results, ex-plored how the Banks strategies translate intoresults on the ground. It demonstrated that de-velopment interventions are more successfulwhen they are judiciously selected so as tomatch country circumstances and leverage thecomparative advantage of the Bank and its part-ners. Selectivity does not necessarily mean doingless. Instead, it implies making informed choicesabout the right kind and number of activities toincrease development effectiveness. In this yearsARDE, Making Choices, with a special focus oninstrument selection, selectivity takes centerstage as a privileged driver of developmenteffectiveness. The Bank exercises selectivity through thechoices it makes in three dimensions, as mod-eled in figure 1.1: its corporate goals, strategiesfor countries, and specific activities or instru-ments.2 Selectivity at each level contributes to de-velopment effectiveness. 2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2T h r e e D i m e n s i o n s o f S e l e c t i v i t yF i g u r e 1 . 1Corporate CountryInstruments CDF/PRSI processes Corporate goals Sector strategies(SSPs) Corporate constraints(e.g., PBA, exposure, budget) Effective mix andsequence of financialand nonfinancialinstruments Instruments/Policyframework PRSP/CDF support Country AssistanceStrategies (CASs) Partnerships and aidcoordinationTodays corporate goals are more clearlyarticulated and understood than ever before.The overarching missionto reduce povertythrough a focus on resultshas been translatedinto corporate processes (e.g., Sector Strategy Pa-pers, new budget priorities). The Bank has alsotaken on a broad range of institutional initiativesin support of its core goals. At the country level,exposure limits and the administrative budgethave served to define the overall financial en-velopes for what the Bank can take on. Selec-tivity at the corporate level, based on anappreciation of the Banks unique knowledgeand partnership assets, is an important tool tohelp ensure that corporate initiatives enrich, butdo not overcrowd, the institutional agenda.However, the Banks adoption of a countryfocus and of a comprehensive developmentperspective means that operational selectivity isessentially a country-level challenge. Country Assistance Strategies are expectedto reflect the Banks corporate goals, while at thesame time matching the needs and comparativeadvantages of partners in particular settings, asprovided for under the Banks business model.3This involves the resolution of inevitable tensionsbetween client ownership and corporate prior-ities as well as between responsiveness andbudget limits. As the 2000 ARDE demonstrated,it is therefore critical to understand what strate-gies succeed in different country circumstancesand to fully exploit the potential of partnershipsin shaping and delineating the Banks own role. Even with the right strategies in place, resultson the ground depend on the effectiveness ofthe instruments available within the Bankstoolkit. Over the past decade, the Bank has de-voted effort and creativity to adapting its finan-cial and nonfinancial services to meet a widevariety of client needs and to implement cor-porate initiatives. New instruments have beencreated, others phased out. What has stimu-lated this attention to instruments? First, manyBank clients have gained access to a spectrumof alternative financing sourcessome involv-ing transaction costs lower than the Banks.Second, for poor and highly aid dependent bor-rowers, there is a continuing drive to enhancethe impact of development assistance on povertyand growth. Third, most borrowers can accessglobal knowledgewhether through the Inter-net or through local knowledge networksmore readily than in the past, thereby raising thebar for the value added of the Banks knowledgeproducts. Finally, it has become increasinglyclear (see 2000 ARDE, Aid and Reform in Africa)that development outcomes depend on countrypolicies and capacities, making a correct un-derstanding of the country context critical to thechoice of instruments and their development ef-fectiveness. Accordingly, this years ARDE focuseson effective choice of activities and instrumentsto implement poverty reduction strategies. This Review describes the Banks toolkit andits evolution (Chapter 2), tracks the performanceof Bank instruments and operations (Chapter 3),examines the use of instruments at the countrylevel (Chapter 4), provides a sector/thematicperspective (Chapter 5), and summarizes thefindings and implications of the analysis (Chap-ter 6). In addition to sources outside OED andthe World Bank, it draws upon OED evaluationfindings from: Project performance assessment reports (for-merly audit reports) and evaluation sum-maries for 348 projects evaluated since the lastReview. In addition, over 5,000 previouslyevaluated projects in OEDs database allowassessment of long-term trends. Country Assistance Evaluations of 50 bor-rowing countries, especially the 15 completedmost recently. Studies of sectors and thematic areas, a studyof the IDA10 through 12 replenishmentperiods, and the Annual Report on OperationsEvaluation.M a k i n g D e v e l o p m e n t C h o i c e s3This Page Intentionally Left Blank5Instruments ofBank AssistanceDevelopment effectiveness depends on selecting the right instrumentsand deploying them in appropriate sequence and combination, inlight of country and sector characteristics. This chapter sets the stagefor later discussions of performance, by providing an overview of theBanks instrument toolkit and analyzing recent instrument choices. Innovationshave been introduced over the 1990s, in both the financial and nonfinancialareas. To leverage these improvements, effective instrument selection in di-verse environments requires clear operational guidance in a comprehensiveand consistent framework.The Banks development effectiveness de-pends on the judicious choice of instrumentsfrom its diversified toolkit. At the broadest level,Bank assistance is either financial or nonfinan-cial (figure 2.1). Traditionally the Bank has fo-cused on the former. More recently attention hasbeen paid to the role of nonfinancial assistance,especially in poorly performing countries.Greater differentiation is being introduced be-tween diagnostic services and client-driven ad-visory services, and within diagnostic servicesclearer standards and structure of core productsare emerging.Financial ServicesInvestment, Adjustment, and Other InstrumentsThe use of financial services is prescribed in theBanks Articles of Agreement. These Articles arefacilitative rather than restrictive, promoting theuse of funds for productive purposes while high-lighting that Bank funds must be used only forthe purposes intended. They are sufficientlybroad and flexible to permit a wide range of lend-ing instruments to have evolved over time as cir-cumstances have changed and understandingof the development process has improved. Thisflexibility allowed the introduction of adjustmentlending in the early 1980s, and the subsequentclassification of Bank lending into two basictypes of instruments: investment and adjustment.Specific instruments reflect variations withinthese broad groups. The specific policies andguidance governing the instruments used andtheir design are left to the Board and manage-ment to elaborate as needed. Over the past 1012 years, the Banks menuof financial instruments has changed in response22to an increasingly complex economic and socialenvironment. As figure 2.2 shows, investmentlending remains the Banks lending workhorse,and, with the exception of the period of the EastAsia crisis, has always constituted the largestshare of Bank lending activities. Innovations in specific investment lend-ing instruments. Within the broad investmentcategory of investment lending, several specificinstruments have been developed. Sector In-vestment and Maintenance Loans (SIMs) madeup about a quarter of investment lending untilthe 1990s. During the 1990s their share droppedto less than 10 percent of total lending as theywere partially replaced by new adaptable lend-ing such as Adaptable Program Loans (APLs) andLearning and Innovation Loans (LILs), intro-duced by the Bank in 1998. The main motiva-tion for introducing APLs and LILs was toincrease the Banks flexibility, to reduce risks toborrowers and the Bank when operating inareas of uncertainty, and to facilitate exit fromfloundering operations. The APL supports long-term development programs with a phased ap-proach, which is more easily adapted to changesin the country environment. This allows subse-quent phases to leverage lessons learned andbuild necessary capacity or consensus in earlyphases. The purpose of LILs is to pilot innova-tions, build consensus, and learn in order to scale2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s6T h e I n s t r u m e n t T o o l k i tF i g u r e 2 . 1FinancialServicesNonfinancialServicesAdvisoryDiagnosticOtherInvestmentSpecificInstruments: Sector Studies,Special StudiesPoverty AssessmentsCountry Economic Mem.Public Expend. ReviewsCtry Finan. Acct. AssmtsCtry Procure. AssmtsOther Diagnostic ReportsSAL, PSAL,PRSC,SECAL,RIL, SNAL,SSAL, DDO,DRLSIL, SIM,APL, LIL,TAL, FIL, ERLGuarantees,GrantsAdjustmentNote: Abbreviations and acronyms for specific investment and adjustment lending instruments are defined in Annex A.Other Inv $1 billionAPL $4 billionERL $6 billionFIL $8 billionSIM $18 billionSECAL $28 billionSAL $40 billionSIL $143 billionTAL $5 billionOther Adj$7 billionL e n d i n g I n s t r u m e n tC h o i c e ( F Y 9 0 0 1 )F i g u r e 2 . 2Note: Abbreviations and acronyms are defined in Annex A.up and apply promising models to other re-gions in the country. These new instruments nowaccount for a quarter of active investment op-erations, or 10 percent by dollar volume, withannual commitments of about US$1 billion forAPLs and US$100 million for LILs. The volumefor LILs is lower than expected. An effort isunder way to reduce processing requirements,preparation time, and costs for LILs, which havebeen greater than originally anticipated.Experience on completed APLs and LILs islimited, with only a few APLs having completedtheir first phase, including projects in Indiapower, Philippines urban water, and Tunisiatransport. One of these, the Haryana PowerProject in India, was discontinued after it failedto meet trigger requirements, an illustration ofthe automatic exit feature of APLs. Innovations in specific adjustment lend-ing instruments. The broad instrument cate-gory of adjustment lending has been dominatedby use of the Structural Adjustment Loan (SAL),covering more than half of the adjustment lend-ing projects approved between FY98 and FY00.SALs and Sector Adjustment Loans (SECALs)constitute the Banks traditional adjustmentlending instrument set, to which, in the early1990s, the Bank added two new instrumentsthe Debt Reduction Loan (DRL) and the Reha-bilitation Import Loan (RIL). The use of theseinstruments has been modest given the recentintroduction of new programmatic lending in-strumentsthe Special Structural AdjustmentLoan (SSAL), the Sub-National Adjustment Loan(SNAL), the Programmatic Structural AdjustmentLoan (PSAL) and its specific derivative for IDAcountries, the Poverty Reduction Support Credit(PRSC). The focus of the PSAL is on medium-term policy and institution building, while thePRSC supports the implementation of povertyreduction strategies. Guarantees. The Banks 14 approved guar-antees represent roughly US$2 billion in con-tingent liabilities. Ten of the 14 are accompaniedby Bank loans, and most of these are targetedto the power sector. A review of the guaranteeprogram prepared for the Board in December2000 suggested that the Bank guarantees havenot only helped catalyze private finance to im-portant transactions, but also improved the fi-nancial viability of transactions, and helpedclient countries to access or re-enter financialmarkets, establish a track record, and build mar-ket confidence. Another review of the IDA par-tial risk guarantees program prepared for FRMin the context of the IDA13 replenishment in Oc-tober 2001 points to similar conclusions. Toclarify the relation of the IDA/IBRD guaranteewith IFC and MIGA instruments, the December2000 Board report included a matrix that set outthe role of the various World Bank Group guar-antee instruments. The Board called for greatercollaboration among the Bank, IFC, and MIGAto ensure that guarantees and other Bank ser-vices are used in a synergistic fashion. Recent Lending ChoicesInstrument choice is influenced by many factors,including the riskiness of the countrys imple-menting environment and internal considera-tions such as corporate priorities and budgetimplications. The following section examineslending instrument choice along these relateddimensions.As part of its risk management procedures, theBank takes into account the countrys risk envi-ronment in deciding whether and how much tolend and with what instrument. Lending alloca-tions to countries with low Country Policy andInstitutional Assessment (CPIA) ratings have tobalance the possible rewards against the risks.Where the policy environment is weak, non-lending services are often a more appropriate op-tion. The Bank is also proactive in seeking toachieve policy reform through lending. Amongthe 44 IBRD borrowers rated by Standard andPoors, it is those with the lowest sovereign rat-ings that received the majority of recent Bankcommitments. For higher-rated countries, whichhave better access to international capital mar-kets, Bank commitments have been limited. Asfigure 2.3 shows, the amount of adjustment lend-ing in low CPIA performers has been larger thanfor investment, even though outcomes for ad-justment lending interventions in low CPIA coun-tries have been particularly weak (Chapter 3).Costs differ by specific lending instruments. Fig-ure 2.4 provides a breakdown of the total costsI n s t r u m e n t s o f B a n k A s s i s t a n c e7(preparation and supervision) by lending in-struments per projects approved as well as perdollar amount disbursed. These cost differencesare highly relevant to country managers facingchoices among instruments and relatively rigidbudget ceilings. Per project, SILs, SIMs, andSECALs have the highest lending costs. LILshave the highest costs per dollar disbursed,whereas SALs, given their large size, had the low-est costs. As mentioned earlier, the LILs higher-than-expected costs and preparation time haveresulted in a more modest roll-out of LILs thanexpected. A noteworthy application of adjustment lend-ing has been in response to major financial cri-sis. During the 1990s, developing countries inAsia and Latin America experienced economicand financial crises that halted their economicgrowth and severely threatened the gains fromprevious reforms. The Bank responded to IBRDborrowers demands for quick disbursing lend-ing with adjustment operations. Overall theBank approved 15 crisis lending operations.1All were to IBRD borrowers and involved largeresource transfers. Commitments, totaling US$17billion, were fully disbursed. The objectives ofthese operations were to supply balance of pay-ments support, weather short-term risks,2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s8L e n d i n g R i s k s , b y C P I A a n d C r e d i tR a t i n g s ( F Y 9 6 0 1 A p p r o v a l s )F i g u r e 2 . 3010203040506070Commit. US$ Billion010203040506070Commit. US$ BillionLow(41 Countries)Medium(69 Countries)High(24 Countries)BBB+ or Better(7 Countries)BB+ to BBB(14 Countries)BB or Lower (24 Countries)AdjustmentInvestment134 IBRD & IDA Countries 45 IBRD CountriesStandard & Poors Sovereign RatingsCPIAAdjustmentInvestmentC o s t sD i f f e r b yL e n d i n gI n s t r u m e n t ( F Y 9 6 0 1E x i t s )F i g u r e 2 . 401002003004005006007008009000 10 20 30 40 50 60 70 80 90Total Costs by Project ($000s)Total Costs Per US$1 MillionLent ($000s)SIMLILERLTALSALSECAL APLSILstrengthen social safety nets, and induce struc-tural improvements in the policy framework. Nonfinancial ServicesFormal nonfinancial services encompass knowl-edge services of two main types: diagnostic andadvisory, as categorized in figure 2.1.2 The di-agnostic reports refer to the standardized non-financial products that have their own toolkits andquality assurance arrangements, and are dividedat the specific instrument level into five core di-agnostic products which underpin the analysisfor the CAS and the Banks overall policy dia-logue, and other diagnostic reports that addresssector-specific or thematic issues of high prior-ity and provide upstream analysis to support fu-ture lending opportunities.3 The advisory reportsare customized to provide advice on special top-ics, and include sector and special studies. ESW Reflecting Corporate PrioritiesThrough the 1980s, most economic and sectorwork (ESW) was advisory in nature; the type andnumber of ESW products were driven by theneed to support the lending program and by bor-rowers particular needs for technical advice.Beginning in the early 1990s, economic andsector work became increasingly influenced bycorporate mandates and priorities. As table 2.1shows, there has been a surge in the productionof poverty and environmental assessments in di-rect response to the priority given by the IDAdonors, the Board, and Management to thebroadening of the Banks development agenda.The number of PERs tripled between the 1980sand 1990s, reflecting concerns about growingemphasis on reviewing a countrys overall pub-lic expenditure allocations and expenditure man-agement, rather than tracking specific loanproceeds. Gender assessments have likewiseshown a large jump following the issuance ofoperational directives and increased attention byBank stakeholders to issues of gender inequal-ity. During FY9801, there has been a significantchange in the allocation of ESW expenditures;the share of core diagnostics reports has in-creased, while the shares of other diagnostic andadvisory reports have declined.Poverty Reduction Strategy Papers, althoughproduced by the borrower authorities, typicallyrequire extensive Bank support. The source-books and other materials that underpin PRSPsI n s t r u m e n t s o f B a n k A s s i s t a n c e9Fiscal year FY8089 FY9099 FY00 FY01 (est)Core Diagnostics:Poverty Assessments 1 112 11 14Public Expenditure Reviews 29 89 14 14Country Profile of Financial Accountability 40 4 0Country Financial Accountability Assessments 15 1 11Country Procurement Assessment Reports 15 15Other Diagnostics:Environmental Assessments 10 87 3 3Private Sector Assessments 40 55 1 2Gender Assessments 1 30 5 8Financial Sector Assessment Programs 12 20Social and Structural Reviews 4 2I-PRSP/PRSP 9 32Note: Gender assessments for FY9000 exclude regional gender action plans and country gender profiles and are likely to be undercounts, as many reports neverreached the final stage.Source: OPCS. E v o l u t i o n o f D i a g n o s t i c E S WT a b l e 2 . 1have helped to provide a poverty-focused frame-work for much of the ESW program. Table 2.1also reveals the quick ramp-up of CFAAs andCPARs, two core fiduciary diagnostic productswhich, along with an integrative, crosscutting as-sessment of a countrys social, structural, and keysectoral development policies, are expected tounderpin the development of Poverty ReductionSupport Credits. Finally, the FSAP is an ESWproduct stimulated by the corporate response tothe East Asia crisis. It is a collaborative Bank/ IMFinstrument that attempts to identify vulnerablecountries and to recommend remedial meas-ures to reduce their vulnerability to financialcrises. Initially funded with exceptional re-sources, its continuation and coverage will de-pend on the future availability of resources fromBank budgets and trust funds.Clear Instrument Guidance Is KeyFor financial instruments, the policy framework,as it has evolved, gives considerable discretionto Bank managers in the selection of instru-ments. Optimal selection of instruments requiresclear guidance on the appropriateness of eachinstrument to specific objectives and conditions,as well as consistent coverage of all instruments(investment and adjustment) in an explicit frame-work. Functionally driven choices require clearoperational guidance.The current guidance on instruments could bestrengthened along three main dimensions. First,the distinctions among the different instrumentscurrently linked mainly to different internal pro-cessing proceduresshould instead be anchoredin their differing development objectives. In-vestment and adjustment instruments should becovered within a common policy framework, sothat their costs, synergies, and complementaritiesare clear. Instrument choices should be driven bydevelopment considerations, without regard to thevarying applicability of safeguards or other pro-cedures. Second, each instrument should be as-sociated with the sector and country conditionsto which it has proven suitable or successful. Itis important for staff and borrowers to receiveguidance which draws on available experienceof the effectiveness of instrument variations in dif-ferent country circumstances. Third, the rules,procedures, and guidance concerning different in-struments, which are now set out in a non-uniform variety of OPs, BPs, memos, andwebsites, should be consolidated and clarified.For nonfinancial instruments, budget andstrategic considerations are important. In an in-dividual country program, the core diagnosticproducts such as poverty assessments, CEMs, andPERs produce important baselines for monitor-ing progress on the outcomes of assistance onbasic objectives. At the corporate level, the over-2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s1 0# of countries CPAR CFAA PERBy Lending Volume:Largest Adjustment Borrowers 10 30% 30% 50%Largest IBRD Borrowers 10 60% 40% 80%Largest IDA Borrowers 10 50% 50% 80%By Policy Environment:Low CPIA Countries 24 42% 4% 42%Medium CPIA Countries 60 57% 28% 52%High CPIA Countries 13 38% 31% 31%Note: It is generally recommended to update fiduciary diagnostics every five years. Largest adjustment borrowers are based on total net commitment value ofadjustment operation approvals in FY0001. Largest IDA/IBRD borrowers are based on net commitments values in the FY01 ARPP. CPIA breakdown excludessmall countries or borrowers under suspension or those with net commitments below US$50 million as of June 30, 2001.Source: OPCS. C o u n t r y C o v e r a g e o f F i d u c i a r yD i a g n o s t i c E S W ( F Y 9 7 0 1 )T a b l e 2 . 2all coverage of core diagnostics has increased inFY01. Attention is needed to ensure these di-agnostics cover country programs involving thegreatest fiduciary risk and the largest vulnerabilityto governance dysfunctions. As shown in table2.2, in terms of volume of lending, fewer thanhalf of the 10 largest IDA and IBRD borrowershave been subject to a CFAA. Only 4 percent ofthe Banks active borrowers with low CPIAscores have had a CFAA, compared with 2831percent by higher-scoring borrowers. In theAfrica Region, only 16 percent of borrowershave a current CFAA.4 Among large IBRD bor-rowers, Brazil has the least coverage amongcore ESW productsit does not have an updatedPER, CPAR, or CFAA. All such gaps for active bor-rowers are expected to be addressed in FY0204.While increased production of CPARs andCFAAs is underway and considerable effort hasbeen devoted to establishing the standards andscope of diagnostic instruments, it is paradoxi-cal that those completed to date have not beenfocused on poor policy environments. Of the 22CFAAs carried out between FY98 and FY01,only one was conducted in a poor policy envi-ronment (borrowers with a low CPIA score).5 Ofthe 49 CPARs conducted by end FY01, 10 (20 per-cent) were undertaken in borrowers with lowCPIA scores; the corresponding figure for the 45PERs is 10 (22 percent). With the figures in table2.2, this suggests that fiduciary diagnostics mayneed to be better aimed at countries where thepotential vulnerability to development risks is thegreatest. I n s t r u m e n t s o f B a n k A s s i s t a n c e1 1This Page Intentionally Left Blank1 3Performance at theInstrument Level The Strategic Compact targets of 75 percent satisfactory outcomes forlending and 85 percent satisfactory quality for nonlending have beenmet. The latest project evaluation data confirm a significant improve-ment in the Banks lending performance, especially for FY00 exiting projects.The upward trend continues into FY01. The first half of FY01 exiting proj-ects evaluated by OED have 82 percent satisfactory outcome ratings. Thereare solid improvements in sustainability and institutional development impactas well. Recent evaluations also show strong performance from the AfricaRegion following its internal portfolio improvement drive. Self-evaluation resultsfor nonlending instruments show a broad improvement in quality, with par-ticular gains in the likely impact of economic and sector work as it becomesmore participatory, client-oriented, and result-focused. There remains roomfor improvement in the poverty focus of these instruments, as well as in thequality of fiduciary reviews and of ESW in poorly performing countries.Bank lending has a mature evaluation sys-tem, grounded in self-evaluation and inde-pendent desk review by OED of everyIDA/IBRD operation, as well as OED fieldperformance assessments for one in four oper-ations. The evaluation results produced bythis system provide the basis for the firstsection on lending performance. The sectionon nonlending performance relies on self-evaluation evidence as well as selected OEDstudies on public expenditure reviews andpoverty assessments.Performance of Lending AssistanceSince last years ARDE, OED has evaluated 348closed projects covering US$33.8 billion in nom-inal commitments and US$29.5 billion in nom-inal disbursements. The evaluations, most ofwhich cover projects exiting the Banks portfolioduring FY00 and FY01, comprise 279 ICR reviewsof completed projects and performance assess-ments of 69 completed projects. This evaluatedcohort includes the first closed APL as well asthe second-ever-evaluated LIL. The majority ofthe recently evaluated adjustment operations33were approved in the past three years, as shownin figure 3.1, while most of the investment proj-ects were approved during the mid-1990s. This section is divided into two parts. Thefirst part describes the trends of the Banksmain performance dimensionsoutcome, sus-tainability, and institutional development (ID)impactfor the projects that exited duringthe FY9601 period. The second, focusing onDeterminants of Success, discusses the fac-tors contributing to the lending performancetrends.Performance TrendsOutcome TrendsThe Strategic Compact target of 75 percent sat-isfactory outcomes has been met. Of the projectsthat exited the portfolio in FY00, 76 percenthad satisfactory outcomes, meeting the StrategicCompact target outcome rate of 75 percent. Asfigure 3.2 shows, the upward trend continues intoFY01.1 Of the first half of FY01 exits evaluatedby OED, 82 percent had satisfactory outcomes.Results for the entire FY01 exiting portfolio willbe available by the end of FY02 and are expectedto achieve roughly 80 percent satisfactoryoutcomes.Since achieving a peak of 84 percent satis-factory for FY99 exits, satisfactory outcomesweighted by disbursements have been lower forthe FY0001 period. Strong performance fromthe second half of FY00 exits, as forecast in lastyears Review, brought satisfactory outcomesfor the FY00 exiting portfolio to 80 percent. Sat-isfactory outcomes for the preliminary FY01disbursement-weighted portfolio continue tofall below the project-weighted portfolio, at 78percent satisfactory, as shown in figure 3.2. The overall results weighted by disburse-ments confirm the portfolios sensitivity to theeffects of jumbo loans. The outcome of a US$1billion operation can make a difference of up to4 percentage points in the disbursementweighted aggregate outcome. The unsatisfactoryperformance of two Russia adjustment opera-tions, as assessed by the Region and validatedby OEDs independent review, has depressed theaggregates not only for ECA but also for adjust-ment lending and for the economic policy andsocial protection sectors.2 This volatility is likelyto continue given the number of large adjustment2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s1 4A p p r o v a l Y e a r s o f R e c e n t l yE v a l u a t e d P r o j e c t s F i g u r e 3 . 1Pre-1990 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000010203040506070Number of Projects EvaluatedApproval FYInvestmentAdjustmentloans in the active portfolio. As risk managementin the Bank receives more attention, it may beuseful to consider adding special quality assur-ance arrangements for very large projects. Institutional Development ImpactOne of every two projects evaluated during theFY0001 exit cohort had substantial or better IDimpact. Improvement in the ability of the bor-rower to make more efficient, equitable, and sus-tainable use of its resources is fundamental tomove further toward full borrower ownership oflending operations. As figure 3.3 shows, 52 per-cent of projects that exited the portfolio duringthe FY0001 period had substantial or better insti-tutional development ratings, an increase of 15percentage points over the FY9699 period aver-age. The results for the first half of FY01 exitsare the highest on record, weighted by projectsand by disbursements.Analysis of institutional development objec-tives by lending instrument shows that adjust-ment operations in general have given moreemphasis to reform of policies and the institu-tional environment, whereas investment opera-tions have tended to address skills formation andorganizational capacity. All lending instrumentsaddressed both aspects of institutional devel-opmentstrengthening organizations andreforming the institutional environment. Thissuggests that appropriate use of instruments isbeing made from an institutional developmentperspective. Sustainability Sustainability continues to improve for theFY0001 exit cohort. Seventy-one percent ofprojects exiting during the FY0001 period wererated to have likely or highly likely resilience tofuture risks, using a recently introduced 4-pointrating scale.3 This represents continuedimprovement since the mid-1990s, though thehigher risks of the operating environment andthe partial sample of FY01 exits raise concernsabout the durability of those gains, as figure 3.4shows. Similar results are found when weightedby disbursements.Aggregate Project Performance Index (APPI)The APPI combines ratings of outcome, sus-tainability, and ID impact into a bottom linefor development effectiveness of lending.P e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l1 5T r e n d s i n O u t c o m e sF i g u r e 3 . 21990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001*2030405060708090Percent SatisfactoryExit FYBy ProjectWeighted by Disbursements*Preliminary; see chapter 3 endnote 1.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s1 6T r e n d s i n I n s t i t u t i o n a lD e v e l o p m e n t I m p a c tF i g u r e 3 . 31990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001*2030405060708090Percent Substantial or BetterExit FYBy ProjectWeighted by Disbursements*Preliminary; see chapter 3 endnote 1.T r e n d s i n S u s t a i n a b i l i t yF i g u r e 3 . 41990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001*2030405060708090Percent Likely or BetterExit FY*Preliminary; see chapter 3 endnote 1.By Project 3-point scaleBy Project 4-point scaleIntroduction of the 4-point scale (June 2000).Weighted by projects, the APPI index is at a 12-year high for the projects exiting during theFY0001 period, with an average of 7.1. Thisincrease largely reflects the improvement in sat-isfactory outcomes. Using OEDs 6-point out-come scale, there has been a modest decreasein the share of moderately satisfactory projectsand a subsequent increase in the number of proj-ects rated fully or highly satisfactory. For theFY0001 exits, the APPI index weighted by dis-bursements also stands at a 12-year high. Performance by Region Africa narrows the performance gap. The AfricaRegion continues its record of improvementwith 71 percent satisfactory outcomes for the pre-liminary FY01 exits. This reduces the Regionssatisfactory outcomes gap with the Banks otherRegions, as figure 3.5 shows. For the FY0001 exiting portfolio, MNA andSAR are the top performers, followed by EAP,LCR, and ECA. Comparing the latest evaluatedcohort, FY0001, with the active regional port-folio, AFR shows the largest improvement, 19percentage points, between the latest satisfactoryoutcome ratings for the evaluated cohort and thenot at risk ratings for the active cohort, as fig-ure 3.6 shows. MNA is the only region for whichdeterioration in future outcomes is suggested bythe Quality Assurance Group (QAG) ratings.This is mostly due to a difficult operating envi-ronment in Lebanon, with a US$729 millionlending portfolio, the fourth largest in the region,65 percent of which is at risk.Performance by SectorTransportation, education, and public sectormanagement are the best-performing sectorsfor the FY0001 exiting portfolio, as figure 3.7below shows, while the energy sector continuesto lag. The high ratings for projects in the trans-portation sector are partially due to an increasedfocus on the independence of borrowers finan-cial resources, together with improved budgetallocations for maintenance works. A similarsectoral performance pattern is obtained whenweighted by disbursements.Using the QAG projects at risk ratings as aleading indicator of future satisfactory outcomes,the results for the active portfolio show a con-siderable improvement in the agriculture andenergy sectors. The closing and restructuring ofP e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l1 7T r e n d s i n O u t c o m e s f o r A f r i c aa n d O t h e r R e g i o n sF i g u r e 3 . 51996 1997 1998 1999 2000 2001*2030405060708090Percent Satisfactory by ProjectExit FYOther RegionsAfrica Region*Preliminary; see chapter 3 endnote 1.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s1 8T r e n d s i n O u t c o m e s b y R e g i o nF i g u r e 3 . 6AFR EAP ECA LCR MNA SAR Bankwide Average020406080100Percent Satisfactory by ProjectFY9699 exits FY0001* exits Active portfolioNot at Risk*Preliminary; see chapter 3 endnote 1.T r e n d s i n O u t c o m e s b y S e c t o rF i g u r e 3 . 7Trans. Educ. PSM HNP Agr. Energy Bankwide Average020406080100Percent Satisfactory by ProjectFY9699 exits FY0001* exits Active portfolioNot at Risk*Preliminary; see chapter 3 endnote 1.poorly performing energy projects largelyexplains the expected improvement in this sec-tor. For the agriculture sector, increased atten-tion to community participation together withincreased knowledge sharing through the Banksinternal thematic groups underlies the forecastimprovement.Performance by Lending InstrumentAdjustment lending outcomes slightly declined forthe FY0001 exit period. Satisfactory outcomesof adjustment operations fell from 82 percent forFY9699 exits to 79 percent for FY0001 whenweighted by projects, and from 87 percent to 70percent when weighted by disbursements. Forthe adjustment operations that exited the port-folio during FY0001, outcome performanceweighted by disbursements is mixed and volatile:satisfactory outcomes for FY00 reached 93 per-cent but the preliminary FY01 exits fell belowthe 50 percent mark, as figure 3.8 shows. Thepreliminary results for FY01 exits are based ononly 13 adjustment operations, hence not robustenough to draw any inference for this exit yearalone. Additionally, as mentioned above, theunsatisfactory outcome performance of the twolarge Russia adjustment operations (one SALand one SECAL) severely depresses the aggre-gates for the partial FY01 adjustment cohort.The above figures are based on OEDs cur-rent evaluation methodology for adjustmentlending. Last years ARDE pointed to the diffi-culty of providing reliable ratings in the absenceof monitorable goals set at the approval stage.Since many operations do not yet specify veri-fiable performance indicators, ratings for theseprojects can be based only on specified inter-mediate objectives. In addition, the timing ofevaluations frequently makes it difficult to useprojected impacts or even genuine outcomes forrating purposes. Hence, until adjustment oper-ations are designed so as to be evaluable, e.g.,through the use of a logical framework, evalu-ation ratings for such lending will continue tobe geared more to compliance with condition-ality and achievement of intermediate outcomesthan to final outcomes and impacts. OED seeksto work in tandem with management on over-coming these limitations, for example throughthe upcoming revision to the Banks policy onadjustment lending. These reservations do notdetract from the substantial progress which hasP e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l1 9T r e n d s i n O u t c o m e s o f A d j u s t m e n tO p e r a t i o n sF i g u r e 3 . 81996 1997 1998 1999 2000 2001*405060708090100Percent SatisfactoryAll Adjustment LendingBy ProjectWeighted by Disbursements1996 1997 1998 1999 2000 2001*405060708090100Percent SatisfactorySECALsSALsSelected Specific Adjustment Instruments(Weighted by Disbursements)*Preliminary; see chapter 3 endnote 1.been made in enhancing the overall efficacy ofadjustment lending. Investment project outcomes show improve-ment for the FY0001 exit period. After a constantoutcome performance throughout the late 1990s,satisfactory outcomes for investment projectshave shown some recent improvements. Satis-factory outcomes for investment projects rosefrom 69 percent for FY9699 exits to 78 percentsatisfactory for FY0001 exits, as shown in figure2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2 0A separate review of performance assessments wasconducted by OED to consider the achievement effec-tiveness of individual project objectives. While OEDevaluation methodology focuses on overall outcome atthe project level, this exercise considered the multipledevelopment objectives within projects, as summa-rized in the figures shown. The sample included 45performance assessments performed by OED duringthe last four exit fiscal years. Based on the assess-ment of performance on project objectives in the PARs,a rating of 1 (Highly Unsatisfactory) to 6 (Highly Satis-factory) was assigned for each objective in each sam-ple project. The review yields multiple ratings forindividual projects, which were then categorized bytime frame into intermediate and long-term goals.Intermediate ObjectivesIntermediate objectives were classified into five cat-egories: Financial Intermediation, Skills and Organi-zational Capacity, Policy and Institutional Environment,Short-Term Emergency, and Physical Outputs. Analysisof the sample suggests that the Banks developmentimpact was most successful in achieving physical out-puts, which averaged 4.3, followed by short-term emer-gency objectives. Performance for the two institutionaldevelopment objectives was roughly even, while per-formance in financial intermediation was weakest,averaging only 2.5. Longer-Term GoalsThe pattern for longer-term goals mirrors that for inter-mediate objectives, with more success on physicaloutputs than institutional change. Long-term goals wereclassified into five categories: Public Sector Perfor-mance, Human and Social Development, Private SectorDevelopment, Natural Environment, and Physical Infrastructure.The Bank was most successful in achieving goals in physicalinfrastructure, with an average score of 4.8, and natural envi-ronment, with an average score of 4.5. Human and socialdevelopment and private sector development scored roughlyeven. The Bank was least successful in impacting public sec-tor performance, with an average of 3.6, in contrast with theimproved record of freestanding PSM projects.W h i c h O b j e c t i v e s H a v e B a n k P r o j e c t sB e e n M o s t E f f e c t i v e i n A c h i e v i n g ?B o x 3 . 1Average Performance on Intermediate ObjectivesAverage Performance on Longer-Term GoalsFinancial IntermediationSkills and Organizational CapacityPolicy and Institutional EnvironmentShort-Term EmergencyPhysical Outputs0 1 2 3 4 5 6Objective/GoalAverage Outcome Rating2. Sector PerformanceHuman and Social DevelopmentPrivate Sector DevelopmentNatural EnvironmentPhysical Infrastructure0 1 2 3 4 5 6Objective/GoalAverage Outcome Rating3. Regions achieving the largest improvementwere MNA (23 percentage points), SAR (22 per-centage points), and AFR (9 percentage points). Ninety percent of the adjustment operationsevaluated by OED during the FY9601 periodwere either SALs or SECALs. With the reserva-tions noted above, as figure 3.8 shows, weightedby disbursements, SECAL performance improvedfairly steadily over the mid-1990s, surpassingthat of SAL, which has been more volatile, forFY00 exits. No operations using the new adjust-ment lending instrumentsSSALs, PSALs,PRSCshave been evaluated so far by OED. About 75 percent of investment disbursementsevaluated by OED for the FY9601 exit cohortwere supported by SILs. The remainder were dis-tributed among SIMs (15 percent), FILs (5 per-cent), ERLs (4 percent), and TALs (2 percent)projects. As figure 3.9 shows, the satisfactoryoutcome ratings for SILs and SIMs indicate astrong performance for both lending instrumentssince FY99. The performance of SIMs since FY99is also strong, with 89 percent satisfactory out-come ratings for the preliminary FY01 exits. Outcomes for other specific investment instru-ments are uneven. For the FY9601 exits, 96percent of ERL disbursements were rated satis-factory. For FILs, the percentage was only 46.Projects were most successful in achieving objec-tives related to physical infrastructure and to thenatural environment. The primary objectives ofERLs are to support physical objectives such asthe restoration of assets and production levelsimmediately after an extraordinary eventsuchas war, civil disturbance, or natural disasterthathas seriously disrupted a borrowers economy.OED has so far evaluated only one APL andtwo LILs. These three evaluations do not con-stitute a large enough sample to draw any accu-rate analysis of their performance. Looking at theactive projects supported by APLs and LILs, QAGassessments indicate that of the 96 active APLsin the Banks portfolio, 93 percent are considerednot at risk. Similarly, QAG assessments indicatethat of the 84 LILs currently active in the port-folio, 89 percent are considered not at risk.Instruments have different risks and rewards,which can be factored into their choice. Figure3.10 places major lending instruments into fourrisk/reward quadrants, where rewards are meas-ured by the average APPI and risk by the APPIsstandard deviation. This comparison is not meantP e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l2 1T r e n d s i n O u t c o m e s o f I n v e s t m e n tO p e r a t i o n sF i g u r e 3 . 91996 1997 1998 1999 2000 2001*405060708090100Percent Satisfactory1996 1997 1998 1999 2000 2001*405060708090100Percent SatisfactoryAll Investment LendingBy ProjectWeighted by DisbursementsSelected Specific Investment Instruments(Weighted by Disbursements)SIMsSILs*Preliminary; see chapter 3 endnote 1.to imply substitutability of lending instrumentsbut rather to provide a relative risk versus rewardframework in which to assess the potentialstrengths and weaknesses of these instruments,in conjunction with country conditions and sec-tor choices. The cost dimension. The varying perform-ance of lending instruments in differing coun-try environments also has a cost dimension.Figure 3.11 shows the Banks cost per US$1million disbursed (in terms of preparation andsupervision) related to achievement of objectives.The results confirm the finding that outcomes forobjectives pursued through adjustment lendingshow a higher sensitivity to country conditionsthan those for investment. Costs of adjustmentlending in terms of dollars lent do not show ahigh level of variation. On the other hand, invest-ment interventions cost considerably more tocomplete in low-CPIA countries. Determinants of SuccessThis section follows on the model of determinantsof project outcomes used in previous Reviews. Thismodel, which has been econometrically tested,confirms that country conditions and the qualityof project-level inputs are the key determinants forsuccess. These determinants are highly relevantfor the implementation of the approximately 1,500active lending projects, and for design and imple-mentation of projects in the future.Country Conditions MatterThe outcomes of projects are closely correlatedwith the borrower country policy and institutionalenvironment, as measured by the CPIA ratings.For FY9601 exits, low CPIA countries had only48 percent satisfactory disbursements, whereasmedium CPIA countries had 81 percent andhigh CPIA had 93. Recent evaluations for stand-alone TechnicalAssistance Loans (TALs) showed this instrumentfared better in low CPIA countries than any otherform of lending. This reflects a change fromfindings in last years Review, due to recentresults from ECA which now accounts for two-thirds of evaluated TALs exiting during theFY9601 period. As figure 3.12 shows, investmentprojects that used SILs or SIMs as their mainlending instrument averaged 47 percent satis-factory outcome rates, heavily influenced, though,by eight large disbursements operations in threecountries. On the other hand, stand-alone tech-nical assistance averaged 75 percent satisfactory.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2 2R i s k s a n d R e w a r d s o f L e n d i n gI n s t r u m e n t sF i g u r e 3 . 1 0-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6Low RiskLow RewardRewardHigh RiskHigh RewardLow RiskHigh RewardHigh RiskLow RewardFILSECALTALSALSILRisk- 0.0While the goals against which such operationswere judged were not always geared to criticalpolicy reform areas, their broadly satisfactoryperformance in low CPIA countries suggests thatthe Bank should focus on increasing capacity andinstitution building in these countries, ratherthan using a traditional lending strategy. Satis-factory outcomes for low CPIA countries at thesector level showed that those projects imple-mented in the transportation and agriculture sec-P e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l2 3I n s t r u m e n t P e r f o r m a n c e I sS e n s i t i v e t o C o u n t r y C o n d i t i o n s( F Y 9 6 0 1 E x i t s )F i g u r e 3 . 1 2SECAL SAL SIL SIM TAL Bankwide020406080100Percent Satisfactory by DisbursementsHigh CPIA Medium CPIA Low CPIAT h e C o s t D i m e n s i o n , b y C P I AG r o u p s ( F Y 9 0 F Y 0 1 E x i t s )F i g u r e 3 . 1 105,00010,00015,00020,00025,0000 20 40 60 80 100Cost per US$1 Million DisbursedPercent Satisfactory Outcome (Weighted by Disbursements)INVLow CPIAINVMedium CPIAINVHigh CPIAADJLow CPIAADJMedium CPIAADJHigh CPIAtor performed on average about 10 percentagepoints higher than the average.For medium CPIA countries, projects usingsectorwide instruments (SECALs and SIMs) wereless successful than others. As figure 3.12 shows,the satisfactory outcome rates, weighted by dis-bursements, of SECALs in medium CPIA coun-tries were 12 percentage points lower than theaverage for SALs in the same countries. Thelower-than-average satisfactory outcome is pri-marily due to the performance of several largeSECALs in the finance sector. The satisfactory out-come of SIMs fares similarly, under-performingthe satisfactory outcome rating average for SILinvestment projects by 10 percentage points;the lower-than-average results are driven by thepoor performance of projects in the educationsector. One of every two dollars disbursed ineducation sector projects of medium CPIA coun-tries, using a SIM or a SECAL, received unsatis-factory outcome ratings from OED.In addition to country-specific conditions,exogenous regional variables can be a significantfactor affecting the outcome of adjustment oper-ations. The Communaute Financiere Africaine4(CFA) franc devaluation of January 1994 playeda significant role in the economic recovery ofmember countries during the mid-1990s5 and thesatisfactory outcomes of adjustment operations.Before 1994, the overvalued CFA franc exacer-bated the effects of a terms of trade deteriora-tion, making the economic downturn of 198793more severe in CFA countries than in the otherAfrican countries. Figure 3.13 shows that theshare of satisfactory adjustment operations6 inCFA countries prior to the devaluation was 54percent, compared with the 62 percent rate fornon-CFA countries. After the devaluation, satis-factory outcomes of adjustment operations in CFAcountries increased by over 30 percentage points,to 84 percent satisfactory outcomes, outstrippingthe 76 percent rate for adjustment operations innon-CFA countries.Project-Level Performance Also MattersIn addition to country-specific conditions, proj-ect quality and direct implementation supportfrom the Bank and borrower are key determi-nants of success. This is supported by a separateanalysis focusing on 45 projects evaluated sincelast years Review.7 Project design weaknesseswere identified as important factors in everysample project rated unsatisfactory, as shown intable 3.1. Weak monitoring and evaluation werealso cited as factors in more than one in threeprojects. For projects with highly satisfactoryoutcomes, project design was found to be themost widespread factor, followed by borrowerimplementation performance. Among projectdesign weaknesses, lack of realism (in scopeand complexity of objectives and/or in proposedtime frame) was by far the most important, con-tributing to poor project performance in no lessthan 72 percent of the unsatisfactory sampleprojects, including all the adjustment projectsrated unsatisfactory. The next most prevalentfactors were the quality of technical and/or insti-tutional design, which affected investment oper-ations more than adjustment.Both QAG and OED assess the quality atentry (QAE) of projects. For projects that havealready closed and been evaluated, OED findsthat QAE improved steadily through the 1990s.And for projects still under implementation,QAG documents a continuing upward trend.These trends are shown in figure 3.14.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2 4Pre-Devaluation Post-Devaluation020406080100Percent Satisfactory OutcomeCFA Non-CFA AfricaA d j u s t m e n tO u t c o m e si n A f r i c aR e l a t e d t o t h e C FAD e v a l u a t i o nF i g u r e 3 . 1 3OED evaluators assess preparation, imple-mentation, and compliance with legal covenantswhen considering borrower performance. Thesecond most prevalent reason for unsatisfactoryoutcome performance, weak borrower commit-ment, affected more than half the projects andwas especially significant for adjustment opera-tions. Satisfactory performance for borrowerpreparation and implementation increased by 5percentage points each between the FY9699 andthe FY0001 periods. The pattern is similar whenweighted by projects. Compliance performanceshows an increase between the two periodswhen weighted by projects but a decrease whenweighted by disbursements, highlighting onceagain the influence that very large projects haveon the FY0001 exit evaluated cohort.Performance of Nonlending AssistanceAs discussed in Chapter 2, nonlending assistanceincludes an evolving range of activities. Some areaimed at improving the Banks knowledge basein order to provide a solid foundation for theBanks policy dialogue with clients and the devel-opment of country strategies. Others directlyrelate to supporting lending assistance. Perfor-mance of the first set of these activities is not asrigorously measured as lending assistance, as theframework in the Bank for evaluation of individualnonlending instruments is less mature. A caveatis in order: as noted in the recent AROE, evalu-ation gaps still exist with respect to the knowl-edge and partnership Bank, and it is not possibleto present reliable performance trends for allnonlending services provided by the Bank (e.g.,research, WBI, aid coordination, grants). Thissection draws from the self-evaluation systemused by the QAG, which covers a sample ofindividual ESW reports, as well as several OEDstudies of individual nonlending instruments. As the Bank has moved toward country-based approaches and increased programmaticlending, country-level fiduciary work hasassumed greater strategic relevance. In FY01,management has recognized three assessmentsPublic Expenditure Reviews (PERs), CountryProcurement Assessment Reports (CPARs), andCountry Financial Accountability Assessments(CFAAs)as the core fiduciary diagnosticsrequired in every client country. Considerationshould be given to classifying safeguard policyreviews at the country level as core ESW.QAGs most recent evaluation of traditionalESW tasks (which is focused on formal andinformal economic and sector reports) high-lights an improvement in overall quality from 73percent satisfactory in 1998 to 86 percent satis-factory in 2000.8 This exceeds the Strategic Com-pact target of 85 percent. Furthermore, QAGnotes that as the Banks ESW has evolved to beparticipatory, client-oriented, and result-focused,the likely impact of the tasks has also improvedsignificantly since 1998. The QAG reviews cite four key areas wherethere is room to further enhance quality: inpoverty reduction strategies and poverty-relatedP e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l2 5Factor Investment Adjustment Total %Weakness in project design 100 100 100Weak monitoring and evaluation 46 13 38Weak borrower commitment 46 88 56Poor borrower implementation performance 42 50 44Poor Bank supervision performance 38 0 28Negative exogenous factors 33 0 25Source: Review of recent PPARs for projects with unsatisfactory outcomes.R e a s o n s f o r U n s a t i s f a c t o r yO u t c o m e s ( p e r c e n t o f s a m p l eu n s a t i s f a c t o r y p r o j e c t s , f o r w h i c h t h e f a c t o r i s i d e n t i f i e d )T a b l e 3 . 1analysis; in the quality of conclusions and in pri-oritizing recommendations; in improving thequality of ESW undertaken in poor policy envi-ronments; and in fiduciary ESW. Poverty focusof ESW is identified as a key concern. In 2000,for the first time, QAG began rating ESW taskson their analysis of implications on the poorand found 61 percent of tasks to be satisfactory.It cited several weaknesses such as poor analy-sis and links to policy options; as well as lackof attention to regional differences in income andpoverty, to the impact of private sector partici-pation and other policy shifts on the poor, andto the impact of reforms on the most vulnera-ble groups. This is also evident from the com-prehensive OED evaluations and QAG ratings ofPoverty Assessments (box 3.3). There is con-siderable room to clarify the extent to which dif-ferent kinds of ESW are expected to addresspoverty-related issues and to mainstream povertyanalysis into the scope of the ESW.9A second area mentioned by QAG for fur-ther attention is the clarity and substance of con-clusions (one in three tasks rated less than2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2 6I m p r o v e d B a n k P e r f o r m a n c e a sE v a l u a t e d b y O E D a n d Q A GF i g u r e 3 . 1 41990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001405060708090100Percent SatisfactoryQuality at EntryOEDQAG1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001405060708090100Percent SatisfactoryQuality of SupervisionOEDQAGApproval FYExit FYsatisfactory) and the sequencing and prioriti-zation of recommendations. QAG rates 50 per-cent of ESW tasks satisfactory on prioritizingrecommendations. Selectivity thus remains anissue at the instrument level and could affectactual impact on the ground. Improving thequality of the conclusions and recommen-dations, along with enhancements to the qual-ity of presentation and improved user-friendliness of ESW could contribute to betterdissemination, and increase the likelihood ofsignificant impact. A third area for improvement is the quality ofESW in poor policy and institutional environ-ments. In 2000, QAG rated 92 percent of ESWtasks in high CPIA countries satisfactory; thescore for low CPIA countries is 72 percent. Thisis confirmed by OEDs analysis of the quality ofESW as reflected in CAEs.10 The analysis indicatesthat in countries with poorer average CPIA rat-ings, ESW tends to have more limited impact onBank products and is barely satisfactory overall.The impact on country dialogue is also on aver-age not satisfactory. By contrast, for countrieswith medium CPIA ratings, the impact of ESWboth on Bank products and on country dia-logue is satisfactory, and for countries with goodCPIA ratings is fully satisfactory. The fourth area of focus mentioned by QAGis the quality of core fiduciary ESW. The PER isthe oldest instrument and is the only fiduciaryESW to benefit from systematic evaluation. Therelative dearth of evaluative evidence on theexperience with and impact of CPARs and CFAAsis largely on account of the evolving nature ofthe Banks thinking in the area of country-levelfiduciary management. There has been little for-mal evaluation of quality and content of thesecore fiduciary products. OEDs 1998 Review noted that PERs had amodest impact on Bank lending strategies, clientexpenditure policies, and aid coordination. Itnoted that the quality of analyses in the PER hasimproved and lauded the impact of the informalPER. The review suggested improving both thePER content and the process.11 Since then, therehave been significant changes to the nature ofthe PER process and its contents. However,QAG 2000 notes that the quality of Public Expen-diture Reviews continues to be mixed. Wherequality was poor, reasons included inadequatefunding, low-quality teams and/or management,P e r f o r m a n c e a t t h e I n s t r u m e n t L e v e l2 7This year for the first time OED reports on the evaluation of operations financed under three special programs. Forty-six such oper-ations exited the Banks portfolio during the FY9601 period. Global Environmental Facility (GEF): The GEF was established in 1991 as a pilot program to assist in the protection of the globalenvironment and to promote environmentally sound and sustainable economic development. Overall, 143 full-size projects (withcommitments greater than US$1 million) have been approved, of which 32 have been evaluated by OED.Montreal Protocol Fund (MT): The World Bank is one of four implementing agencies (along with UNDP, UNIDO, and UNEP) forthe Multilateral Fund for the Implementation of the Montreal Protocol to reduce ozone-depleting substances. Eleven projects havebeen approved under this fund, of which OED has evaluated four.Special Financing Grants (SF): The SF grants cover special emergency assistance provided to recent post-conflict countriessuch as Bosnia-Herzegovina, East Timor, Kosovo, and West Bank and Gaza.Outcome Performance of Bank-Managed Special Programs% Satisfactory Outcome % Satisfactory OutcomeSpecial Program Type (Weighted by Projects) (Weighted by Disbursements)GEF 84 85MT 100 100SF 100 100B a n k - M a n a g e d S p e c i a l P r o g r a m sB o x 3 . 2unclear goals, and inadequate coverage (in somecases several PERs for the same country).Although there is no formal evaluation of theCFAA and CPAR, there is evidence to suggest thattheir findings are yet to be fully integrated intothe CAS process and influence the Banks assis-tance. Internal Audit Department 2001 notes thatweaknesses in the capacity for sound financialaccountability identified through the CFAA arenot being addressed and that the expectation tolink CFAA findings and recommendations to theCAS has not been met. The Annual ProcurementReview for FY00 (draft) also notes that few CPARscarried out since 1998 have resulted in specificreform actions (although the review argues thatthere has not been enough time for significantreforms to take place). It however notes that amonitoring system would be implemented toensure that the CPAR coverage supports the CASand other strategic requirements regarding lend-ing, country risk assessment, and capacity build-ing. Work is currently under way in OPCS toendure greater linkage and integration within thefiduciary documents as well as with the CASprocess. Until FY01, the CPAR and CFAA werenot treated as formal ESW and hence did notcome under the purview of the institutional ESWreview mechanism or QAG. QAG is currently inthe process of rating the quality of the CFAAs andCPARs undertaken in FY01.The launch of an ESW reform effort in July1999 has led to significant improvements to thequality of ESW. Yet, QAG notes that theseimprovements are not mirrored in commensu-rate gain in the effectiveness of Bank processessuch as sustained managerial attention and peerreviews. For instance, there has been significantimprovement in the quality and process of peerreviews, but peer review advice does not oftenhave an impact on the final product. The200001 AROE notes that the networks currentsupport role does not fully exploit their com-parative advantage. Task teams need to drawadequately upon the analytical expertise orknowledge base in the networks, DevelopmentEconomics Department, or World Bank Institute.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s2 8The Poverty Assessment (PA) is a key instrument to serve theBanks overarching objective of poverty reduction. Evaluativeevidence suggests considerable scope to increase the Bankscapacity to effectively analyze and develop poverty reductionstrategies through the PA. OEDs review of 46 Poverty Assess-ments in 1996 noted that 54 percent of the assessments met therequirements set forth in OD 4.15. Since then, there has been someimprovement. The 1998 QAG review of 15 Poverty Assessmentsrated 43 percent of them satisfactory and OEDs follow-up reviewin 1999 of the assessments completed between FY96 and FY98rated 63 percent of them satisfactory. QAG 1998 raised several weaknesses in poverty-relatedanalysis such as lack of access to reasonable data and inade-quate data mining; poor technical skills and inadequate men-toring; lack of attention to the most salient policy issues; and poorquality assurance mechanisms.OEDs 1999 follow-up evaluation of Poverty Assessmentsfound that 62 percent of the assessments did not adequatelyspecify goals and where multiple goals were specified, therewas seldom any prioritization. The OED review also noted sys-temic weaknesses in areas such as: design of Poverty Assess-ments and inclusion of all relevant groups; fostering and sus-taining ownership; rigor and meaningfulness of the policy analy-sis; and quality of and prioritization among recommendations.The OED 1999 review also notes that surveyed stakeholdersviewed knowledge transfer, and local consultation and part-nerships as the least satisfactory aspects. Forty-two percent ofstakeholders felt that the PA had not provided even a moderateincrease in local capacity.QAG 2000 reports a further improvement in overall quality to69 percent satisfactory in 2000, but the Banks processes asrelated to Poverty Assessment are satisfactory 38 percent ofthe time. QAG noted that most PAs lack a coherent (Bank assis-tance) strategy for poverty reduction. This weakness is expectedto be addressed through the PRSP process; the PRSP processis likely to make the PA a more technical background report thatfeeds into in-country discussions of the most effective strategyto reduce poverty. *Note: Inclusion of a set of related poverty indicators; the scope and depth of povertydiagnosis; the breadth and scope of policy prescriptions for poverty reduction; andthe strategic content of such prescriptions.Q u a l i t y o f t h e B a n k s P o v e r t y A s s e s s m e n t sB o x 3 . 32 9Country Strategies,Instruments, and OutcomesEffective country strategies are shaped by corporate priorities andrequire strategic instrument choices. Equally, instrument design mustbe tailored to country operating environments. Recent evaluative evi-dence yields lessons on instrument choice and use in a range of country con-ditions. This chapter identifies areas in which the Bank can further increaseits development effectiveness at the country level. It points to performance-based allocations, strategic selectivity in line with comparative advantage,results-based strategies backed by verifiable performance indicators, trans-parent CAS linkages with PRSP), and agreed policy frameworks as the majorchallenges of country strategy. Strengthened risk management, reliance onbusinesslike partnerships, and adoption of participatory approaches emergeas the best antidotes to risk aversion, especially in post-conflict situations andin countries where policy reform and capacity building are uphill tasks.Individual instruments of the Banks assis-tance need to complement each other within anappropriate strategy in order to achieve qualityoutcomes. At the country level, this requires theBank to adapt assistance to the country contextwhile maintaining the link between corporateselectivity, the country assistance strategy process,and the appropriate use of instruments. Coun-try program outcomes have been measured in thefifty-odd CAEs conducted thus far by OED.1Drawing primarily from these evaluations, thischapter assesses development effectiveness at thecountry level by addressing three questions: How does corporate selectivity help definecountry strategies? How do country assistance strategies selectand combine instruments to maximize devel-opment effectiveness? How do lending and nonlending instrumentscontribute to country program outcomesunder differing country conditions? Corporate Selectivity and CountryStrategiesCountry assistance programs are shaped withinthe constraints of corporate priorities and44resource envelopes. Corporate selectivity definesthe institutions goals and priorities, reflecting itscore competencies, its global mandate, and itsareas of comparative advantage for countryassistance. The corporate framework aims atpoverty reduction through a country-drivenframework. It recognizes that development effec-tiveness depends on the country policy andinstitutional environment, and on judicioussequencing between lending and nonlendingactivities. Corporate choices are conveyed toindividual country programs through three mainmechanisms: the policy and assistance framework pro-vided by the CDF and PRSP inter-country lending allocations for IDA andIBRD countries the allocation of administrative resources toprepare and supervise lending and nonlend-ing activities.The CDF and PRSI processes are designed to cen-ter the Banks mission on poverty reductionthrough support of a country-driven, results-oriented framework jointly owned by the pub-lic, private, and voluntary sectors. In turn, theindividual countrys framework (as articulated inthe PRSP for low-income countries) providesthe grounding for the Banks assistance strat-egy, and constitutes a key platform for articulatingthe countrys development goals. These rela-tively new mechanisms are beginning to influ-ence country assistance strategies. Initially seenas a Bank requirement for further adjustmentlending, the April 2001 Interim PRSP for Vietnamdeveloped into a central component of the gov-ernments own 10-year development strategy.In Ghana, the governments Ghana Vision 2020document provided a long-term holistic vision ofdevelopment, which has underpinned the CDF,and has helped to solidify country ownership. The performance-based allocation of lendingresources is the Banks principal mechanism forexercising inter-country selectivity. Since thelate 1970s, the allocation of IDA resources hasbeen influenced by country performance. Ascurrently defined, performance is assessedthrough explicit indicators reflecting the sound-ness of country policies and institutional arrange-ments and the development risk profile of theongoing IDA operations portfoliocombined inan overall performance rating in an 80:20 ratio.In recent years, the combined performance rat-ing has been reduced by one-third for borrow-ers with severe governance problems. Evaluationresults confirm that the CPIA ratings are stronglyassociated with project outcomes, as shown infigure 4.1, for both IDA and IBRD countries.Countries with poor CPIA ratings show well-below-average portfolio results. OEDs recent IDA Review2 finds that theperformance-based allocation system hasevolved over the last decade to reflect newdevelopment knowledge and evolving corpo-rate priorities. These changes have strength-ened the link between countries policy andinstitutional performance and lending levels.Today, the system directs more resources togood performance than it did a decade ago. TheReview proposes several additional adjustmentsto further strengthen the link, achieve greaterconsistency of treatment among countries, andincrease transparency: establishment of a writ-ten record of country ratings; adjustment ofspecific CPIA criteria to ensure that they assesspolicy performance rather than level of devel-opment; revision of the governance discount;and broader disclosure of ratings to borrowersand the wider international community. IDAmanagement has initiated enhancements to theperformance-based allocation system drawingon these recommendations.3For IBRD borrowers, lending allocations arenecessarily shaped by global financial systemconsiderations, Bank exposure, and credit-worthiness. Nevertheless, the Bank has sought toincrease the weight of performance in its alloca-tion decisions for middle-income countries inrecognition of the better development prospectsexpected from lending under good country poli-cies and institutions. Figure 4.2 shows that bothIDA and IBRD lend more on a per capita basisto countries with better project results. The scope and scale of the Banks activitiesin a country are necessarily constrained bythe countrys administrative budget. The sizeof country budgets is still closely linked to thesize of the countrys lending portfolio. Non-2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s3 0lending allocations tend to be residually derivedafter accommodating the fixed costs associ-ated with lending and supervision of the ongo-ing portfolio. This feature of the budget processis being reviewed as it may hamper country-level development effectiveness, especiallywhere lending is limited such as in new bor-rowers or countries in lending hiatus. Severalrecent CAEs emphasized the need for soundcountry knowledge before initiating or resum-C o u n t r y S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s3 1B e t t e r L e n d i n g O u t c o m e sA s s o c i a t e d w i t h H i g h e r P o l i c ya n d I n s t i t u t i o n a l Q u a l i t y( F Y 9 6 0 1 E x i t s )F i g u r e 4 . 10102030405060708090100Percent Satisfactory(Weighted by Disbursements)CPIA Average FY9800Low Medium HighBankwide AverageBankwide AverageM o r e L e n d i n g t o C o u n t r i e s w i t hB e t t e r P r o j e c t O u t c o m e s( F Y 9 6 0 1 E x i t s )F i g u r e 4 . 201020304050607080901000 5 10 15 20 25 30 35 40Lending per Capita (FY9901 Approvals)Bankwide AverageBankwide AveragePercent Satisfactory(Weighted by Disbursements)ing lending. Thus, maintaining a base ofstandard ESW coverage may be more cost-effective than the steep learning curve, lostopportunities, and costly project restructuringwhich usually result from inadequate lendingpreparedness (e.g., Chile in the 1980s). Inaddition, the World Bank knowledge bank isexpected to support countries where knowl-edge, not financing, is the priority (e.g., Bhutanand Chile in the 2000s), or where the policyand governance environment needs nurturingupstream of lending (e.g., Haiti now).The close link between budgets and lendingvolumes poses a special challenge in very smallcountries. In the Maldives, for instance, the Bankmakes one loan only every four or five yearswhile carrying out basic economic work inbetween. The CAE found that this ESW hadhigh value to the client, but that budget con-straints prevented the Bank from undertakingadditional high-priority work on private sectordevelopment and environmental issues in atimely fashion. Special arrangements to fundand implement such work in collaboration withlike-minded partners may be required to meetsuch needs.Country Strategies and InstrumentChoiceWithin the countrys budget and lending allo-cation, the Country Assistance Strategy providesthe vehicle for the Bank and borrower to decideon the right combination of instruments basedon the countrys development objectives. Asnoted in the IDA Review, selectivity in lendinginstrument choice is a strategic matter and oughtto be treated as such in the CAS.4 This is becom-ing more important given the desire of mostdevelopment partners to demonstrate a sharperdivision of labor through the use of CDF prin-ciples. For borrowers as well as their externalpartners, instrument choices must be based ona sound understanding of the full menu of instru-ments available. OED evaluations find that there is untappedpotential to use the choice of instruments morestrategically in country programs. In 15 CAEscompleted last year, country programs were ratedfor their treatment of instrument relevance andperformance. Eight programs were rated sub-stantially or highly attentive to instrument selec-tivity, and seven were rated modest to negligible.Most of the strategies dealt extensively with thebalance between lending and nonlending, butfewer than half stated the strategic rationale forthe particular mix of lending instruments adoptedin relation to program objectives. The rationalefor specific lending instrument choice was madeexplicit only in a few cases, usually in connec-tion with the use of an APL or LIL. The Reviewalso highlights a lack of specific instrument pref-erence on the part of some borrowers, especiallyfor new borrowers or countries borrowing afteran extended period of disengagement, wheregovernments were not aware of, or had limitedexperience with, the full menu of Bank instru-ments. This emphasizes the need for the Bank tokeep borrowers fully informed of changes in theBanks toolbox so as to encourage full ownershipof instrument choice decisions. What are the obstacles to more strategic useof instruments in country programs? One issue,already explored in Chapter 2, is the lack of rel-evant and sufficiently detailed operational guid-ance in matching instruments with developmentobjectives. Three other factors emerge at thecountry level. First, partnerships may not alwaysbe adequately leveraged. Second, a logical frame-work and results chain linking instruments withobjectives is not a standard feature of countrystrategies. And third, country circumstances mayrequire unanticipated changes in the assistanceprogram in between the preparation of periodicCASs. This makes it important to use CAS updatesand progress reports to provide an updatedstrategic context for Bank activities. The fol-lowing sections describe these obstacles andhow they might be addressed. Defining Comparative Advantage Depends onPartnersTo enhance the effectiveness of country strate-gies, the Bank needs to address program selec-tivity in consultation with borrowers and otherassistance partners. The IDA Review emphasizesthat the Bank should define more sharply its com-parative advantage as an institution and followthrough on the implications for its programs at2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s3 2the country and sector levels. At a country levelthis involves leveraging its comparative advan-tage and identifying areas in which other donorsmay effectively take the lead. Partnerships are key to improving the impactof aid resources, enabling areas of comparativeadvantage to be identified and exploited. Areview of OEDs recent CAEs provides evidenceof modest progress made toward more effectivepartnering at the country level. For example, cul-tivating partnerships and a focus on aid coor-dination were strong elements of the Banksassistance to Vietnam, particularly in nonlend-ing services. The Bank and UNDP workedclosely together in Vietnam, with the Bank exe-cuting a number of UNDP-financed projectswhen it was unable to lend itself. According tothe Vietnam CAE, the Banks work in aid coor-dination created a collaborative spirit accordingto CDF principles well before it was selected asa CDF pilot country. Sector working groupswere established along with businesslike part-nerships in technical assistance and ESW. Expe-rience in Vietnam also highlights the tradeoffoften required between improved coordinationand the additional time required to reach a con-sensus. Partnership means letting the borrowerand other donors lead even if the resulting pacedoes not fully mirror Bank preferences.Effective aid coordination is particularly impor-tant in countries with a large volume of assis-tance and many donors, especially countrieswith limited administrative capacity and post-conflict countries. Uncoordinated aid programsimpose heavy burdens on recipient countries andlimit the impact of aid agencies programs. InWest Bank and Gaza, the Bank established aunique structure to coordinate the activities ofa large number of donors delivering high vol-umes of aid. In addition to the usual Consulta-tive Group, the aid coordination architectureincludes several Liaison Committees. These pro-vide a forum for donors and authorities toaddress policy questions and to coordinate activ-ities at the operational level.Logical Framework Application in the CASThe CAS should trace through the linkages fromthe countrys long-term development goals tosector/thematic objectives and to instruments.The CDF and PRS processes provide the forwardlinkages that relate the CAS process to medium-or longer-term outcomes such as the Millen-nium Development Goals and tailor-made goalscustomized to countries own developmentvisions (e.g., Ghana Vision 2020). With theCDFs and PRSPs providing the links to thepoverty reduction framework, the CAS focuseson providing the business model relating Bankinstruments to sector and thematic objectives.Recent decisions for costing of CAS programs andlinking to the budget are steps in the right direc-tion. Increased partnerships are also critical forCASs to be appropriately selective in the Banksassistance.The CAS guidelines clearly distinguishbetween Bank and country priorities. Wherethere are differences, the risk of misreading theborrowers commitment is high and this may leadto inappropriate initiatives or to the choice of thewrong instrument. In Morocco, the FY97 CASproposed a program of predominantly invest-ment lending to target social and rural devel-opment. Investment lending was perceived to bethe best means to affect public expenditure forcritically important health, education, and ruralprograms. The proposed strategy, however, mis-judged the level of government commitment tosocial spending. The government was not eagerto borrow from the Bank for programs withhigh social content, for which funds on con-cessional terms were readily available from othersources. In addition, investment lending had apoor track record in Morocco. As a result, theactual lending program differed substantiallyfrom that presented to the Board in the CAS withthe bulk of Bank lending going to adjustmentlending for policy reform. Establishing clear and sequential linksbetween each instrument and the CAS objectivesis crucial to achieve efficacy. This requires astrong link between instruments and perform-ance indicators based on upfront analysis whichincludes recognition of risks as well as learningfrom past resultsespecially for the adaptableand programmatic lending instruments. Evalu-ative evidence would allow the Bank to learnfrom the past in terms of what instruments haveC o u n t r y S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s3 3had what results, but CASs seldom give atten-tion to past instrument performance in order toilluminate lending pipeline choices. It is alsoimportant to adjust strategies for changing coun-try conditions, through timely CAS updates andprogress reports that provide an agreed strate-gic context for Bank activities.Country Strategies Are Dominated by LendingSatisfactory country outcomes require more thanjust successful projects. Table 4.1 shows thatfor a sample of CAEs, project performance (asmeasured by the aggregate outcome ratings)often went hand in hand with country outcomesin the CAEs, but deviations served as remindersthat country outcomes are determined by thecombination of lending and nonlending instru-ments, as well as other factors. An emphasis on lending appears to have lim-ited the Banks effectiveness in some countries.Lending pressures were reported in 5 out of 13recently evaluated countries. Three of thesewere large or higher-income borrowers, in whichgovernment instrument preferences for financialassistance were readily accommodated (Chile,Mexico, India). In some cases lending was usedto achieve goals better met with nonlendinginstruments. During the 1980s and early 1990s,the Indian governments unwillingness to acceptpolicy-based instruments restricted Bank assis-tance to investment lending. This led to prepa-ration of some investment projects of lowrelevance and weak effectiveness. Similarly, theBank opened its dialogue on transport reformwith Kazakhstan in the context of a loan that hada primary focus on misguided support for astate-owned transport enterprise. And in Bulgaria,because of the authorities limited interest inprimary health care, the dialogue on health sec-tor reforms was launched via the financing ofambulances. This approach, while pragmatic,generated disproportionate Bank investment forlow-priority activities. The effectiveness of Bank lending has provedto be influenced by the quality and coverage ofnonlending. Accordingly, the amount and typeof nonlending assistance should be carefullycalibrated to enhance Bank lending and improve2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s3 4PortfolioPerformance% Disbursements Country program outcomeSatisfactory Fully Satisfactory Partially Satisfactory Marginal UnsatisfactoryUruguay (199000) Kazakhstan (199199) Costa Rica (199000)Vietnam (198800) Kyrgyz (199300) Bulgaria (199097)Bulgaria (199801)Chile (198599)Burkina Faso (198999) Indonesia (199398)Egypt (199000) Lesotho (199000)Mexico (198999)Lower Middle El Salvador (198900) India (199000)(6274% sat) Ghana (199500) Cameroon (199600)Uganda (198699) Tanzania (199600) Morocco (199700) Cameroon (198295)Tanzania (198695) Papua NG (198900)Paraguay (199000)Haiti (198601)Note: Portfolio performance quartiles are based on Bank-wide country averages of OED evaluated project performance for projects exiting FY9001, weighted by disbursements. Countryprogram outcome ratings are taken from the relevant CAEs. All CAEs have been disclosed with the exception of Bulgaria, which is expected to be disclosed by March 2002. Source: OED. C o u n t r y P r o g r a m a n d P o r t f o l i oP e r f o r m a n c eT a b l e 4 . 1Upper Middle(7587% sat)Low(< 62% sat)High(> 87% sat)the relevance of Bank assistance strategies. Non-lending was rated supply driven in 8 of the 15countries programs reviewed. In most casesCAE task managers cited a lack of borrowerpreference and inadequate consultation by theBank as reasons for the supply-driven services.There is considerable scope for the Bank toincrease borrowers awareness of its menu ofnonlending instruments. Recent evaluative evi-dence offers examples of the way synergybetween nonlending and lending programs con-tributes to satisfactory outcomes. For example,box 4.1 illustrates how lending to the Lesothoeducation sector had a highly satisfactory out-come based on relevant and timely nonlendingservices. CAEs found instances where gaps in ESWcoverage reduced the relevance and effective-ness of Bank lending. Lending in spite of poor,outdated, or insufficient knowledge was a fea-ture of assistance programs to a number of newand renewed Bank borrowers. In Kazakhstanand the Kyrgyz Republic, for example, insuffi-cient analytic work in some areas preventedclear diagnoses, although ESW was notablyeffective in others (such as pension reform).Reductions in resources devoted to nonlendingposed special problems in transition economies,where limited experience with reforms meantthat changes needed quickly to be captured. Theneed to update and maintain a substantial knowl-edge base is a priority also in countries that haveexperienced long interruptions in Bank lending,because they lack a network of officials famil-iar with Bank policies. For its part, the Bank mayneed to update staff knowledge of current coun-try contexts. In Kenya, the Banks assistancestrategy underwent a major change in an effortto link lending to improved governance. Lend-ing pressures, however, were a significant fac-tor in undermining the strategy (box 4.2).Country Strategies in the Absence of a LendingProgramThere are at least 30 countries in which the Bankis not providing lending support for a wide rangeof reasons. This situation poses special chal-lenges for country strategy formulation. OEDsrecent country evaluations yield lessons forenhancing the Banks assistance strategy in threescenarios under which lending is constrained.First time or renewed borrowers. An initialperiod of nonlending before lending to new orrenewed borrowers can help develop bothgood relations with the government and otherpartners and a sound basis for lending. In Viet-nam, the Bank carried out substantial ESW fromFY88 until it resumed lending in FY94. Antici-pating that lending would eventually resume, theBank conducted work in sectors where Bankcredits were later approved (health, finance,and energy). The recent CAE found this ESW tohave been very effective in its impact on boththe selection and the design of Bank credits aswell as the country dialogue by helping to winthe confidence of Vietnamese officials. If thelapse in lending is protracted, the payoff fromcontinued nonlending engagement may take along time to materialize. The Bank has sus-pended lending to Haiti twice in the past 10years, from 1991 to 1994 and from 1997 to thepresent, but carried out nonlending activitiesboth times. During the first hiatus, ESW was partof a multi-donor effort to examine developmentC o u n t r y S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s3 5In Lesotho, ESW in education had significant impact on the outcomes of projects. Bank analytical services in the education sectorwere highly relevant, timely, and in tune with government and CAS objectives. The FY90 report Improving Quality and Efficiencyin Education helped to establish a coherent policy framework for education reform in the 1990s. This alleviated past problems withprioritizing goals and coordinating donor activities in the sector. Strong analytic work also provided a platform for subsequentBank lending, leading to satisfactory outcomes in education, despite the weaker quality of the rest of the country-lending pro-gram. It also led to a strong and sustained relationship with the Ministry, even through a time of political turmoil. S y n e r g y B e t w e e n S e c t o r W o r k a n d L e n d i n gB o x 4 . 1and assistance needs. In the more recent period,the Bank focused on social sector issues in thehope that they would have some impact onpolicies and with the expectation that lendingwill some day be resumed. The recent CAE con-sidered these activities to have had negligibledirect impact on policy dialogue thus far, aswould be expected in light of the extremelyunstable country context. Benefits could beexpected to accrue upon resumption of lending,in addition to the more immediate donor coor-dination benefits. Nonlending, properly designed,should be viewed as a long-term investment insuch cases, requiring regular maintenance.Stop-and-go-reformers. Maintaining ESW dur-ing breaks in lending enables the Bank torespond quickly to changes in political commit-ment or economic environments. Bulgaria fol-lowed a stop-and-go pattern of policyimplementation from the onset of its transitionin 1989 to mid-1997. The Bank reduced its ana-lytic work and policy dialogue when conditionsin Bulgaria deteriorated and Bank lendingdeclined. This meant that when a new reformistgovernment took over in 1997, the Bank was notfully ready to pick up its part of the dialogue. Theexperience in Bulgaria suggests that the Bankshould find instruments to continue dialogueand promote reforms when not lending. Thereis, in fact, considerable room for innovation ofnonlending assistance. For example, the Bankmight usefully systematize its engagement withclient countries to ensure a stable and reliabletwo-way channel between decision-makers thatwould, if warranted, prepare countries to borrow.No clear intent to lend. In some cases, theBank engaged in substantial nonlending whereit did not have a clear expectation of lending.In West Bank and Gaza, effective nonlendingassistance resulted in a coordinated program offinancial assistance to the territory. The result-ing consultations enabled the Bank to demon-strate its political neutrality and created a networkfor further dialogue in the territory. Greaterresort to this option might be considered incountries with poor policy environments andweak institutions. In addition, clients may onlywant ideas. The Bank should continue to sup-port countries where knowledge, rather thanfinancing, is required.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s3 6Bank assistance to Kenya over the last two decades has beencharacterized by weak government compliance and poor per-formance in the context of dismal economic governance. The out-come of Bank assistance deteriorated in recent years to one ofthe lowest levels in the Bank, and in 1998 lending to the countrywas halted. A complete re-evaluation of Bank assistance toKenya resulted in a landmark assistance strategy in the September1998 CAS. It proposed a primarily nonlending strategy, with lend-ing linked to improvements in economic governance. This moveto nonlending and the focus on governance was highly relevant,particularly as past Bank strategies had continued to push lend-ing in spite of poor portfolio results and under-funded analyticand advisory services. While the focus on economic governanceto trigger lending levels was a first in Bank history, the Kenya CASillustrates a more general trend of strengthened links betweencountry policy performance and Bank lending.This highly relevant and innovative strategy was not fullyimplemented. Initial steps toward reform were rewarded withincreased lending in the absence of real and sustained progressimplementing conditions set out in the Bank strategy. Lendingfurther increased with the approval of emergency operationswhich moved Kenya to a high case lending scenario that,according to the CAS, should have been triggered by substantialreforms. Action plans are no substitute for action in countries withweak or inconsistent track records of reform. Further, experiencein Kenya highlights the need for greater discussion of Bank pol-icy on emergency lending to poor performers. The need foremergency lending stemmed in part from weak enforcementand implementation of past Bank projects in Kenya. In the longrun, emergency lending may undermine the effectiveness ofBank conditionality and may not fit well with institutional devel-opment in poor performers. At a minimum, all unenvisagedlending to poor performers should be accompanied by a one-yearCAS or country progress report, to anchor it within overall Bankstrategy.K e n y a : E n h a n c i n g L e n d i n g S t r a t e g y i n aP o o r P o l i c y E n v i r o n m e n tB o x 4 . 2Risk Management Through GradualEngagement/DisengagementCountry evaluations find that a modulatedresponse to engagement may increase the rel-evance and effectiveness of Bank assistance.Early and extensive pre-lending engagementprovides a sound basis for lending, while thephased graduation of countries through con-tinued nonlending and innovative lendingarrangements allows progress and sustainabilityto be assessed on a timely basis. The highly sat-isfactory outcome of the lending program in ElSalvador was based to a large extent on a grad-uated engagement strategy. The Bank pro-posed a strategy of progressive involvement inorder to deal with the risks inherent in a post-conflict setting following 10 years of disen-gagement. An extensive period of nonlendingduring the peace process was followed bylending for adjustment and social programs.The recent CAE for El Salvador notes that thisapproach allowed the Bank to gauge thegovernments commitment to peace and reformand provided a coherent framework for the gov-ernment and the Bank to establish priorities.Grounding the CAS in recently conducted ESWresulted in good project quality at entry, andcontributed to a highly satisfactory program. Recent country evaluations (and recent events)emphasize that the Bank also needs to refrainfrom abrupt disengagement. There is consider-able demand for Bank involvement in countrieseven after a lending program winds down (e.g.,Chile). The Chile CAE recommends gradual with-drawal from Bank assistance and, given thecountrys susceptibility to external shocks, themaintenance of a nonlending program to mon-itor and to assist the country. The recentlyapproved IBRD Deferred Drawdown Option(DDO) provides one mechanism by which coun-tries can extend their financial engagement withthe Bank. Available to both IBRD and blendcountries to whom the Bank makes a single-tranche adjustment loan, the DDO gives bor-rowers access to long-term IBRD resources tomanage ongoing structural programs if marketborrowing becomes difficult and unforeseenfinancing needs materialize. The DDO expandsthe menu of risk management tools available toclient countries no longer continuously or heav-ily dependent on IBRD financing. How Instruments Contribute to CountryOutcomes Even though country contexts vary dramatically,recent evaluative evidence identifies countrycapacity and borrower commitment as key driv-ers of instrument effectiveness. The evidence sug-gests that the Bank can do more to tailor itsassistance within the context of these two vari-ables. Three specific tools are recommended toimprove instrument performance in all contexts.Enhancing the effectiveness of country strategiesin poorly performing environments is discussedin box 4.3. Country CapacitySimpler project design. Several recent CAEs makesimilar recommendations to improve Bank lend-ing effectiveness. Projects should have simpledesigns even when addressing complex prob-lems. Time frames should be less ambitious andthe Bank should make more realistic assess-ments of implementation capacity. This is par-ticularly important for new borrowers, transitioneconomies, capacity constrained countries, andpoor performers. In the Kyrgyz Republic andLesotho, Bank assistance was less well calibrated.The initial Bank strategy in Kyrgyz emphasizeda flexible learning-by-doing approach, but indi-vidual operations overestimated the govern-ments implementation capacity and commitment.A series of large, ambitious, and complex proj-ects was approved in an uncertain economicenvironment. This led to only a moderately sat-isfactory country outcome. Bank assistance toLesotho was also delivered in the context ofeconomic and political uncertainty. While theassistance program was based on a correct diag-nosis of the countrys problems, the outcome wasalso moderately satisfactory over the 1990 to2001 period, largely as a result of over-ambitiousobjectives which stressed weak government own-ership and stretched implementation capacity.Most projects with unsatisfactory outcomes haddesigns that were too complex in relation tolocal capacity and failed to involve beneficiariesearly in the process. C o u n t r y S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s3 7Tailor nonlending to country characteris-tics. Nonlending activities, like lending, havemore impact when adapted to country charac-teristics. In Lesotho and Kyrgyz, the analyticcapacity in government has in the past been solimited that the impact of Bank country reportswas restricted to a few officials and scholars.The impact of the Banks nonlending activitiesis dampened in countries with ample domes-tic capacity. In India, Bank recommendationswere often lost among competing domesticanalyses of comparable quality. The MexicoCAE recommends that the Bank put into per-spective the relative weight of its financial andadvisory services in countries where advancedin-country analytic capacity exists. This suggestsa need to be far more selective as recom-mended by the Task Force on Middle-IncomeCountries, which emphasized the need to workas much as possible in collaboration with clientsand other partners. In weak institutional envi-ronments, on the other hand, nonlending part-nerships should involve particular attention tohelping build quality home-grown analysis.The recent Chile CAE offers a model for sup-porting countries where knowledge, not financ-ing, is needed. It recommends using shortpolicy notes that inform and guide the decisionof the moment, and more detailed sector workonly if lending operations require it. Whilecost recovery is an option, the full cost pricingpolicy of the Bank has proven to be a signifi-cant obstacle. When charged for, Bank advicemust be of extremely high standards in orderto satisfy demand, and it should be limited toareas where the knowledge market has failedand the Bank has a unique capacity to help itsmember countries.Borrower Commitment and Policy Environment Borrower commitment is the second strongdeterminant of country outcomes. In Bulgaria,the Bank took appropriate action in the face ofweak government commitment. A key element2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s3 8Strengthening the Banks assistance in poorly performing coun-tries hinges on looking at what has and has not worked well.Both lending and nonlending instruments are undermined inpoor policy and institutional environments. For lending, projectoutcomes are closely tied to country policy and institutionalenvironments, as measured by the CPIA ratings. CAEs also sug-gest the quality and impact of nonlending are lower in poor pol-icy environments. These results underscore the need for morestrategic selection of instruments in poorly performing countries. Stand-alone Technical Assistance Loans (TALs) fared betterin low CPIA countries than any other form of lending, due largelyto recent evaluations in ECA, suggesting that the Bank shouldcontinue focusing on capacity and institution building, a criti-cal objective for low-capacity countries. Operational resultsconfirm that major policy reforms supported by adjustment lend-ing had a particularly weak success rate in countries withpoorly committed governments. While true generally for alllending, adjustment lending results are relatively more sensi-tive to country conditions as proxied by CPIA results. Recent CAEsidentify more general enhancements across instruments to raiseeffectiveness in poor performers. For example, instrument designshould be tailored to capacity levels, with a series of simplerinterventions providing better results than a single complexundertaking. Pilot programs also provide a valuable instrumentfor the Bank to test uncertain environments and to build capac-ity. The success of pilot initiatives suggests considerable scopefor their expanded use in poor performers.In terms of nonlending instruments, diagnostics are criticalin poor performers for the assessment of ownership and risks.However, the balance between core diagnostic versus advisorywork should be related to country conditions. Coverage of diag-nostic exercises should be aimed at countries where potentialvulnerability is the greatest. There is also a need for greaterfocus on the nonlending process and product. Remaining engagedin poor policy environments requires, not prescriptive ESW, buta focus on knowledge services that encourage debate and engagestakeholders. Another important lesson is the need to leverageanalytic work conducted by partners. This is particularly impor-tant in Bank budgetary environments constrained by poor per-formance. Drawing on analytic work conducted by partnersallows the Bank to strategically direct its resources to maximizeimpact. Finally, new forms of enclave assistance focused on out-comes may be experimented with, tapping the expertise of pri-vate companies and nongovernmental organizations.G e t t i n g B e t t e r I n s t r u m e n t R e s u l t s i nP o o r l y P e r f o r m i n g C o u n t r i e sB o x 4 . 3of the Banks assistance program was a Finan-cial and Enterprise Sector Adjustment Loan(FESAL). When it became clear that the gov-ernment would not deliver on its commitmentto follow through with reforms the Bank tooka prudent stance and delayed the FESAL. TheBank then focused on supporting institutionalchanges and specific sectoral needs throughinvestment lending. The FESAL was put on holdfor five years until a more appropriate environ-ment existed for the reforms. The Bulgaria CAEnotes that in hindsight, the move to delay FESALwas highly relevant and the project outcome wasrated satisfactory by OED. In Mexico, the Bankapproved two single-tranche SALs for pensionreform. Policy actions were agreed upon andtaken in advance of the approval of each loan.Use of simple single-tranche loans allowed theBank to maintain an appropriately low profileon politically charged issues.The effectiveness of nonlending varies by coun-try policy environment. There is a strong con-nection between nonlending effectiveness andthe quality of country policy environments. Astable 4.2 illustrates, ESW carried out in countrieswith lower average CPIA ratings had less impacton Bank products and on country dialogue thanit did in countries with medium and high CPIAratings. However, the returns of ESW whichlead to policy turnarounds are so high that alower rate of success may be acceptable providedthe activity is carried out with the clear objec-tive of nurturing reform and capacity building.Remaining engaged in poor policy environ-ments also requires a focus on activities beyondprescriptive ESW. The Bank must use the fullrange of its ESW toolkit to encourage debate andengage stakeholders. Innovation in nonlendingapproaches and partnerships should be a keyfeature of risk management with respect to poorperformers and a powerful antidote to unwar-ranted risk aversion. Directions for Effective Instrument Use inCountry Strategies Diagnostics and due diligence. The Banks movetoward a country-based approach and increasedprogrammatic lending is being supported by amovement to standardize analytic coverage ineach client country. Project-level fiduciary con-trols are being complemented with country-leveldiagnostics and capacity-building initiatives. InFY01, management further enhanced the role offiduciary assessments in its operational work byincluding Country Financial Accountability Assess-ments (CFAAs), Country Procurement Assess-ment Reports (CPARs), and the Public ExpenditureReview (PERs), together with Poverty Assess-ments (PAs) and Country Economic Memoranda(CEMs) as part of core diagnostic reports.The Bank could do more to clarify theintended impact and consequences of its coreassessments. The guidelines for the CPAR, CFAA,and PER suggest that findings and recommen-dations are to be fed into the CAS process, butdo not elaborate on how the instruments areexpected to achieve this linkage. There is no con-sensus in the Bank on the minimum fiduciarystandards for borrowing countries. More impor-tantly, the consequences of the fiduciary assess-ment findings for Bank assistance programs areunclear. The Banks policy currently does notrequire fiduciary assessments to be carried outin all borrower countries, and coverage to dateC o u n t r y S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s3 9CPIA ratings Relevance Impact on Bank products Impact on country dialogueLow 4.9 3.7 2.9Medium 4.5 4.2 3.9High 5.4 5.2 5.1Average rating 4.7 4.3 3.9Note: The analysis is on a scale of 16, with above 3.5 representing overall satisfactory rating.Ratings based on a review of ESW quality in 50 countries with CAEs.Q u a l i t y o f E S W b y C o u n t r y P o l i c ya n d I n s t i t u t i o n a l E n v i r o n m e n tT a b l e 4 . 2appears to be based on demand from countryteams, the budget, or available skills. Currentguidelines do not provide criteria for selectingand prioritizing across countries; in particular thevolume of adjustment loans in countries does notappear to be an explicit criterion.Piloting projects allows uncertain environ-ments to be tested. Evidence in the most recentcountry evaluations suggests that on the whole,pilot projects contributed greatly to the effec-tiveness of lending, by building institutionalcapacity or convincing stakeholders of the ben-efit to reform. In Chile, positive results frompilots in health care stimulated public debate andhelped to launch institutional reforms in thesector more generally. In the Kyrgyz Republic,limited Bank experience in the country ham-pered the relevance and effectiveness of theearly lending program. The Kyrgyz CAE notesthat piloting new approaches through LILs mighthave resulted in more realistic perceptions ofgovernment capacity, simpler project designs,and more effective lending. OED evaluations forIndia also reinforce the need for a better under-standing of capacity constraints and the contextof reforms before project preparation. Recom-mendations include using small-scale pilots toprovide insight into the current policy and insti-tutional framework and to test reforms and inno-vations. OED findings in India suggest that Bankteams that adopted intensive policy dialogueand pilot projects improved the relevance of theirinterventions and achieved better quality atentry. Positive experience with pilot investmentlending has resulted in an expanded scope forthe application of LILs.Dissemination and outreach. Recent CAEsshow a mixed quality of dissemination of ESWas a weakness in many Bank programs. Themodest impact of earlier Bank reports in Mex-ico and India was influenced by governmentreticence over the dissemination of externalanalysis and recommendations. Conversely,the payoff from wide dissemination and out-reach is substantial. In Vietnam, the earliesteconomic and sector reports were translatedinto Vietnamese and sold in street kiosks. Sincethe introduction of the CDF in 1999, analyticwork has been produced in closer partnershipwith many stakeholders and dissemination hasbeen extended through workshops and con-ferences. The FY97 Morocco CAE noted that thelarge amount of good-quality ESW produced didnot have an impact, largely because the effortsexcluded building a constituency for the analy-sis and recommendations. But recently there hasbeen a successful change of approach, andthe FY01 CAE notes that Bank efforts to dis-seminate studies have contributed to the pub-lic policy debate.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s4 04 1Sector and ThematicStrategies, Instruments,and OutcomesThe evolution of the Banks corporate priorities is reflected in the recentexpansion of lending for social protection, economic policy, publicsector management, and finance, and in a stronger focus on institu-tional reform. Declines in lending for rural development and education, bycontrast, seem inconsistent with defined priorities and highlight the needfor updated operational strategies in these sectors. Specific investmentloans continue to be the predominant lending instrument, and the use ofnew programmatic lending instruments is growing. Innovations in lendingarrangements aiming to support corporate priorities include sectorwideapproaches and social funds. The Banks crosscutting thematic objectives,such as environmental sustainability and gender, can best be achievedthrough complementary use of both lending and nonlending tools.This chapter reviews recent trends in thelending among sectors in relation to corporatepriorities and addresses the following questions: How is corporate selectivity reflected in sec-toral strategies and lending commitments? How effective have been the choice, combi-nation, and deployment of instrumentsbetween sectors? What instruments are used to support theBanks crosscutting thematic strategies andwith what results?Selectivity Across SectorsThe Banks Strategic Directions give emphasis tofive priorities for corporate advocacy: empower-ment, security, and social inclusion; the investmentclimate; public sector governance; education; andhealth. Recent shifts in the structure of lending arebroadly consistent with these priorities, and withthe defined core competencies of the organi-zation. The biggest sectoral shifts over the lastdecade, as measured by the Banks own sectorclassification,1 are highlighted in figure 5.1.Between FY9092 and FY9901, lending for eco-nomic policy quadrupled, growing to a full one-fifth of total lending. Social protection and publicsector management were other fast-growing sec-tors. Meanwhile, lending for agriculture shrank tojust 9 percent, and lending for electric powerand energy was at one-quarter the earlier level. 55The declines in agricultural and infrastructurelending represent a reaction to policy shifts sincethe late 1980s in response to the disappointingrecord of public agencies in agricultural market-ing and extension and in support of privateinvestment in several sectors, particularly electricpower and energy, oil and gas, industry, andtelecommunications. Together, these four sectorsaccounted for 20 percent of lending in FY9092but only 4 percent in FY9901. In the case ofpower and energy, the decline represents a delib-erate shift by the Bank toward nonlending activ-ity designed to create new regulatory and policyenvironments. Lending for transportation, whichcontinues to depend more heavily on publicinvestment,2 was maintained at 1114 percentthroughout the period, with a shift in the focus oflending from purely infrastructure investment toreform of the institutional framework for betterservice delivery and sustainability of investments. The expansion of lending for social protec-tion, from 1 percent of commitments in FY9092to 9 percent in FY9901, matches the prioritynow given by the Bank to social safety. Similarly,the priority of public sector governance isreflected in the doubling of lending for publicsector management (PSM) to 10 percent of theBanks portfolio, with increasing attention togovernance and anti-corruption issues. Lend-ing in the finance sector increased in the latterhalf of the 1990s, supporting the priority ofimproving the investment climate. Within lend-ing for finance, the balance has shifted from sup-porting the development of individual financialinstitutions and markets to encouraging thereform of financial systems. Rapid growth inthe volume and share of lending for economicpolicy is also consistent with the Banks corecompetencies in economic management andfinancial systems, and with the corporate advo-cacy priority of the investment climate (althoughthe levels of lending for both economic policyand finance also reflect the Banks role in emer-gency lending to countries in crisis).Lending to two of the sectors identified as cor-porate advocacy prioritieshealth and educa-tionhas not grown. The health portfolioremained at about 5 percent of total commit-ments from FY90 to FY01, while lending for edu-cation fell to only 5 percent in FY9901: annualeducation sector commitments dropped sharplyin the last two years to less than half their aver-age for FY9099. This measure does not reflectthe inclusion of health or education compo-nents in multi-sectoral investment or adjustmentoperations. The share of policy conditions2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s4 2M a j o r S h i f t s i n S e c t o r a l C o m m i t m e n t s( F Y 9 0 9 2 t o F Y 9 9 0 1 A p p r o v a l s )F i g u r e 5 . 1Agriculture Energy Social ProtectionPSM Finance Economic Policy02468101214Commitments (US$ Billions)FY9092FY9901applied in adjustment lending to the social sec-tors expanded rapidly in the 1990s to 18 percentin FY9800, with human development account-ing for 15 percent of tranche conditions.The sharp fall in lending for agriculture alsoappears inconsistent with corporate priorities,considering that rural development is one of thecrosscutting themes within the Banks definedcore competencies and is critical to povertyreduction in many client countries. A decreasein some traditional lending areassuch as large-scale irrigation and drainage and agriculturalcredit to large borrowersaccounts for part ofthe decline. Even though a growing share oflending for agricultural operations has beenassigned to other categories such as environment,there is a real decline, given also the relativelysmall share of lending directed to rural devel-opment as a whole. A portfolio review of all sec-tors/categories of lending for FY9900 reportsa decline in rural lending in most regions inFY00, from the average for FY9899, attributedto a fall of 50 percent in expenditure on ESWrelative to lending costs, the high-risk and high-cost image of rural operations, and a decline instaff resources combined with skill shortages.OEDs rural poverty review identifies additionalfactors: reduced demand from borrowers whogive higher priority to social sectors when foodsupplies appear secure; changing incentives forcountry directors faced with pressure to lend atlow cost; and a lack of analytical methodologieswithin the Bank to adequately demonstratepoverty impact and social returns to rural andagriculture investment.3Instrument Choice and Performance Instrument Choice Among SectorsThe recent expansion of adjustment lending inthe Banks portfolio has occurred primarily infour sectorseconomic policy, finance, publicsector management, and social protection. Fig-ure 5.2 shows the distribution of investmentand adjustment commitments in each sector inFY9601. Adjustment operations have been usedonly on a limited scale in agriculture and on avery small scale in education, health, energy, andtransportation. While most investment projectsincorporate technical assistance, significant useS e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s4 3D i f f e r e n t S e c t o r s R e l y o n D i f f e r e n tI n s t r u m e n t G r o u p s ( F Y 9 6 0 1 A p p r o v a l s )F i g u r e 5 . 2Urban Water Trans. Energy Oil & GasEdu. Env. HNP Agr. Soc Prot.PSD Mult. Finance PSM Econ Pol020406080100Share of IDA/IBRD AmountInvestment Adjustment Investment (TAL)of stand-alone Technical Assistance Loans hasoccurred in parallel with adjustment lending. Within investment lending, SILs remain thedominant instrument in all sectors. Some sectors,though, have made significant changes in theiruse of instruments over the 1990s, as sectorstrategies have been enhanced and the newadaptable lending instruments introduced. Theuse of FILs, predominantly in finance and on asmaller scale in agriculture, urban development,and other sectors, has dropped sharply. Theuse of SIMs has also declined in all the sectorswhere their use had been significant, primarilyin transport, education, agriculture, and healththe same sectors that have made most use ofAPLs since their introduction in 1998. Socialprotection has taken the lead in the use of LILs,which accounted for more than 20 percent ofprojects and commitments in the sector inFY9801, followed closely by education.Sector-Specific Issues in Instrument Choice The following section looks at selected sectorsof Bank operation and how the different instru-ments perform in achieving sectoral goals. Wherephysical assets are a significant intermediateobjectiveas in many transport, urban devel-opment, environment, and water projectsSILsare the predominant instrument. Where policyreform or fiscal management is the goal, adjust-ment lending dominates. And when institutionaldevelopment combined with policy reform is animportant objective, both investment and adjust-ment instruments are used. Adjustment loans,often supported by Technical Assistance Loans,have been particularly effective when the IDfocus is on policy and regulatory reform, whereasinvestment operations appear to offer advantageswhen sector conditions require substantial con-sultation and consensus building among multi-ple stakeholder groups and institutions, andwhen piloting, monitoring, and evaluation activ-ities warrant high priority. In the finance sector, the move from invest-ment to adjustment lending (figure 5.3) hasmatched the concurrent shift in the Banks objec-tives, from expanding access to investmentfinance through individual financial institutionsto the reform of financial systems. FinancialIntermediary Loans (FILs) performed poorlybecause of weaknesses in the policy and insti-tutional environment, shifting attention to over-coming these weaknesses. Financial adjustmentloans (denoted as FSALs and including SECALs2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s4 4T h e M o v e t o A d j u s t m e n t L e n d i n gi n F i n a n c e ( F Y 9 0 0 1 A p p r o v a l s )F i g u r e 5 . 31990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 200102004006008001,0001,200IBRD/IDA Lending US$ MillionsFILsFSALNote: Time trends reflect three-year moving averages, and exclude several outlier jumbo loans that were in response to severe financial crises.or finance-focused SALs) have proven moreeffective than FILs to support wide-rangingreforms in the financial sector, and their per-formance has improved (to more than 85 per-cent satisfactory for projects exiting FY9601)since OEDs review of the sector in 1998. Still,FSALs have been more successful in removingdistortions and improving financial infrastructurethan in restructuring institutions or improvingcompetition, and their effectiveness in support-ing the difficult financial restructuring neededafter a crisis is over is still uncertain. To addressthese longer-term issues, the Banks financial sec-tor strategy proposes that loans should be fre-quently monitored (as recommended by OED),4and that vehicles such as technical assistancecomponents, adaptable program lending, andpartnerships with other donors should be usedto maintain continuing dialogue.In the social protection sector, adjustmentloans account for more than half of all commit-ments for the 59 evaluated projects exiting inFY9701. Ninety-two percent of the adjustmentoperations had satisfactory outcome ratings,compared with 84 percent for investment oper-ations in the sector. The recent movement of theBanks clients toward multi-pillar reforms cre-ated the opportunity to focus adjustment loansdirectly on pension reform.5 Client countrieshave called on the Bank to finance the initialexpenditures involved in honoring existing pen-sion commitments, allowing them to divert con-tribution revenue from the public pay-as-you-gosystem to new funded schemes. When well-designed, the reform results in reduced costs tothe government in the future, providing theresources to repay the loan. The first adjustmentloan based purely on pension reform took placein December 1996 to Argentina, quickly fol-lowed by operations in Mexico, Peru, Uruguay,and Kazakhstan. The Mexican loan helpedfinance transition costs and improve the regu-latory framework for the funded pillar. The proj-ect in Uruguay fostered increased efficiencyamong the second pillar pension fund admin-istrators and promoted the development of theprivate securities market.6In the public sector management sector,both investment projects and adjustment loans (thelatter accounting for about one in four operations)have performed above the Bank average. TheBanks strategy for the sector envisages a signif-icant complementary role for programmaticinvestment and adjustment lending in supportingpublic sector reform.7 Long-term institutional con-cerns do not fit easily in a traditional investmentproject with limited scope and the need to dis-burse against actual project expenditures. APLshave been approved or are under considerationfor Ghana, Bolivia, Tanzania, and Zambia, tofacilitate a longer-term focus on institution build-ing and to link disbursements more closely withgovernments needs and with improvements inmonitorable indicators. Traditional adjustmentlending, on the other hand, may focus on systemicinstitutional concerns, but its typically short timeframe and irregular disbursement pattern are notwell-suited to sustained efforts at institution build-ing. The new programmatic adjustment loans areexpected to overcome some of these limitationsby encouraging a longer-term and more system-atic approach to public sector reform through amedium-term program of annual single-trancheoperations.In the agriculture sector, a significant changein instrument use has been the decline in the useof FILs, from 15 percent of projects exiting inFY94 to none in FY0001. Adjustment lendingcontinues to play a role, accounting for 14 per-cent of commitments for projects exiting inFY9701, with 16 of the 19 operations rated sat-isfactory on exitstronger performance on aver-age than for investment lending in the sector.OEDs 1997 review of agriculture SECALs8 foundthat performance had improved in loansapproved from the early 1990s (when the Bankspolicies for the sector shifted from the public-production-and-control model to liberalizationand competitive marketing), with more attentiongiven to borrower ownership and the timeneeded to implement reforms, but that furthermeasures were needed to match the Banksadjustment lending instrument to the long-termneeds of institutional reform. These measuresincluded frontloading of policy reforms, andsystematic nurturing of borrower commitmentthrough nonlending services and participationprior to lending. Investment loans were foundS e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s4 5to have been effective in reform of public expen-diture in the agriculture sector but had rarely cov-ered such comprehensive policy reformprograms. The Banks strategy for rural and agri-cultural development in the ECA Region envis-ages little future use of Sector Adjustment Loans,despite their relatively good outcome ratings: thebulk of lending for policy reform will be throughstructural adjustment operations, with agriculturaland rural reform components, while muchgreater weight will be given to investment lend-ing with particular concern for community-baseddevelopment and poverty reduction.9Adjustment loans have rarely been used in thehealth and education sectors despite the pol-icy constraints often present in these sectors.The mixed performance of a few educationSECALs in the 1990s (five of the nine operationswere rated unsatisfactory at completion) hasbeen an inhibiting factor: the instrument has notbeen used for an education project exiting since1997. The education sector strategy paper envis-ages greater use of the new investment lendinginstruments because of the importance in this sec-tor of process-driven goals that involve manystakeholders and institutions.10 APLs and LILsare permitting more open-ended lending, basedon specific objectives and a long-term develop-ment strategy, and allowing for piloting andinnovation over a shorter time frame.11 Of the twohealth SECALs exiting in the 1990s, one wasrated satisfactory. When Bank investments rep-resent only a small share of health sector spend-ing, their impact depends on leveraging widerchanges in the sector. In this respect, adjust-ment operations may help focus attention on pol-icy dialogue, but they are not good vehicles fortesting new approaches through piloting, for rig-orous monitoring and evaluation to provide evi-dence on what works, or for helping to scale upsuccessful approaches. Nevertheless, there isroom for them, and the Banks strategy envisagesa more significant role for SECALs in health andeducation in some regions in the future. Traditional specific investment lending con-tinues to dominate Bank operations in the watersector. SIMs and APLs have not been used on asignificant scale, although activities in the sectorhave included complex programmatic opera-tions. OEDs review of the sector recommendsgreater use of adaptable lending instruments andthe development of new, cost-effective,performance-based approaches in deployingthese instruments.12 Since 1989, Bank strategyhas focused on facilitating the privatization processin water supply and sanitation. For this purpose,lending operations have been increasingly sup-ported by other instruments, leveraging the Bankscross-country experience, relationships and capac-ity to connect clients with additional sources offinance, technical expertise, and partnerships.The cost dimension of sectoral operations. Atthe sectoral level, country conditions influenceboth the performance and the costs of lendingoperations. Figure 5.4 illustrates costs and out-comes at the sectoral level for selected adjustmentand investment instruments under different coun-try conditions. The figure demonstrates that costsfor investment lending are considerably higheracross all sectors in low CPIA countries. Thehealth, nutrition, and population (HNP) sectordemonstrates considerable movement in termsof cost of lending and outcome between coun-try contexts. HNP investment lending in lowCPIA countries has among the highest costs andlowest outcomes, but cost is halved and outcomeranks among the highest when undertaken inmedium/high CPIA countries. Investment lend-ing in the education sector exhibits an equallystrong movement, with lending costs decreasingdramatically from low to medium/high CPIA andoutcomes improving with country conditions.The costs associated with sectoral interventionsusing adjustment lending are not as sensitive tocountry environment. Figure 5.4 shows that forall sectors the cost of lending for SALs/SECALsonly modestly decreases between low and higherCPIA countries. Country environment does, how-ever, have a strong positive impact on outcomefor adjustment lending in all sectors. The agri-culture sector is the only exception, with costsfor adjustment lending increasing in medium/highCPIA countries, though this is also accompaniedby higher outcome ratings.Combining and Sequencing Instruments Few Bank-funded projects stand alone. Manyare one of a sequence of similar loans and/or are2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s4 6supported by concurrent or overlapping opera-tions with similar or related objectives. Thesequencing of operations, and the synergiesbetween different operations, may be particularlyimportant when project objectives include com-plex institutional reforms, and when the courseof these reforms cannot be clearly predicted.Successful adjustment lending in these conditionsrequires the use of complementary instrumentsto ensure continuity in policy dialogue and thenecessary analytical work over a long period,together with support to piloting and imple-mentation (box 5.1). This was demonstratedeffectively, for example, in Armenias educationsector. The Bank supported the governments far-reaching reform program from 1996 through twokinds of instrumentsa SIL and two SALsnei-ther of which could have been fully effective onS e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s4 7C o s t s a n d O u t c o m e s A r e I n f l u e n c e db y Q u a l i t y o f O p e r a t i n g E n v i r o n m e n tF i g u r e 5 . 4010,00020,00030,00040,00050,0000 20 40 60 80 100Cost per US$1 Million DisbursedPercent Satisfactory Outcome (Weighted by Disbursement)AgrEducEnergyHNPTransUrbanAgrEducEnergyHNPTransUrbanAllAll02,0004,0006,0008,00010,0000 20 40 60 80 100Cost per US$1 Million DisbursedPercent Satisfactory Outcome (Weighted by Disbursement)AgrEcon Multi PSMAgrEcon Multi PSMAll Low-CPIA Countries Medium/High-CPIA CountriesAll Low-CPIA Countries Medium/High-CPIA CountriesInvestmentSILsAdjustmentSALs/SECALsits own. Policy conditionalities in the SALs sup-ported the necessary legal and regulatorychanges, as well as crucial provisions in the edu-cation budget needed to finance the reformsand encourage public support. The SIL, mean-while, supported the Ministry of Educationscapacity for analysis and policy making, decen-tralized school management, and textbookproduction.Even though most sector strategies focus onthe use of lending to achieve sector objectives,nonlending instruments also play a critical role.Most directly, sector analytical work drives theeffectiveness of lending assistance. OEDs sec-tor reviews confirm the role of economic andsector work in gaining government commit-ment and producing good results in lendingoperations. In the water sector, for example,recent economic and sector work has had a sub-stantial positive impact on the design of Bankwater projects. Good analysis and policy dia-logue have been instrumental in gaining gov-ernment commitment and improving policies forthe environment in a number of countries,including China, Mozambique, Costa Rica, India,and Vietnam. The importance of diagnostic ana-lytical work is further enhanced by the CDFframework, which calls for coordination ofdonor support on the basis of comparativeadvantage within a country-owned develop-ment strategy. Yet, OEDs rural poverty studynotes the modest and declining spending onrural ESW, while the environment review findsa decline from the 1990s in both the number ofstudies and budgetary allocations. Innovative Approaches to Sector Assistance:SWAPs and Social Funds The design of Bank interventions is not limitedto the choice of lending instruments. The avail-able instruments can be applied to a wide vari-ety of approaches designed to match specific2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s4 8Two examples from the ECA Region illustrate effective combi-nation of investment and adjustment lending for sectoral reformobjectives.Reform of social protection system in Bulgaria. The BulgariaSocial Protection Adjustment Loan (SPAL), in 1999, supported anambitious and comprehensive program to establish the legalbasis for reform of the pension system, labor markets, and socialassistance. Although the full scope of the program proved unre-alistic and not all the measures were accomplished, the progressmade on pension reform was strong. The success of the SPALin its objectives on reform of the pension system can be attrib-uted in large part to the support provided through an invest-ment project, the Social Insurance Administration Project (SIAP),effective two years prior to the SPAL, at a time when the gov-ernment was committed to reform but consensus was lackingon specific policies. Contrary to usual Bank practice in theregion, this project included no policy conditionality. It focusedsolely on building technical and administrative capacity in thecountryin the National Social Security Institute (NSSI)forpension policy development and implementation. With the ben-efit of high-quality technical assistance through the project,the NSSI was instrumental in developing policy proposals thatwere technically sound and publicly acceptable, allowing quickpassage through parliament of the legislation needed to supportrestructuring of the system.Restructuring of the mining sector in Ukraine. The UkraineCoal Pilot Project, rated highly satisfactory, played a crucial rolein setting in motion a major restructuring program, in combina-tion with the (almost) parallel Coal SECAL. Although the launch-ing of the SECAL did not wait for the results of the pilot project,the SECAL was restructured to incorporate lessons from thepilot. The achievement of the pilot in demonstrating a feasibleprocess for closure of mines was sufficient to trigger an accel-erated mine closure program and thus substantially reduce thedrain on the national budget for operation of uneconomicalmines. Taking a flexible approach (the project was amendedthree times to reallocate resources as priorities changed) inadverse institutional conditions, the investment project focusedon the technical, political, and social issues for mine closure,building understanding and consensus on the need for mineclosures through consultation, information sharing, and socialassessments, and by demonstrating approaches that were tech-nically, financially, environmentally, and socially viable. TheSECAL facilitated acceleration and a scaling-up of the program.C o m b i n i n g I n v e s t m e n t a n d A d j u s t m e n tL e n d i n g f o r I n s t i t u t i o n a l R e f o r mB o x 5 . 1development needs. Sectorwide approaches(SWAPs) and social funds are two approaches thathave assumed growing significance in the Bankslending. They provide mechanisms to coordi-nate the use of donor resources in support,respectively, of sectorwide reforms and small-scalecommunity-level investments. Table 5.1 outlinesthe objectives and core features of each approach.SWAPs aim to achieve long-term improvement insector performance. The prerequisites for suc-cessful use of a SWAP include macroeconomic sta-bility in the recipient country and countryleadership of a sound sector strategy and expen-diture framework. Success also depends criticallyon donor willingness to align and coordinateassistance in support of the government pro-gram, using a joint annual review process. Build-ing the consensus necessary to set up a SWAP istime-consuming, and the anticipated benefits arelong-term. Social funds, in contrast, can be estab-lished to produce visible results rapidly in adverseeconomic and institutional conditions. They maybe appropriate when action is needed urgentlyto address small-scale infrastructure and otherneeds at the community level and existing agen-cies lack the capacity to respond. But they offerlimited long-term impact on sector performanceor wider institutional development and they canbe misused to undermine budget discipline andsector policy standards.SWAPs signify a processnot a program orspecific financial instrument. Bank support toSWAPs, in some 20 countries, has taken theform of SILs, SIMs, and increasingly, APLs. Theapproach was introduced in the early 1990s inresponse to the proliferation of poorly coordi-nated donor-supported projects, weak govern-ment commitment to programs and reforms,and failure to budget adequately for recurrentcosts. The approach has the potential both toreduce aid transaction costs and to strengthencountries management and accountability struc-tures. In practice, SWAPs have had some successin introducing common implementation arrange-ments and pooled donor funding, greater coher-ence in sector policy and planning, enhancedcountry ownership of a long-term strategy andpriority programs, improved allocation of pub-lic resources, and elimination of imbalances incapital and recurrent spending. Reviews of theGhana health program have documentedprogress in program transparency, in repro-ductive health services, and in the quality of basichealth care. Still, no SWAP has yet resulted in allexternal funding being supported through a sin-gle unified donor-coordinated lending instru-ment. Moreover, challenges remain in buildingcountry ownership and capacity beyond centralgovernment, establishing baseline data andagreed indicators for monitoring program per-formance and progress, and getting significantharmonization of donor procedures. Social funds are multi-sectoral and usuallyhave broad, cross-sectoral objectives such asalleviating poverty and strengthening communityorganization. Bank lending to social funds beganin 1989 and has expanded rapidly to some 58countries. Social fund projects create autonomousimplementing agencies to finance small projectsin several sectors, based on proposals submit-ted by local groups. Most social funds were setup as temporary mechanisms to channelresources to needy communities at a time of cri-sis, but virtually all are still in operation, sup-ported by successive Bank SILs and grants frommultiple donors, and have assumed longer-termobjectives. OEDs social funds review13 findsthat the approach has been effective in respond-ing rapidly in emergency situations, mobilizingnongovernment resources, and delivering small-scale infrastructure in poor communities,although performance has varied across sec-tors. Projects have been less successful in achiev-ing consistent improvements in developmentor welfare indicators, or in facilitating institutionaldevelopment.SWAPs and social funds can be used to meetcomplementary sector objectives at national andgrassroots levels although some tension existsbetween the two approaches. A social fund oper-ating on a significant scale in a given sector isnormally channeling funds outside theexpenditure framework that is at the core of asectorwide reform program, and risks under-mining this programeven when steps are takento promote coordination. In Zambia, for exam-ple, the Bank has been supporting a social fundwhich has allocated some 70 percent of its expen-S e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s4 9diture to schools, while at the same time usinga SWAP in the education sector to coordinatefunding from different sources within a strategicframework for improving the quality and effi-ciency of basic education. Zambias Social Invest-ment Fund has proved effective in constructionand rehabilitation of school buildings and inhelping to coordinate activities at the local level.And it has been training district-level educationofficers in community development and self-help techniques, under the auspices of the SWAP.As a temporary institution, it has effectively com-pensated for some of the weaknesses of publicsector reform. But, in substituting for some func-2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s5 0Feature Sectorwide Approaches Social FundsSectoral scopePrimary sectors of operationSectoral objectives Institutional developmentfocusInstitutional design Mechanism for determiningresource allocation Strengths Limitations Lending instrument choiceC o r e F e a t u r e s o f S W A P s a n d S o c i a l F u n d sT a b l e 5 . 1Single sector or subsectorHealth, roads, education, water, agriculture Improve efficiency and quality throughsectorwide reforms, correct balance of capitaland recurrent expenditures, and build publicsector capacity Central and provincial/regional government Integration of management of donor resources ingovernment systemsCommon implementation arrangements. Jointannual reviewSector strategy and sectorwide expenditureframework Government ownership and accountabilityPublic sector ID impact. Increased cohesion insector policy with improved allocation ofexpenditureReduction in aid transactions costsRequires: government ownership of sound sectoralreform program macroeconomic stability agreement among major donors time and resources to reach agreementSILs/ SIMs/ APLs/ SECALsMulti-sectoralEducation, health, water, roadsIncrease access to and quality of basic services,primarily through improvement of small-scaleinfrastructureLocal governments or community-basedorganizationsEnclavesemi-autonomous implementingagency manages funds of multiple donorsaccording to each donors requirements andprocedures, although mostly using a commonOperational ManualCommunity proposalsQuick visible resultsRapid implementation of small infrastructureprojects in difficult conditionsMobilization of nongovernment resources andcapacityDifficulty in meeting technical and institutionalrequirements in multiple sectorsDifficulties of coordination and accountability vis--vis government agenciesLimited ID impact at central levelReliant on resource management capacity ofpublic agencies and/or communities for financingof recurrent costsSeries of SILstions of central and local officials, it has tendedto inhibit development of the permanent insti-tutional capacity and accountability structuresneeded for sustainable sectorwide improvements.OEDs social funds review recommends strongcountry team coordination to ensure consistencyof social fund activities with a countrys sectorreform strategies, as well as the Banks sectoralpolicies and technical standards. It also cautionsagainst allowing lending through social funds todisplace sector policy-intensive initiatives inwhich the Bank has comparative advantage.Optimally, exit strategies for social fund pro-grams can be facilitated by SWAPs. The sectoral focus of SWAPs can be comple-mented and enhanced by the new PRSCs focuson cross-sectoral challenges of poverty reductionand institutional development. In this context,SWAPs provide a means of advancing the sectorreforms and addressing the capacity constraintsfor implementing poverty reduction strategies.The interface between SWAPs and PRSCs needscareful design in the context of a countryspoverty reduction strategy. The implications needto be drawn as the Banks new business modelfor low-income countries is refined.Policies and Actions in Support ofThematic StrategiesThe Bank has enunciated strategies in a num-ber of crosscutting thematic areas. These thematicareas are not sectors in the traditional sense,but represent dimensions that enhance the qual-ity and equity of the development process acrosssectors. To implement these strategies, the Bankseeks to mainstream the operational emphasesthey imply at the sector and country levels bydrawing on a wide range of tools. OED hasrecently reviewed four such thematic strate-giesenvironment, gender, culture, and partic-ipation.14 These reviews have documented theinstitutional constraints to mainstreaming and thedifficulties (often underestimated) of using lend-ing, corporate nonlending activities, and coun-try-level nonlending activities individually andtogether to achieve thematic objectives. Table 5.2summarizes the specific combinations used ineach of the four areas, in which XX denotesstrong use of the activities, X denotes some use,and blank indicates no or negligible use.Of the four strategies reviewed by OED, envi-ronmental sustainability has exploited the widestrange of actions and instruments. Direct lend-S e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s5 1Environment Gender Culture ParticipationTargeted lendingDedicated investment projects XX X XInvestment project components XX X XAdjustment lending components XCorporate nonlending activitiesDo no harm policies (incl. safeguards) XX XDo good policies XX X X XMonitoring and structures for consistent compliancewith policies/guidelines XX XStaff training and capacity building XX X X XGlobal/regional partnerships XX X XCountry-level nonlending activitiesCAS/PRSP X X X XEconomic and sector work (ESW) XX X XCountry-level partnerships X X X XT o o l s t o S u p p o r t T h e m a t i c S t r a t e g i e sT a b l e 5 . 2ing has been particularly effective in improvingnatural resource management. Lending is com-plemented by a strong implementation of cor-porate policies, especially do no harmsafeguard policies. Provisions for monitoringand supporting the implementation of environ-mental safeguards were introduced in fiscal2001, through creation of the Quality Assuranceand Compliance Unit. The environment strategyis also supported by country-level nonlendinginstruments, including National EnvironmentalAction Plans, as well as diagnostic work on spe-cific issues. This not only supports direct Banklending, but also serves to integrate awarenessof environmental issues into national programs.In Morocco, for example, the Banks involvementled to mainstreaming of environmental issues inline ministries. Country environmental assess-ments are increasingly being prepared with theparticipation of local experts and through in-country partnerships with other donors.Leveraging regional and global partner-ships has been a key element of the Banksstrategy for the environment. The Bank hasused its advisory services and convening powerto raise awareness of global environmental con-cerns among members, and it supports multi-country partnerships for sustainable waterresource development, forest issues, and envi-ronment conservation. One example is theCEOs Forum on Forests, launched by PresidentWolfensohn in 1997, to forge a working part-nership between international forest industries,environmental and social development organ-izations, and the Bank. Beyond its own lend-ing, the Bank is a major implementing agencyfor the Montreal Protocol and the Global Envi-ronment Facility which help the institution fur-ther its environment agenda at the local as wellas the global level.Partnerships have also been used with par-ticularly good results in cultural heritage. TheBank has done a limited amount of lending forpreservation of cultural heritage sites, but thestrategy for culture has focused mainly on coor-dinating efforts with other multilateral and bilat-eral donors, and through the Culture Assets forPoverty Reduction Group, on fostering partner-ships with technical agencies, NGOs, govern-ments and foundations, with the objective ofmainstreaming culture in development. All the thematic strategies have effectivelyused analytical work to support thematic objec-tives. Gender assessments at the country levelhave increasingly involved local consultants andpartners and in some cases (as in Ghana,Ethiopia, and Yemen) have relied on participa-tory assessments. These assessments are notmandatory. Where they have been carried out,they have contributed substantially to integrationof Women in Development (WID) and genderissues in the Banks CAS and lending portfolioas in India. In the absence of a timely genderassessment, these issues can be neglected in theCAS. This issue is acknowledged in the recentlycompleted Gender Mainstreaming SSP, whichenvisages that all countries with an active Bankprogram will be covered by a gender assessmentwithin the next three years.15The gender strategy has been less effectivethan the environment strategy in linking cor-porate nonlending activities with lending andnonlending operations. The significant effortsmade in the past to establish internal struc-tures, notably the Gender Sector Board andThematic Groups, were not accompanied by aclear implementation strategy. Policy on genderwas scattered among several different docu-ments, leaving the operating environment unclearand subject to different interpretations. Althoughthe Bank has sponsored sectoral training andknowledge services through its WBI and Gen-der Sector Boardincluding specific training(such as integrating gender considerations inmicro-level economic activities), conferencesand publicationsno systematic training onBank gender policy and implementation strat-egy is available to staff. In fact, a recent exter-nal evaluation found the World Bank to be theleast developed among multilateral organiza-tions in mainstreaming gender goals.16 The newGender Mainstreaming Strategy Paper focusessharply on addressing these weaknesses andaims to clarify accountabilities, costing, fundingmechanisms, and M&E arrangements for imple-mentation across the Bank. An interim Opera-tional Memo has already been made availableto staff.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s5 2Participation is a strategy that has exploitedonly a limited range of instruments, with theresult that the quantity of participation by primarystakeholders has increased substantially in Bank-assisted projects and CAS and PRSP processes,but without a corresponding increase in quality.OEDs participation review notes that the mainactivity has been building staff capacity,through the Sourcebook and Learning Group, anddraws attention to the need for a more system-atic and strategic approach to participation. Par-ticipation, unlike environment and gender, hasbeen viewed as a means of improving the qual-ity and effectiveness of the Banks own opera-tions rather than a development objective initself. Some attention has been given to policiesas a means to advance participation, but on-going work has not been strategically coordinatedor reviewed for quality. Relevant safeguard poli-cies are limited to the issues for indigenous peo-ples and involuntary resettlement, throughenvironmental assessments.Two directions for future work emerge fromOEDs evaluations of thematic initiatives. First,successful mainstreaming requires the strategicuse of a set of instruments as well as clearerRegional accountability and greater networkauthority to achieve results. As the participationstudy showed, staff guidelines and training arenot sufficient. Even in the Banks most maturethematic strategy, the policy framework is farclearer for the do no harm than the do goodaspects of the strategy. For example, the corpo-rate goal of environmental sustainability remainsto be translated into comprehensive operationalguidance and monitorable performance goals. Tomore fully mainstream its thematic goals, theBank should give more attention to monitoringoutcomes; clarifying accountability for promotion/performance on a particular thematic strategy; andcreating processes, operating environments, andincentives that promote internal compliance withthematic strategies. A second challenge going forward concernsthe application of safeguard policies, as animportant instrument in promoting crosscutting,thematic objectives throughout Bank operations.Issues for the future of these policies are beingaddressed by the Bank in a forthcoming paper.The challenge for the Banks do no harm poli-cies is to strike the right balance between com-pliance and results, with serious and independentoversight to ensure that safeguards meet accept-able international standards combined with rea-sonable adaptation to national conditions andpriorities so as to avoid the creation of a risk-averse mentality among Bank managers andstaff. OEDs water review notes, for example, thatsome countries (including Nepal) have found theBanks do no harm policies too demanding andhave turned to alternative sources of finance. TheOED forestry review finds that the Banks strat-egy fostered an overly cautious approach thatwas ineffective in slowing the global rate oftropical forest destruction. A pilot program to har-monize the countrys enabling regulatory envi-ronment with the Banks safeguard policyframework has been proposed for Vietnam.High-Potential Approaches forImplementationTo pursue its sector objectives, the Banks basicbusiness model is lending based on sectorknowledge. New approaches have emerged forfinancing. To implement its thematic strategies,on the other hand, the Bank draws on a widerrange of activities, including country and globalpartnerships. The following broad lessonsemerge for both sector and thematic strategies. Selecting lending instruments to supportinstitutional development. All the Banksinstruments can be effectively used to help buildinstitutional capacity and support reform. Mak-ing the right choice, though, requires clarityabout the operations objectives and good under-standing of country and sector conditions.Adjustment lending, supported as necessary bytechnical assistance, can be effective whennational ownership and consensus on sectorreforms is strong. Programmatic investment lend-ing may be the preferred approach, supportedif relevant by LILs, when the range of stake-holders and institutions involved is large, andwhen there are challenges for consultation, con-sensus building, piloting, monitoring, and eval-uation. More work is needed to assess theeffectiveness of multi-sectoral adjustment instru-ments in addressing sector-specific issues, espe-S e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s5 3cially considering the growing significance ofPRSCs. In this context, potential complementarityand synergy between PRSCs and SWAPs shouldbe exploited. Finally, a combination of adjust-ment lending (supporting relevant budgetaryand regulatory changes) and investment lend-ing (supporting piloting and implementation ofsector reforms) may be the best prescription.Importance of partnerships. The Bankspartnershipsat the global, regional, and coun-try levelshave played an increasingly signifi-cant role, both in building understanding andconsensus on sector strategies within the devel-opment community and in setting the pace anddirection of reform in client countries. The Bankhas used its convening power and knowledgebase to advance dialogue and joint initiatives insupport of thematic objectives. The Banks part-nerships with international and national stake-holders have helped the institution to improvethe relevance of its policies and to build a broadconsensus around some of its thematic priorities.Over the years, the Bank has also built internalcapacity to manage sectoral knowledge andsupport these priorities. At the country level, part-nerships have a crucial role in setting the paceand direction of reform, particularly in largecountries where selectivity is essential to lever-aging the Banks relatively small share in exter-nal assistance. This has been demonstrated in thewater sector, for example, in China, India, Mex-ico, and Brazil. While such partnerships areclearly an essential and valuable instrument ofthe Bank in promoting sector and thematicobjectives, the transaction costs in some casescan be high and more strategic attention may beneeded in the future to cost effectiveness andselectivity in the use of partnerships.Better ESW. The value of economic and sec-tor work in support of sector and thematic objec-tives can be greatly enhanced through attentionto three aspects: timing; institutional assessment;and follow-up through dissemination, consul-tation, and policy dialogue. The timing of diag-nostic studies is crucial to ensure that sector andthematic priorities are reflected in the CAS, ashighlighted in OEDs gender review. This Reviewalso finds that the Bank has been weak in assess-ing and improving the borrowers institutionalframework for gender, thereby reducing theoverall effectiveness of its assistance at the coun-try level. The participation review notes thatlack of follow-up was one of the weakest aspectsof participation in CAS preparation. Follow-upand dissemination of gender assessments havebeen variable. The Gambia, Cote dIvoire, andZambia WID assessments remained internal doc-uments and, in Poland, even womens NGOswere unaware of the in-depth treatment of gen-der issues in the Poverty Assessment. In contrast,the process of preparing gender and WID assess-ments in Yemen and Ecuador, with the govern-ment in the lead and substantial stakeholderparticipation, resulted in increased gender aware-ness and capacity in the country, enhanced therelevance of the reports for the clients, andensured timely dissemination. The environmentreview cites China, Mozambique, and Costa Ricaas examples where substantial progress hasbeen achieved in gaining government commit-ment and improving the design and applicationof a countrys own environmental policiesthrough high-quality analytical work combinedwith sustained policy dialogue. Participation in lending and nonlendingoperations. The effectiveness of lending oper-ations for sector and thematic objectives is influ-enced by the extent and quality of stakeholderparticipation. This has been particularly evidentin the environment and water sectors, whereimproved results have been closely associatedwith participation. Participation has also helpedin the design and implementation of cultureprojects and has been a key element in a num-ber of social fund projects, as well as commu-nity-based sector projects. It has played animportant role in the Banks gender objectives.In Morocco, for example, where the CAS calledfor a strategy note on gender issues to be devel-oped through a participatory process, the Bankplayed a catalytic role by funding consultants andworkshops that brought together governmentinstitutions and womens associations in publicmeetings. Much of the participation, however,has been too limited or rushed to make a dif-ference. OEDs audit on Nepal forestry, forexample, found that lack of sufficient time forbuilding the spirit of community forestry at the2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s5 4grassroots level negatively affected communityforestry in the country. OEDs social funds studyfound that the nature of community participa-tion was sufficient for executing subprojects butnot for building significant community capacity.A shift by the Bank toward a long-term capacity-building approach to participation, as recom-mended in the Participation Review, would takemore time and resources for consultation, train-ing, and technical assistance, and greater depend-ence on local partners who have appropriateexpertise.S e c t o r a n d T h e m a t i c S t r a t e g i e s , I n s t r u m e n t s , a n d O u t c o m e s5 5This Page Intentionally Left Blank5 7Findings and ImplicationsAs the Bank works with partners to tackle the increasingly complexchallenge of development, selectivity has become more important.In the continuing drive to enhance the impact of development assis-tance on poverty and growth, this Review has highlighted areas in whichthe Bank can further increase its development effectivenessby making theright choices in line with country performance and potential, corporatepriorities, and comparative advantage. Continued Gains in PerformanceGiven the adverse operating environment andthe increased external risks associated with thecurrent global economic decline, it is fortunatethat the internal constraints to developmenteffectiveness are being overcome. The StrategicCompact targets of 75 percent satisfactory out-comes for lending and 85 percent satisfactoryquality for nonlending have been met. The lat-est project evaluations point to further qualitygains for lending instruments beyond these tar-gets. There are also solid improvements in thesustainability of project achievements and theirinstitutional development impact. Recent eval-uations show encouraging improvements in theAfrica Region following its internal drive for port-folio improvement. Nonlending instrumentsalso show a broad improvement in quality, asESW becomes more participatory, client-oriented, and result-focused. The improvedcraftsmanship of both financial and nonfinan-cial instruments is an encouraging sign of thepayoff from heightened attention in the Bankto quality and results, supported by strengthenedevaluation.Implications for PolicyAs the quality of individual operations improves,judicious selectivity can further leverage thesegains into country-level development impact.This Review identifies three broad areas wherebetter selectivity can produce even better results. Corporate SelectivityCorporate selectivity defines the institutionsgoals and priorities, in the context of support-ing overall client country priorities, and reflectsthe institutions core competencies and areas ofcomparative advantage. The transmission ofthese goals and priorities to individual countryprograms occurs through the CDF and corporateresource allocation, and through the PRSPs for66low-income countries. These processes are work-ing well, but can be fully leveraged using thePRSP platform for harmonizing country-specificgoals and CAS business plan objectives. Track-ing the results will require improved capacitiesfor monitoring and evaluation both in countriesand within the Bank. The broad framework governing instrumentshas proven flexible, allowing innovations tomeet new challenges. Within the framework ofthe broad instrument groups of investment andadjustment lending, sharper operational guidancefor specific instruments would help countryassistance strategies choose instruments appro-priate to specific objectives and to sector andcountry conditions. Instead of the current vari-ety of OPs, BPs, memos, and websites, a uniformtreatment covering each instruments role, design,and lessons from past performance could con-structively inform instrument choice for countryand sector managers as well as borrowers. Thecurrent update and conversion process underway for the operational policy and guidance onadjustment lending, to be followed by investmentlending, will offer a good opportunity to incor-porate these improvements.Enhancing Country StrategiesMore strategic treatment of instruments in CAS.The CAS is a strong strategic vehicle whosepotential for achieving selectivity has not yetbeen fully tapped. CASs should deal with selec-tion of instruments more directly and transpar-ently. A logical framework should be used (andresults chains specified) to link specific instru-ments with country objectives and the Millen-nium Development Goals as adapted to countryconditions and taking full account of past per-formance. Unanticipated changes to countryprograms should be captured in timely CASupdates and progress reports to provide anagreed-upon strategic context for Bank activities.The CAS needs to leverage catalytic and scaling-up effects among instruments as well. Interac-tions between investment and adjustment, forinstance, can greatly enhance the potentialimpact of individual interventions. The forth-coming CAS retrospective may offer insight onhow to approach these tasks. Partnerships can be better leveraged. To enhancethe effectiveness of country strategies, the Bankneeds to address selectivity of programs in con-sultation with borrowers and other assistancepartners. The CDF pilot countries have broken newground in this area, but continued focus on busi-nesslike partnerships is needed, laying out moreexplicitly shared objectives, distinct accountabil-ities, and reciprocal obligations among partners. Better strategies in poorly performing countries.The Bank is devoting renewed attention toachieving better results in countries with poorpolicy and institutional environments, with acurrent focus on nonlending support. ThisReview presents evaluation findings that aregermane to this difficult task, and may be of useto the Task Force on World Bank Assistance toLow-Income Countries Under Stress. Effective-ness of both lending and nonlending activitiesis undermined in poor policy and institutionalenvironments. However, some instruments per-form better than others in poorly performingcountries. For example, there are cases wheretechnical assistance lending geared to the devel-opment of domestic capacities has been effec-tive. On the other hand, adjustment lendingmay be particularly ill-suited for these countriesuntil they have demonstrated the consensus andthe conviction to reform based on concreteupfront actions. In addition, over-complexity ofprojects when country capacity is limited leadsto excessive risks. More conscious tailoring ofinstruments and partnerships to country con-ditions should bring precious gains in thesedifficult environments. Experimentation withoutcome-based operations and innovative part-nerships with private and voluntary sector organ-izations should be encouraged. The role of nonfinancial activities in poorlyperforming countries deserves special attention.Diagnostic work is especially needed in poor per-formers for the assessment of ownership andrisks, thus; in general, coverage of diagnosticexercises should be aimed at countries wherepotential vulnerability is the greatest. The bal-ance between core diagnostic and advisory workshould be related to country conditions. Further,coordination of fiduciary products should beimproved and more targeted country diagnos-2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s5 8tics toward countries likely to benefit from adjust-ment lending would help the Bank improve itsrisk management.Nonlending activities should be strategicallychosen. The amount and type of nonlendingassistance should be carefully calibrated toenhance Bank lending and improve the relevanceof Bank assistance strategies. Resource allocationto ESW should not be exclusively linked to acountrys lending program because an up-to-date knowledge base is a prerequisite to goodlending even for new or renewed borrowers.Instrument Selection in Support of SectorStrategiesDifferent instruments perform differently in dif-ferent sectors. This should be a consideration forinstrument choice and deliberate experimenta-tion with new instruments and partnerships.Sector strategies must be based on reliableknowledge about what works and what doesnot, to inform choices at the country level.Selecting lending instruments to support sec-tor reforms. With appropriate product design,most Bank instruments can be used to helpbuild institutional capacity and support sectorreform. Making the right choice, though, requiresclarity about the operations objectives and agood understanding of country and sector con-ditions. Adjustment lending, supported as nec-essary by capacity-building assistance, can bemore effective when national ownership andconsensus on sector reforms are strong, whenthe range of stakeholders and institutionsinvolved is relatively small, and when monitor-ing and evaluation arrangements are in place. Insectors where these conditions are largely absent,such as in education and health in some coun-tries, programmatic investment lending may bethe preferred approach, supported where rele-vant by LILs and targeted capacity building. Themulti-sectoral approach has had limited successin addressing sector-specific issues, and thisshould be factored into the use of PRSCs. In thiscontext, potential complementarity and synergybetween PRSCs and SWAPs should be exploited.Finally, a combination of adjustment lending(to support relevant budgetary and regulatorychanges) and investment lending (to supportgradual implementation of sector reforms) maybe the best prescription. This might be pre-ceded by LILs or other small-scale interventionsin collaboration with partners to ensure thatadequate knowledge and commitment are inplace upstream of large-scale lending.Implications for EvaluationThis Annual Review has confirmed the impor-tance of nonfinancial activities. ESW has beena key driver of performance improvements inlending over recent years, as well as being crit-ically important for effective country programs.A strong evaluative framework is needed toensure the continued effectiveness of theseinstruments, building on the current practice byQAG to assess their quality at entry. PRSPs are critical to the Banks country strate-gies and development effectiveness becausethey lay out the pace of achievements of devel-opment outcomes. Monitoring progress onselected indicators is already a strong emphasisin PRSPs. This should be complemented bycomparable attention to systematic programevaluation of PRSP interventions and their results,for example in the upcoming Bank/IMF com-prehensive review of the PRSP approach.As business models, CASs and SSPs can bestrengthened through a more transparent andobjective record of past performance. A strongerindependent and self-evaluation focus would befacilitated by a closer alignment of inputs toresults, using a logical results chain and verifi-able performance indicators. Finally, the Review has presented projectevaluation findings supporting the importanceof instrument selection for development effec-tiveness, and potential gains to developmentimpact from leveraging synergies among instru-ments and their sequencing. Accordingly, coun-try, sector, and thematic evaluations shouldaddress more systematically whether the rightinstruments were used for the developmentgoals selected, and also whether the comple-mentarity of instruments was exploited judi-ciously. In addition, project-level evaluationshould capture synergies from complementaryor sequenced instruments (e.g., adjustment loansand stand-alone technical assistance).F i n d i n g s a n d I m p l i c a t i o n s5 9This Page Intentionally Left Blank6 1This annex describes the various types of lend-ing used by the World Bank. There are twotypes of lending facilitated by the Bank: Invest-ment Lending and Adjustment Lending.INVESTMENT LENDINGSpecific Investment LoansSpecific Investment Loans (SILs) support thecreation, rehabilitation, and maintenance of eco-nomic, social, and institutional infrastructure.In addition, SILs may finance consultant servicesand management and training programs.Sector Investment and Maintenance LoansSector Investment and Maintenance Loans (SIMs)focus on public expenditure programs in partic-ular sectors. They aim to bring sector expenditures,policies, and performance in line with a countrysdevelopment priorities by helping to create anappropriate balance among new capital invest-ments, rehabilitation, reconstruction, and main-tenance. They also help the borrower develop theinstitutional capacity to plan, implement, andmonitor an expenditure or investment program.Adaptable Program LoansAdaptable Program Loans (APLs) provide phasedsupport for long-term development programs.They involve a series of loans that build on thelessons learned from the previous loan(s) inthe series.Learning and Innovation LoansLearning and Innovation Loans (LILs) supportsmall pilot-type investment and capacity-building projects that, if successful, could leadto larger projects that would mainstream thelearning and results of the LIL.Technical Assistance LoansThe Technical Assistance Loan (TAL) is used tobuild institutional capacity in the borrower coun-try. It may focus on organizational arrange-ments, staffing methods, and technical, physical,or financial resources in key agencies.Financial Intermediary LoansFinancial Intermediary Loans (FILs) providelong-term resources to local financial institu-tions to finance real sector investment needs. Thefinancial institutions assume credit risk on eachsubproject.ANNEX A: THE BANKS LENDING INSTRUMENTSANNEXESInvestment Lending Adjustment LendingSpecific Investment Loan (SIL) Structural Adjustment Loan (SAL)Sector Investment and Maintenance Loan (SIM) Sector Adjustment Loan (SECAL)Adaptable Program Loan (APL) Programmatic Structural Adjustment Loan (PSAL)Learning and Innovation Loan (LIL) Special Sector Structural Adjustment Loan (SSAL)Technical Assistance Loan (TAL) Rehabilitation Import Loan (RIL)Financial Intermediary Loan (FIL) Debt Reduction Loan (DRL)Emergency Recovery Loan (ERL) Poverty Reduction Support Credit (PRSC)Deferred Drawdown Option (DDO)Sub-National Adjustment Loan (SNAL)Emergency Recovery LoansEmergency Recovery Loans (ERLs) support therestoration of assets and production levels imme-diately after an extraordinary eventsuch aswar, civil disturbance, or natural disasterthatseriously disrupts a borrowers economy. Theyare also used to strengthen the managementand implementation of reconstruction efforts, andto develop disaster-resilient technology and earlywarning systems to prevent or mitigate theimpact of future emergencies.ADJUSTMENT LENDINGStructural Adjustment LoansThe Structural Adjustment Loan (SAL) supportsreforms that promote growth, efficient use ofresources, and sustainable balance of paymentsover the medium and long term.Sector Adjustment LoansThe Sector Adjustment Loan (SECAL) supportspolicy changes and institutional reforms in aspecific sector.Programmatic Structural Adjustment LoansThe Programmatic Structural Adjustment Loan(PSAL) is provided in the context of a multi-yearframework of phased support for a medium-termgovernment program of policy reforms and insti-tution building.Special Structural Adjustment LoansThe Special Structural Adjustment Loan (SSAL)supports structural and social reforms by cred-itworthy borrowers approaching a possible cri-sis, or already in crisis, and with exceptionalexternal financing needs. These loans help coun-tries to prevent a crisis or, if one occurs, to mit-igate its adverse economic and social impacts.Rehabilitation Import LoansThe Rehabilitation Import Loan (RIL) supportsgovernment policy reform programs aimed atcreating an environment conducive to privatesector investment, where foreign exchange isrequired for urgent rehabilitation of key infra-structure and productive facilities. The focus ison key short-term macroeconomic and sectorpolicy reforms needed to reverse declines ininfrastructure capacity and productive assets.Debt Reduction LoansThe Debt Reduction Loan (DRL) supports gov-ernment policy reform programs aimed at cre-ating an environment conducive to private sectorinvestment, where foreign exchange is requiredfor urgent rehabilitation of key infrastructureand productive facilities. The focus is on keyshort-term macroeconomic and sector policyreforms needed to reverse declines in infra-structure capacity and productive assets.Poverty Reduction Support CreditThe Poverty Reduction Support Credit (PRSC)program is expected to consist of a series of oper-ations, typically two or three, which together sup-port IDA countries medium-term policy andinstitutional reform programs to help implementtheir poverty reduction strategies. Its specificstructure depends on country circumstances,including the objectives and nature of the coun-trys reform program that it supports and the tim-ing of the requirement for assistance.Deferred Drawdown OptionThe Deferred Drawdown Option (DDO) is avail-able to both IBRD and Blend countries to whomthe Bank makes a single-tranche adjustmentloan. The DDO gives borrowers access to long-term IBRD resources to manage ongoingstructural programs if market borrowing becomesdifficult and unforeseen financing needsmaterialize. Sub-National Adjustment Loans The Sub-National Adjustment Loan (SNAL) sup-ports reforms that promote growth, efficient useof resources, and sustainable balance of pay-ments at a sub-national level. * Drawn from World Bank Lending Instruments:Resources for Development Impact, OPCS, July2000.2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s6 26 3ANNEX B: A LIST OF DISCLOSED OED COUNTRY ASSISTANCEEVALUATIONSRegion Country Evaluation FYAfrica (AFR) Ghana 1995, 2000Zambia 1996Cote dIvoire 1998Mozambique 1998Togo 1998Kenya 1998, Malawi 1998Ethiopia 1999Burkina Faso 2000Tanzania 2000Cameroon 2000Uganda 2000Botswana 2001Lesotho 2002East Asia and Pacific (EAP) Philippines 1998Thailand 1998Indonesia 1999Cambodia 1999Papua New Guinea 2000Vietnam 2002South Asia (SAR) Bangladesh 1998Sri Lanka 1999Nepal 1999Maldives 1999India 2001Europe and Central Asia (ECA) Poland 1997Albania 1998Ukraine 1999Azerbaijan 1999Kazakhstan 2001Kyrgyz 2001Bulgaria 20022 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s6 4Region Country Evaluation FYMiddle East and North Africa (MNA) Morocco 1997, 2001Yemen 1999Egypt 2000Latin America and the Caribbean (LCR) Argentina 1996, 2000Bolivia 1998Jamaica 1999Ecuador 1999Uruguay 2000Costa Rica 2000Paraguay 2001Mexico 2001Chile 2001El Salvador 2001Haiti 2002Peru 20026 5ANNEX C: STATISTICAL TABLESExit FY9699 Exit FY0001* Active PortfolioSust %Sust % ID Impact Sust % likely or ID Impact %Projects Share Outcome likely or % sub or Projects Share Outcome likely or better % sub or Projects Share Projects# % % Sat better better APPI # % % Sat better (Old Scale)1 better APPI # % not at RiskSector GroupAgriculture 206 20 67 45 38 6.4 63 16 69 49 43 43 6.5 233 16 92Economic Policy 83 8 77 56 28 6.7 15 4 87 93 80 47 7.3 36 2 86Education 90 9 74 44 28 6.6 34 9 82 75 53 47 7.2 160 11 86Electric Power & Other Energy 67 7 64 47 38 6.4 31 8 58 57 51 52 6.5 80 6 85Environment 24 2 71 54 42 7.0 13 3 69 69 67 46 6.8 95 7 82Finance 66 7 67 56 44 6.7 15 4 80 87 63 67 7.3 59 4 98Health Nutrition & Population 59 6 63 49 31 6.3 26 4 73 56 44 46 6.7 144 10 83Mining 9 1 78 67 78 7.3 3 1 100 67 67 67 8.3 14 1 100Multisector 15 1 73 33 7 6.2 5 1 100 75 67 33 7.8 19 1 95Oil & Gas 22 2 73 64 36 6.8 13 3 62 58 50 38 6.5 18 1 89PSD/Industry 45 4 58 55 36 6.2 20 5 58 63 50 42 6.6 63 4 81Public Sector Management 56 6 84 65 45 7.0 33 9 82 84 79 55 7.2 89 6 85Social Protection 48 5 83 35 42 6.8 25 7 92 86 63 52 7.7 98 7 92Telecommunications & Informatics 18 2 78 72 56 7.4 6 2 100 100 100 83 8.8 12 1 100Transportation 90 9 85 61 53 7.3 48 13 93 87 79 76 8.0 165 11 85Urban Development 55 5 74 44 31 6.4 21 5 90 71 61 48 7.0 76 5 87Water Supply & Sanitation 54 5 55 36 25 5.8 13 3 69 75 50 46 6.7 87 6 84NetworkEnvironmentally & Socially Sustainable Development 228 23 68 46 38 6.4 76 20 69 53 47 43 6.6 328 23 89Finance, Private Sector & Infrastructure 414 41 70 52 41 6.6 168 44 78 75 64 58 7.2 59 4 98Human Development 194 19 73 43 31 6.5 85 22 82 72 53 48 7.2 531 37 87Poverty Reduction & Economic Mgmt. 171 17 79 59 33 6.8 55 14 82 84 76 50 7.2 128 9 86Lending TypeAdjustment 161 16 82 63 38 7.0 39 10 79 84 73 56 7.2 51 4 88Investment 846 84 69 48 37 6.5 345 90 78 70 59 51 7.1 1397 96 87Lending SourceIBRD 488 48 75 58 41 6.8 172 45 78 80 68 58 7.3 648 45 87IDA/blend 519 52 68 43 34 6.4 212 55 78 64 54 47 6.9 800 55 87RegionAfrica 304 30 57 34 31 6.0 111 29 66 51 46 41 6.4 357 25 85East Asia and Pacific 153 15 81 59 43 7.1 57 15 80 71 66 61 7.2 260 18 91Europe and Central Asia 130 13 82 65 45 7.1 90 23 81 81 69 53 7.3 289 20 88Latin America and Caribbean 215 21 81 60 44 7.1 62 16 81 79 66 53 7.3 289 20 88Middle East and North Africa 78 8 71 44 29 6.3 26 7 88 92 58 60 7.6 115 8 83South Asia 127 13 68 51 32 6.4 38 10 89 81 61 63 7.7 138 10 89Income GroupHigh 11 1 70 90 50 7.0 1 0 100 100 100 100 8.3 2 0 100Upper middle 192 19 85 71 49 7.3 63 16 80 86 68 58 7.6 231 16 84Lower middle 301 30 74 56 38 6.7 123 32 85 84 74 61 7.5 510 35 90Lower 503 50 64 38 32 6.2 197 51 72 58 49 44 6.7 705 49 86Grand Total 1007 100 71 50 37 6.6 384 100 78 71 61 52 7.1 1448 100 87Notes: Exit FY denotes the year in which the project leaves the World Banks active portfolio, normally at the end of disbursements. Percentages exclude projects not rated. Sector designations are taken from the World Banks Common Data Stores as of end FY01. Income groups designationsare taken from the World Development Indicators 2001. Active Portfolio data reflect projects active as of July 1, 2001, and is provided by the Quality Assurance Group (QAG). For details on QAGs Projects at Risk indicator please refer to the FY01 Annual Review on Portfolio Performance.1. See endnote number 14 of the ARDE Report for differences between the old and the current sustainability scale.* The data for FY01 exits represent a partial IBRD/IDA lending sample (131 out of 275) and reflect all OED project evaluations through October 15, 2001. The processing of the remainder of the FY01 exits is ongoing, expected to be completed by spring 2002.Outcome, Sustainability, Institutional Development (ID) Impact and Aggregate by VariousDimensions, Weighted by Projects, FY9699 and FY0001* Exits; Projects at Risk,Similarly Disaggregated, for the Active PortfolioT a b l e C . 12 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s6 6Exit FY9699 Exit FY0001* Active PortfolioSust %Sust % ID Impact Sust % likely or ID Impact %Disburse Share Outcome likely or % sub or Disburse Share Outcome likely or better % sub or Disburse Share Disburse$ Millions % % Sat better better APPI $ Millions % % Sat better (Old Scale)1 better APPI $ Millions % not at RiskSector GroupAgriculture 10,544 13 76 55 47 6.7 3,395 12 81 69 52 62 7.1 15,551 15 92Economic Policy 12,947 15 79 64 40 6.9 1,480 5 72 99 88 36 6.7 3,537 3 97Education 5,159 6 78 52 32 6.8 2,017 7 81 84 66 60 7.5 10,469 10 80Electric Power & Other Energy 6,804 8 70 57 43 6.7 3,805 14 63 55 59 59 6.5 10,537 10 87Environment 968 1 72 62 42 7.0 240 1 79 56 75 20 6.5 5,081 5 83Finance 13,478 16 81 74 46 5.5 933 3 96 97 71 85 8.0 4,399 4 89Health Nutrition & Population 3,428 4 79 67 42 6.9 913 3 75 56 46 54 6.6 9,336 9 88Mining 844 1 97 96 62 7.6 619 2 100 51 51 51 7.4 1,113 1 100Multisector 722 1 96 62 25 7.3 541 2 100 83 49 2 3.0 1,548 1 91Oil & Gas 2,080 2 75 81 18 5.9 1,377 5 36 35 30 31 5.9 1,057 1 77PSD/Industry 4,131 5 69 67 47 6.8 1,268 5 81 84 74 76 6.2 2,429 2 73Public Sector Management 3,216 4 93 56 45 6.2 1,286 5 81 86 71 48 7.1 4,269 4 92Social Protection 3,817 5 97 66 46 7.4 2,208 8 63 97 89 41 6.8 4,294 4 90Telecommunications & Informatics 839 1 90 92 79 8.8 762 3 100 100 100 73 8.3 521 1 100Transportation 7,718 9 89 60 50 6.8 4,659 17 97 94 83 86 8.2 18,780 18 88Urban Development 3,875 5 87 58 30 6.9 1,181 4 89 80 56 44 6.9 5,950 6 91Water Supply & Sanitation 3,759 4 54 26 18 5.6 830 3 81 84 59 41 6.9 4,977 5 85NetworkEnvironmentally & Socially Sustainable Development 11,476 14 76 56 46 6.8 3,635 13 81 68 55 59 7.1 20,632 20 90Finance, Private Sector & Infrastructure 42,282 50 77 63 41 6.3 15,238 55 80 75 68 66 7.3 50,666 49 88Human Development 12,099 14 83 60 38 7.0 5,138 19 72 85 72 51 7.0 24,642 24 85Poverty Reduction & Economic Mgmt. 18,475 22 84 64 42 6.9 3,503 13 80 91 79 38 5.9 7,906 8 94Lending TypeAdjustment 31,875 38 87 72 44 6.4 5,714 21 70 86 80 49 7.0 9,368 9 95Investment 52,457 62 75 56 41 6.7 21,800 79 81 76 65 62 7.0 94,478 91 87Lending SourceIBRD 60,710 72 81 65 44 6.5 18,504 67 75 79 71 60 7.0 62,207 60 88IDA/blend 23,622 28 76 54 36 6.7 9,009 33 87 77 66 57 7.2 41,640 40 87RegionAfrica 10,344 12 67 41 39 6.3 3,466 13 73 57 57 32 6.4 14,346 14 83East Asia and Pacific 23,587 28 89 72 74 6.2 5,973 22 85 79 72 49 7.6 28,197 27 93Europe and Central Asia 11,780 14 73 70 42 6.9 6,757 25 61 73 69 40 6.2 15,610 15 84Latin America and Caribbean 20,597 24 86 64 63 7.1 6,173 22 85 86 78 52 7.4 22,499 22 87Middle East and North Africa 5,703 7 74 42 78 6.3 1,412 5 95 97 60 29 7.5 5,642 5 81South Asia 12,320 15 70 57 67 6.4 3,733 14 90 85 68 29 7.6 17,553 17 91Income GroupHigh 319 0 80 94 59 6.2 5 0 100 100 100 100 8.3 25 0 100Upper middle 29,566 35 89 77 51 6.5 6,899 25 88 90 74 68 7.3 25,028 24 85Lower middle 23,751 28 76 65 44 6.9 10,157 37 70 82 79 54 6.9 36,656 35 90Lower 30,695 36 73 45 32 6.4 10,453 38 82 67 56 58 7.0 42,138 41 88Grand Total 84,331 100 79 62 41 6.6 27,514 100 78 78 69 58 7.0 103,846 100 88Notes: Exit FY denotes the year in which the project leaves the World Banks active portfolio, normally at the end of disbursements. Percentages exclude projects not rated. Sector designations are taken from the World Banks Common Data Stores as of end FY01. Income groups designationsare taken from the World Development Indicators 2001. Active Portfolio data reflect projects active as of July 1, 2001, and are provided by the Quality Assurance Group (QAG). For details on QAGs Projects at Risk indicator please refer to the FY01 Annual Review on Portfolio Performance. 1. See endnote number 14 of the ARDE Report for differences between the old and the current sustainability scale.* The data for FY01 exits represent a partial IBRD/IDA lending sample (131 out of 275) and reflect all OED project evaluations through October 15, 2001. The processing of the remainder of the FY01 exits is ongoing, expected to be completed by spring 2002.Outcome, Sustainability, Institutional Development (ID) Impact and Aggregate by VariousDimensions, Weighted by Disbursements, FY9699 and FY0001* Exits; Disbursements atRisk, Similarly Disaggregated, for the Active PortfolioT a b l e C . 26 7ANNEX D: OED EVALUATION METHODOLOGYOEDs evaluation work utilizes methodologyand criteria developed by OED over the past 30years. OED evaluates development interven-tions by assessing their results based on theirown stated objectives. At the project level, OED methodologyfocuses on the outcome, sustainability, and insti-tutional development impact of Bank opera-tions. This approach has been extended tocountry, corporate, sector, thematic, and globalpolicy evaluations by making suitable adjust-ments to the criteria. Evaluation CriteriaOED evaluates outcomes by considering threefactors: Relevance of the interventions objectives inrelation to country needs and institutionalpriorities; Efficacythe extent to which the develop-mental objectives have been (or are expected tobe) achieved; andEfficiencythe extent to which the objectiveshave been (or are expected to be) achievedwithout using more resources than necessary. Inaddition, an economic rate of return of at least10 percent is a benchmark for rating investmentprojects as satisfactory. The assessment of relevance is critical becauseit identifies excessively or inadequately ambitiousobjectives. Combining these three factors, over-all outcome is rated on a 6-point scale, rangingfrom highly satisfactory to highly unsatisfactory(see box). OEDs sustainability measure assesses theresilience to risk of net benefit flows over timeby answering these questions: At the time of eval-uation, what is the resilience to risks of futurenet benefit flows? How sensitive is the inter-vention to changes in the operating environment?Will the intervention continue to produce netbenefits as long as intended, or even longer?How well will the intervention weather shocksand changing circumstances?Highly satisfactory: All relevant developmental objectives are (or are expected to be) achieved and/or exceededefficiently, with no shortcomings. Satisfactory: Most of the relevant development objectives are (or are expected to be) achieved efficientlywith only minor shortcomings. Moderately satisfactory: Most of the major relevant objectives, on balance, are (or are expected to be) met,although significant shortcomings are observed. Moderately unsatisfactory: Many of the major relevant objectives are not (or are not expected to be) met;major shortcomings are observed.Unsatisfactory: Most major relevant objectives are not (or are not expected to be) met and/or most objec-tives are not relevant. Highly unsatisfactory: None of the relevant objectives is (or is expected to be) met and/or objectives are notrelevant.O E D s O u t c o m e R a t i n g S c a l eThe institutional development impactmeasure evaluates the extent to which an inter-vention improves the ability of a country orregion to make more efficient, equitable, and sus-tainable use of its human, financial, and natu-ral resources. Such improvements can derivefrom changes in values, customs, laws and reg-ulations, and organizational mandates. Account-ability, good governance, the rule of law, andthe participation of the civil society and the pri-vate sector are prominent characteristics of aneffective institutional environment.Glossary of Evaluation DocumentsImplementation Completion Report (ICR)[formerly Project Completion Report (PCR)]. TheBank prepares an Implementation CompletionReport for each lending operation it finances. TheICR is prepared at the time of project comple-tion by the staff of the responsible regionaloffice (within six months of the final disburse-ment of the Bank loan). It assesses: (a) thedegree to which the project achieved its devel-opment objectives and outputs as set out in theproject documents; (b) other significant out-comes and impacts; (c) prospects for the proj-ects sustainability; and (d) Bank and borrowerperformance, including compliance with relevantBank safeguard and business policies. It also pro-vides the data and analysis to substantiate theseassessments, and it identifies the lessons learnedfrom implementation. The borrower prepares and provides to theBank its own evaluation report on the projectsexecution and initial operation, its cost and ben-efits, the Banks and borrowers performance,and the extent to which the purposes of the loanwere achieved. The borrowers report is attachedunedited to the ICR. OED evaluates about 250operations each year.Evaluation Summary (ES) [formerly Evalu-ative Memorandum (EVM)]. Once sent to theBoard of Executive Directors, each ICR is eval-uated by OED, which validates or adjusts the rat-ings based on the information provided in thecompletion report and other operational docu-ments. OED summarizes its findings in anevaluation summary (formerly evaluative mem-orandum). This memorandum conveys the OEDratings, comments on the lessons to be drawnand on the quality of the ICR, and suggestswhether the project is a candidate for a PPAR (seebelow). Bank regional staff have an opportunityto review this summary before it is completed.OED enters its findings and ratings in a databaseused for aggregate analysis.Project Performance Assessment Report(PPAR) [formerly Project Performance AuditReport]. OED conducts Project PerformanceAssessments (formerly called Audits) for 25 per-cent of all completed projects. The purpose ofthe Assessment is to validate the findings andaugment the information in the ICR, and toexamine issues and lessons of broad applicability.Some Assessments are intended to serve asbuilding blocks for broad sector studies or Coun-try Assistance Evaluations. They provide inde-pendent, field-based post-completion verificationof a projects implementation and results. Theyincorporate the views of the borrower and mainstakeholders, and analyze the operation in its sec-toral and country context. The operational staffand borrower representatives have an opportu-nity to comment on the draft report. The finalreport is submitted to the Banks Board and iswidely distributed within the Bank and the bor-rowing country. Country Assistance Evaluation (CAE)[formerly Country Assistance Review (CAR)].Country Assistance Evaluations are program eval-uations that concentrate on the impact and devel-opment effectiveness of the Banks countrywideactivities, including nonlending services. Theyevaluate the effectiveness of the Banks countryassistance strategy, taking into account countryconditions, the impact of external factors, and therole of other development agencies and finan-ciers. They also assess the effectiveness of the var-ious instruments of Bank assistance, includinginvestment project lending, adjustment lending,technical assistance, economic and sector work,policy dialogue, and aid coordination. They offerlessons and recommendations for the Banksregional staff, and many are timed to feed intothe design of Country Assistance Strategies. Sector Studies. Sector Studies compare expe-rience across countries, assessing the Banksassociation with a sector over a multi-year time2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s6 8A n n e x e s6 9period covering lending and nonlending workand completed operations, as well as opera-tions that are still being implemented or pre-pared. They seek to review the relevance,efficacy, and efficiency of the totality of theBanks work in a sector.Impact Evaluations. OED Impact Evalua-tions take a second independent look at proj-ects and programs several years after completion,to assess what lasting contributions the Bank hasbeen making to borrowing countries develop-ment. An Impact Evaluation study covers theimpacts on beneficiaries, losers, and other stake-holders in the project or program, preferably withtheir active involvement. These evaluations arealso intended to examine recent projects orthose still undergoing implementation, and toassess the relevance of the current strategy inlight of past experience.Process Studies. Process Studies focus on theBanks business processes to assess their effi-ciency and effectiveness. Examples includereports on aid coordination (included in thisreview), participation in Bank operations, andthe implementation of the IDA1012 replenish-ment agreements.This Page Intentionally Left Blank7 1The Committee on Development Effectiveness(CODE) met on January 14, 2002 to discuss the2001 Annual Review of Development Effective-ness (2001 ARDE). The 2001 ARDE confirms thatthere has been continued progress in the Bankslending performance. Overall operational out-comes improved. The Strategic Compact targetof 75 percent satisfactory outcomes for lendinghas been met. Solid progress has been made insustainability and institutional developmentimpact both for investment and adjustment lend-ing. Improvements were also achieved in thequality of economic and sector work (ESW). CODE commended Management and stafffor the significant gains in the performance ofthe Banks lending portfolio and the quality ofESW. However, members saw no room for com-placency and urged Management to sustain thefocus on quality and to explore ways to encour-age calculated risk-taking and innovation. TheCommittee supported the reports findings andre-affirmed the high value it placed on contin-uing to review the report on an annual basis.The Committee especially focused on the fol-lowing issues:Selectivity and Instrument Choice. TheCommittee appreciated the 2001 ARDE sfocus on the importance of linking instrumentchoice to development objectives. Membersstressed the need to: further rationalize the number of instru-ments and gain a more complete under-standing of the conditions under whicheach performs well; make instrument selection in close con-sultation with the clients; further differentiate between weak per-formers and understand better how theBank can assess and capitalize on bor-rower commitment; and sharpen the costing of each instrument,especially safeguards.In addition, the Committee highlighted theimportance of improving evaluation meth-ods to assess the impact of adjustment oper-ations. One member expressed concernregarding the wide use of adjustment lend-ing in countries with low CPIA ratings in theperiod FY9601. Prior ARDE findings haddisplayed a shift of adjustment lending towardbetter performing countries. Clearly, moreprogress in this direction was needed.Good Quality Outcomes and Risk. TheCommittee was pleased to note the contin-ued upward trend in the Banks perform-ance documented in the report. Yet it wasconcerned about the sustainability of thegains, given the already high outcome levelsand the higher risks in the operating envi-ronment. The Committee members discussedwhether risk-aversion not yet reflected in theOED performance data might be growing.They asked Management to explore, in par-ticular, how incentives for risk-bearing and riskmanagement affect operational choices. Onemember noted the disconnect between thehigh satisfactory outcomes for Bank projectsand the failure of many client countries to gen-erate rapid, equitable growth. Other points raised in the discussion by Com-mittee members:ESW. A member noted that non-lending assis-tance was a critical tool for the Bank toANNEX E: MANAGING DEVELOPMENT EFFECTIVENESS: AN OVERVIEWFROM THE CODE CHAIRPERSONremain engaged in poorly performing coun-tries, as highlighted in the 2001 ARDE. How-ever, he stressed that parallel actions were alsonecessary, including investments to supportpoverty reduction, such as rural develop-ment, education, and agriculture and askedManagement what more could be done toenhance the performance of projects in thesesectors in low-income countries under stress.Management informed the Committee thatplans are in place to expand ESW in IDAcountries, both in the areas of core diagnos-tic and advisory studies. Furthermore, theproposed LICUS strategy was designed todeal with the issues which had been raised. Scope and Timing of the ARDE. Most speak-ers supported the current system of annualreporting as useful in alerting Managementand the Board to performance issues in atimely manner and in identifying innovativemethodologies for improving measurement ofdevelopment objectives, risk, and other rel-evant indicators. One member requested thatthe 2001 ARDE attach a standard Annex out-lining the methodologies used. OED agreedto do so in the disclosed version as well asin future reports.Pieter Stek, Chairman2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s7 2Chapter 11. 20002001 Annual Report on OperationsEvaluation. OED, June 2001.2. Instruments refer to specific forms of theBanks development assistance, and can be clas-sified into two broad groups, financial and non-financial, as expanded on in the next chapter.Financial services include IDA and IBRD lend-ing, guarantees, and grants. Nonfinancial serv-ices include a diverse group of activities includingdiagnostic and advisory economic and sectorwork, research, aid coordination, and other part-nership services.3. Supporting Country Development: WorldBank Role and Instruments in Low- and Middle-Income Countries. Development Committee, Sep-tember 25, 2000.Chapter 21. There is no formal designation in the Bankfor crisis lending. These operations were selectedon the basis of two criteriaaccelerated or veryrapid preparation and meeting immediate bal-ance of payments needs. They cover the 1994Mexico financial crisis, the 199798 East Asiancrisis and the most recent financial crisis inTurkey during 2000. 2. Informal services include products,process tasks, and events such as seminars,workshops, conferences, WBI activities, andpolicy notes. The primary focus of this report ison formal nonfinancial activities.3. The five core diagnostics include PovertyAssessments (PAs), Country Economic Memo-randa (CEMs)/Social and Structural Reviews(SSRs), Country Financial Accountability Assess-ments (CFAAs), Country Procurement AssessmentReports (CPARs), and Public ExpenditureReviews (PERs). Other diagnostics include theFinancial Sector Assessment Program (FSAP),Institutional and Governance Review (IGR), etc.4. The Bank used the Country Profile ofFinancial Accountability (CPFA) extensivelybefore FY98, particularly in the Africa Region.5. These figures do not include diagnosticsundertaken in small borrowers and in those insuspension. The bulk of the CFAAs (17 out of22) were conducted in countries with mediumCPIA environments.Chapter 31. The data for FY01 exits represents a par-tial sample of lending exits (131 out of 275) andreflects all OED project evaluations throughOctober 15, 2001; this partial coverage is notedwith dashed lines in all the figures in thischapter.2. These two operations are the third Struc-tural Adjustment (SAL III) and the Social Pro-tection Adjustment Loan (SPAL), totaling US$1.2billion in disbursements.3. In July 2000, the rating scale for the sus-tainability criterion was changed from a 3-pointscale (Likely, Uncertain, Unlikely) to a 4-pointscale (Highly Likely, Likely, Unlikely, HighlyUnlikely), with the new scale available for proj-ects exiting in FY00 and FY01. The 3-pointscale is still monitored within OED to validatethe longer-term trends. 4. The CFA zone consists today of 14 coun-tries: Benin, Burkina Faso, Cameroon, CentralAfrica Republic, Chad, Congo, Cote dIvoire,Equatorial Guinea, Gabon, Guinea-Bissau, Mali,Niger, Senegal, Togo. For the purpose of thisanalysis, Guinea-Bissau is not included giventhat it joined in 1997 after the CFA devaluation.5. World Bank Assistance to CFA Countries.An Evaluation of Selected Social, Economic andRegional Aspects of the Banks Performance.OED, November 9, 2000. 6. These evaluations cover adjustment oper-ations approved in FY9001 and evaluated byOED.7. This separate review drew from a sampleof projects found to have had unsatisfactory andhighly unsatisfactory outcomes.8. QAG assesses the quality of ESW on fourmain criteria: objectives, scope, and strategic rel-evance; internal quality; presentation; and likelyimpact. It uses a 4-point scale: Highly Satisfac-tory, Satisfactory, Marginal, and Unsatisfactory.Additionally, QAG also rates the Banks inter-nal processes related to ESW: adequacy ofinputs, quality of managerial attention, andother contributions such as client capacity, peerreviews, etc.9. QAGs methodology for assessing analy-sis of implications for the poor is currentlybeing refined.7 3ENDNOTES10. The analysis covers three dimensions:timeliness and relevance of ESW; internal qual-ity (clarity of presentation; practicality of rec-ommendations; dissemination); impact of ESWon Bank products; and impact of the ESW onpolicy dialogue.11. In particular, the review called for betteridentification of objectives and target audiences,improved coverage of institutions and withgreater depth, and a greater emphasis on out-comes and results. In terms of process, it empha-sized the need for better timeliness and greaterconsultation and coordination with all key part-ners in the client country as well as the IMF.Chapter 41. See Annex B for a complete list. The timeperiods covered for the more recent CAEs areprovided in table 4.1.2. See IDAs Partnership for Poverty Reduc-tion (FY94FY00)An Independent Evalua-tion OED, 2000, and the background paper forthis report, entitled Review of the Performance-Based Allocation System, IDA1012. OED, Feb-ruary 14, 2001.3. Enhancing IDAs Performance-Based Allo-cation System. IDA, September 24, 2001. 4. IDAs Partnership for Poverty Reduction.OED, para. 7.12.Chapter 51. This classification allows some overlapbetween categories, which limits the precisionof the analysis. For example, a pension reformproject might be classified as either a financeor a social protection project. In addition, manyprojects include components in other sectors.The classification is based on the primary sec-toral designation of an operation, rather than itscomponents (for investment lending) or con-ditions (for adjustment lending). For example,the water and sanitation portfolio shows adecline in commitments in the second half ofthe 1990s, but this does not necessarily indicatea drop in the overall level of Bank activity inthe sector because of increasing incorporationof W&S components in agriculture, urban devel-opment, and social fund projects. The Bank iscurrently undertaking a re-definition of thesesector and thematic codes, to overcome thesedifficulties and relate tracking of operationalactivities more closely to corporate priorities.Under the new coding system, themes areexpected to be clearly differentiated from sec-tors, and every operation to be coded on bothdimensions, with multiple codes possible foreach operation.2. The proportion of Bank-funded projects inthe transportation sector that included privatesector development components declined from76 percent in 1996 to 48 percent in 2001, in par-allel with the global drop in private investmentin the sector from its peak in 1997.3. Toward Sharpening the Focus on RuralPoverty: A Review of World Bank Experience.OED, October 2001.4. Financial Sector Reform: A Review of WorldBank Assistance. OED, 1998.5. Social Protection Sector Strategy: FromSafety Net to Springboard. HDN, 2001.6. Social Protection Sector Strategy. WorldBank, 2001.7. Reforming Public Institutions and Strength-ening Governance, A World Bank Strategy.November 2000.8. Reforming Agriculture: The World BankGoes to Market. OED, 1996.9. Rural Development Strategy: Eastern Europeand Central Asia. World Bank Technical PaperNo. 484, ECSSD, 200110. Joan Nelson, in her book ReformingHealth and Education (1999), also underlinesthe particular difficulties that are involved in sup-porting complex reforms of health and educa-tion systems (as compared, for example, withpension reform), and which make adjustmentlending more difficult to apply effectively inthese sectors.11. Education Sector Strategy. World Bank,1999.12. Bridging Troubled Waters: Assessing theWater Resources Strategy Since 1993. OED, 2001.13. Social Funds: A Review of World BankExperience. OED, 2001.14. The four OED thematic studies includeIntegrating Gender in World Bank Assistance(2001), which covers the period from 1990 to2000; Cultural Properties in Policy and Practice:2 0 0 1 A n n u a l R e v i e w o f D e v e l o p m e n t E f f e c t i v e n e s s7 4E n d n o t e s7 5A Review of World Bank Experience (2001),which covers the period from 1972 to 2000;Promoting Environmental Sustainability inDevelopmentAn Evaluation of the World BanksPerformance (2001), which covers the periodfrom 1990 to 2000; and a review of participation.15. Integrating Gender into the World BanksWork: A Strategy for Action. World Bank, Janu-ary 2002.16. Evaluation Report 1.99. Christen MichelsenInstitute, Oslo, Royal Ministry of Foreign Affairs,1999.This Page Intentionally Left BlankThis report has drawn on a wide range of WorldBank documents and outside sources. WorldBank sources include OEDs major studies,Country Assistance Evaluations, Project Per-formance Assessment Reports, and ongoingresearch. Buckley, R. 1999. 1998 Annual Review of Devel-opment Effectiveness. No. 19062. World BankOperations Evaluation Department. Wash-ington, D.C.Christen Michelsen Institute. 1999. WID/GenderUnits and the Experiences of Gender Main-streaming in Multilateral Organizations:Knights on White Horses. Evaluation Report1.99. Oslo: Royal Ministry of Foreign Affairs.Devarajan, S., Dollar, D. and Holmgren, T. 2001.Aid and Reform in Africa. World Bank. Wash-ington, D.C.Gautam, Madhur. 2001. 20002001 AnnualReport on Operations Evaluation. No. 22482.World Bank Operations Evaluation Depart-ment. Washington, D.C.Hanna, Nagy. 2000. 1999 Annual Review ofDevelopment Effectiveness. No. 19905. WorldBank Operations Evaluation Department.Washington, D.C.Johnston, Timothy. 2001. 2000 Annual Reviewof Development Effectiveness. No. 21550. WorldBank Operations Evaluation Department.Washington, D.C.Nelson, Joan M. 1999. Reforming Health andEducation: The World Bank, The IDB, andComplex Institutional Change. OverseasDevelopment Council, Washington, D.C.Pitman, Keith. 2001. Bridging Troubled Waters:Assessing the Water Resources Strategy Since1993. World Bank Operations EvaluationDepartment. Washington, D.C.UNCTAD (United Nations Conference on Tradeand Development). 2000. The Least Devel-oped Countries 2000 Report. New York andGeneva.World Bank. 2001a. Social Funds: A Review ofWorld Bank Experience. Operations Evalua-tion Department. Washington, D.C._____. 2001b. Social Protection Sector Strategy:From Safety Net to Springboard. Human Devel-opment Department. Washington, D.C._____. 2001c. Integrating Gender in World BankAssistance. Operations Evaluation Depart-ment. Washington, D.C._____. 2001d. Promoting Environmental Sus-tainability in DevelopmentAn Evaluation ofthe World Banks Performance. OperationsEvaluation Department. Washington, D.C._____. 2001e. Cultural Properties in Policy andPractice: A Review of World Bank Experience.Operations Evaluation Department. Wash-ington, D.C._____. 2000a. Reforming Public Institutions andStrengthening Governance: A World BankStrategy. Poverty Reduction and EconomicManagement Network. Washington, D.C._____. 2000b. World Bank Assistance to CFACountries: An Evaluation of Selected Social,Economic and Regional Aspects of the BanksPerformance. Operations Evaluation Depart-ment. Washington, D.C._____. 1999. Education Sector Strategy. HumanDevelopment Department. Washington, D.C._____. 1999. Reforming Agriculture: The WorldBank Goes to Market. Operations EvaluationDepartment. Washington, D.C.7 7BIBLIOGRAPHY7 8The Operations Evaluation Department (OED), anindependent evaluation unit reporting to the WorldBanks Executive Directors, rates the developmentimpact and performance of all the Banks completedlending operations. Results and recommendations arereported to the Executive Directors and fed back intothe design and implementation of new policies andprojects. In addition to the individual operations andcountry assistance programs, OED evaluates the Bankspolicies and processes.Summaries of studies and the full text of the Prcisand Lessons & Practices can be read on the Internet athttp://www.worldbank.org/html/oedHow To Order OED PublicationsOperations evaluation studies, World Bank discussionpapers, and all other documents are available from theWorld Bank InfoShop.Documents listed with a stock number and price codemay be obtained through the World Banks mail orderservice or from its InfoShop in downtown Washington,D.C. For information on all other documents, contactthe World Bank InfoShop.For more information about this study or OEDsother evaluation work, please contact ElizabethCampbell-Pag or the OED Help Desk.Operations Evaluation DepartmentPartnerships & Knowledge Programs (OEDPK)E-mail: ecampbellpage@worldbank.orgE-mail: eline@worldbank.orgTelephone: (202) 458-4497Facsimile: (202) 522-3200Ordering World Bank PublicationsCustomers in the United States and in territories notserved by any of the Banks publication distributors maysend publication orders to:The World BankP.O. Box 960Herndon, VA 20172-0960Fax: (703) 661-1501Telephone: (703) 661-1580The address for the World Bank publication databaseon the Internet is: http://www.worldbank.org (selectpublications/project info).E-mail: pic@worldbank.orgFax number: (202) 522-1500Telephone number: (202) 458-5454The World Bank InfoShop serves walk-in customersonly. The InfoShop is located at:701 18th Street, NWWashington, DC 20433, USAAll other customers must place their orders through theirlocal distributors.Ordering by e-mailIf you have an established account with the World Bank,you may transmit your order by electronic mail on theInternet to: books@worldbank.org. Please include youraccount number, billing and shipping addresses, the titleand order number, quantity, and unit price for eachitem.OPERATIONS EVALUATION DEPARTMENT PUBLICATIONSStudy Series2001 Annual Review of Development Effectiveness: Making ChoicesAgricultural Extension: The Kenya ExperienceAgricultural Extension and Research: Achievements and Problems in National SystemsBangladesh: Progress Through PartnershipDeveloping Towns and Cities: Lessons from Brazil and the PhilippinesThe Drive to Partnership: Aid Coordination and the World BankFinancial Sector Reform: A Review of World Bank AssistanceFinancing the Global Benefits of Forests: The Banks GEF Portfolio and the 1991 Forest Strategy and Its ImplementationFiscal Management in Adjustment LendingIDAs Partnership for Poverty ReductionIndia: The Challenges of DevelopmentIndia: The Dairy RevolutionInformation Infrastructure: The World Bank Groups ExperienceInvesting in Health: Development Effectiveness in the Health, Nutrition, and Population SectorMainstreaming Gender in World Bank Lending: An UpdateNongovernmental Organizations in World BankSupported Projects: A ReviewPaddy Irrigation and Water Management in Southeast AsiaPoland Country Assistance Review: Partnership in a Transition EconomyPoverty Reduction in the 1990s: An Evaluation of Strategy and PerformancePromoting Environmental Sustainability in DevelopmentReforming Agriculture: The World Bank Goes to MarketSocial Funds: Assessing EffectivenessUganda: Policy, Participation, PeopleThe World Banks Experience with Post-Conflict ReconstructionThe World Banks Forest Strategy: Striking the Right BalanceZambia Country Assistance Review: Turning an Economy AroundEvaluation Country Case SeriesBosnia and Herzegovina: Post-Conflict ReconstructionBrazil: Forests in the Balance: Challenges of Conservation with DevelopmentCameroon: Forest Sector Development in a Difficult Political EconomyChina: From Afforestation to Poverty Alleviation and Natural Forest ManagementCosta Rica: Forest Strategy and the Evolution of Land UseEl Salvador: Post-Conflict ReconstructionIndia: Alleviating Poverty through Forest DevelopmentIndonesia: The Challenges of World Bank Involvement in ForestsUganda: Post-Conflict ReconstructionProceedingsGlobal Public Policies and Programs: Implications for Financing and EvaluationLessons of Fiscal AdjustmentLesson from Urban TransportEvaluating the Gender Impact of World Bank AssistanceEvaluation and Development: The Institutional Dimension (Transaction Publishers)Evaluation and Poverty ReductionMonitoring & Evaluation Capacity Development in AfricaPublic Sector PerformanceThe Critical Role of EvaluationMultilingual EditionsAssessing Development Effectiveness: Evaluation in the World Bank and the International Finance CorporationApprciation de lefficacit du dveloppement :Lvaluation la Banque mondiale et la Socit financire internationaleDeterminar la eficacia de las actividades de desarrollo :La evaluacin en el Banco Mundial y la Corporacin Financiera InternacionalCte dIvoire : Revue de laide de la Banque mondiale au paysPhilippines: From Crisis to OpportunityFilipinas: Crisis y oportunidadesRebuilding the Mozambique Economy: Assessment of a Development PartnershipReconstruir a Economia de Moambiquehttp://www.worldbank.org/oedOED PUBLICATIONSContentsAcknowledgmentsForeword, Prefacio, Avant-ProposExecutive Summary, Resumen, Rsum AnalytiqueAbbreviations and Acronyms1. Making Development Choices2. Instruments of Bank AssistanceFinancial ServicesInvestment, Adjustment, and Other InstrumentsRecent Lending ChoicesNonfinancial ServicesESW Reflecting Corporate PrioritiesClear Instrument Guidance Is Key3. Performance at the Instrument LevelPerformance of Lending AssistancePerformance TrendsDeterminants of SuccessPerformance of Nonlending Assistance4. Country Strategies, Instruments, and OutcomesCorporate Selectivity and Country StrategiesCountry Strategies and Instrument ChoiceDefining Comparative Advantage Depends on PartnersLogical Framework Application in the CASCountry Strategies Are Dominated by LendingCountry Strategies in the Absence of a Lending ProgramRisk Management Through Gradual Engagement/DisengagementHow Instruments Contribute to Country OutcomesCountry CapacityBorrower Commitment and Policy EnvironmentDirections for Effective Instrument Use in Country Strategies5. Sector and Thematic Strategies, Instruments, and OutcomesSelectivity Across SectorsInstrument Choice and PerformanceInstrument Choice Among SectorsSector-Specific Issues in Instrument ChoiceCombining and Sequencing InstrumentsInnovative Approaches to Sector Assistance: SWAPs and Social FundsPolicies and Actions in Support of Thematic StrategiesHigh-Potential Approaches for Implementation6. Findings and ImplicationsContinued Gains in PerformanceImplications for PolicyCorporate SelectivityEnhancing Country StrategiesInstrument Selection in Support of Sector StrategiesImplications for EvaluationAnnexesAnnex A: The Bank's Lending InstrumentsAnnex B: A List of Disclosed OED Country Assistance EvaluationsAnnex C: Statistical TablesTable C.1. Outcome, Sustainability, Institutional Development (ID) Impact and Aggregate by Various Dimensions, Weighted by Projects, FY9699 and FY0001 ExitsTable C.2. Outcome, Sustainability, Institutional Development (ID) Impact and Aggregate by Various Dimensions, Weighted by Disbursements, FY9699 and FY0001 ExistsAnnex D: OED Evaluation MethodologyAnnex E: Managing Development Effectiveness: An Overview from the CODE ChairpersonEndnotesBibliographyBoxesTablesFigures