REPORT on the report from the Commission on the Mid-term Review of Structural Interventions Objectives 1 and 6 (1994-99) (COM(1998)0782 - C4-0032/99) (Delegation of the power of decision - Rule 52 of the Rules of Procedure)
22 April 1999
Committee on Regional Policy
Rapporteur: Jean Baggioni
By letter of {7/01/1999}7 January 1999 the Commission forwarded to Parliament its report on the Mid-term Review of Structural Interventions Objectives 1 and 6 (1994-99)(COM(1998)0782 - C40032/99).
At the sitting of {28/01/1999}28 January 1999 the President of Parliament announced that the Conference of Presidents had referred this report to the Committee on Regional Policy as the committee responsible, with power of decision pursuant to Rule 52 of the Rules of Procedure.
At its meeting of 26 November 1998 the Committee on Regional Policy had appointed Mr Jean Baggioni rapporteur.
It considered the Commission report and the draft report at its meetings of 25 March, 7 and 21 April 1999.
At the last meeting it adopted the draft legislative resolution unanimously.
The following were present for the vote: Miguel Arias Cañete, chairman; Howitt, Decourrière, Napoletano, vice-chairmen; Baggioni, rapporteur; Azzolini, Bontempi (for Darras), Botz, Chichester (for Berend), Crampton, de Lassus, Girão Pereira, Hatzidakis, Klaß, Karamanou, Lage, Marinucci (for Frutos Gama), McCarthy, Myller, Nicholson, Otila, Peijs (for Costa Neves), Rapkay (for Izquierdo Collado), Ryynänen (for Monfils), Schiedermeier (for Viola), Schroedter, Vallvé, Varela Suanzes-Carpegna and Walter.
The report was tabled on 22 April 1999.
A MOTION FOR A RESOLUTION
Report from the Commission on the Mid-term Review of Structural Interventions Objectives 1 and 6 (1994-99)(COM(1998) 782 - C4-0032/99)
The European Parliament,
- having regard to the report from the Commission on the Mid-term Review of Structural Interventions Objectives 1 and 6 (1994-99)(COM(1998) 782 - C4-0032/99),
- having regard to Article 158 of the EC Treaty (former Article 130a of the EC Treaty),
- having regard to the Court of Auditors' Special Report No 15/98 on the assessment of Structural Fund intervention for the 1989-1993 and 1994-1999 periods[1],
- having delegated the power of decision to the Committee on Regional Policy pursuant to Rule 52 of its Rules of Procedure,
- having regard to the report of the Committee on Regional Policy (A4-0217/99),
A. having regard to the requirement of efficiency in the pursuit of the objective of economic and social cohesion,
B. having regard to the major role that must be played by evaluation at every stage of the implementation of Community structural interventions,
C. having regard to the fundamental nature of precise democratic scrutiny of the smooth functioning of the Structural Funds,
D. whereas evaluation may also represent an important source of information for individual citizens as regards the utilisation and local impact of Community aid, and whereas, in this respect, the relevant transparency must be strengthened,
E. whereas mid-term evaluation is provided for in the chapter on implementing provisions included in every CSF and SPD, with a view to strengthening the monitoring of interventions,
F. noting with satisfaction that all the programmes under Objectives 1 and 6 for the period 19941999, amounting to over a hundred, have been the subject of mid-term evaluation,
G. whereas these evaluations, which are in general of a better quality than those carried out during the previous programming period, have enabled the undeniable contribution to economic and social cohesion of the structural interventions to be identified, precisely in many cases, from both the macroeconomic and the microeconomic point of view, in the lessdeveloped regions eligible under Objectives 1 and 6 where the largest volume of Community funding is concentrated,
H. whereas evaluation of the implementation of programmes at three stages - ex ante, mid-term and ex post - is a key element in the revision of the programmes, being based on a strong need for efficiency and transparency,
I. whereas these evaluation exercises make a useful contribution to reinforcing partnership within the monitoring committees, increasing the sense of responsibility of the authorities concerned and strengthening a culture of good management of Community funds,
J. whereas the evaluations, in virtually all cases, have justified the mid-term reprogramming deemed essential to improve the effectiveness of structural interventions by the Community,
K. whereas the Commission gave the assessors an extremely flexible framework within which to define their criteria or indicators,
L. whereas, in so doing, the Commission acknowledges the limited scope of some of the evaluation reports, and whereas that admission casts doubt on the quality of its own report,
Quality of the evaluation criteria and indicators
1. Regrets the limited scope of some of the evaluation reports which addressed only the implementation and management of the programmes rather than their results and impact;
2. Calls on the Commission to indicate which aspects of regional policy need more particularly to be evaluated: additionality, convergence, partnership, concentration, employment, endogenous economic development or social welfare; reiterates, in this connection, that per capita GDP must not be the only standard used to judge the effectiveness of structural interventions;
3. Considers it essential that all necessary measures should be taken to improve the quality of the evaluation criteria in all the respects in which it is currently insufficient, with greater use being made of simulation techniques in order to gauge more accurately the impact of the interventions on GDP, investment, employment, prices and consumption, budget deficits or imports and exports;
4. Takes the view, therefore, that it would be advantageous to agree on a selection of the indicators in order to reduce their number and increase their homogeneity, reliability and readability, thus allowing the added value of the interventions to be evaluated and more accurately compared;
5. Recommends the Commission, therefore, to ensure that the methodological and statistical research carried out under the MEANS programme (Method of Evaluating Measures of a Structural Nature) and in the Regional Policy Technical Evaluation Group is developed with that in mind;
Scope of the evaluations
6. Stresses the need to ensure that the assessors are totally independent and to provide them with the technical and financial resources required to improve the quality of their performance, in accordance with the specific obligations set out in the project specifications;
7. Notes with some concern that all the assessors agree that the impact of the interventions on employment and sustainable development is always less than on growth and calls for systematic account to be taken of this observation by all the programme managers as a priority area of action when the revisions are made;
8. Takes the view that it is undesirable to focus purely on the expenditure incurred when judging the effectiveness of Structural Fund interventions and that, instead, the impact of their social benefits on the economy, and, in particular, on employment, should be measured;
9. Notes, in this connection, that the evaluation of the effects of interventions on employment should not concentrate exclusively on the notion of 'net job creation' but should also take account of the 'number of jobs saved' and the 'number of jobs created indirectly';
10. Encourages the assessors to fine tune in a specific manner the results and impact indicators relating to the structural interventions - in order to optimise their scope and efficiency - with regard to access to basic infrastructure, new technologies and the information society, as well as to environmental protection and promotion of sustainable development, bearing in mind, inter alia, their importance when account is taken of the permanent handicaps of outlying or island areas;
11. Recommends to the Commission that it enhances exchanges of experience between assessment professionals and Structural Fund programme managers in order to achieve a close synergy at the various stages - ex ante, mid-term and ex post - of the evaluations;
12. Urges the Member States which are still failing to do so to ensure optimum financial implementation in future, within the timescale provided for in the programming, without which the mid-term evaluation exercise loses much of its meaning;
Revision of the programmes
13. Calls on the Commission to simplify the revision procedures as far as possible in order to establish a closer link between the production of reliable, usable results and their actual incorporation into the programmes;
14. Notes that reallocations have usually concentrated on how the funds are being spent, giving rise to a transfer of resources from the programmes which are slower to spend their appropriations to those that spend them more rapidly; notes, therefore, that these adjustments may penalise certain programmes which, although they may experience some difficulty in using all of their funds, are nevertheless of the greatest usefulness in the general framework of regional programming;
15. Calls on the Commission, therefore, when carrying out the revisions, to apply the regulations more flexibly to certain programmes or subprogrammes which, although they may have difficulty in using all of their appropriations, are nonetheless crucial to balanced, sustainable regional development, by granting them, as an exception, more favourable cofinancing opportunities;
16. Expects that the Commission will also make available to the EP the evaluators' detailed evaluation reports;
17. Regrets that the Commission report does not give more details concerning certain key elements of the mid-term revision, such as the national and regional political contexts and changing needs since the evaluations cannot be regarded as the only data to be taken into account;
Forthcoming programming period
18. Draws attention to the fact that these strategic analyses are designed to prepare for future Structural Fund interventions since they emphasise the longer-term problems which cannot be addressed during the current programming period; calls, therefore, on the Commission to make optimum use of these data when drawing up future programmes;
19. Notes the significant risk, emphasised in some of the evaluation reports, of a return to slower growth after the completion of certain SPDs or CSFs and calls on the Commission, during the current reform of the Structural Funds, to take all the appropriate measures to restrict that phenomenon to a minimum, with particular regard to the programming of interventions in the regions which have reached the 'phasing-out' stage;
20. Hopes that, at the Mid-Term Review of the forthcoming programming period (2000-2006), the criterion of utilisation of appropriations will not constitute an essential condition for the release of the performance reserve provided for in the Commission proposals and that the decision will instead be tailored in such a way as to give priority to the quality of the programmes under way and the degree to which the initial objectives have been attained;
21. Insists, finally, on the importance which must be attached to the analyses of and lessons learned from the evaluation reports in the forthcoming implementation of the European Spatial Development Perspective (ESDP) in order to ensure the balanced development of the European area which fully integrates the principle of economic, social and territorial cohesion;
22. Instructs its President to forward this resolution to the Council and Commission.
- [1] OJ C 347, 16.11.1998, p. 1.
B EXPLANATORY STATEMENT
The obligation to evaluate Community structural policies was introduced by the 1988 reform of the Structural Funds, as amended in 1993.
Evaluation is defined as an assessment of effectiveness at three levels: the general impact of structural interventions on cohesion (Article 130a of the EC Treaty), on the implementation of the CSFs and the specific impact of each actual intervention.
The report under consideration deals with the mid-term review of structural interventions Objectives 1 and 6 (1994-99). The methods of evaluation will be considered first, followed by an assessment of the effects of evaluation as they emerge from the Commission's report, before a few proposals for the future are put forward to wind up this report.
I. METHODS OF EVALUATION
1. The aim of mid-term reviews
A mid-term review is designed to ensure maximum possible efficiency in the implementation of the CSFs and SPDs. It must be carried out because of the constraints on public spending. In this connection, it becomes increasingly essential to ensure that the appropriations allocated to the Structural Funds are used in full and that the interventions are of high quality.
These mid-term reviews are designed in general terms to identify the state of and trends in regional disparities. More precisely, mid-term reviews enable the requisite changes to be made which will make the programmes more effective, may also enable new priorities such as employment to be tackled and account to be taken of unforeseeable events (such as the emergency measures required to cope with the recent earthquakes in Umbria and the Italian Marches). Evaluation constitutes a major tool for the carrying out of these reviews. Sound management must take account of evaluation: it is key element thereof.
2. The evaluation procedure
All the Member States carried out a mid-term review, i.e. of more than one hundred programmes in connection with Objectives 1 and 6. The Common Guide on Monitoring and Interim Evaluations published by the Commission in 1996 lays down the substance of such evaluations They must include a critical analysis of all of the data collected, indicate to what extent the objectives were being achieved and explain any discrepancies between the actual and expected results of the intervention. In the case of programmes lasting more than three years, a mid-term evaluation is carried out at the end of the third year of implementation.
Responsibility for mid-term evaluations is shared among the Commission, the Member States and the regions concerned[1]. Evaluation costs, borne by the technical assistance budget allocated to each programme, have generally amounted to less than 0.1% of the total cost of the programmes. If it is to be usefully applied, an evaluation presupposes that indicators and quantified objectives laid down in advance have been finalised so that they may continue to apply throughout the duration of the programmes. Mid-term evaluations must also take as their point of reference the ex ante evaluations carried out on the basis of the projects submitted by the Member States.
As the Member States have agreed, it is important to ensure that the evaluation reports are produced by external assessors independent of the authorities responsible for managing the Structural Funds.
On the other hand, it is for the monitoring committees to define the substance of the evaluation reports and, subsequently, to analyse the conclusions thereof in order to use them as the basis for possible proposals for making changes to the programmes.
II. THE EFFECTS OF EVALUATION
The evaluation exercises include conclusions which, where appropriate, may lead to a rescheduling of appropriations.
1. Conclusions
The evaluation reports note a fairly satisfactory degree of financial implementation of the programmes relating to Objectives 1 and 6 (1994-1997) since, by the end of 1997, 64% of the appropriations had been committed and 48% of the payments made. However, with regard to Objective 1, there were serious delays in the implementation of commitments in the Netherlands, France and Belgium (less than 50%) and of payments in Italy and Austria (only 32%). With regard to Objective 6, the outturn in Sweden (32% for commitments and 26% for payments) was much lower than in Finland (57% for commitments and 38% for payments).
As for the conclusions proper, they cover both the macroeconomic and the microeconomic impact of the structural interventions.
From the macroeconomic point of view, the evaluation reports include, for example, data relating to the impact on growth and employment. Accordingly, in 1999 in Spain, the CSF resulted in a 5.1% increase in GDP, i.e. an additional average annual growth of 0.8% of GDP, and a 2.4% increase in employment compared with the reference situation, i.e. without a CSF.
Nevertheless, the Commission acknowledges that it is difficult to compare these figures because the methods used vary from Member State to Member State. The Commission also takes the view that the challenge facing future evaluations of the macroeconomic impact will be to use more developed methodologies which include a section on supply and a section on demand. At present, the latter section is frequently notable by its absence. Similarly, evaluation in small regions causes problems relating to the compilation of regional data at a very fragmented level. According to the model put in place in Hainault, Belgium, the SPD should give rise to an additional annual growth in investment of 0.7% (1994-2005) and an additional annual growth in employment of 0.3% (i.e. 5100 net jobs created in 1999 and 15 800 in 2005).
Microeconomic evaluation consists of an analysis of the actual effectiveness of these interventions. Basic infrastructure is the area where the impact of Community intervention is the most visible and most easily measured. Accordingly, in Spain, CSF resources cover on average more than 25% of national infrastructure. Box 6 on page 16 of the report gives a number of examples of important achievements in Ireland, Greece and Portugal as regards road networks, telecommunications, energy diversification and improvements in water-supply systems.
As regards the productive environment and support for employment, the evaluation reports quote impressive job-creation figures with respect to Hainault, Belgium, to the Länder in the east of Germany and to Italy (see Box 7 on page 17).
Specific evaluations have also been carried out in the field of human resources, i.e. the impact of training programmes on the beneficiaries (in Spain, for example, completion of a training course increases an individual's chances of finding a job to 48%; in Germany, 60% of trainees found or kept a job after six months' training).
Given the specific nature of the programmes and the fact that they are not really comparable, the Commission notes that it is difficult to present in a unified form these microeconomic data, quantification of which is variable depending on the areas of intervention. Similarly, the issue of efficiency (comparison of the financial resources allocated to a programme with the results achieved), is rarely addressed.
The Commission also acknowledges that, despite a number of improvements, the definition and quantification of the indicators are still far from perfect. The Common Guide on Monitoring and Interim Evaluations drawn up by the Commission provided a boost in this field by ensuring greater consistency. In particular, the Guide distinguishes between achievement indicators (e.g. the number of kilometres of road constructed) and results indicators (time saved thanks to a road-building project; the number of people who have successfully completed a training course) and general indicators (social and economic effects, increase in the number of jobs and in GDP).
2. Reprogramming
According to the Commission, evaluations have proved useful to justify a number of reprogramming decisions.
Reprogramming is not necessarily linked to the success or failure of the programmes concerned. It is, after all, always possible to fine tune without changing the major priorities. Quite frequently (for example in Italy and Portugal), changes are accompanied by a transfer of resources from programmes or subprogrammes that are slow to spend their funds to programmes which spend their funds more rapidly. In terms of volume, the major changes affected the CSFs in Italy (EUR 700 million), in Spain (EUR 600 million) and in Greece (EUR 400 million). However, those amounts represent no more than between 2% and 5% of the allocations concerned. On the other hand, fairly small programmes have been subject to radical change: in Hainault (Belgium), 19% of available resources were redirected to assist SMUs. In Flevoland (the Netherlands), transfers of appropriations amounted to 13%. Funds were redirected in this way largely with a view to assisting projects relating to infrastructure, access to the information society, enhancement of human resources and equal opportunities.
Although their quality has been enhanced, evaluations must continue to be improved so that they may make a more telling contribution to the decision-making process. With that in mind, the Commission emphasises the need to establish a new set of indicators which are more suitable for measuring the results and the impact of the programmes. The Commission should submit a paper setting out guidelines on methodology issues, in the knowledge that no single method of evaluation can cover every eventuality but that a wide range of methods is required. From that point of view, a substantive increase in multilateral exchanges of information can only be beneficial. The Technical Evaluation Group, which consists of representatives of the Member States and of the Commission, is a useful forum for the exchange of good practice.
III. PROPOSALS FOR THE FUTURE
On the basis of this report and of Court of Auditors' Special Report No 15/98 on the assessment of Structural Fund intervention for the 1989-1993 and 1994-1999 periods, we may formulate the following proposals for the future with respect to methodology, the scope of evaluation and the role of the assessors:
1. Methodology
A better methodological framework is required. Evaluation methods need to be improved even further so that we may establish the most appropriate models for measuring the social and economic impact of the programmes.
Efforts must also be geared to the establishment of pertinent indicators and the compilation of reliable statistics on which the very validity of those indicators depends. The current plethora (hundreds, in some cases) of more or less pertinent indicators is prejudicial to the quality of the evaluation. We need to apply a smaller number of impact indicators which have been carefully selected (on the basis of regional constraints, of the very variable scale of the CSFs and SPDs, of the various programmes and subprogrammes, and of the heterogeneous nature of the geographical and thematic contexts) and are reliable so that results may be compared more effectively.
2. Scope of the evaluation
In future, the scope of the evaluation needs to be made more precise. The Commission should indicate which aspects of regional policy should be evaluated more particularly: not only per capita GDP but also additionality, convergence, employment. endogenous economic development, impact on the environment, and sustainable development.
3. The role of the assessors
If it is to be useful, evaluation must be genuinely independent. More than is the case at present, the independence of the assessors vis-à-vis the managers and public authorities concerned must be maintained.
The assessors' project specifications should clearly indicate the operational objectives pursued (regional, sectoral and national aspects, and short-term and long-term impact) and the most appropriate methodology.
The improvements made to the various aspects of evaluation will be a guarantee of better comparability. In this way, the evaluation results will be all the more useful for the managers, who will have the task of making any corrections to the direction of the programmes, and to the budgetary authorities concerned, especially the European Parliament, whose task it is to assess the results of structural policy. These improvements to methodology are also indispensable for the reliability of the exchanges of good practice. In general terms, evaluation will be a key element in the success of the reform of the Structural Funds which principally seeks greater efficiency.
- [1] The significance of the role of the regions in the evaluation exercise depends on the degree of decentralisation in each Member State.