External, Monitoring and Evaluation
The Project Management Unit (PMU) will monitor and supervise project implementation according to the project implementation arrangements designed for the project.
The project implementation plan (PIP) envisages external monitoring and evaluation of NDP I by external agencies in addition to internal monitoring and evaluation. The services of Development and Research Services (P) Ltd., New Delhi has been hired by PMU for external monitoring and evaluation of NDP I.
The main objectives of the consultancy are to conduct comprehensive and annual surveys for collecting data on various outcome indicators, and to create a database for project monitoring and evaluation.
The comprehensive surveys include; (a) baseline survey before the inception of the project, (b) mid-term survey in year 3, and (c) end of project survey in year 6. The annual surveys will be undertaken in year 1, 2, 4, and 5 of the project. The comprehensive surveys - baseline, mid-term, and end of project - will cover a large sample size, while the annual surveys will be conducted over a sub-sample of the comprehensive surveys.
Data collected through the surveys would be used for the following purposes:
- To generate baseline data for planning and project design.
- To improve decision making process by following the action plan developed for project implementation on the basis of survey.
- To identify key indicators, methods of measuring them and give baseline figures against the indicators to enable an objective assessment of impact of the project at regular intervals and end of the project.
- To assess social and environmental performance outcomes, including any potential adverse impacts resulting from the projects interventions.
Main Tasks of the Consultancy
The selected consultant will be responsible for the following activities, in collaboration with PMU:
(a) To develop a sample frame and sampling methodology for the surveys. This would involve the following tasks:
- Recommending a methodology for sampling, developing the sample frame, and selecting the sample. The sample design should ensure an adequate sample size to detect reasonably small changes in the indicators in each participating state and across other classifications of the project area. The final sample and details of the statistical methodology used to select the sample need to be cleared by PMU.
- The sample design and sample size determination should rely on good secondary data on PDO indicators and other variables (including data on number of dairy animals owned, milk production, selling markets etc).
- The selected firm will develop sample weights and provide documentation on the methodology used to develop the weights.
(b) To develop survey questionnaires and other documents for guiding field surveys.
- The Consultant will be responsible for revising and translating draft questionnaires to ensure they are adequate to capture data on all indicators, including (i) PDO indicators, (ii) intermediate outcome indicators, and (iii) indicators on characteristics of dairy production in India.
- Revisions of the questionnaire shall be made in consultation with PMU and the final version of the questionnaire should be cleared by PMU.
- The Consultant will be responsible for preparing all support documentation including coding guides, interviewer and supervisor manuals, and the data entry manual. Hard copies and electronic versions of all documentation would to be provided to PMU.
- The Consultant will also be responsible for pilot-testing the questionnaires, implementing the surveys, field data collection and supervision, data entry, and data cleaning. The arrangements for these survey processes would be discussed and agreed with the PMU before the start of each survey.
- As part of data collection, each household will be positioned using a GPS (Geographical Positioning System) device.
(c) To assess compliance with Environment and Social Management Framework. The assessment would be carried during mid-term and end of project survey rounds.
(d) To assess compliance of Governance and Accountability Action Plan (GAAP) framework developed for the project. The assessment would be carried during mid-term and end of project survey rounds.
(e) Capacity building. Since the assignment requires long-term involvement, it is imperative that the consultant earmarks dedicated professionals for this assignment to work closely with the PMU with a view to sharing knowledge and their capacity building.
Outputs of the Consultancy
- Inception report: The report will describe methodology for conducting the survey, data collection tools such as formats and questionnaires, guidelines for survey, list of sampled locations, and procedures for data collection.
- Interim report: This report will describe the following; methodology issues, progress achieved in data collection and other survey processes (e.g. data entry), problems encountered and how they were resolved.
- Electronic survey data: Once data collection, data entry, and data cleaning activities are completed, the consultant will provide validated data in SPSS/ STATA/SAS format and with a proper data dictionary.
- Draft report: The report will include summary of data on all indicators as stipulated, together with findings of data analysis. The report shall be developed in a clear, logical, and readable manner.
- Final report: A draft final report shall incorporate all comments on the draft report.
In addition to Development and Research Services (P) Ltd., New Delhi; the services of a few more specialized agencies would be hired by PMU for conducting specialized studies like Methane emission reduction studies, waste management studies etc. to monitor and evaluate the respective indicators of the project.
Review of Outputs of the Consultancy
The Inception Report, Draft Report and Final Report will be reviewed jointly by PMU and the World Bank. The report will also be subject to any other review mechanism in place for NDP I.