Monitoring & Evaluation for Jobs Operations
Jobs M&E Toolkit
Job Quality
Introduction
The ‘Jobs M&E Toolkit’ provides a package of resources for project teams and clients to support mainstreaming the jobs agenda in World Bank Group (WBG) lending operations and beyond.
The Toolkit contains a set of definition and guidance on indicators for key results on jobs, data collection forms and manuals, which are tailored by beneficiary type: individuals and firms.
The availability of measurable indicators should encourage a more systematic assessment of jobs outcomes. Many projects in the WBG portfolio are tackling jobs challenges, but the lack of resources available on jobs measurement has often discouraged teams from articulating links to jobs in their Project Development Objectives (PDOs) and/or results frameworks. Application of defined indicators and standard methodologies will help address accountability for results attributable to WBG and other development partners’ projects upon project completion.
The Jobs M&E Toolkit provides resources to be used throughout the entire project cycle. It is best applied ex-ante in the design of projects and their M&E systems, so that data collection can support implementation progress and reporting from the outset. Regular monitoring and data availability will underpin project completion to assess achievements in job results ex-post. The indicators and data collection forms may also be useful for related mid-term, final or impact evaluations.
The Jobs Group M&E Team is available as a resource for questions on accessing, adapting, pilot testing, and implementing the tools made available through the Jobs M&E Toolkit. Please share your experience and project examples to revise, adapt and refine the Toolkit with Raphaela Karlen and Siv Tokle.
Jobs Indicators
The Jobs M&E Toolkit provides a ‘menu’ of jobs indicators for project teams to select from according to the design of a specific intervention and expected results related to jobs.
The Jobs M&E Toolkit includes indicators to measure key results across: (a) Job Creation; (b) Job Quality; and (c) Job Access. The Jobs M&E Toolkit is aimed for use by Investment Project Financing (IPF), and Program for Results (P4Rs) within the WBG and jobs-related project by any other development partner. The selection of the indicators for a specific operation depends on its Jobs related objectives and the activities the project team plans to implement to achieve them.
For corporate reporting purposes, WBG project teams are encouraged to use the Corporate Results Indicator (CRI) on jobs. The CRI ‘Number of project beneficiaries reached by jobs-focused WBG interventions’ is part of the WBG’s Corporate Scorecard. The use of the CRI is required for all Jobs-related WBG operations of more than $5 million, irrespective of the instrument (Number of project beneficiaries reached by jobs-focused WBG interventions).
The data collection methodology depends on availability of data sources, access to online tools by project beneficiaries and the country context. For the Jobs M&E Toolkit, data may either be collected in field visits by enumerators or self-reported by project beneficiaries through an online survey. The choice of the appropriate methodology depends on the data to be collected and whether project beneficiaries have access and are responsive to online surveys.
The interactive Jobs M&E Toolkit includes the following resources:
| Jobs M&E Toolkit | |
|---|---|
| Project Design | A. Jobs Indicator Definitions and Guidance – for use at the PDO and/or intermediate level in Results Framework Includes standard definition, unit of measure and guidance on baseline for each Jobs Indicator; guidance on the types of projects in which each Jobs Indicator may be used; and examples of WBG projects where the Jobs Indicators (or those similar in nature) are used. The Definition and Guidance for each Indicator may be accessed through the menu of indicators. |
| Project Implementation | B. Jobs Data Collection Forms – are short administrative forms for reporting on Jobs Indicators. Each form provides a comprehensive set of questions linked to the Jobs indicators per beneficiary types – Individuals and Firms. Each Data Collection Form has a Data Processing and Aggregation Table providing assistance on calculating the value of the selected Jobs Indicators based on the data collected.Data Collection Form – Individual Beneficiaries Data Collection Form – Firm Beneficiaries C. Jobs Data Collection Manuals for enumerators include Guidance on the use of the data collection forms when administering in the field among project beneficiaries – by beneficiary type: D. Template of Terms of References for the recruitment of a consultant / firm to implement the Jobs Data Collection Forms, which may be adapted by TTLs and the PIU for a given project and country. |
Jobs Outcomes
Key outcomes on jobs are grouped under Job Creation, Job Quality, and Job Access. Outcomes and related indicators may be included as results at the PDO-level, or as intermediate results, depending on project design and objectives. Figure 1 below illustrates these three dimensions of Jobs Outcomes, and includes brief descriptions of sub-outcomes (Job M&E Toolkit Definition of Jobs Outcomes) that tend to be most relevant at the PDO level.
Job Creation
Job Creation
Direct or indirect jobs
Short-or long-term jobs
New Enterprises
Creation of new firms, often through interventions making it easier to do business, such as access finance.
Entrepreneurs / self-employed
Existing and new enterprise owners who employ at least one non-family worker, farmers, and self-employed.
Job Access
Labor Force Participation
Extent to which population is economically active (employed or actively looking for work)
Working of Labor Market
Functioning of labor market in balancing supply of labor (workers) and demand for labor (employers)
Access and Opportunity for Jobs
Access to employment opportunity for disadvantaged groups (e.g. women, youth, bottom 40 percent)
Job Quality
Worker productivity
Ratio of volume measure of output (GDP or gross value added) and labor input use (total hours worked on total employment)
Working conditions and benefits
Physical work conditions
Social insurance / protection
Earnings / livelihoods
Improved income or savings from livelihood / income generation activities
Job Creation: Outcomes relate to the creation of more jobs for project target beneficiaries at an individual level or within firms. Jobs may be created by private sector firms, either by new firm entry, or by growth and expansion of existing firms leading to an increase in hired labor. While these jobs often cover wage employment, another outcome refers to an increase in the number of entrepreneurs and self-employed.
Job Access: Outcomes relate to connecting people to jobs from inactivity or unemployment; helping people move from one job to another; and improving performance of the labor market. Asking the question “Jobs for whom?” allows targeting of specific vulnerable or traditionally disadvantaged groups in the labor market, especially women, youth, and the bottom 40 percent of households. Targeting can also be spatial – for example creating jobs in a rural, lagging region, or in fragile or vulnerable settings.
Job Quality: Outcomes focus on the ability of beneficiaries to achieve better labor market results, through enhanced productivity levels, improved working conditions, and higher earnings. Many of the poor already have jobs, but these may be poor quality jobs in low productivity, low income activities. These types of results are therefore especially relevant for projects targeting the informal sector, and livelihoods projects.
Intermediate outcomes are facilitating factors that contribute to achieving the final outcomes towards more, better, and inclusive jobs. These include: Access to/Working of Product Markets, Firm performance/investment, Improved Human Capital.
Definitions
A “Job” is defined as “activities that generate actual income, monetary or in kind, and that do not violate fundamental rights and principles at work” (WDR 2013). A job is often associated with formal wage employment. Yet, other types of jobs including informal wage employment, farm-related work or own-account work are prevalent in middle- and low-income countries.
A “Jobs Operation” is a World Bank lending activity or project by any other development partner that has an explicitly stated and substantive link to creating jobs, improving the quality of existing jobs, and helping individuals connect to jobs or move to better jobs. This means that activities are selected or designed with a focus on jobs or on addressing a jobs challenge.
Beneficiaries of Jobs Operations are most often Individuals and Firms. They may be direct or indirect (e.g. in infrastructure projects) beneficiaries of projects.
Individual beneficiaries include project participants who seek a job or aspire to improve their employability or earnings, subsistence farmers, and self-employed individuals. Interventions targeting individual beneficiaries include Labor Programs such as public employment services, wage subsidies and public works, as well as interventions promoting sustainable livelihoods and income-generating activities. In certain circumstances, households can be captured as individual beneficiaries.
Firm beneficiaries refer to the entity directly participating in the project intervention. The term “firm” in this toolkit refers to the establishment: an establishment may be part of a larger firm, but data collected under the projects must correspond to the specific establishment receiving support/targeted by the intervention. If a project seeks data at the establishment level or many establishments of the same firm, then data must be aggregated for reporting on indicators. Ideally, project results reporting should be consistent with the Enterprise Survey (ES) methodology. The ES defines ‘firms’ as business organizations composed of one or more establishments that are specified under common ownership or control; and ‘establishments’ as the physical location where business is conducted or where services or operations are performed. TTLs may need to adapt the terminology most relevant for the country, legal, and beneficiary specific context of the project. Further guidance for the PIU and data collection firm is provided in the Jobs M&E Toolkit.
Interventions targeting firm beneficiaries generally provide firms or entrepreneurs with productive assets such as access to finance or securing access to land, in addition to direct investments and technical assistance. Many interventions at the firm-level also aim at fostering small scale entrepreneurship, and SME development.