SDMX Metadata Concepts

Download PDF

0.a. Goal

Goal 16: Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels

0.b. Target

Target 16.3: Promote the rule of law at the national and international levels and ensure equal access to justice for all

0.c. Indicator

Indicator 16.3.3: Proportion of the population who have experienced a dispute in the past two years and who accessed a formal or informal dispute resolution mechanism, by type of mechanism

0.d. Series

Applies to all series

0.e. Metadata update

2023-03-31

0.g. International organisations(s) responsible for global monitoring

United Nations Development Programme (UNDP), United Nations Office on Drugs and Crime (UNODC) and Organization for Economic Cooperation and Development (OECD)

1.a. Organisation

United Nations Development Programme (UNDP), United Nations Office on Drugs and Crime (UNODC) and Organization for Economic Cooperation and Development (OECD)

2.a. Definition and concepts

Definition:

Number of persons who experienced a dispute during the past two years who accessed a formal or informal dispute resolution mechanism, as a percentage of all those who experienced a dispute in the past two years, by type of mechanism.

Concepts:

A dispute can be understood as a justiciable problem between individuals or between individual(s) and an entity. Justiciable problems can be seen as the ones giving rise to legal issues, whether or not the problems are perceived as being “legal” by those who face them, and whether or not any legal action was taken as a result of the problem.[1]

Categories of disputes can vary between countries depending on social, economic, political, legal, institutional and cultural factors. There are, however, a number of categories that have broad applicability across countries, such as problems or disputes related to:[2]

  • Land or buying and selling property
  • Family and relationship break-ups
  • Injuries or illnesses caused by an intentional or unintentional act or omission of another person or entity
  • Occupation/employment
  • Commercial transactions (including defective or undelivered goods or services)
  • Government and public services (including abuse by public officials)
  • Government payments
  • Housing (Tenancy and landlord)
  • Debt, damage compensation, and other financial matters
  • Environmental damage (land or water pollution, waste dumping, etc.)

Dispute resolution mechanisms vary across countries around the world. While in many countries courts represent the main institution dealing with disputes of civil nature, the same may not be true in countries or societies where the first point of reference in such cases are informal systems, traditional or religious leaders. The formulation of the indicator, and the formulation of the questions in the survey, have to account for these differences and make sure to include all relevant institutions or mechanisms that are generally recognized and used.

A list of dispute resolution mechanisms could include:

  • Lawyer or third-party mediation
  • Community or religious leaders or other customary law mechanisms
  • A court or tribunal
  • The police
  • A government office or other formal designated authority or agency
  • Other formal complaints or appeal procedure

To improve the accuracy of the indicator it is important to define precisely the denominator (the population at ‘risk’ of experiencing the event of interest, i.e. accessing a dispute-resolution mechanism) by identifying the ‘demand’ of dispute resolution mechanisms. This demand is composed of those who use dispute resolution mechanisms (users) and those who - despite needing them - do not have “access” to such mechanisms for various reasons such as lack of knowledge on how to access them, lack of trust in institutions, lack of legal advice/assistance, lack of awareness about justice mechanisms, geographical distance or financial costs, to mention a few. It is important to exclude from the demand those who experience disputes and do not turn to dispute resolution mechanisms because they do not need them (voluntarily self-excluded). This refers to cases where the dispute is simple or when respondents solve the issue with the other party through direct negotiation.

1

Genn, G, Paths to Justice: What People Do and Think About Going to Law (Oxford: Hart, 1999), 12. ↑

2.b. Unit of measure

Percent (%)

2.c. Classifications

Not applicable

3.a. Data sources

The Indicator is based on four questions to be included in a household survey. The four questions can be part of an add-on access to justice survey module, to be incorporated into other ongoing general population surveys (such as surveys on crime victimization, corruption, governance, quality of life, public attitudes or surveys on other topics) or be part of dedicated surveys on access to justice and legal needs.

Data should be collected as part of a nationally representative probability sample of adult population residing in the country, irrespective of legal residence status. The sampling frame and sample design should ensure that results can be disaggregated at sub-national level. The sample size should be sufficiently large to capture relevant events and compute needed disaggregation.

3.b. Data collection method

  • Data are collected by the United Nations Development Programme (UNDP) through a standardised questionnaire sent to countries. This questionnaire provides specific definitions of data to be collected and it collects a set of metadata to identify possible discrepancies from standard definitions and to assess overall data quality (e.g. sample size, target population, agency responsible for the data collection, etc.).
  • Data for multiple years are collected to assess data consistency across time.
  • Countries can use the collected data to calculate the indicators based on the proposed module or using other data sources (e.g. SDG 16 Survey Initiative, crime victimization surveys among others).

3.c. Data collection calendar

Countries are encouraged to conduct surveys on access to justice through the proposed module in regular intervals, but at least every four years to reflect progress between each of the quadrennial reviews of Goal 16 at the High Level Political Forum (HLPF).

3.d. Data release calendar

Data on relevant SDG indicators are collected, compiled and sent back to countries for data review annually. Data are then reported to United Nations Statistics Division (UNSD) through the regular reporting channels annually.

3.e. Data providers

Data are collected through official nationally representative surveys. In most countries and most cases, such surveys are conducted by National Statistical Offices (NSOs). In some cases, other national institutions or other entities may conduct surveys on access to justice according to the same methodological standards.

3.f. Data compilers

Data will be compiled by the co-custodians for this indicator- United Nations Office on Drugs and Crime (UNODC), United Nations Development Programme (UNDP) and Organization for Economic Cooperation and Development (OECD).

3.g. Institutional mandate

UNDP - Strengthening the rule of law and promoting human rights are cornerstones of UNDP’s work to achieve structural transformation for sustainable human development, build resilience to prevent and withstand shocks and eradicate extreme poverty. UNDP supports national partners to expand access to justice, especially for women, youth, persons with disabilities, marginalized groups and displaced communities. This includes advancing legal aid mechanisms and the use of mobile courts to resolve criminal and civil matters in hard-to-reach areas.

UNODC – as custodian of the UN standards and norms in crime prevention and criminal justice, UNODC assists Member States in reforming their criminal justice systems in order to be effective, fair and humane for the entire population. UNODC develops technical tools to assist Member States in implementing the UN standards and norms and supports Member States through the provision of technical assistance in crime prevention and criminal justice reform. It does so through a number of Global programmes and through the UNODC field office network.

OECD – The OECD supports its Member and partner countries in achieving more responsive and people-centred justice services and access to justice as core components of inclusive growth, sound democracies and a thriving investment climate. Enhanced access to justice is also a fundamental piece of the OECD’s work to shape policies that foster equality, opportunity and well-being for all, given its significant impacts on people’s ability to participate in the economy, health, employment and relationships. Additional areas of support include digital and data-driven transformation of justice, justice for businesses, child-friendly justice, justice for women and people-centred measurement of justice performance.

4.a. Rationale

While there is no standard definition of access to justice, it is broadly concerned with “the ability of people to defend and enforce their rights and obtain just resolution of justiciable problems in compliance with human rights standards; if necessary, through impartial formal or informal institutions of justice and with appropriate legal support.”[3] For citizens in need of justice, a number of conditions should be met for their rights to be recognised, such as access to adequate information, access to justice services and legal advice, and access to institutions of justice that provide fair and impartial treatment. The rationale of this indicator is to focus on one step of the process and in particular on the accessibility of justice institutions and mechanisms (both formal and informal) by those who have experienced a justiciable problem. The indicator can provide important information about the overall accessibility of civil justice institutions and processes, barriers, and reasons for exclusion of some people. The disaggregation by type of dispute resolution mechanism provides additional information about the channels used by citizens in need of enforcing or defending their rights.

This indicator has several advantages:

  1. It is people-centred, as it measures the experience of justiciable problems from the perspective of those who face them.
  2. It provides a broad assessment of people’s approach to address problems they face, both inside and outside of formal institutions or mechanisms.
  3. It focuses on the experience of accessing justice mechanisms or institutions when in need
  4. It is easy to interpret.
  5. It can be produced on the basis of few survey questions, which can be easily incorporated into ongoing national surveys.
  6. It is well suited to monitor public policies aimed at improving the functioning of formal or informal dispute resolution mechanisms (top-down policies) and to those aimed at empowering the population (bottom-up policies).
  7. It can be disaggregated by various socio-demographic (such as age, sex, migratory background, etc.) and geographical variables and thus be used to identify vulnerable groups/areas.
  8. It draws on methodological guidelines derived from a comprehensive review of more than 60 national surveys conducted by governments and civil society organizations in more than 30 jurisdictions in the last 25 years.
3

Praia Group Handbook on Governance Statistics: Access to and Quality of Justice (forthcoming 2019). ↑

4.b. Comment and limitations

A major challenge is that the concept of dispute (justiciable problem) is subject to different interpretations and the propensity to consider a disagreement or conflict in terms of a justiciable problem can vary greatly across individuals and between societies. A way to address this issue is to focus on a number of possible disputes that can be considered of justiciable nature across most countries, as for example the one listed in the section above[4]. Standardised descriptions of the most common types of disputes are also to be used in surveys in order to maximise comparability across different legal systems and countries.

In order to identify the group of people in demand of dispute resolution mechanism it is necessary to identify the group of people voluntarily self-excluded. A way to identify this group is by including an additional question about the reasons why people did not use a dispute resolution mechanism. This question would allow to differentiate cases of voluntary and involuntary exclusion and define the denominator as the population who experienced a problem minus the voluntarily self-excluded.

Another challenge refers to identifying possible dispute resolution mechanisms as they vary considerably in different countries around the world. The formulation of the questions in the survey has to take into account these different possibilities and make sure to include all relevant institutions generally recognized in the community. This proposed list of dispute resolution mechanisms identifies those that are common in most countries in the world but it can be adapted to the country context.

The share of population experiencing the disputes under investigation can be of relatively small size and this can influence the statistical significance of results. A way to address this is to increase the question’s reference period, recognizing that respondents’ ability to recall specific issues becomes increasingly unreliable the further back in time it extends. For these reasons, this proposal follows the recommendation from the Legal Needs Surveys and Access to Justice methodological guidance and suggests a reference interval of two years. With such reference period resulting data would be suitable for monitoring recent changes in contexts/policies while being based on a sufficient number of cases to ensure statistical significance of analyses.[5] Possible telescoping effects (the effect of misplacement in time of events taking place in the past) need to be addressed properly by bounding in clear terms the time interval of reference in relevant questions.

4

These types of disputes have broad applicability across countries as reflected in Legal Needs Surveys and Access to Justice , OECD (2019), which builds upon a review of more than 60 large-scale legal need surveys conducted over the past 25 years. ↑

5

Experimental evidence indicates that increasing a legal needs survey’s reference period from one to three years has only “a fairly modest” impact on problem reporting [Pleasence et al. (2016)] ↑

4.c. Method of computation

Number of persons who experienced a dispute during the past two years who accessed a formal or informal dispute resolution mechanism (numerator), divided by the number of those who experienced a dispute in the past two years minus those who are voluntarily self-excluded (denominator). The result would be multiplied by 100.

16 . 3 . 3 =   N u m e r   o f   p e o p l e   w h o   a c c e s s e d   f o r m a l   o r   i n f o r m a l   d i s p u t e   r e s o l u t i o n   m e c h a n i s m   N u m b e r   o f   p e o p l e   w h o   e x p e r i e n c e d   a   d i s p u t e   i n   t h e   p a s t   2   y e a r s   a n d   d i d   n o t   v o l u n t a r i l y   s e l f - e x c l u d e X 100

This is a survey-based indicator that emphasizes citizens’ experiences over general perceptions. Both numerator and denominator are measured through sample surveys of the general population.

The computation of this indicator requires the inclusion of a short module of four questions in a representative population survey. The following table illustrates the content of the four questions needed to compute the indicator.

Content of question

Instruction
  1. Experience of a dispute over past 2 years, by type of dispute

If no dispute was experienced, skip to END, otherwise go to 2.
  1. Most recent dispute experienced, by type of dispute

Continue with 3.
  1. Access to dispute resolution mechanism, by type of mechanism

If no DRM was accessed go to 4., otherwise skip to END
  1. Reason why no dispute resolution mechanism was accessed

Go to END.

4.d. Validation

The data for the indicator is collected through Household Surveys conducted by National Statistics Offices (NSOs) or other institutions following tight survey protocols and complying with the metadata. Data producers are encouraged to strictly follow the data quality practices, protocols and frameworks in relation of data quality. In addition to the data, countries are requested to report on the metadata which serves as one additional layer of validation and verification of the data by confronting with the metadata used and the recommended for global reporting. For survey-based indicators, metadata are assessed in relation to the representativeness and coverage of the survey as well as alignment of question wording and answer options with international standards. Before publication by custodian agencies, a standardised “pre-publication process” is implemented, where national stakeholders can verify and review the data before publication.

4.e. Adjustments

Not applicable

4.f. Treatment of missing values (i) at country level and (ii) at regional level

• At country level

National data are not imputed if data derived from surveys conducted at country level are not available

• At regional and global levels

There is no imputation of missing values.

4.g. Regional aggregations

Regional aggregates are produced only when available data cover at least a certain percentage of countries of the region and the population of these countries account for a certain percentage of the regional population.

4.h. Methods and guidance available to countries for the compilation of the data at the national level

Methodological documentation from surveys conducted at national level is available (e.g. household survey in Nigeria conducted by the National Bureau of Statistics (NBS) and UNODC; Governance, Public Safety and Justice Survey conducted by Statistics South Africa in 2019, Kenya Integrated Household Budget Survey 2015-2016 conducted by KNBS; Argentina - Unmet Legal Needs and Access to Justice conducted by the Subsecretaría de Acceso a la Justicia Ministerio de Justicia y Derechos Humanos; or Colombia – Survey of Citizen Security and Coexistence conducted by DANE).

Furthermore, the Legal Needs Surveys and Access to Justice methodological guidance published by OECD in 2019 provides methodological guidance for developing the questionnaires and conducting such surveys. This guide brings together the experience gained through more than 60 national surveys conducted by governments and civil society organizations in more than 30 jurisdictions in the last 25 years.

In 2022, the United Nations Office on Drugs and Crime (UNODC) together with the United Nations Development Program (UNDP) and the Office of the United Nations High Commissioner on Human Rights (OHCHR) published the SDG 16 Survey Questionnaire[6] and Implementation Manual[7], which contain internationally standardised survey question wording (in the five official UN languages) as well as implementation guidance related to this indicator.

4.i. Quality management

The three custodian agencies have statistical units with dedicated staff to support the data collection through technical assistance, collating and verifying the received data and continuously improve data collection mechanisms including guidelines.

4.j. Quality assurance

It is recommended that NSOs serve as the main contact for compiling and quality assuring the necessary data to report on SDG 16.3.3, in close coordination with Ministries of Justice and/or other relevant bodies in the country. Automated and substantive validation procedures are in place when data are processed by custodian agencies to assess their consistency and compliance with standards.

4.k. Quality assessment

See Section 4.d Validation

5. Data availability and disaggregation

Data availability:

A growing number of countries are implementing surveys using similar methodologies in order to assess legal needs, improve justice services, and strengthen linkages across sectors. However, the scale and methods of administration have varied. More than 60 national legal needs surveys have been conducted in more than 30 countries over the course of the last 25 years.

Many of those surveys contain the questions needed to compute this indicator (experience of dispute, use of resolution mechanism - either formal or informal – and reasons for not taking action to resolve the dispute).

Time series:

Not applicable

Disaggregation:

Recommended disaggregation for this indicator are:

- type of dispute resolution mechanism

- sex

- disability status

- ethnicity

- migration background

- citizenship

The disaggregation by type of dispute resolution mechanism is of fundamental importance to assess the type of justice institutions and mechanisms available for citizens and for this reason it is part of the indicator itself.

6. Comparability/deviation from international standards

Sources of discrepancies:

Data for this indicator are based on four standardised survey questions. If data from more than one survey are available for the same country, discrepancies may be due to different wording of the questions, different structure of the questionnaire, different survey methods and operations, different sample design and sample size. As a rule, data from national surveys complying with recommended standards are used, when available.

7. References and Documentation

https://www.sdg16hub.org/topic/sdg-16-survey-initiative-questionnaire

https://www.sdg16hub.org/topic/sdg-16-survey-initiative-implementation-manual

  • Legal Needs Survey and Access to Justice. Available at:

https://www.oecd.org/governance/legal-needs-surveys-and-access-to-justice-g2g9a36c-en.htm