Guide

The development and application of an appropriate system of indicators is considered as one of the key elements to monitor effectively the implementation of RIS3 at national and regional level. The aim should be to ensure that strategic planning is implemented in pursuit of the objectives set, and that funding is allocated in accordance with the design, promoting the production of the desired outputs, immediate outcomes and impacts.

The ‘RIS3 Definition of Output and Result Indicators’ method allows regional stakeholders to effectively monitor the implementation of the RIS3 strategies in each region. It facilitates the process of the identification of output and result indicators that are responsive to policy, normative, robust, and are available to be collected or calculated over time, enhancing the quality of the monitoring and evaluation processes. The data used intend to measure and monitor the achievement of the objectives of individual strategies of development programs / actions of RIS3 and will play a key role in decision-making by providing information on the current each time mode. The procedural use of indicators is to create a system of measurable parameters that will function as directional guide for determining objectives and priorities at national and regional level to achieve the objectives of smart specialization.

Output indicators measure the progress of actions undertaken to achieve the expected results, being selected through a common indicators’ set, defined by the regulations of the Structural Funds ERDF and ESF. Output indicators are measured at the level of actions and projects, and thus, their baseline value is 0. Result indicators measure the changes made to the intervention area and due to the impact of funded activities.

The application supports the adoption of output and result Indicators linked to thematic objectives; investment priorities; with defined targets and baselines. The RIS3 Output and result indicators’ application aims to assist regional stakeholders to define in concrete and justified way the RIS3 for each region to achieve the changes and evolution of the regional productive structure towards activities that (a) are globally competitive and (b) have a greater potential for value added.

One of the outputs of this application is the visualisation of the indicators in multiple formats. The indicators can be visualised and downloaded in tabular format or in different graphical representations. The application also performs statistical calculations on the output and result indicators such as correlation analysis and multivariate regression analysis.

Figure 1 provides the rationale behind this ONLINE S3’s application. There are six (6) discrete stages in the “Definition of Outputs and Results Indicators” application.

Figure 1 Rationale behind this ONLINE S3 Definition of Outputs and Result indicators  

Given the fact that Output and Result Indicators is a critical method in RIS3, while at the same time poses significant conceptual challenges in practice, Output and Result Indicators aims to assist RIS3 stakeholders to access information promptly and in formal methods to design and monitor actions that will support European regions pursue towards Smart Specialisation.

The logical steps of the application, based on its methodological description are:

·Step 1. Definition of the challenges, thematic objectives, investment priorities and measures addressed in the context of each RIS3 strategy

·Step 2. Definition of specific actions based on the previously thematic objectives, investment priorities and measures.

·of the appropriate output indicators per defined actions in Step 2

·Step 4. Selection of the appropriate results indicators per defined actions in Step 2 or covering the total RIS3

·Step 5. Data analysis of the outputs and results indicators through different visualisations; correlations analysis and multivariate regression analysis.

·Step 6. Synthesis of Consolidation Report

Following the application architecture described in previous ONLINE S3 deliverables, and the proposed methodology for the definition of the Output and Result Indicators the information flows within the application are given in Figure 2, below. As it is illustrated, the initial input coming from user defines the necessary parameters, regarding the challenges, thematic objectives, and investment priorities and in sequence the actions that will be addressed. By filtering this information through the selected more relevant output and results indicators, the corresponding data is pulled from the main database. The processing of these data, in terms of visualization, is based statistically processed findings on table or graphic format, which can be sent to the final report. Individual tables or graphics can also be extracted through the application.

The main output of the application is the consolidation report of the Output and Result Indicators strategy analytics, which will give the user a comprehensive picture regarding the progress of the RIS3 strategy in his/her region. Moreover, individual information that might be needed to some other phases of the RIS3 development can also be extracted through this application.

Figure 2 Roadmap for the selection of output and result indicators for monitoring and evaluation.

The potential users of the application 6.2 are considered high level executives that are actively participating in the design, implementation and monitoring of Smart Specialisation Strategies in European Regions.

One the main benefits of the application is that users can use the application to formalise the process of design, and monitor the actions in their adopted Smart Specialisation strategies. All the data are stored in a relational database hence each user can retrieve the information easily at different time intervals. The application is implemented to formalise the process but the content of each action and indicators that is included is completely open to users’ input.

Adopting a common set of outputs and result indicators, from which each region can select the most appropriate for their actions gives the ability to each region to perform advanced statistics analysis including correlation and multivariate linear regression analysis.

Each region can access solely their own data while data of the result indicators that are used in the correlation and multivariate linear regression analysis are gathered from Eurostat and are public available and open for any kind of use.

The following Figure 3 display the key benefits of the application that are considered to greatly support regional actors and stakeholders for the further development of their regions and the successful and cost-effective implementation of RIS3.

Figure 3 Benefits to stakeholders when using the definition of outputs and results indicators.

For the use of the Application 6.2 a valid user account is required. The application is web based and all data are stored in a MYSQL database.

The data that are required so that the users can work effectively have been already uploaded to the database. Required data are considered

  • the list with relative information of the fifty-two (52) common output indicators as presented in the following Table 1 List of common output indicators of this guide.
  • All values of fifty-three (53) results indicators, for all the years and the European regions (NUTSII level) that are available in Eurostat. (http://ec.europa.eu/eurostat/web/regions/data/database).The list of results indicators and a description of each one is included in Table 2 List of available result indicators

The output and results indicators are being grouped in the same categories as of other ONLINE S3 applications such as the Regional assets mapping and the Benchmarking. The categories are (i) Geography, (ii) Demography & Society, (iii) Economy & Labor, (iv) Sectoral structure, (v) Business Characteristics and (vi) Innovation System.

An outline illustrating the overall process of the “Definition of Outputs and Results indicators” application is given in Figure 4, in order to present more clearly the steps that are included in this application.

Figure 4 Key steps when using this ONLINE S3 Definition of Output and result indicators application.

How to use this application step-by-step?

Each page in the “Definition of Output and Results indicators” 6.2 application has two main menu areas:

·         The first area includes three buttons: i) to create a new strategy analytics; ii) to view the list of each users’ strategies analytics (Figure 6 My Strategies analytics) and iii) button to log out from the application.

·         The second menu displays on the left side the name of the strategy that user is working on. On the right side there are five (5) buttons with reports icons that the user can  click to go to specific steps in the process of the Definition of Output and Results indicators in the specific strategy analytics.

Figure 5 Main menu areas

My strategies analytics displays a list of all the strategies analytics that each user has created. Each user can create more than one strategies analytics. The user can view / edit or delete each strategies analytics by clicking on the respective links. The Complete / Incomplete status links show if the user for each report has submitted each step in the database. For a report to be complete all five steps have to submitted. The Process link drives the user in the following Figure 7 The process.

Figure 6 My Strategies analytics

In this Process page the status of each step is highlighted. The completed i.e. submitted steps are colored while the steps that are not submitted for each report are grey.

 

Figure 7 The process


 

Step 1: Definition of challenges.

In Step 1. the challenges addressed by the regional smart specialisation strategy are defined. The user specifies the referring country and region of the strategy analytics is doing to develop. All the thematic objectives and the all the investment priorities addressed are also defined in this step. Acknowledging that each region might use Measures as well as the required by RIS3 Guide thematic objectives and investment priorities each region can add each own measure for the grouping of actions and output and result indicators.

In each step a Description section is included so that users can enter formatted text related to the step.

Figure 8 Definition of challenges

The user must click on the button save and continue to store the entered information in the application / database.

Step 2: Definition of actions

An action is defined by the following information:

·        Title of the action

·        Short description of the action

·        Related thematic objective (TO) to the action (The list includes only the TOs selected in Step 1)

·        Related Investment Priority (IP) to the action (The list includes only the IPs selected in Step 1)

·        Related Measure from the Measures defined in Step 1.

·        Related output indicators

·        Related result indicators.

 

The user must click on the button save and continue to store the entered information in the application / database.

 

Figure 9 Add an action

 

The related output indicators are first selected by the user from the provided list of the 52 common output indicators ( Figure 10 Add action Select Outputs indicators).

The user by clicking on the left categories can see the available indicators per category. To relate / link an indicator to the action has to click on the specific title. Then the indicator will appear as a box in the right column as a distinct area. To delete the indicator can click on the (x). Special attention must be given that by deleting an output indicator all its related data are also deleted.

Figure 10 Add action Select Outputs indicators

 

Figure 11 Add Output indicator

 

Following the selection of output indicators than for each one the user ought to enter the following information. In the popup “Add output indicator” most of the fields are automatically filled in from the database. The user can modify those fields if considered required since are referring to the specific output indicator for the action.

The most important information that the user should enter in this popup are: The target year; The target value; and the realized outcomes for each year.

The user must click on the button “Update output indicator” to store the entered information in the application / database.

The data for each year are stored automatically in the database on input field change. After the data for each field is stored then the field is changing its white color to greenish, so that the users understands that data is stored successfully.

The action related result indicators follow same procedure as the relation of common outputs indicators to action. Results indicators are first selected by the user from the provided list of the result indicators ( Figure 12 Add action Select Results indicators).

The user by clicking on the left categories can see the available result indicators per category. To relate / link an indicator to the action has to click on the specific title. Then the indicator will appear as a box in the right column “Selected result indicators” as a distinct area. To delete the result indicator can click on the (x). Special attention must be given that by deleting a result indicator all its related data are also deleted.

 

Figure 12 Add action Select Results indicators

 

Following the selection of result indicators than for each one the user ought to enter the following information ( Figure 13 Add Result indicator). In the popup “Result indicator data” most of the fields are automatically filled in from the database. The user can modify those fields if considered required since are referring to the specific output indicator for the action.

The information the user should enter in this popup is basically only the baseline year. If in the database, the baseline value for the selected year exists then should appear in the field baseline value. This feature depends upon the data availability from Eurostat otherwise the user enters a baseline value.

 

 

Figure 13 Add Result indicator

The user must click on the button “Update result indicator” to store the entered information in the application / database.

 


 

Step 3. Output indicators

Smart specialisation strategies can be very complex and include large number of actions and output indicators. To further assist the definition of output indicators in this step all the output indicators are presented as lists in tabs. The content of the tabs appear when the user clicks on the titles. The available lists of the used common indicators in the strategy analytics are:

·         All Common indicators

·         Grouped per Thematic Objectives

·         Grouped per Investment priorities

·         Grouped per actions

·         Grouped per measures

As in all steps a Description section is included so that users can enter formatted text related to the Output indicators. The user must click on the button save and continue to store the text in the application / database.

Figure 14 Output indicators All

Figure 15 Output indicators per thematic objectives.

The “Description” text area refers to the Output indicators step and not for each specific tab.

Figure 16 Output indicators per investment priorities

Figure 17 Output indicators per actions

 

Figure 18 Output indicators per measures


 

Step 4. Definition of result indicators

As of described in previous Step 3 Definition of output indicators the definition of results indicators is facilitated in this step. The results indicators are presented as lists in tabs. The content of each tab appears when the user clicks on the titles. The available lists of the used result indicators in the strategy analytics are:

·         All Result indicators

·         Results indicators related to actions

·         Results indicators non-related to actions

As in all steps a Description section is included so that users can enter formatted text related to the Result indicators. The user must click on the button save and continue to store the text in the application / database.

 

Figure 19 All Results indicators


 

In the second tab the results indicators that are selected in different actions in Step 3 are grouped and presented per actions.

Figure 20 Results indicators related to actions

In the third tab the user can define extra results indicators, that would be used in the strategy analytics but cannot be directly associated to specific actions.

The non-related to actions results indicators related follow same procedure as the relation of results indicators to action. Results indicators are first selected by the user from the provided list of the result indicators (Figure 21 Results indicators non-related to actions).

The user by clicking on the left categories can see the available result indicators per category. To relate / link an indicator to the action has to click on the specific title. Then the indicator will appear as a box in the right column “indicators non-associated to actions” as a distinct area. To delete the result indicator can click on the (x). Special attention must be given that by deleting a result indicator all its related data are also deleted.

 

Following the selection of result indicators than for each one the user ought to enter the following information (Figure 13 Add Result indicator). In the popup “Add result indicator” most of the fields are automatically filled in from the database. The user can modify those fields if considered required since are referring to the specific output indicator for the action.

The information the user should enter in this popup is basically only the baseline year. If in the database, the baseline value for the selected year exists then should appear in the field baseline value. This feature depends upon the data availability from Eurostat otherwise the user enters a baseline value.

Figure 21 Results indicators non-related to actions


 

Step 5. Data analysis

In this step, all the data entered are analysed using customised tables; visualizations and advanced analytics tools. Data analysis contains five (4) main tabs: The Outputs Indicators; The results indicators; Correlation analysis; Regression analysis; and Two indicators linear regression

 

Figure 22 Indicators application Data analysis

As in all steps a Description section is included so that users can enter formatted text related to the Data analysis. The user must click on the button save and continue to store the text in the application / database.

 

Output indicators

Output indicators can be analysed in tabular, bar and line charts formats. To create a chart the user must select the Years and the common outputs indicators to be included.

There are five (5) grouping queries for the analysis of the output indicators: i) per output indicator per year; ii) per actions per years; iii) per measures per years; iv) per thematic objectives per years and v) per investment priorities per years.

The results of the queries can be downloaded as CSV files by clicking the respective button.

Figure 23 Data analysis Output indicators per years

Figure 24 Data analysis Output indicators per actions per years

In the following example CO10 is related to the action titled “Action not fully completed” which is not related to any measure.

Figure 25 Data analysis Output indicators per measures per years

 

Figure 26 Data analysis Output indicators per thematic objectives per years

 

Figure 27 Data analysis Output indicators per investment priorities per years

 

The charts can be downloaded by clicking the button “Export as image”

Figure 28 Data analysis Output indicators bar chart

 

Figure 29 Data analysis Output indicators line chart

Results indicators

Figure 30 Data analysis Result indicators


The Results indicators can be analysed also in tabular, bar and line charts formats. The table includes all the results indicators as defined in Annex I Table 2 List of available result indicators.

To create a chart the user must select the Years and the results indicators to be included.

The created chart can be exported as PNG file.

The Description section refers to the Data analysis step and not to each tab.

 

 

Figure 31 Data analysis Result indicators select indicators to be included in the chart

Figure 32 Data visualisation of Result indicators


 

Correlation analysis


Figure 33 Data analysis Correlation


The correlation analysis examines the relation of an output and a result indicator considering data from different regions for specific years. The scope is to calculate the correlation coefficient, a number that quantifies a type of correlation and dependence, meaning statistical relationships between two or more values in fundamental statistics.

The user first selects the year of the results indicator to be examined. The value that is taken into the correlation analysis is the difference of the selected year and the following one, since the result of an output indicator is considered as this difference and not the results indicators actual value.

Following the user must select the regions that wants to be included in the analysis.

Finally, the output and result indicator that wishes to be examined are selected and displayed as a pair.

By clicking the button “Examine the possible correlation” the correlation coefficient is being calculated and the chart created.

 


 


 

The following chart uses random created data so the interpretation of them is useless, though the chart in the X and Y axis shows the values of each variables while the line plots the correlation coefficient. The negative value of the correlation coefficient is displayed as the slope of the line top left corner to the bottom right corner.

Figure 34 Data analysis Correlation details


 

Regression analysis

In this tab the user can perform multi variate regression analysis to a number of outputs indicators, defined as independent variables and a result indicator defined as dependent variable. As in the previous described Correlation analysis the user must select the: i) Results indicators reference year; ii) Outputs indicators reference year; iii) the European regions to be considered; iv) select output indicators as independent variables and v) select a result indicator as the one dependent variable. To perform the multivariate regression analysis the use clicks on the button “Examine the possible multilinear regression”.

Figure 35 Data analysis multi variate regression

The outcome of the performed multivariate regression analysis is the calculation of The Fitted Model and the following key diagnostics for Data Transformation Decisions

R-Square; F-Statistic; Mean; Variance; Mean: The first half; Mean: The second half; Variance: The first half; Variance: The second half; First order serial-correlation; Second order serial-correlation; Durbin-Watson statistic; Mean absolute errors;

Finally, there is an estimation of the Normality Condition and the calculation of ith Residuals from the fitted model

Figure 36 Data analysis multi variate regression

Table 1 List of common output indicators

SFC2014 Code

Name (in application)

Description

Unit

CO01

Number of enterprises receiving support

Number of enterprises receiving support in any form from ERDF (whether the support represents state aid or not).

enterprises

CO02

Number of enterprises receiving grants

Number of enterprises receiving support in forms of non-refundable direct financial support conditional only to completion of project (grants).

enterprises

CO03

Number of enterprises receiving financial support
other than grants

Number of enterprises receiving non-grant type financial support, in forms of loan, interest subsidy, credit guarantee, venture capital or other financial instrument.

enterprises

CO04

Number of enterprises receiving non-financial support

Number of enterprises receiving support that does not involve direct financial transfer (guidance, consultancy, enterprise incubators, etc.). Venture capital is considered as financial support.

enterprises

CO05

Number of new enterprises supported

Number of enterprises created receiving financial aid or support (consultancy, guidance, etc.) from ERDF or ERDF financed facility. The created enterprise did not exist three years before the project started but the Managing Authority or national legislation may set lower the time criterion. An enterprise will not become new if only its legal form changes.

enterprises

CO06

Private investment matching public support to enterprises (grants)

Total value of private contribution in supported project that qualifies as state aid where the form of support is grant (see Common Indicator 2 'Number of enterprises receiving grants'), including non-eligible parts of the project.

EUR

CO07

Private investment matching public support to enterprises (non-grants)

Total value of private contribution in supported project that qualifies as state aid where the form of support is other than grant (see Common Indicator 3 'Number of enterprises receiving  inancial support other than grants'), including non-eligible parts of the project.

EUR

CO08

Employment increase in supported enterprises

Gross new working positions in supported enterprises in full time equivalents (FTE). Essentially a 'before-after' indicator which captures the part of the employment increase that is direct consequence of project completion (workers employed to implement the project are not counted). The positions needs to be filled (vacant posts are not counted) and increase the total number of jobs in the enterprise. If total employment in the enterprise does not increase, the value is zero – it is regarded as realignment, not increase. Safeguarded etc. jobs are not included.
Gross: Not counting the origin of the jobholder as long as it directly contributes to the increase of total jobs in the organisation. The indicator should be used if the employment increase can plausibly be attributed to the support.
Full-time equivalent: Jobs can be full time, part time or seasonal. Seasonal and part time jobs are to be converted to FTE using ILO/statistical/other standards.
Durability: Jobs are expected to be permanent, i.e. last for a reasonably long period depending on industrial-technological characteristics; seasonal jobs should be recurring. Figures of enterprises that went bankrupt are registered as a zero employment increase.
Timing: Data is collected before the project starts and after it finishes; Managing Authorities are free to specify the exact timing. Using average employment, based on 6 months or a year, is preferred to employment figures on certain dates.

FTE

CO09

Increase in expected number of visits to supported sites of cultural or natural heritage and attractions

The ex ante estimated increase in number of visits to a site in the year following project completion. Valid for site improvements that aim to attract and accept visitors for sustainable tourism. Includes sites with or without previous tourism activity (e.g. nature parks or buildings converted to museum). One visitor can make multiple visits; a group of visitors count as many visits as many members the group has. The Managing Authorities set the methodology for estimating the expected number that can be based on demand analysis.

Visits/year

CO10

Additional households with broadband access of at least 30 Mbps

Number of households with internet access with a download speed of at least 30 Mb/sec and who before only had more limited access or did not have access at all. The capacity to access must be a direct consequence of the support. The indicator measures households with the possibility to access, not whether the people living in the households actually choose to be connected or not.

households

CO11

Total length of new railway line

Length of railroads constructed by the project where no railroad existed before

Km

CO11a

Total length of new railway line within TEN-T

Total length of new railway line within TEN-T

Km

CO12

Total length of reconstructed or upgraded railway line

Length of railroads of which quality or capacity have been improved. This can include electrification, developing single track railroad into double track, increasing the possible speed on the track, or any combination of these, but excludes installation of signalling systems (incl. ensuring ERTMS (European Rail Traffic Management System) compatibility).

Km

CO12a

Total length of reconstructed or upgraded railway line within TEN-T

Total length of reconstructed or upgraded railway line within TEN-T

Km

CO13

Total length of newly built roads

Length of roads (in kilometres) constructed by the project where:
• no road existed before
or
• as a consequence of project completion, the capacity and quality of the previously existing local/secondary road is significantly improved to reach a higher classification (e.g. national road or equivalent); in this case the road cannot be counted under indicator “Total length of reconstructed or upgraded roads”.

Km

CO13a

Total length of newly built roads within TEN-T

Total length of newly built roads within TEN-T

Km

CO14

Total length of reconstructed or upgraded roads

Length of roads where the capacity or quality of the road (including safety standards) was improved. If the upgrade is significant enough for the road to qualify as new road, it will be counted under “Total length of newly built roads” and not under this indicator (see above).

Km

CO14a

Total length of reconstructed or upgraded roads within TEN-T

Total length of reconstructed or upgraded roads within TEN-T

Km

CO15

Total length of new or improved tram and metro lines

Length of metro, tram or suburban train lines constructed or upgraded. The service along the upgraded lines must significantly improve as a consequence of the project completion.

Km

CO16

Total length of new or improved inland waterway

Length of inland waterway with new or improved navigation capacity. The improvement may concern improved transport capacity or safety aspects.

Km

CO17

Additional waste recycling capacity

Annual capacity of newly built waste recycling facilities. It also includes xtension of existing facilities.

tonnes/year

CO18

Additional population served by improved water supply

Number of persons provided with drinking water through drinking water supply network as a consequence of increased drinking water production/transportation capacity built by the project, and who were previously not connected or were served by sub-standard water supply. It includes improving the quality of the drinking water. The indicator covers persons in households with actual (i.e. not potential) connection to the water supply system. It includes reconstruction projects but excludes projects aiming to create/improve irrigation systems.

persons

CO19

Additional population served by improved wastewater treatment

Number of persons whose wastewater is transported to wastewater treatment plants through wastewater transportation network as a result of increased waste water treatment/transportation capacity built by the project, and who were previously not connected or were served by sub-standard wastewater treatment. It includes improving wastewater treatment level. The indicator covers persons in households with actual (i.e. not potential) connection to the wastewater treatment system.

population equivalent

CO20

Population benefiting from flood protection measures

Number of people exposed to flood risk where vulnerability decreased as a direct consequence of a supported project.

persons

CO21

Population benefiting from forest fire protection measures

Number of people exposed to forest fire hazards where vulnerability decreased as a direct consequence of a supported project.

persons

CO22

Total surface area of rehabilitated land

Surface of remediated or regenerated contaminated or derelict land made available for economic (except non-eligible, e.g. agriculture or forestry) or community activities.

Hectares

CO23

Surface area of habitats supported in order to attain a better conservation status

Surface of restored or created areas aimed to improve the conservation status of threatened species. The operations can be carried out both in or outside of Natura 2000 areas, capable of improving the conservation status of targeted species, habitats or ecosystems for biodiversity and the provisioning of ecosystem-services. Areas that receive support repeatedly should be counted only once.

Hectares

CO24

Number of new researchers in supported entities

Gross new working positions (that did not exist before) to directly perform R&D activities, in full time equivalents. The post must be a consequence of project implementation or completion, be filled (vacant posts are not counted) and increase the total number of research jobs in the organisation. Support staff for R&D (i.e. jobs not directly involved in R&D activities) is not counted. The indicator focuses on employed personnel; the supported entity may be new or already existing.
Gross: Not counting the origin of the jobholder as long as it directly contributes to the increase of total research jobs in the organisation.
Full-time equivalent: Jobs can be full time, part time or seasonal. Seasonal and part time jobs are to be converted to FTE using ILO/statistical/other standards. In the field of RTD the duration of jobs tends to be shorter (“project support”). The jobs created for different projects should be added up (provided that all projects receive support); this is not regarded as multiple counting.

FTE

CO25

Number of researchers working in improved research infrastructure facilities

Existing working positions in research infrastructure facilities that (1) directly perform R&D activities and (2) are directly affected by the project. The posts must be filled (vacant posts are not counted). Support staff for R&D (i.e. jobs not directly involved in R&D activities) is not counted. If more researchers will be employed in the facilities as a consequence of the project, thus the numbers of research jobs increases, the new posts are included (see also “Number of new researchers in supported entities”). The facilities may be private or public. The project must improve the facilities or quality of equipment, i.e. maintenance or replacement without quality increase is excluded.
Full-time equivalent: Jobs can be full time, part time or seasonal. Seasonal and part time jobs
are to be converted to FTE using ILO/statistical/other standards.

FTE

CO26

Number of enterprises cooperating with research institutions

Number of enterprises that cooperate with research institutions in R&D projects. At least one enterprise and one research institution participates in the project. One or more of the cooperating parties (research institution or enterprise) may receive the support but it must be conditional to the cooperation. The cooperation may be new or existing. The cooperation should last at least for the duration of the project. Enterprise: Organisation producing products or services to satisfy market needs in order to reach profit. The origin of the enterprise (inside or outside of the EU) does not matter. In case one enterprise takes the formal lead and others are subcontractors but still interacting with the research institution, all enterprises should be counted. Enterprises cooperating in different projects should be added up (provided that all projects receive support); this is not regarded as multiple counting. Research institution: an organisation of which R&D is a primary activity.

enterprises

CO27

Private investment matching public support in innovation or R&D projects

Total value of private contribution in supported innovation or R&D projects, including noneligible parts of the project.

EUR

CO28

Number of enterprises supported to introduce new to the market products

The indicator measures if an enterprise receives support to develop a 'new to the market' product in any of its markets. Includes process innovation as long as the process contributes to the development of the product. Projects without the aim of actually developing a product are excluded. If an enterprise introduces several products or receives support for several projects, it is still counted as one enterprise. In case of cooperation projects, the indicator measures all participating enterprises.
A product is new to the market if there is no other product available on a market that offers the same functionality, or the technology that the new product uses is fundamentally different from the technology of already existing products. Products can be tangible or intangible (incl. services). Supported projects that aimed to introduce new to the markets products but did not succeed are still counted. If a product is new both to the market and to the firm, the enterprise should be counted in both relevant indicators (see indicator 29 ‘Number of enterprises supported to introduce new to the firm products’). The boundaries of the market (either geographical or other) are defined by the Managing Authority based on the business activity of the enterprise receiving support.

enterprises

CO29

Number of enterprises supported to introduce new to the firm products

The indicator measure if an enterprise is supported to develop a 'new to the firm' product. Includes process innovation as long as the process contributes to the development of the product. Projects without the aim of actually developing a product are excluded. If an enterprise introduces several products or receives support for several projects, it is still counted as one enterprise. In case of cooperation projects, the indicator measures all participating enterprises to which the product is new. A product is new to the firm if the enterprise did not produce a product with the same functionality or the production technology is fundamentally different from the technology of already produced products. Products can be tangible or intangible (incl. services). Supported projects that aimed to introduce new to the firm products but did not succeed are still counted. If a product is new both to the market and to the firm, the enterprise should be counted in both relevant indicators (see indicator 28 ‘Number of enterprises supported to introduce new to the market products’).

enterprises

CO30

Additional capacity of renewable energy production

Increase in energy production capacity of facilities using renewable energy resources, built/equipped by the project. Includes electricity and heat energy. Renewable energy resource: Any energy source that is not fossil or nuclear. See regulation 2009/28, art. 2(a).

MW

CO31

Number of households with improved energy consumption classification

Number of households in improved energy class – see Directive 2010/31/EU. Improved class must be the direct consequence of the project completion.

households

CO32

Decrease of annual primary energy consumption of public buildings

Calculations are based on the energy certificate of buildings (see Art.12.1.b of Directive 2010/31/EU). In line with the deadlines set in the Directive, the indicator must apply to all public buildings above 500m2 total useful area and were reconstructed using Structural Funds support. If the construction starts after 9 July 2015, the threshold for public buildings decreases to 250m2 total useful area. The Managing Authority may include buildings in the calculation with less than 250m2 (or 500m2 before 9/7/2015). Value will be calculated from the energy certificates issued before and after the reconstruction. The indicator will show the total decrease of annual consumption, not the total saved consumption.

kWh/year

CO33

Number of additional energy users connected to smart grids

Smart grid: Electricity network that integrate the actions of energy users by exchanging digital information with the network operator or supplier. An energy user can be consumer, generator, or both. Enterprises can be users too.

Users

CO34

Estimated annual decrease of GHG

This indicator is calculated for interventions directly aiming to increase renewable energy production (see indicator 30) or to decrease energy consumption through energy saving measures (see indicators 31 and 32), thus its use is mandatory only where these indicators are relevant. Uses for other interventions with possible GHG impact are optional with methodology developed by the Managing Authority. The indicator will show the total estimated of annual decrease by the end of the period, not the total decrease throughout the period.
In case of renewable energy production, the estimate is based on the amount of primary energy produced by supported facilities in a given year (either one year following project completion or the calendar year after project completion). Renewable energy is supposed to be GHG neutral and replacing non-renewable energy production. GHG impact of nonrenewable energy is estimated through the MS total GHG emission per unit of non-renewable energy production.
In case of energy saving measures, the estimate is based on the amount of primary energy saved through in a given year supported operations (either one year following project completion or the calendar year after project completion). Saved energy is supposed to be replacing non-renewable energy production. GHG impact of non-renewable energy is estimated through the MS total GHG emission per unit of non-renewable energy production.

tons of CO2 equivalent

CO35

Capacity of supported childcare or education infrastructure

Number of users who can use newly built or improved childcare or education facilities. “Users” in this context mean the children, pupils, or students, not teachers, parents or other persons who may use the facilities too. It includes new or improved buildings, or new equipment provided by the project. It measures nominal capacity (i.e. number of possible users which is usually higher than or equal to the number of actual users).

persons

CO36

Population covered by improved health services

Population of a certain area expected to benefit from the health services supported by the project. It includes new or improved buildings, or new equipment for various type of health service (prevention, outpatient or inpatient care, aftercare).
The indicator excludes multiple counting even if the intervention benefits more services targeting the same persons: one person still counts as one even if that person will use several services which were supported by Structural Funds. For example, an aftercare facility is developed in a city with a population of 100,000 inhabitants. It will serve half the city’s population, thus the indicator value will increase by 50,000. If later a prevention service is developed in the same city that will serve the whole population, the indicator value will increase by another 50,000.

persons

CO37

Population living in areas with integrated urban development strategies

Population living in areas with integrated urban development strategies within the meaning of Article 7 of Regulation 1301 / 2013 (ERDF). Use the indicator only once for each area.

persons

CO38

Open space created or rehabilitated in urban areas

Size of renovated / newly developed publicly accessible open-air areas. It does not include developments covered by the ‘standard’ common indicators (e.g. roads, rehabilitated land, schoolyards, etc.)

square meters

CO39

Public or commercial buildings newly built or renovated in urban areas

Size of renovated / newly developed public and commercial areas

square meters

CO40

Rehabilitated housing in urban areas

Number of renovated / newly developed housing units in residential areas, as part of urban rehabilitation.

housing units

CO41

Number of enterprises participating in crossborder, transnational or interregional research projects

Variant of indicator 26 'Number of enterprises cooperating with research institutions' with the difference that the research project must qualify as crossborder, transnational or interregional. If a participating enterprise has departments operating in different places, the location of the participating department(s) should be taken into account to qualify as crossborder project.

enterprises

CO42

Number of research institutions participating in  rossborder, transnational or interregional esearch projects

Variant of indicator 41 'Number of enterprises participating in crossborder, transnational or interregional research projects' with the difference that it counts cooperating research institutions instead of enterprises. If a participating organisation has departments operating in different places, the location of the participating department(s) should be taken into account to qualify as crossborder project.

organisations

CO43

Number of participants in cross-border mobility initiatives

Cross-border mobility initiatives are those supported under the investment priority set out in art. 7 (a) (i) of the ETC regulation. Participants are those who start in such initiatives. Managing authorities are encouraged to exclude double counting due to multiple participations.

persons

CO44

Number of participants in joint local employment initiatives and joint training

Joint local employment initiatives are those supported under the investment priority set out in art. 7 (a) (i) of the ETC regulation. Participants are those who start in such initiatives. Managing authorities are encouraged to exclude double counting due to multiple participations.

persons

CO45

Number of participants in projects promoting gender equality, equal opportunities and social inclusion across borders

Projects supported under the investment priority set out in art. 7 (a) (ii) of the ETC regulation. Participants are those who start in such initiatives. Managing authorities are encouraged to exclude double counting due to multiple participations.

persons

CO46

Number of participants in joint education and training schemes to support youth employment, educational opportunities and higher and vocational education across borders

Joint education and training schemes are those supported under the investment priority set out in art. 7 (a) (iii) of the ETC regulation. Participants are those who start in such initiatives. Managing authorities are encouraged to exclude double counting due to multiple participations.

persons

Table 2 List of available result indicators

Variable Name and unit (in application)

Description

Gross domestic product (GDP) (Million euro)

Gross domestic product (GDP) at current market prices

Gross value added (Million euro)

Gross value added at basic prices (all NACE activities)

Economically active population 15-64 years old (Thousand)

Economically active population 15-64 years old (all sexes). The economically active population comprises employed and unemployed persons.

Employment rates in population 15-64 years old (%) (%)

Employment rates in population 15-64 years old (%) (all sexes). Employed persons are all persons aged 15 and over who, during the reference week, worked at least one hour for pay, profit or family gain, or were not at work but had a job or business from which they were temporarily absent.

Unemployment rates in population 20-64 years old (%) (%)

Unemployment rates in population 20-64 years old (%) (all sexes). Unemployed persons comprise persons aged 15-74 who were (all three conditions must be fulfilled simultaneously): 1. without work during the reference week; 2. available for work at the time (i.e. were available for paid employment or self-employment before the end of the two weeks following the reference week); 3. actively seeking work (i.e. had taken specific steps in the four-week period ending with the reference week to seek paid employment or self-employment), or who found a job to start within a period of at most three months.

Unemployment rates in population 15-64 years old tertiary education graduates (%)

Unemployment rates in population 15-64 year old tertiary education graduates (male and female) as percentage of the population

Scientists and engineers as % of active population (% of active population)

Total number of scientists and engineers aged 25 to 64 as a share of the active population.

Researchers in all sectors (Full-time equivalent (FTE))

Number of researchers (male and female) in both the public (government, higher education) and private (business enterprise, private non-profit) sectors

Employment in Knowledge Intensive Activities (KIA) as % of total employment (% of total employment)

Employment in Knowledge Intensive Activities (KIA) as % of total employment

Employment in high and medium high-technology manufacturing  (% of total employment)

Employment in medium-high and high-tech manufacturing as percentage of total employment (male and female)

Persons employed in Manufacturing (C) (persons employed)

Number of persons employed in Manufacturing (C) according to NACE Rev. 2 (from 2008 onwards)

Units in Manufacturing (C) (local units)

Number of local units in Manufacturing (C) according to NACE Rev. 2 (from 2008 onwards)

Employment growth in Manufacturing (C) (% employment growth rate)

Growth rate of employment (%) in Manufacturing (C) according to NACE Rev. 2 (from 2008 onwards)

Persons employed in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) (persons employed)

Number of persons employed in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) according to NACE Rev. 2 (from 2008 onwards)

Units in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) (local units)

Number of local units in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) according to NACE Rev. 2 (from 2008 onwards)

Employment growth in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) (% employment growth rate)

Growth rate of employment (%) in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) according to NACE Rev. 2 (from 2008 onwards)

Persons employed in Transportation and storage (H) (persons employed)

Number of persons employed in Transportation and storage (H) according to NACE Rev. 2 (from 2008 onwards)

Units in Transportation and storage (H) (local units)

Number of local units in Transportation and storage (H) according to NACE Rev. 2 (from 2008 onwards)

Employment growth in Transportation and storage (H) (% employment growth rate)

Growth rate of employment (%) in Transportation and storage (H) according to NACE Rev. 2 (from 2008 onwards)

Persons employed in Accommodation and food service activities (I) (persons employed)

Number of persons employed in Accommodation and food service activities (I) according to NACE Rev. 2 (from 2008 onwards)

Units in Accommodation and food service activities (I) (local units)

Number of local units in Accommodation and food service activities (I) according to NACE Rev. 2 (from 2008 onwards)

Employment growth in Accommodation and food service activities (I) (% employment growth rate)

Growth rate of employment (%) in Accommodation and food service activities (I) according to NACE Rev. 2 (from 2008 onwards)

Persons employed in Information and communication (J) (persons employed)

Number of persons employed in Information and communication (J) according to NACE Rev. 2 (from 2008 onwards)

Units in Information and communication (J) (local units)

Number of local units in Information and communication (J) according to NACE Rev. 2 (from 2008 onwards)

Employment growth in Information and communication (J) (% employment growth rate)

Growth rate of employment (%) in Information and communication (J) according to NACE Rev. 2 (from 2008 onwards)

Net business population growth ()

Net business population growth in industry, construction and services except insurance activities of holding companies for all company sizes

Innovative enterprises in Manufacturing (C) (% of total enterprises)

Innovative enterprises as % of total enterprises in Manufacturing (C) according to NACE Rev. 2 (from 2008 onwards), all size classes

Innovative enterprises in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) (% of total enterprises)

Innovative enterprises as % of total enterprises in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) according to NACE Rev. 2 (from 2008 onwards), all size classes

Innovative enterprises in Transportation and storage (H) (% of total enterprises)

Innovative enterprises as % of total enterprises in Transportation and storage (H) according to NACE Rev. 2 (from 2008 onwards), all size classes

Innovative enterprises in Accommodation and food service activities (I) (% of total enterprises)

Innovative enterprises as % of total enterprises in Accommodation and food service activities (I) according to NACE Rev. 2 (from 2008 onwards), all size classes

Innovative enterprises in Information and communication (J) (% of total enterprises)

Innovative enterprises as % of total enterprises in Information and communication (J) according to NACE Rev. 2 (from 2008 onwards), all size classes

VC investment as % of GDP (% of GDP)

Venture capital investment as percentage of gross domestic product (GDP) on all maturity levels (seed, start-up and later stage)

GERD in Business enterprise (% of GDP)

Total intramural R&D expenditure (GERD) in Business enterprise sector as % of GDP

GERD in Government (% of GDP)

Total intramural R&D expenditure (GERD) in Government sector as % of GDP

GERD in Higher Education  (% of GDP)

Total intramural R&D expenditure (GERD) in Higher Education sector as % of GDP

GERD in Private non-profit sector  (% of GDP)

Total intramural R&D expenditure (GERD) in Private non-profit sector as % of GDP

GBAORD as % of general government expenditure (%  of government expenditure)

Government budget appropriations or outlays on R&D (GBAORD) as % of general government expenditure

GERD financed by all sectors (% of GDP)

Total intramural R&D expenditure (GERD) financed by all sectors as % of GDP

GERD financed by Business enterprise (% of GDP)

Total intramural R&D expenditure (GERD) financed by the Business enterprise sector as % of GDP

GERD financed by Government (% of GDP)

Total intramural R&D expenditure (GERD) financed by the Government sector as % of GDP

GERD financed by Higher education (% of GDP)

Total intramural R&D expenditure (GERD) financed by the Higher education sector as % of GDP

GERD financed by Private non-profit sector (% of GDP)

Total intramural R&D expenditure (GERD) financed by the Private non-profit sector as % of GDP

GERD financed by abroad (% of GDP)

Total intramural R&D expenditure (GERD) financed by abroad as % of GDP

Enterprises cooperating with academia (% of enterprises)

Share of enterprises cooperating with universities or other higher education institutions in innovation core activities (Com.Reg. 995/2012) (all enterpise sizes).

Households with broadband internet access (% of households)

Households having access to broadband internet.

Graduates from tertiary education (graduates)

Total number of graduates  from first and second stage of tertiary education (levels 5 and 6)

Students in tertiary education (students)

Total number of students in first and second stage of tertiary education (levels 5 and 6) in all fields

High-tech exports  (% of total exports)

Total high-tech exports to all countries of the world as a percentage of total exports (from 2007, SITC Rev. 4)

GVA in Manufacturing (C) (% change of price index on previous period)

Gross Value added in Manufacturing (C) according to NACE Rev. 2 (from 2008 onwards), Price index (implicit deflator), percentage change on previous period, euro

GVA in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) (% change of price index on previous period)

Gross Value added in Wholesale and retail trade; repair of motor vehicles and motorcycles (G) according to NACE Rev. 2 (from 2008 onwards), Price index (implicit deflator), percentage change on previous period, euro

GVA in Transportation and storage (H) (% change of price index on previous period)

Gross Value added in Transportation and storage (H) according to NACE Rev. 2 (from 2008 onwards), Price index (implicit deflator), percentage change on previous period, euro

GVA in Accommodation and food service activities (I) (% change of price index on previous period)

Gross Value added in Accommodation and food service activities (I) according to NACE Rev. 2 (from 2008 onwards), Price index (implicit deflator), percentage change on previous period, euro

GVA in Information and communication (J) (% change of price index on previous period)

Gross Value added in Information and communication (J) according to NACE Rev. 2 (from 2008 onwards), Price index (implicit deflator), percentage change on previous period, euro