Skills2Graph: Processing million Job Ads to face the Job Skill Mismatch Problem

1 Introduction andMotivation

Given the very high number of job positions and applicants on online job portals, the problem of person-job fit has become relevant in recent literature, both as a skill measuring system [Xu et al., 2018] and as a job recommendation system [Zhang et al., 2016]. Recommender systems in the labor market domain rely strongly on handcrafted features and expert knowledge, and these characteristics make them costly, difficult to update, and error-prone. For that reason, we propose skills2graph, a job recommendation system based on a knowledge-poor and data-driven approach, which can be adapted to different countries/industries and easily updated over time. skills2graph was realized as part of the research activity of an EU project 1 , which aims at realizing the first EU real-time labor market monitor, by collecting and classifying Online Job Vacancies (OJVs) from all 27+1 EU

1 This work is partially supported within the research activity of an EU Project entitled EUROSTAT 2020: Towards the European Web Intelligence Hub - European system for collection and analysis of online job advertisement data (WIH-OJA) in which some of the authors are involved as coordinators and researchers. https://tinyurl. com/eurostat-lm-2020

countries, extracting the requested skills from job descriptions, see, e.g. [Boselli et al., 2017; Boselli et al., 2018a; Boselli et al., 2018b; Colombo et al., 2019; Giabelli et al., 2020b]. skills2graph is designed as a tool to support decision making activities of citizens and labor market analysts facing the skill match/mismatch problem by processing 2.5M+ OJVs to answer the following questions:

• Q1. Skills-Occupation match. How relevant are my skills for my current occupation?
• Q2. Reskilling. Which are the relevant skills for my actual position that I should acquire?
• Q3. Career-path. Which are the occupations that fit the skills I already have in a specific country? Which skills should I learn to have a better fit with one of those occupations, given its characteristics in that specific country?
• Q4. Upskilling. Given a new skill that I want to acquire, what are other similar/complementary skills that I can get too?

The approach is based on the rca [Alabdulkareem et al., 2018], a count-based measure that assesses the importance of a skill for an occupation based on their respective appearances in OJVs, and on the cosine similarity between the vectors representing skills and occupations in an embedding space. We generate several embeddings, and we perform an intrinsic evaluation of them, selecting the one which better expresses the semantic similarity between words within the European Standard Taxonomy of Occupations and Skills (ESCO 2 ). This way, we produce a unified representation that preserves distributional and lexical-based semantic features.

Contribution. The contributions of skills2graph are:

• (i) We exploit web labor market data using distributional semantics (embeddings), knowledge-based representations (ESCO), and a count-based measure of skill relevance (rca). We melt those representations to measure the personjob fit starting from the user's skills;
• (ii) We organize the previous resources as a graph database for performing graph-traversal queries;
• (iii) We present skills2graph: a recommender system that exploits the resources developed in (i) and (ii) to suggest the most suitable occupations starting from the user's skills in a certain context.

2 https://tinyurl.com/sv4squr

2 An Overview of skills2graph

The workflow of skills2graph, presented in Fig. 1, can be divided into four main steps.

S.1: Select a taxonomy-aware word vector representation.

We train several embedding models through FastText and, to select the model which better preserves the semantic similarity between taxonomic elements, we perform an intrinsic evaluation following [Baroni et al., 2014]. The evaluation consists of choosing the model which maximizes the correlation between cosine similarity between vectors and a benchmark measure of similarity. In [Baroni et al., 2014] the authors use a human-crafted similarity dataset as the benchmark, however, those kinds of resources are costly to update, limited in size and coverage, and struggle to distinguish between semantic similarity and relatedness [CamachoCollados et al., 2017]. Hence, we employ an automated measure of pairwise semantic similarity between taxonomic elements, namely HSS (developed in [Giabelli et al., 2021]).

Compared with HSS, previous measures of semantic similarity (see [Aouicha et al., 2016]) suffer from two main limitations. First, when a word has multiple senses, those methods compute a value of similarity for each word sense and then consider only the highest, which is the self-information of the least frequent lowest common ancestor. As a result, more specific senses will have a higher value of similarity, but this does not reflect the use of words in OJVs. Second, though they consider the taxonomy's structure (i.e., the relationships between concepts), they do not acknowledge the number of child entities (i.e., words) belonging to those concepts. That is crucial in our case as ESCO includes generic concepts that contain many different occupations, while some specific concepts can be represented by a few occupations which are highly informative.

S.2: Compute the skill-occupation relevance. We compute the relevance of the skills for each occupation through the rca, originally used in the US context [Alabdulkareem et al., 2018], where the authors used the measure onet, provided by the O*NET taxonomy 3 , to take into account the importance of each skill for each occupation. ESCO does not provide such a measure, hence we employ the skill frequency, which for occupation oi ∈ ¯O and skill sl ∈ ¯ S is defined as:

where I denotes the indicator function.

3 https://www.onetonline.org/

The rca is computed as:

where sf is the skill frequency of skill sl for occupation oi. In order to have a measure which is more easily understandable, we compute the normalized rca, normalizing the rca with respect to the maximum value obtained for the occupation taken into account:

so that rcaN(oi, sl) ∈ [0, 1] and the most requested skill for each occupation has a normalized rca equals to 1.

S.3: Store data as a DAG. The information extracted through word embeddings and rca is stored in our graph database, called S2JGraph, which is formalized as a directed labeled multi-graph, and the formalization is inspired by [Giabelli et al., 2020a; Mercorio et al., 2019]. Notice both the rca and the best embedding are computed for each country, capturing the differences between the requested skills and the occupations as they are used in different countries.

S.4: Rank occupations. Given a starting set of skills S, we rank the occupations on the basis of (i) the cosine similarity between the skills in S and the most required skills for occupation oi and (ii) the rca of the skills in S for oi:

with Si being the set of skills required by the occupation oi with a normalized rca of at least 0.6.

3 Skill Based Recommendations

Having S2JGraph stored as a DAG in Neo4j allows to query it using the Cypher query language [Francis et al., 2018]. S2JGraph is helpful for those who may want to know the importance of the skills they own for their actual occupation or other occupations, in their country or in a different one. They may also want to know which skill gap they have to fill to comply with the requirements of a specific occupation. Finally, they may want to know, given a new skill they would like to acquire if there are similar or complementary skills they might be interested in. We can say that S2JGraph helps the user in the problem of skill match/mismatch by answering the questions we draw in the Introduction, that are (Q1) Skills-Occupation match, (Q2) Reskilling, (Q3) Career-path, and (Q4) Upskilling.

The principal use case - Q3 - recommends a series of occupations in a target labor market based on the user's skills, matching all the occupations in the target country cA which require at least one of the starting skills in S. Then the query matches all the skills which are required by the target occupation with a rca > α and which have a cosine similarity with all of the starting skills in S < β. These are the skills of the skill gap, which are relevant for the target occupation (high rca) and different enough from the starting skills (low cosine similarity). These are skills that the user should acquire to do that job in the target country. An example of query Q3 is reported in Tab. 1 (α = 0.6, β = 0.7). The starting parameters are the following: S =["implement front-end website design", "CSS","C#", "use markup languages"]; oS ="Web and multimedia developers"; cS = UK; cA = DE. Fig. 2 shows the results in the user interface.

4 User Evaluation

The results of skills2graph were evaluated through a user study following [Kanakia et al., 2019]. We asked ten labor Market experts belonging to the European Network on Regional Labour Market Monitoring to judge whether the starting skills are relevant for the occupations provided by Q3, using a Likert scale ranging from 1 to 5, with 1 being not relevant and 5 being completely relevant. The participants were all confident in their ability to correctly evaluating the recommendations, since they are active in the Labor Market Intelligence field, and well-acquainted in the ICT domain. The evaluation of skills2graph was performed on the British labor market, using ten different starting sets of four skills, three of which are shown in Tab. 2. The sets were chosen within the most popular skills in the ICT labor market, selecting the clusters with high similarity with each other - with a few changes when they were too similar. As three recommendations were presented for each item, we decided to use Precision@3 (P@3), assuming either a user score of at least 3 is a true positive (P@3-3) or of at least 4 (P@3-4). The normalized Discounted Cumulative Gain (nDCG) has also been computed, which measures the usefulness of an item based on its position in the list of recommendations.

5 Results and Discussion

All the ten experts responded to the user evaluation, and there were no missing votes. The results (see Tab. 3) show a high degree of correlation between the experts' evaluations and the recommendation ranking, showing that the system is effective in identifying the correct jobs given a set of user's skills. skills2graph can process any dataset of OJVs in any EU language; here we used 2.5M+ OJVs form France, Germany, and the United Kingdom, processed through distributional semantics and co-occurrence statistics, organized in a graph database. skills2graph identifies the most suited job based on a set of user's skills, encoding the skill relevance as it emerges from the labor market demand.

A demo video of the tool is available at https://youtu.be/ Fiz9z 4FSbA.