Automation of companies’ recruitment process: development of an algorithm capable of ranking CVs according to job offers

Universidade do Minho Escola de Engenharia Departamento de Informática Beatriz de Freitas Rocha Automation of companies’ recruitment process Development of an algorithm capable of ranking CVs according to job offers December 2022 Universidade do Minho Escola de Engenharia Departamento de Informática Beatriz de Freitas Rocha Automation of companies’ recruitment process Development of an algorithm capable of ranking CVs according to job offers Master dissertation Integrated Master’s in Informatics Engineering Dissertation supervised by Pedro Rangel Henriques Hugo Paulino Santos December 2022 i AUTHOR COPYRIGHTS AND TERMS OF USAGE BY THIRD PARTIES This is an academic work which can be utilized by third parties given that the rules and good practices internationally accepted, regarding author copyrights and related copyrights. Therefore, the present work can be utilized according to the terms provided in the license bellow. If the user needs permission to use the work in conditions not foreseen by the licensing indicated, the user should contact the author, through the RepositóriUM of University of Minho. License provided to the users of this work Attribution-NonCommercial CC BY-NC https://creativecommons.org/licenses/by-nc/4.0/ ii STATEMENT OF INTEGRITY I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Code of Ethical Conduct of the University of Minho. Beatriz de Freitas Rocha ABSTRACT This document presents a Thesis and describes the underlying work which was developed along the second year of the Master Degree in Informatics Engineering offered by Departamento de Informática of Universidade do Minho and accomplished at Syone SBS Software – Tecnologia e Serviços de Informática, S.A.. In the past few years, some attempts to automatically screening CVs with resource to Natural Language Processing have been made not only to save recruiters’ time, but also to spare them the most tedious task of the recruitment process and, consequently, smooth their job. However, the majority is still very primitive, misclassifies a lot of CVs and needs a deeper study. Therefore, the aim of this Master’s Project is precisely to develop an algorithm that is capable of automatically ranking candidates’ CVs according to their similarity regarding the job offer they applied for. Thus, a general architecture was proposed where CVs and job offers are preprocessed, in order to obtain the respective texts proper to be further processed. That said, two different approaches were followed, in order to find the similarity between the documents in question. To do so, the first approach resorted to several Machine Learning algorithms and similarity measures, while the second approach structured the initial documents to compare their respective information. After that, tests were conducted to evaluate both approaches and enable the comparison between them. Finally, the conclusions were drawn and also reported in this dissertation. keywords Curriculum Vitae (CV), CV screening, Machine Learning (ML), Natural Language Processing (NLP). iii RESUMO Este documento apresenta uma Tese e descreve o trabalho subjacente que foi desenvolvido ao longo do segundo ano do Mestrado em Engenharia Informática do Departamento de Informática da Universidade do Minho e realizado na Syone SBS Software – Tecnologia e Serviços de Informática, S.A.. Nos últimos anos, algumas tentativas de triagem automática de currículos com recurso a Processamento de Linguagem Natural foram feitas não só para economizar o tempo dos recrutadores, mas também para os poupar da tarefa mais entediante do processo de recrutamento e, consequentemente, suavizar o seu trabalho. Contudo, a maioria ainda é muito primitiva, classifica incorretamente muitos currículos e necessita de um estudo mais aprofundado. Sendo assim, o objetivo deste Projeto de Mestrado é precisamente desenvolver um algoritmo capaz de classificar automaticamente os currículos dos candidatos de acordo com a sua similaridade relativamente à oferta de emprego a que se candidataram. Deste modo, foi proposta uma arquitetura geral onde os CVs e as ofertas de emprego são pré-processados, de forma a obter os respetivos textos adequados para posterior processamento. Dito isto, foram seguidas duas abordagens distintas, de forma a encontrar a semelhança entre os documentos em questão. Para tal, a primeira abordagem recorreu a diversos algoritmos de Aprendizagem Automática e medidas de similaridade, enquanto a segunda abordagem estruturou os documentos iniciais para comparar as suas respetivas informações. De seguida, foram realizados testes para avaliar ambas as abordagens e possibilitar a comparação entre elas. Por fim, as conclusões foram tiradas e também relatadas nesta dissertação. palavras-chave Aprendizagem Automática, Curriculum Vitae (CV), Processamento de Linguagem Natural (PLN), triagem de currículos. iv CONTENTS 1 introduction 1 1.1Context 1 1.2Motivation 2 1.3Objectives 2 1.4Research hypothesis 2 1.5Research methodology 3 1.6Document structure 3 2 state of the art 5 2.1CV sources and formats 5 2.1.1Europass 5 2.1.2LinkedIn profile as a CV 8 2.1.3Curriculum Lattes 11 2.1.4CIÊNCIAVITAE 14 2.1.5Syone’s internal CV 25 2.1.6Considerations 28 2.1.7Ontology definition 28 2.2Semantic information extraction 31 2.2.1Entities and relationships 31 2.2.2Information Extraction workflow 32 2.2.3Named Entity Recognition approaches 34 2.2.4Information Extraction workflow with spaCy 34 2.3Similarity algorithms 39 2.3.1Jaccard Similarity 39 2.3.2TF-IDF 40 2.3.3Doc2Vec 40 2.3.4BERT 42 2.3.5USE 44 2.3.6Algorithm comparison 44 2.4Similarity measures 45 2.4.1Cosine similarity 45 2.4.2Euclidean distance 46 2.4.3Manhattan distance 46 2.4.4Chebyshev distance 47 2.5Summary 47 v contents vi 3 proposed approach 48 3.1System architecture 48 3.2Summary 51 4 cvs processing 52 4.1Text extractor and XML extractor 52 4.2Knowledge extractor 53 4.3Document builder 64 4.4Summary 64 5 similarity algorithm (approach a)65 5.1Implementation and results 65 5.2Summary 79 6 similarity algorithm (approach b)80 6.1Implementation and results 80 6.2Summary 89 7 conclusion 90 a sample job offer 97 b sample cv 99 c json file corresponding to the sample job offer 108 d json file corresponding to the sample cv 112 LIST OF FIGURES Figure 1Europass - Personal Information 6 Figure 2Europass - Work Experience 6 Figure 3Europass - Education and Training 6 Figure 4Europass - Personal Skills (part 1)7 Figure 5Europass - Personal Skills (part 2)7 Figure 6Europass - Additional Information 8 Figure 7LinkedIn profile as a CV - Summary 8 Figure 8LinkedIn profile as a CV - Experience 8 Figure 9LinkedIn profile as a CV - Education 9 Figure 10 LinkedIn profile as a CV - Contact 9 Figure 11 LinkedIn profile as a CV - Top Skills 9 Figure 12 LinkedIn profile as a CV - Languages 9 Figure 13 LinkedIn profile as a CV - Certifications 10 Figure 14 LinkedIn profile as a CV - Honors-Awards 10 Figure 15 LinkedIn profile as a CV - Publications 10 Figure 16 LinkedIn profile as a CV - Patents 10 Figure 17 LinkedIn profile as a CV - Personal Information 10 Figure 18 Curriculum Lattes - Personal Information 11 Figure 19 Curriculum Lattes - Address and Contact 12 Figure 20 Curriculum Lattes - Academic Education 13 Figure 21 Curriculum Lattes - Professional Occupation 13 Figure 22 Curriculum Lattes - Occupation Area 14 Figure 23 CIÊNCIAVITAE - Identification: Personal Information (part 1)14 Figure 24 CIÊNCIAVITAE - Identification: Personal Information (part 2)15 Figure 25 CIÊNCIAVITAE - Identification: Personal Information (part 3)15 Figure 26 CIÊNCIAVITAE - Identification: Contacts (part 1)15 Figure 27 CIÊNCIAVITAE - Identification: Contacts (part 2)16 Figure 28 CIÊNCIAVITAE - Identification: Contacts (part 3)16 Figure 29 CIÊNCIAVITAE - Identification: Contacts (part 4)17 Figure 30 CIÊNCIAVITAE - Identification: Acting Domains 17 Figure 31 CIÊNCIAVITAE - Identification: Languages 18 Figure 32 CIÊNCIAVITAE - Identification: Summary 18 Figure 33 CIÊNCIAVITAE - Education (part 1)19 vii Acronyms xiv JSON JavaScript Object Notation. K KE Knowledge Engineering. M ML Machine Learning. N NER Named Entity Recognition. NLP Natural Language Processing. P POS Part-Of-Speech. PV-DBOW Distributed Bag Of Words version of Paragraph Vector. PV-DM Distributed Memory version of Paragraph Vector. T TF-IDF Term Frequency — Inverse Document Frequency. U UPOS Universal Part-Of-Speech tag set. URL Uniform Resource Locator. USE Universal Sentence Encoder. 1 INTRODUCTION In this first chapter of the dissertation, the Master’s Project is introduced along with its context and motivation. Furthermore, the necessary objectives to complete it are also detailed, the research hypothesis and the corresponding research methodology are specified and the document structure is presented. 1.1 context Recruitment consists in hiring the best candidates for a given job offer according to their skills and the first task of this process is to screen the CVs of all the job applicants. However, this can be a very exhausting and tedious chore for recruiters, considering they have to manually narrow down the most appropriate candidates from an immensely large pool. (Amin et al.,2019) As a proof, the screening process can be condensed as stated bellow: (Catherine et al., 2010) 1. Understand the job offer’s requirements in terms of the skills that are mandatory (such as certain technologies) and those that are optional, but preferable (for example, speaking a particular language), the experience criteria if any, the preference for the location of the candidate, etc.; 2. Go through each and every of the applications and discard those which do not satisfy the skills required for the job; 3. Out of the remaining candidates, find the best match for the job. To do this, the recruiter has to read the CVs in detail and compare them with the job offer. In addition, since the number of candidates who can be interviewed is limited, the recruiter has to make a relative judgement on them. Therefore, when using such a sequential and relative selection procedure, manual screening may miss potentially better candidates for the job opening which, in turn, may result in a suboptimal pool of potential employees. (Mehta et al.,2013) 1 1.2. Motivation 2 Moreover, with less than 5% of people being selected from these applications, it is impractical for the recruiters to go through the CVs one by one for these limited number of openings. (Daryani et al.,2020) Another problem faced by recruiters is the fact that the CVs do not follow any specific pattern, i.e., beyond the fact that the CVs are in various formats, such as .pdf, .docx, .jpg, etc., they do not have a standard style of presenting their content. As an attempt to solve this problem, many job portals decided to provide an online form so that the job applicants could fill up all the information of their CV in a structured manner, creating the so called "candidate metadata". Nevertheless, this solution requires redundant efforts from the candidates, which often leads them to fill that template with incomplete information. (Daryani et al.,2020) 1.2 motivation In the past few years, there have been developed some automatic CV screening tools, in order to automate that recruitment process task. However, most of them show vulnerabilities: either in precision, the lack of empirical validation or even the presentation of bias (ending up discriminating candidates). Enachescu (2019), for instance, states some negative impacts on the accuracy of the system. On the other hand, Laumer and Eckhardt (2009), for example, did not validate their conceptual approach with real unstructured data. Lastly, the American multinational technology company, Amazon, e.g., developed an Artificial Intelligence (AI) recruiting tool that was not rating candidates for software developer jobs and other technical posts in a gender-neutral way (ama). This Master’s Work entails the creation of a system that will overcome those issues, never forgetting the principal aim: being capable of screening CVs automatically. 1.3 objectives The main objective of this Master’s Work is to identify which applicants are most suitable for a specific job offer. To attain this objective, the steps below will be followed: • Extract crucial information about the candidates and the job offers; • Apply a function capable of calculating the similarity between a CV and a job offer. 1.4 research hypothesis The research hypothesis that will be proved with this Master’s Work is stated bellow: 1.5. Research methodology 3 “It is possible to develop an algorithm capable of automatically matching the most suitable applicants to a certain job offer”. 1.5 research methodology In order to accomplish this Master’s Work, an iterative methodology based on literature review, solution proposal, implementation and testing will be carried on. To achieve this approach, the following steps will be performed: • Bibliographic study to deeply understand the state of the art in CV sources and formats, Information Extraction (IE) components and steps, Natural Language Processing algorithms and similarity metrics; • Synthesis and description of the products of that study; • Analysis of the sample of CVs and job offers that will be used as a case study; • Extraction of the important information from those CVs and job offers; • Implementation of an algorithm where the recruiters will be able to choose the job offer and the respective candidates and get as a result the x best candidates that applied for that job, according to the number of applicants they are willing to interview; • Testing of the developed algorithm and evaluation and discussion of the results. After the pursuance of these steps, if the results are still not favorable, then iterate all over again. 1.6 document structure This document is composed of seven different chapters with the following contents: 1. Introduction - In the first chapter the problem is contextualized and the motivation to proceed with its resolution is exposed. In addition, the main goals of this work are detailed and the research hypothesis to be proved is stated. Lastly, the research methodology containing the steps to be followed is proposed; 2. State of the art - In this chapter the concepts of CV sources and formats, semantic information extraction, similarity algorithms and similarity measures are studied and the results of that study are presented; 3. Proposed approach - In the third chapter, a solution to the problem is proposed, which, in turn, is illustrated with a diagram for better understanding; 1.6. Document structure 4 4. CVs processing - In this chapter the engines responsible for processing the CVs and the challenges that arose with their development are explained; 5. Similarity algorithm (approach A) - In the fifth chapter the general coding strategy for calculating the affinity between job offers and CVs in the first similarity algorithm approach is presented, along with the elucidation of the various techniques applied to achieve that affinity and their respective results and conclusions; 6. Similarity algorithm (approach B) - In this chapter the second similarity algorithm approach that was followed to calculate the affinity between job offers and CVs is explained and the results and conclusions of that approach are presented; 7. Conclusion - The last chapter contains a recap of all the work done throughout this Master’s Project, the obstacles that emerged during the development of the project, the conclusions obtained and the future work that could be done in the following iterations. 2 STATE OF THE ART Before proposing an approach to solve the problem under discussion, it is important to analyze the current state of the art regarding the scope of this Master’s Work. Therefore, in this second chapter, the concepts of CV sources and formats, semantic information extraction, similarity algorithms and similarity measures are explored. 2.1 cv sources and formats ACurriculum Vitae (Latin for "course of life"), as the name implies, is a document containing the information about an individual’s career, education and skills and it is usually used when they want to apply to a certain job offer. However, there is still no standard format globally used and accepted at the moment and, as a result, there are several different types of CVs, according to the area of expertise or country. Despite that, there is a set of patterns that can be seen in the various types of CVs. Therefore, in this section, the Europass (Eur), LinkedIn profile as a CV (Lin), Curriculum Lattes (Lat), CIÊNCIAVITAE (Cie) and Syone’s internal CV formats will be presented and analyzed and, after that, an ontology supporting all the information it will be necessary to extract from each CV will be developed. 2.1.1Europass The Europass was created by the Directorate General for Education and Culture of the European Union so that citizens could follow a common pattern among all the countries of the European Union (EU). It is a very complete format and can be divided into five main sections: Personal Information, Work Experience, Education and Training, Personal Skills and Additional Information. The first section, Personal Information (Figure 1), contains the individual’s name, picture, contact information (such as home address, telephone/mobile number, e-mail address, websites and messaging accounts), sex, date of birth and nationalities. It also contains the description of the job position they have applied for. 5 2.1. CV sources and formats 6 Figure 1: Europass - Personal Information The Work Experience section (Figure 2) holds the previous occupations the individual has had in the past and the respective period of time, company and location in which they had performed them enumerated chronologically. In addition, the responsibilities the person had and the sector where they worked are detailed. Figure 2: Europass - Work Experience The Education and Training section (Figure 3) is very similar to the previous one. It contains the individual’s qualifications (and corresponding European Qualification Framework (or other) levels) and, once again, the respective period of time, organization and location in which they achieved them enumerated chronologically. Moreover, the main subjects covered are also described. Figure 3: Europass - Education and Training 2.1. CV sources and formats 7 The next section, Personal Skills (Figure 4and Figure 5), starts with the individual’s language skills and the respective proficiency level regarding understanding, speaking and writing. Then, it also displays the individual’s communication skills, organisational/managerial skills, job-related skills, computer skills and other skills. In the end, the person can also state whether they have a driving license and, if so, its category. Figure 4: Europass - Personal Skills (part 1) Figure 5: Europass - Personal Skills (part 2) Finally, the last section, Additional Information (Figure 6), has any other information about the individual that was not stated above and that they might consider relevant to mention, such as publications, presentations, projects, among others. 2.1. CV sources and formats 8 Figure 6: Europass - Additional Information 2.1.2LinkedIn profile as a CV LinkedIn is the world’s largest professional network on the Internet. This platform allows people to find job opportunities through their profile which showcases past professional experiences, skills, licenses and certifications, etc.. Currently, it is also possible to export a person’s LinkedIn profile as a CV which, in turn, has ten main sections: Summary, Experience, Education, Contact, Top Skills, Languages, Certifications, Honors-Awards, Publications and Patents. The Summary section (Figure 7) presents a brief summary about the person. That summary usually contains a description about their skills, achievements or previous job experiences. Figure 7: LinkedIn profile as a CV - Summary The Experience section (Figure 8) contains the previous work experiences with the corresponding start and end dates. It also includes the company’s name, the job role, the country where they took place and a brief description. Figure 8: LinkedIn profile as a CV - Experience 2.1. CV sources and formats 9 The Education section (Figure 9) contains the previous education miles which include the respective school, degree, field and start and end dates. Figure 9: LinkedIn profile as a CV - Education The Contact section (Figure 10) contains the person’s contact information, such as address, mobile number, email and website. Figure 10: LinkedIn profile as a CV - Contact The Top Skills section (Figure 11) contains the best skills of the individual. Figure 11: LinkedIn profile as a CV - Top Skills The Languages section (Figure 12) contains the individual’s language skills and the corresponding proficiency level. Figure 12: LinkedIn profile as a CV - Languages In the Certifications section (Figure 13) the individual can state the respective certifications obtained. 2.1. CV sources and formats 16 Figure 27: CIÊNCIAVITAE - Identification: Contacts (part 2) Figure 28: CIÊNCIAVITAE - Identification: Contacts (part 3) 2.1. CV sources and formats 17 Figure 29: CIÊNCIAVITAE - Identification: Contacts (part 4) The subsection Acting Domains (Figure 30) contains the knowledge area, the topic and the keywords. Figure 30: CIÊNCIAVITAE - Identification: Acting Domains In the subsection Languages (Figure 31) the individual needs to choose the language, whether it is their mother tongue and the proficiency levels regarding speaking, reading, writing, comprehension and peer-review. 2.1. CV sources and formats 18 Figure 31: CIÊNCIAVITAE - Identification: Languages In the last subsection, Summary (Figure 32), the person can provide a brief description about theirself. Figure 32: CIÊNCIAVITAE - Identification: Summary 2.1. CV sources and formats 19 In the next section, Education (Figure 33, Figure 34 and Figure 35), the individual starts by stating whether the education mile was obtained in Portugal or abroad. Then, they have to choose the level of education and provide the organization, course and corresponding code, specialization, description, grade, status and start and end dates. Then, the person can also supply the dissertation title, its ID, the organization where the thesis was defended and the information about the supervisors (name and role). Lastly, the knowledge area can also be included. Figure 33: CIÊNCIAVITAE - Education (part 1) Figure 34: CIÊNCIAVITAE - Education (part 2) 2.1. CV sources and formats 20 Figure 35: CIÊNCIAVITAE - Education (part 3) The next section, Professional Path (Figure 36 and Figure 37), starts with the type of job, employer, professional bond, professional category, host institution and start and end dates. Moreover, it has the corresponding activities that can be associated with the Activities section. Figure 36: CIÊNCIAVITAE - Professional Path (part 1) Figure 37: CIÊNCIAVITAE - Professional Path (part 2) 2.1. CV sources and formats 21 The Projects section (Figure 38, Figure 39, Figure 40 and Figure 41) contains the function and the dates between which they were performed. Moreover, it contains the project’s title, the description, the type of financing, the start and end dates, the project’s status and the organization. It also has the information about the financing itself, such as the financing entity, the financing program and the year of assignment. Finally, it has the project’s references (including the reference code, the URL and the type of relation), the information about other participants, the knowledge areas and the keywords. Figure 38: CIÊNCIAVITAE - Projects (part 1) Figure 39: CIÊNCIAVITAE - Projects (part 2) 2.1. CV sources and formats 22 Figure 40: CIÊNCIAVITAE - Projects (part 3) Figure 41: CIÊNCIAVITAE - Projects (part 4) The Productions section hosts every work the individual has produced and it varies according to the category and type chosen. In case of a publication in a journal (Figure 42, Figure 43 and Figure 44), for instance, the individual has to provide the title of the article, the name of the journal, the section, the volume, the edition, the main and final pages, the publication date, the country and city of publication and the URL. Next, the person needs to supply the information about the identifier (the type, the actual identifier and the type of relation). Then, they also need to state the name to cite and whether it is theirs or from another author. Lastly, it has the corresponding projects that can be linked from the section above, the knowledge areas and the keywords. 2.1. CV sources and formats 23 Figure 42: CIÊNCIAVITAE - Productions (part 1) Figure 43: CIÊNCIAVITAE - Productions (part 2) 2.1. CV sources and formats 24 Figure 44: CIÊNCIAVITAE - Productions (part 3) Like the previous section, the Activities section shelters a lot of different details about the activities, according to the type chosen. In the case of tutoring (Figure 45), for example, the details involve the topic, the start and end dates, the name of the student and the description. Figure 45: CIÊNCIAVITAE - Activities The last section, Distinctions (Figure 46 and Figure 47), houses the distinctions’ type, name and year of the attribution and the promoting entity. Additionally, it contains the country, the description and the expiration date (if applicable). Lastly, the person can also state the knowledge areas and the keywords. 2.1. CV sources and formats 25 Figure 46: CIÊNCIAVITAE - Distinctions (part 1) Figure 47: CIÊNCIAVITAE - Distinctions (part 2) 2.1.5Syone’s internal CV At the time of application, candidates to Syone must fill in the company’s internal CV so that the Human Resources (HR) team has access to the most relevant information for the hiring process in one single format. This format has six key sections: Personal Information, Executive Summary, Professional Experience, Education, Languages and Technologies. The first section, Personal Information (Figure 48), contains the individual’s personal data, such as name, job description, age and gender. 2.2. Semantic information extraction 32 Figure 54: Entities found in the text Relationships are defined over two or more related entities. Examples of relationships are "is employee of" between a person and an organization, "location of outbreak" between a disease and a location, among others. Figure 55 shows the same text, where the relationship "Employee_Of" occurs between a person and an organization and the relationship "Located_In" occurs between an organization and a location. (Sarawagi,2008) Figure 55: Relationships found in the text 2.2.2Information Extraction workflow A typical workflow of Information Extraction consists of the following steps: 1. Sentence splitting - The text is split into sentences resorting to sentence boundary indicators (question marks, exclamation points, etc.); (Choi et al.,2021) 2.2. Semantic information extraction 33 2. Tokenization - Each sentence is then broken down along a predefined set of delimiters (such as spaces, commas, dots, etc.) giving rise to tokens which, in turn, are usually words, digits or punctuation marks; (Sarawagi,2008) 3. Part-Of-Speech (POS) tagging - After the tokens are generated, a grammatical category from a fixed set, like a noun, a verb, an adjective, an adverb, an article, a conjunct, a pronoun or other parts of speech, is assigned to each token; (Sarawagi,2008) 4. Lemmatization - The various forms of a token (such as "appeared" or "appears") are mapped to its root form, also known as the lexeme or lemma (e.g., "appear"); (Bird et al.,2009) 5. Named Entity Recognition - Single tokens and/or sets of tokens are recognized as organizations, locations, dates, etc.; (Lee et al.,2004) 6. Relationship extraction - Relationships between different entities, such as "is employee of", "is located in", among others, are extracted. (Bird et al.,2009) Figure 56 illustrates the pipeline and shows how the above mentioned steps interconnect. Figure 56: Information Extraction workflow 2.2. Semantic information extraction 34 2.2.3Named Entity Recognition approaches According to (Choi et al.,2021), Named Entity Recognition approaches are classified into two main types: • Knowledge Engineering (KE), also referred to as rule-based approach, which uses the domain knowledge of human expertise represented in a machine-understandable form, i.e., in the form of production rules. In this approach, rules are iteratively constructed and refined to improve the accuracy of text processing; • Automatic training, also known as the Machine Learning approach, which uses ML algorithms, as expected. It requires a large amount of annotated training data. Comparing the two approaches, it is clear that the efforts required for defining patterns and developing rules in the first approach are less than those required for annotating a large amount of training data in the second approach. Moreover, the first one tends to produce a higher performance, since human expertise results in more accurate patterns. (Choi et al., 2021) 2.2.4Information Extraction workflow with spaCy spaCy (spa) is a modern Python library for industrial-strength Natural Language Processing. It comes with pretrained pipelines and currently supports tokenization and training for more than 60 languages. It features state-of-the-art speed and neural network models for tagging, Named Entity Recognition and more. Therefore, throughout this subsection, the functionalities of spaCy will be explored and applied, performing the steps mentioned in Subsection 2.2.2in the text sample that follows: Apple is looking at buying U.K. startup for $1billion. First of all, the library is imported, the English model is loaded which creates an nlp object and the text in question is processed with that nlp object which, in turn, creates a Doc (short for "document") object. 1import spacy 3nlp = spacy . load ( "en_core_web_sm" ) doc = nlp ( "Apple is looking at buying U.K. startup for $1bil lio n . " ) Listing 2.3: Import library, load English model and create Doc object Then, the text is split into sentences. 2.2. Semantic information extraction 35 for sent in doc . sents : 2print( sent . text ) Listing 2.4: Sentence splitting In this case, since the text itself consists of a single sentence, the code above results in just that sentence. Apple is looking at buying U.K. startup for $1 billion. After that, that sentence is broken into tokens. for token in doc : 2print( token . text ) Listing 2.5: Tokenization The result contains twelve tokens, including words, symbols, numbers and punctuation marks. Apple is looking at buying U.K. startup for $ 1 billion . Next, comes the POS tagging phase where a grammatical category from a fixed set is assigned to each token. for token in doc : 2print( token . text , token . tag_ ) Listing 2.6: POS tagging The result shows the detailed POS tag assigned to each token, including "NNP" (noun, proper singular), "VBZ" (verb, 3rd person singular present), "VBG" (verb, gerund or present 2.2. Semantic information extraction 36 participle), "IN" (conjunction, subordinating or preposition), "NN" (noun, singular or mass), "$" (symbol, currency), "CD" (cardinal number) and "." (punctuation mark, sentence closer). (Pen) Apple NNP is VBZ looking VBG at IN buying VBG U.K. NNP startup NN for IN $ $ 1 CD billion CD . . Then, comes the lemmatization phase where the root form of each token in inflectional or derivational form is derived. for token in doc : 2print( token . lemma_) Listing 2.7: Lemmatization From the result, it can be seen that "is" was derived as "be", "looking" was derived as "look" and "buying" was derived as "buy". Apple be look at buy U.K. startup for $ 1 billion . Later, comes the Named Entity Recognition step where, as the name implies, the tokens are recognized as named entities. 2.2. Semantic information extraction 37 for ent in doc . ents : 2print( ent . text , ent . label_ ) Listing 2.8: Named Entity Recognition The result shows that spaCy found three entities: "Apple" which was recognized as an organization, "U.K." which was recognized as a geopolitical entity and "$1billion" which was recognized as money. Apple ORG U.K. GPE $1 billion MONEY spaCy also allows visualizing the text sample and its named entities with different colours through displaCy. Figure 57: Text sample and its named entities When applied to a much more complex text sample, it is possible to see the great power of spaCy. It not only identifies the previous entities, but also recognizes "Steve Jobs", "Steve Wozniak" and "Ronald Wayne" as people, "weekly" as a date, "University of Minho" as an organization and "Portugal" as a geopolitical entity. Figure 58: More complex text sample and its named entities In addition to the features mentioned so far, spaCy has many more. One of the powerful functionalities that might be useful in the future is the dependency parsing. In this step, the dependencies between the tokens (head token and child token) are identified. for token in doc : 2print( token . text , token . dep_ , token . head . text , token . pos_ , [child for child in token . children ] ) Listing 2.9: Dependency parsing 2.2. Semantic information extraction 38 By the result, it is possible to conclude that "Apple" is a nominal subject with "looking" being its head token; "is" is an auxiliary verb with "looking" also being its head token; "looking" is the root of the sentence, which means that it acts as the head of many tokens ("Apple", "is", "at" and "."), but is not a child of any other token; "at" is a preposition with, once again, "looking" being its head token and "buying" being its child token; "buying" is a complement of preposition with "at" being its head token and "startup" and "for" being its child tokens; "U.K." is a compound noun with "startup" being its head token; "startup" is a direct object with "buying" being its head token and "U.K." being its child token; "for" is a preposition with "buying" also being its head token and "billion" being its child token; "$" is a quantifier with "billion" being its head token; "1" is a compound noun with "billion" also being its head token; "billion" is an object of preposition with "for" being its head token and "$" and "1" being its child tokens; "." is a piece of punctuation with "looking" being its head token. Apple nsubj looking PROPN [] is aux looking AUX [] looking ROOT looking VERB [Apple, is, at, .] at prep looking ADP [buying] buying pcomp at VERB [startup, for] U.K. compound startup PROPN [] startup dobj buying NOUN [U.K.] for prep buying ADP [billion] $ quantmod billion SYM [] 1 compound billion NUM [] billion pobj for NUM [$, 1] . punct looking PUNCT [] Like the named entities, spaCy also allows visualizing the text sample (including the tokens’ simple Universal Part-Of-Speech tag set (UPOS) (UPO) POS tags) and the dependency parse tree with directed, labeled arcs from head tokens to child tokens through displaCy. Figure 59: Text sample and its dependency parse tree 2.3. Similarity algorithms 39 2.3 similarity algorithms The last step of this Master’s Work is comparing the candidates’ CVs with the job offers. Therefore, in this section of the state of the art, the most popular similarity algorithms (Jaccard Similarity, Term Frequency — Inverse Document Frequency (TF-IDF), Doc2Vec, Bidirectional Encoder Representations from Transformers (BERT) and Universal Sentence Encoder (USE)) will be explored and analyzed. 2.3.1Jaccard Similarity Jaccard Similarity is defined as the size of the intersection divided by the size of the union of the sample sets, which can be represented by the following formula: (Bank and Cole,2008) J(A,B) = |A∩B| |A∪B|(1) Adapting this definition to the problem in question, we can translate Jaccard Similarity as the ratio between the cardinality of words in common in both documents and the number of words resulting from the union of both documents. Follows an illustrative example to aid understanding. Figure 60: Jaccard Similarity 2.3. Similarity algorithms 40 2.3.2TF-IDF TF-IDF measures how much a term (word) is relevant to a document (set of words) in a collection of documents, which can be represented by the following formula: TFIDF(t,d) = TF(t,d)×IDF(t)(2) Term Frequency TF(t,d) measures the number of times term t occurs in document d . (Jing et al.,2002) However, this depends on the generality of the term and the length of the document. For example, the term "is" will probably appear more times in a 10000 worded document than in a 100 worded one, but it is not correct to say that the first document is more important. Therefore, the absolute frequency is divided by the total number of terms in the document. TF(t,d) = count of tin d number of terms in d(3) Document Frequency DF(t) is the number of documents in which the term t occurs at least once. (Jing et al.,2002) DF(t) = occurrence of tin document set (4) Similarly to Term Frequency, Document Frequency is divided by the total number of documents N. The main goal is to know the informativeness of a term, which means that, ideally, the more common a term is (e.g., stop words, such as “the”, “a”, “an”, “so”, “what”, etc.), the lower the weight. However, DF(t) N returns precisely the inverse and, so, it is necessary to invert this fraction. Still, a large N will result in a huge value for the latter, hence it is essential to take the log of it which, in turn, originates Inverse Document Frequency IDF(t) . IDF(t) = log(N DF(t))(5) 2.3.3Doc2Vec Doc2Vec, as the name implies, is capable of transforming text documents into a vectorized form and is heavily based on Word2Vec, as expected. Therefore, first, it is important to briefly explain the latter. Like with text documents, words also need to be converted into vectors, so that machines can understand them. A traditional approach is simply one-hot encode the words, which results in vectors (whose length is equal to the size of the total unique vocabulary in the 2.3. Similarity algorithms 41 corpus) with only one target element being 1and the remaining being 0. Conventionally, these words are encoded in alphabetical order, which means that one-hot vectors for words starting with "a" will have value 1at lower indexes while those for words beginning with "z" will have the same value at higher indexes. However, this approach has, at least, two issues. First, it is not possible to infer the relationship between two words. For instance, although they have a similar meaning, "endure" and "tolerate" will have their targets far away from each other. Additionally, there are a numerous redundant 0values in the vectors which, in turn, are wasting a lot of space. Figure 61: One-hot encoding Word2Vec is an efficient solution to these problems, because it is a Word Embedding method and, so, generates a type of mapping that allows words with similar meaning to have similar representation. It can be divided into two subtypes: skip-gram and Continuous Bag Of Words (CBOW). For skip-gram, the input is the target word, while the outputs are the words surrounding it. For instance, in the sentence “I have a cute cat”, if the input was “a”, the corresponding output would be “I”, “have”, "cute" and “cat”, assuming the window size is 2. Figure 62: Skip-gram 3 PROPOSED APPROACH The objective of this Master’s Project is to develop an algorithm capable of identifying the most suitable candidates for a given job offer. Therefore, in this chapter, an architecture proposal for the system was designed and explained, in order to facilitate the understanding of the entire project. 3.1 system architecture Figure 68: General architecture of the system 48 3.1. System architecture 49 Figure 68 depicts the general architecture of the proposed software system to be implemented in this project, in order to accomplish the objectives defined in Chapter 1. The diagram shown sketches the main blocks of the system and its overall characteristics, without considering all details or specific aspects that will be further refined. In this architecture, it is shown that the system will receive a CV (provided in .docx, .pdf, etc. format) which, in turn, will go through the text extractor engine and/or the XML extractor engine, to extract its text and/or XML, respectively, and be properly processed in the next phase. In parallel, the system will also receive a job offer that can be made available in two ways. It will either receive a file (also provided in .docx, .pdf, etc. format) which, in this case, will undergo an analogous preprocessing or the user will introduce the job offer’s relevant information directly. At this point, the CV’s information plus the job offer’s information will enter a comparator module that, as the name suggests, compares the CV’s data and the job offer’s requirements and returns the affinity between them, quantifying how much the individual’s details comply with the company’s requests. Keep in mind that, to develop the comparator module, two different approaches will be tried, aiming at a clear understanding of the pros and cons of each one of them. Figure 69: Architecture of approach A 3.1. System architecture 50 The architecture of the first approach can be seen in Figure 69. In this architecture, it is shown that, with regard to the job offer, the first path mentioned in the general architecture will be followed. Therefore, both the CV text and/or XML and the job offer text and/or XML will go through an engine that will run a Machine Learning-inspired similarity algorithm and a similarity measure on these inputs and return the affinity between them. Figure 70: Architecture of approach B The architecture of the second approach can be seen in Figure 70. In this architecture, it is shown that the knowledge, that is, all the relevant information from the CV text and/or XML (i.e., the attributes associated with the concepts defined in the ontology presented in Subsection 2.1.7(Personal Information, Executive Summary, Professional Experience, Education, Language and Technology)) needs to be extracted, resorting to the knowledge extractor engine. After the data extraction process, the collected data is gathered and properly structured in an internal representation that corresponds to the content of the CV and, from there, the document builder engine creates a structured document (in this case, a JavaScript Object Notation (JSON) document). In parallel, in this approach, regarding the job offer, the second path referred to in the general architecture will be followed, i.e., the user will fill in a form with the relevant information about the job offer in question (such as the area and education level requested) 3.2. Summary 51 and, the associated interface module, in turn, will return the corresponding structured document (in this case, a JSON document). Lastly, the structured documents together with the rules that define the weights of each requirement will be passed through the similarity algorithm engine which will return the final output, i.e., the affinity between the CV and the job offer. The algorithms designed and implemented to achieve the first approach and the second approach will be described and discussed in detail in Chapter 5and Chapter 6, respectively. 3.2 summary In this chapter, it was presented the proposed approach to solve the main challenge. This approach, in turn, was subdivided into two other approaches that will be tried in order to assess the pros and cons of each one of them. To sum up, in order to find the affinity between the CV and the job offer, the first approach will run a ML algorithm and a similarity measure on the inputs while the second approach will compare the inputs by structuring them first. With that in mind, let us start by explaining the approaches’ engines involved in processing CVs. 4 CVS PROCESSING This chapter aims to clarify not only the work carried out in terms of CVs processing, but also to describe the challenges that have arisen along this path. Therefore, in the next sections, a detailed explanation of the text extractor, XML extractor, knowledge extractor and document builder engines mentioned in Chapter 3will be given. Note that the knowledge extractor and the document builder engines are only used in the comparator module of the second approach. It should also be noted that, from this chapter onwards, only Syone’s internal CV format will be considered, but, in the future, other CV formats may be dealt with following an analogous process. 4.1 text extractor and xml extractor As shown in the general architecture of the system, it is necessary to extract the text and/or XML of the document itself. For that, the Extractor class was developed aided by two subclasses: TextExtractor and XMLExtractor. As their names imply, the TextExtractor subclass is used to extract the content of documents in plain text format while the XMLExtractor subclass is used to extract XML from documents, allowing a more structured analysis. Note that the first one takes advantage of the Apache Tika library through the bindings provided by a Python package (tik) which, in turn, is capable of handling different file types, such as .pdf, .docx, etc.. Also note that, regarding the second one, the logic involves accessing the XML from the .docx files (which are actually .zip files with all the XML files associated with the document in question), as the documents to be processed are in this format. If these were in a different file format (like .pdf), the methodology for extracting their XML would be completely different. The class diagram of the text extractor and XML extractor engines can be seen in Figure 71. 52 4.2. Knowledge extractor 53 Figure 71: Text extractor and XML extractor engines’ class diagram 4.2 knowledge extractor After extracting the text and/or XML from each CV, it is now possible to extract its sections (Personal Information, Executive Summary, Professional Experience, etc.), to later build the corresponding structured document. Therefore, first, the Parser class was created followed by one subclass: CVSyoneParser (capable of extracting information from CVs following Syone’s template). Note that, with this architecture, in the future, it will be possible to add more subclasses capable of parsing different CV formats (e.g., EuropassParser specialized in extracting information from Europass CVs, LinkedInParser specialized in extracting information from LinkedIn profiles as CVs, etc.). Regarding CVSyoneParser, the reasoning for extracting text from the Personal Information, Executive Summary, Professional Experience, Education and Languages sections turned out to be always the same: find the word that indicates the beginning of the section and the word that indicates the end of it (which also indicates the beginning of the next section) and extract the text between them. To ensure that the captured terms are effectively headings and not random words throughout the document, these words are searched along with a newline character. This is done with resource to the re module (re) which provides many functions to deal with regular expressions. Let us take a look at an example. Figure 72 shows an example of an Education section which starts with the keyword "EDUCATION" and ends with the keyword "LANGUAGES" (which, in turn, symbolizes the beginning of another section). From here, it is possible to conclude that, in order to extract the Education section’s text, it is necessary to extract the text between those two keywords. 4.2. Knowledge extractor 54 Figure 72: Example of an Education section When it came to extracting the fields from the Personal Information section (in this case, name, job description, age and gender) the logic was similar. The process started to get more complex in the sections that contained more than one entry (Professional Experience and Education). Initially, an approach was followed where the indexes that mark the beginning and the end of all keywords (Dates (from to), Name and type of organization offering education and training, etc.) were stored in data structures (in this case, lists). Then, according to the length of those lists, in each iteration, the text of the different fields was extracted between the indices representing the respective delimiters. However, this method proved to be impractical because, when some fields were missing, the length of all data structures was not the same. Therefore, there was a need to embark on 4.2. Knowledge extractor 55 another approach. This time, those entries were divided by the separator "Dates (from to)" (see how each entry starts with those keywords on Figure 73 and Figure 74). Figure 73: Separator of each Professional Experience section’s entry Figure 74: Separator of each Education section’s entry 4.2. Knowledge extractor 56 Then, the main reasoning for extracting the fields from each entry was similar to the Personal Information section, with special attention to some cases, such as: • Professional Experience entries without the "Software and Environments used" field; Figure 75: Example of a Professional Experience entry without the "Software and Environments used" field • Professional Experience entries without the "Occupation or position held" field; Figure 76: Example of a Professional Experience entry without the "Occupation or position held" field • Education entries without the "Principal subjects / occupational skills covered" field; 4.2. Knowledge extractor 57 Figure 77: Example of an Education entry without the "Principal subjects / occupational skills covered" field • Education entries with multiple "Title of qualification awarded" fields; Figure 78: Example of an Education entry with multiple "Title of qualification awarded" fields • Education entries without the "Title of qualification awarded" field; 4.3. Document builder 64 4.3 document builder Finally, the DocumentBuilder class was created to produce the structured document (in this case, a JSON document) corresponding to each CV. This is done by traversing the internal document representation and merging all sections of the CV together. The structured document corresponding to the sample CV shown in Appendix B, which is returned by the document builder engine, can be seen in Appendix D. The class diagram of this engine is shown in Figure 87. Figure 87: Document builder engine’s class diagram 4.4 summary In this fourth chapter, the engines involved in processing CVs mentioned in the previous chapter (text extractor, XML extractor, knowledge extractor and document builder) were explained along with the obstacles that arose with their development. As such, it is possible to proceed to the description and discussion of the algorithms designed and implemented to achieve the first approach also presented in Chapter 3. 5 SIMILARITY ALGORITHM (APPROACH A) In this chapter of the dissertation, the engine that compares the CV and the job offer in the first approach presented in Chapter 3is explained, starting by the elucidation of the general coding approach and following with the presentation of the techniques attempted and the respective results which, in turn, are also discussed. 5.1 implementation and results Let us start by showing the application of USE with cosine similarity to unpreprocessed documents. Please, bear in mind that, for the other techniques that were applied, the methodology changes very little. First of all, all of the libraries, modules and functions that will be needed are imported. 1import pandas as pd import tensorflow_hub as hub 3from sklearn . metrics . pairwise import cosine_similarity from util s import get_cvs , get_jobs , highlight_max 5 from syextractor . extractors . text_extractor import TextExtractor Listing 5.1: Import libraries, modules and functions Then, the Universal Sentence Encoder’s TF Hub module is loaded. module_url = " https :// tfhub . dev/google/universal −sentence −encoder/4" 2 model = hub . load ( module_url ) Listing 5.2: Load the USE’s TF Hub module After that, the previously defined model is applied to each job offer’s text in order to obtain their respective word embeddings. Then, the same process is repeated for each CV. Next, the cosine similarity is applied to those word embeddings to calculate the affinity 65 5.1. Implementation and results 66 between them and the results are saved into a dataframe. Finally, the best CV for each job offer is highlighted. 1row_values = {} data = [ ] 3 for job in get_jobs ( ) : 5job_vec = model ( [ TextExtractor ( " . . / data/job_offers/"+job ) . get_content ( ) ] ) for cv in get_cvs ( ) : 7cv_vec = model ( [ TextExtractor ( " ../ data/cvs/"+cv ) . get_content ( ) ] ) row_values . update ( { cv : cosine_similarity ( job_vec , cv_vec ) [ 0] [ 0]}) 9data . append( row_values . copy ( ) ) 11 df = pd . DataFrame ( data , index = get_jobs ( ) ) 13 df . style . highlight_max ( color = ' lightgreen ' , axis = 1) Listing 5.3: General code approach to apply the algorithm and the similarity measure (in this case, USE with cosine similarity) Having the general approach when it comes to code, it is now possible to apply the various algorithms explored in Section 2.3to the existing sample data and evaluate the respective results, so that the one that gives the most accurate outcome can be chosen. It should be noted that the sample to be taken into account contains twenty job offers and one hundred and sixteen CVs, which means that, of the algorithms explored, it was not possible to apply Doc2Vec, since this algorithm needs to be trained and there is a limited amount of data. Also note that the candidates presented throughout this document have been anonymized, in order to ensure their privacy. First of all, Jaccard Similarity, TF-IDF, BERT and USE were applied to the whole sample without any preprocessing, in order to find the most adequate CV for each job offer. Note that, TF-IDF was applied using the scikit-learn library (skl). Besides that, take into account that TF-IDF, BERT and USE are vectorization algorithms, therefore, after the vectorization of the CV and the job offer in question, it was necessary to apply a similarity measure to compute the affinity between the vectors (in this case, cosine similarity, which was also applied using the scikit-learn library). Lastly, bear in mind that there already are BERT models tuned to be used for sentence/text embedding generation, so, initially, the bert-basenli-mean-tokens model (ber) from SentenceTransformers library (Sen) was applied. Like with BERT, there are also pre-trained Universal Sentence Encoder models available, so TF2.0 Saved Model (v4) (Ten,a) from TensorFlow Hub (Ten,b) was employed. The CVs chosen for each job offer based on the application of these algorithms can be found in Table 2. Analyzing the results of this table, it appears that, for the same job offer, the same CV was rarely chosen by the different algorithms. 5.1. Implementation and results 67 Job offer Jaccard Similarity TF-IDF BERT (bert-base-nli-mean-tokens) USE Backend Developer GO Sharon Rodgers Daniel Jacobson David Johnson Jennifer Ortiz Backend Developer Sharon Rodgers Brenda Lynch Tyler Garza Richard Brady BE Developer Amanda Johnson Brenda Lynch Katherine Norris Troy Hobbs Business Development Manager Edward Walters Sean Smith Edward Walters Bethany Guerrero Frontend Developer (Angular) Carolyn Hall Daniel Wilcox Sharon Rodgers Jennifer Ortiz Frontend Developer (Vue.js) (1) Robert Harris Brenda Lynch Sharon Rodgers Jennifer Ortiz Frontend Developer (Vue.js) (2) Carolyn Hall Brenda Lynch Sharon Rodgers Jennifer Ortiz Infrastructure Architect and Pre-Sales Edward Walters Sean Smith Rachel Tucker Donald Shaffer IT Recruiter William Thomas Benjamin Patel Benjamin Patel Bethany Guerrero Junior Developers (1) Daniel Wilcox David Johnson Katherine Norris Patrick Carter ML/Data Engineer Mary Roy Thomas Dixon Tyler Garza Thomas Dixon .NET Developer (1) Robert Harris Daniel Wilcox Katherine Norris Brandy Taylor .NET Developer (2) William Thomas Daniel Wilcox Vincent Carrillo Ann Hicks QA Engineer Edward Walters Katherine Bullock Tyler Garza Katherine Bullock Scrum Master (1) Edward Walters Sean Smith Edward Walters Donald Shaffer Scrum Master (2) Edward Walters Brenda Lynch David Johnson Edward Walters Solution Designer - Hardware Carrie Yodes Sean Smith Carrie Yodes Bethany Guerrero Junior Developers (2) Daniel Wilcox David Johnson Katherine Norris Patrick Carter System Administrator Daniel Wilcox Sarah Nichols Christopher Christian Sarah Nichols Tech Lead/Senior Frontend Daniel Wilcox Ashley Moss Tyler Garza Ashley Moss Table 2: CVs chosen for each job offer based on the application of the algorithms As the candidate chosen for each job offer varies greatly from algorithm to algorithm, it was tried to cross the 5best CVs chosen by each algorithm for each job offer, in order to verify if there were candidates in common. The result of crossing the 5best CVs chosen by the various algorithms for each job offer can be found in the Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11, Table 12, Table 13, Table 14, Table 15, Table 16, Table 17, Table 18, Table 19, Table 20, Table 21 and Table 22. It can be seen that some algorithms do have some candidates in common for certain job offers. However, those intersections still vary a lot between each other. Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Sharon Rodgers, Carrie Yodes TF-IDF Thomas Dixon BERT USE Table 3: Intersection of the top 5CVs chosen by the various algorithms for the Backend Developer GO job offer 5.1. Implementation and results 68 Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Carrie Yodes Sharon Rodgers, Carrie Yodes TF-IDF Carrie Yodes Brenda Lynch, Jennifer Ortiz BERT USE Table 4: Intersection of the top 5CVs chosen by the various algorithms for the Backend Developer job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Robert Harris TF-IDF Thomas Dixon, Troy Hobbs, Brenda Lynch, Jennifer Ortiz BERT USE Table 5: Intersection of the top 5CVs chosen by the various algorithms for the BE Developer job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Edward Walters TF-IDF Wendy Brown Bethany Guerrero BERT USE Table 6: Intersection of the top 5CVs chosen by the various algorithms for the Business Development Manager job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Sharon Rodgers TF-IDF Ashley Moss Troy Hobbs, Brenda Lynch BERT USE Table 7: Intersection of the top 5CVs chosen by the various algorithms for the Frontend Developer (Angular) job offer 5.1. Implementation and results 69 Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Carrie Yodes Carrie Yodes Carrie Yodes TF-IDF Carrie Yodes Ashley Moss, Brenda Lynch, Carrie Yodes BERT Carrie Yodes USE Table 8: Intersection of the top 5CVs chosen by the various algorithms for the Frontend Developer (Vue.js) (1) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Sharon Rodgers TF-IDF Ashley Moss Brenda Lynch BERT USE Table 9: Intersection of the top 5CVs chosen by the various algorithms for the Frontend Developer (Vue.js) (2) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Carrie Yodes Edward Walters, Carrie Yodes Carrie Yodes TF-IDF Carrie Yodes Carrie Yodes BERT Carrie Yodes USE Table 10: Intersection of the top 5CVs chosen by the various algorithms for the Infrasctructure Architect and Pre-Sales job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Benjamin Patel Brandy Taylor, Benjamin Patel Brandy Taylor, Benjamin Patel TF-IDF Benjamin Patel Benjamin Patel BERT Brandy Taylor, Benjamin Patel USE Table 11: Intersection of the top 5CVs chosen by the various algorithms for the IT Recruiter job offer 5.1. Implementation and results 70 Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Daniel Wilcox Sharon Rodgers TF-IDF BERT USE Table 12: Intersection of the top 5CVs chosen by the various algorithms for the Junior Developers (1) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Vanessa Ferguson TF-IDF Thomas Dixon, Patrick Carter, Christopher Jimenez BERT USE Table 13: Intersection of the top 5CVs chosen by the various algorithms for the ML/Data Engineer job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Sharon Rodgers, Robert Harris TF-IDF Virginia Berry BERT USE Table 14: Intersection of the top 5CVs chosen by the various algorithms for the .NET Developer (1) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity TF-IDF Troy Nguyen BERT USE Table 15: Intersection of the top 5CVs chosen by the various algorithms for the .NET Developer (2) job offer 5.1. Implementation and results 71 Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Katherine Bullock Katherine Bullock TF-IDF Katherine Bullock, Nicholas Perkins, Sean Smith, Brenda Lynch BERT USE Table 16: Intersection of the top 5CVs chosen by the various algorithms for the QA Engineer job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Sara Larson, Sean Smith Edward Walters Jeffrey Edwards, Sean Smith TF-IDF Sean Smith BERT USE Table 17: Intersection of the top 5CVs chosen by the various algorithms for the Scrum Master (1) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Edward Walters, Stephen Campbell Edward Walters Edward Walters TF-IDF Edward Walters Edward Walters, Sean Smith BERT Edward Walters USE Table 18: Intersection of the top 5CVs chosen by the various algorithms for the Scrum Master (2) job offer 5.1. Implementation and results 72 Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Wendy Brown, Carrie Yodes Wendy Brown, Carrie Yodes TF-IDF Wendy Brown, Carrie Yodes BERT USE Table 19: Intersection of the top 5CVs chosen by the various algorithms for the Solution Designer - Hardware job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Daniel Wilcox Sharon Rodgers TF-IDF BERT USE Table 20: Intersection of the top 5CVs chosen by the various algorithms for the Junior Developers (2) job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity TF-IDF Sharon Rodgers Sarah Nichols BERT USE Table 21: Intersection of the top 5CVs chosen by the various algorithms for the System Administrator job offer Algorithm Jaccard Similarity TF-IDF BERT USE Jaccard Similarity Daniel Wilcox Sharon Rodgers TF-IDF Ashley Moss Ashley Moss, Jennifer Ortiz, Carolyn Hall BERT Ashley Moss USE Table 22: Intersection of the top 5CVs chosen by the various algorithms for the Tech Lead/Senior Frontend job offer 5.1. Implementation and results 73 However, it was found that the bert-base-nli-mean-tokens model is obsolete and, therefore, produces low quality sentence embeddings. Thus, it was necessary to apply another model. For that, the all-mpnet-base-v2(all,a) model (also from SentenceTransformers library) was chosen, since it has the highest average performance according to (Pre). The CVs chosen for each job offer based on the application of this model can be found in Table 23. Analyzing the results of this table, it can be seen that, for each job offer, each CV chosen is different when compared to the previous model, except for the Scrum Master (1) job offer. Job offer BERT (all-mpnet-base-v2) Backend Developer GO Andrew Long Backend Developer Tiffany Orozco BE Developer Sharon Rodgers Business Development Manager Sean Smith Frontend Developer (Angular) Tiffany Orozco Frontend Developer (Vue.js) (1) Tiffany Orozco Frontend Developer (Vue.js) (2) Tiffany Orozco Infrastructure Architect and Pre-Sales John Richard IT Recruiter Roger Barrett Junior Developers (1) Tiffany Orozco ML/Data Engineer Thomas Dixon .NET Developer (1) Troy Nguyen .NET Developer (2) Ann Hicks QA Engineer Katherine Bullock Scrum Master (1) Edward Walters Scrum Master (2) Edward Walters Solution Designer - Hardware Hannah Santos Junior Developers (2) Tiffany Orozco System Administrator Cynthia Williams Tech Lead/Senior Frontend Tiffany Orozco Table 23: CVs chosen for each job offer based on the application of the all-mpnet-base-v2model Despite being the one with the best average performance, the all-mpnet-base-v2model is also one of the slowest, as expected. Thus, the all-MiniLM-L6-v2(all,b) model (also from SentenceTransformers library) was applied as well, since it is one of the models with the best average performance/speed ratio (also according to (Pre)). The CVs chosen for each job offer based on the application of this model can be found in Table 24. Analyzing the results of this table, it can be seen that, for each job offer, each CV chosen is different when compared to the previous two models, except for the Frontend Developer (Angular) job offer which has the same candidate chosen for this and for the all-mpnet-base-v2model. 6 SIMILARITY ALGORITHM (APPROACH B) In this chapter, the engine that compares the CV and the job offer in the second approach presented in Chapter 3is explained and the results of its application are discussed. 6.1 implementation and results In this approach, and similarly to the CVs, it was also necessary to extract a structured document from each job offer. However, unlike the CVs, the existing job offers do not follow a pattern and, for that reason, it was not possible to follow the approach described in Section 4.1, Section 4.2and Section 4.3. Instead, it was necessary to create a form so that the HR team could fill in the necessary information regarding the job offer and, from there, the respective structured document could be generated. For instance, suppose the HR team filled out the form based on the job offer from Appendix A. In this case, the corresponding structured document would be the JSON document from Appendix C. Having the form, it was necessary to create a logic to calculate the percentage of similarity between a job offer and a CV. This logic involved calculating the score obtained by the candidate in each section (in this case, Professional Experience, Education, Languages and Technologies) and also calculating the maximum score that the candidate could obtain in those same sections. After that, the first score is divided by the latter, in order to give the percentage obtained by the candidate in each section. Then, each value is multiplied by the respective weight assigned to the section in question (since a section can have a different importance according to the job offer) and, finally, the sum of each product results in such percentage of similarity between a job offer and a CV. That said, it is important to mention that the reasoning for calculating the score obtained by the candidate in each section is common to all sections, since it is always subdivided into two other calculations: calculation of the score obtained by each professional experience/education/language/technology found and calculation of the score obtained by each professional experience/education/language/technology not found. 80 6.1. Implementation and results 81 Thus, and starting with the Professional Experience section, it is worth recalling that, although a job offer may ask for experience in a specific role (such as Machine Learning Engineer), other roles may be related and, therefore, also valuable (such as Data Engineer, Data Scientist, etc.). As a consequence, it was found useful to have a document for each related role, that is, for the Machine Learning Engineer role there will be a document with those keywords, plus Data Engineer, Data Scientist, etc., for the IT Recruiter role there will be a document with those keywords, plus Head of Human Resources, etc. and so on. Furthermore, it was concluded that the score based on the number of years a candidate has held a role should increase towards a limit, not only because the difference between 1and 5years should be much more significant than the difference between 10 and 15 years, for example, since there is a point where there is not much more to learn, but also because that limit is necessary to find the maximum score that the candidate can obtain in this section, since there is not a maximum number of years of experience a person can have. The function in question and the respective graph can be seen below. Notice how the curve grows exponentially until approximately x=10 and, from there, it becomes much less steep, getting closer and closer to the value 25. f(x) = 25 1−1 x+1(9) Figure 88: Graph of math function that returns score based on number of years That said, regarding the calculation of the score obtained for each professional experience found, for each Professional Experience entry of the job offer, the respective document related to the role in question is opened and if any of the keywords existing in that document is found in any Professional Experience entry of the CV, one of the following applies: • If the duration field exists in the job offer and the duration field in the CV (which is calculated using the start date and the end date with the help of the dateparser library and the dateutil module (dat,c)) is equal to or greater than the required duration, 1 is added to the value resulting from subtracting the duration in the job offer to the duration in the CV and f(x)is applied to this result, so that the candidate is valued; 6.1. Implementation and results 82 • If the duration field exists in the job offer and the duration field in the CV is less than the required duration, the symmetric of f(x) is applied to the absolute value resulting from subtracting the duration in the job offer to the duration in the CV, so that the candidate is devalued; • If the duration field does not exist in the job offer, f(x) is applied to the duration in the CV, so that the candidate is valued (since, in this case, all experience must be valued). On the other hand, with regard to the calculation of the score obtained for each professional experience not found, a methodology similar to that explained above is applied to verify if any role of any Professional Experience entry of the job offer (or similar) is not found in any Professional Experience entry of the CV and, if so, one of the following applies: • If the duration field exists in the job offer, the symmetric of f(x) is applied to the value in that field, since, in this calculation, the candidate always needs to be devalued; • If the duration field does not exist in the job offer, the candidate receives -10 points (because, as mentioned before, they have to be devalued). Finally, when it comes to calculating the maximum score that the candidate could obtain in this section, since the calculation of the score obtained for each professional experience found always resorts to f(x) , that maximum score equals to 25 points (limit of the function) for each Professional Experience entry of the job offer. Continuing with the Education section, it is worth recalling that, like with the Professional Experience section, although a job offer may ask for a certain education level in a specific field, other fields may be related and, therefore, also valuable. For that reason, the methodology involving documents with related fields was also used in this section. Furthermore, please note that a candidate may have more than one level of education for the same required field, i.e., imagine that the job offer asks for a Bachelor Degree in Computer Science and the candidate not only satisfies this requirement, but also has a Master Degree in Computer Science. In this case, only the latter should be considered, as it not only meets the requirement, but also exceeds it (note that a Master Degree is a higher education level than a Bachelor Degree). Lastly, keep in mind that for each education level a numerical value was mapped, in order to be able to relativize the different levels and make calculations between them. Note that each numerical value represents the typical number of years the respective education level generally lasts plus the numerical value of the previous education level (Bachelor Degree was mapped to 3, Post-Graduation was mapped to 4and so on). That said, with regard to the calculation of the score obtained for each education found, for each Education entry of the job offer, a methodology similar to that explained for the Professional Experience section is applied to verify if any field of any Education entry of the job offer (or similar) is found in any Education entry of the CV and, if so, the maximum 6.1. Implementation and results 83 level of education in that field is extracted from the CV. In this situation, one of the following applies: • If the maximum education level in the CV is equal to or greater than the education level in the job offer and the candidate has finished the degree, 1is added to the value resulting from subtracting the latter to the first, so that the candidate is valued; • If the maximum education level in the CV is less than the education level in the job offer and the candidate has finished the degree, the latter is subtracted to the first, in order to devalue the candidate; • If the maximum education level in the CV is equal to or greater than the education level in the job offer and the candidate has not finished the degree, the latter is subtracted to the first, so that the candidate is valued (but not as much as in the first case); • If the maximum education level in the CV is less than the education level in the job offer and the candidate has not finished the degree, 1is subtracted to the value resulting from subtracting the latter to the first, so that the candidate is devalued (even more than in the second case); On the contrary, when it comes to the calculation of the score obtained for each education not found, for each Education entry of the job offer, a methodology similar to that explained for the Professional Experience section is applied to verify if any field of any Education entry of the job offer (or similar) is not found in any Education entry of the CV and, if so, the candidate receives the symmetric of the result of adding 1to the education level requested in the job offer, in order to devalue the candidate (evidently, even more than a candidate who has a degree in the required field, but a lower education level). At last, with regard to the calculation of the maximum score that the candidate could obtain in this section, similarly to the first case in the explanation of the calculation of the score obtained for each education found (which is the one that results in the highest score), for each Education entry of the job offer, 1is added to the value resulting from subtracting the level of education requested to the maximum level of education (in this case, Doctorate Degree which, in turn, is equivalent to 10). Proceeding with the Languages section, take into account that, in this case, there are only five possible language levels, so, for the same reasons pointed out for the Education section, a numerical value was also mapped to each of those levels (Basic was mapped to 1, Average was mapped to 2, Good was mapped to 3, Very Good was mapped to 4and Native was mapped to 5). In addition, all CVs have a proficiency level regarding reading, writing and speaking, so the average of these three values was used as the language level of the CV, in order to be able to compare with the language level of the job offer. 6.1. Implementation and results 84 That said, with regard to the calculation of the score obtained for each language found, it is verified whether the language for each Language entry of the job offer is found in any Language entry of the CV and, if so, one of the following applies: • If the language is required and the language level in the CV is equal to or greater than the language level in the job offer, 1is added to the value resulting from subtracting the latter to the first, so that the candidate receives 1point in case they have the same language level, 2points in case they have one language level above, etc. and, therefore, is valued; • If the language is required and the language level in the CV is less than the language level in the job offer, the latter is subtracted to the first, so that the candidate receives -1point in case they have one language level below, -2points in case they have two language levels below, etc. and, so, is devalued; • If the language is not required, 0.5is added to the result of dividing the result of subtracting the language level in the job offer to the language level in the CV by 10, so that the candidate receives 0.4if they have one language level below, 0.5if they have the same language level, 0.6if they have one language level above, etc., so that the candidate is never devalued, but does not obtain a greater appreciation than in the first case. In contrast, with regard to the calculation of the score obtained for each language not found, it is verified whether the language for each Language entry of the job offer is not found in any Language entry of the CV and, if so and if that language is required, the candidate receives the symmetrical of the language level required by the job offer, as they should only be devalued for the required languages that they do not have. Finally, when it comes to the calculation of the maximum score that the candidate could obtain in this section, for each Language entry in the job offer, there are two possible situations: • In case that language is required, a similar approach to the one explained in the first case of the calculation of the score obtained for each language found (which, for required languages, is the one that results in the highest score) is applied, that is, 1is added to the value resulting from subtracting the language level required by the job offer to the maximum language level (in this case, and as explained before, Native); • In case that language is not required, a similar approach to the one explained in the third case of the calculation of the score obtained for each language found is applied, i.e.,0.5is added to the result of dividing the result of subtracting the language level required by the job offer to the maximum language level by 10. 6.1. Implementation and results 85 Last but not least, let us take a look at the Technologies section. First of all, this section is subdivided into several subsections (Programming Languages, Databases, Operating Systems and Tools) which, in turn, and as for each section, may also have a different importance according to the job offer and, therefore, must have weights assigned to them as well. On top of that, it is important to mention that all of those subsections are very similar to each other, as they all have analogous fields. Thus, both the calculation of the score obtained for each technology (programming language, database, operating system or tool) found and the calculation of the score obtained for each technology not found were done in such a generic way that they can be used by all of those subsections. Taking these two considerations into account, it is easy to see that the calculation of the score obtained by the candidate in this section will involve the calculation of the score obtained by each technology found and also the calculation of the score obtained by each technology not found (as expected), but, now, for each Technologies subsection and, then, multiplying these results by the respective weights mentioned above. In addition, in this case, there are only five possible technology levels, so, once again, for the same reasons pointed out for the Education section, a numerical value was also mapped to each of those levels (Basic was mapped to 1, Average was mapped to 2, Good was mapped to 3, Very Good was mapped to 4and Advanced was mapped to 5). Furthermore, unlike the previously mentioned sections, each subsection of this section has not just one, but two fields that play a major role when it comes to calculating scores. These fields are related to the technology level and duration and, therefore, in this case, the score calculations will always take these two parcels into account. Moreover, f(x) was also used in this section for reasons analogous to those explained for the Professional Experience section. Having that in mind, as to the calculation of the score obtained for each technology found, it is verified whether each entry in the Technologies subsection in question of the job offer is found in any entry in the Technologies subsection in question of the CV and, if so, one of the following applies: • If the technology is required: –Regarding the score related to the technology level, there are two possible cases: * If the technology level in the CV is equal to or greater than the technology level in the job offer, 1is added to the value resulting from subtracting the latter to the first, in order to value the candidate (as explained for the Languages section); * If the technology level in the CV is less than the technology level in the job offer, the latter is subtracted to the first, in order to devalue the candidate (once again, as explained for the Languages section); 6.1. Implementation and results 86 – Regarding the score related to the duration, there are three possible cases (similar to those explained for the Professional Experience section): * If the duration field exists in the job offer and the duration field in the CV (which, once more, is calculated using the start date and the end date with the help of the dateparser library and the dateutil module) is equal to or greater than the required duration, 1is added to the value resulting from subtracting the duration in the job offer to the duration in the CV and f(x) is applied to this result, so that the candidate is valued; * If the duration field exists in the job offer and the duration field in the CV is less than the required duration, the symmetric of f(x) is applied to the absolute value resulting from subtracting the duration in the job offer to the duration in the CV, so that the candidate is devalued; * If the duration field does not exist in the job offer, f(x) is applied to the duration in the CV, so that the candidate is valued (since, in this case, all experience must be valued). • If the technology is not required: – Regarding the score related to the technology level, an analogous calculation to that explained for not required languages is applied here, so that the candidate always receives an appreciation, but not as much as if it was a required technology; –Regarding the score related to the duration, there are three possible cases: * If the duration field exists in the job offer and the duration field in the CV is equal to or greater than the required duration, just 0.5is added to the value resulting from subtracting the latter to the first and f(x) is applied to this result, in order to value the candidate, but, once more, not as much as if it was a required technology; * If the duration field exists in the job offer and the duration field in the CV is less than the required duration, the candidate receives 0points, so that they are not valued (since they do not satisfy the requirements regarding this field) nor devalued (since the technology in question is not required); * If the duration field does not exist in the job offer, 0.5is multiplied by the result of applying f(x) to the duration in the CV, so that the candidate is valued, but, anew, not as much as if it was a required technology; Meanwhile, as to the calculation of the score obtained for each technology not found, it is verified whether each entry in the Technologies subsection in question of the job offer is not found in any entry in the Technologies subsection in question of the CV and, if so and if 6.1. Implementation and results 87 that technology is required (because, once again, the candidate should only be devalued for the required technologies that they do not have), one of the following applies: • If the duration field exists in the job offer, the candidate receives the symmetric of the result of adding the technology level required by the job offer to the result of applying f(x)to the duration required by the job offer; • If the duration field does not exist in the job offer, the candidate receives just the symmetric of the technology level required by the job offer. Ultimately, regarding the calculation of the maximum score that the candidate could obtain in this section, for each entry of each Technologies subsection of the job offer, there are two possible situations: • In case that technology is required, 25 points, which is the limit of the function always used in the calculation of the score obtained for each technology found regarding the score related to the duration, are added to the result of applying a similar approach to the one explained in the first case of the calculation of the score obtained for each technology found regarding the score related to the technology level, that is, 1is added to the value resulting from subtracting the technology level required by the job offer to the maximum technology level (in this case, and as explained before, Advanced); • In case that technology is not required, 25 points, which come from a similar scenario to the one explained above, are added to the result of applying a similar approach to the one explained in the calculation of the score obtained for each technology found regarding the score related to the technology level, that is, 0.5is added to the result of dividing the result of subtracting the technology level required by the job offer to the maximum technology level by 10. After that, all the maximum scores computed for each Technologies subsection have to be multiplied by the respective weights mentioned above (and, of course, all the results are added together to, then, give the maximum score that the candidate could obtain in the Technologies section). Having the calculation of the score obtained by the candidate in each section and the calculation of the maximum score that the candidate could obtain in those same sections, it is now possible to calculate the percentage of similarity between a job offer and a CV, as explained at the beginning of this chapter, paying special attention to the case in which the maximum score that the candidate can obtain is 0(which results in a percentage of 0, as it is not possible to divide by this value) and to the case in which the final calculation results in a negative value (which also results in a percentage of 0, because there can be no negative percentages). 6.1. Implementation and results 88 At this stage, it is important to rank the candidates according to the job offer they applied for from best to worst according to their percentage of similarity and display only the best x , x being a configurable parameter that is stored in a dictionary of settings along with the weights mentioned throughout this chapter (which, of course, are also configurable). That said, applying the developed algorithm to the same one hundred and sixteen CVs referred to in Chapter 5according to the job offer presented in Appendix Awith the configurations shown below, leads to the results presented in Table 30. 1configuration = { " limit " : 5, 3" weights " : { " sections " : { 5"professional_experience": {"weight": 0.2 5 } , " education " : { " weight " : 0.2 5 } , 7" languages " : { " weight " : 0.2 5 } , "technologies": { 9"weight" : 0.2 5 , " subsections " : { 11 " programming_languages " : { " weight " : 0.2 5 } , " databases " : { " weight " : 0.2 5 } , 13 "operating_systems" : {"weight" : 0.2 5 } , " tools " : { " weight " : 0.2 5 } , 15 } , } , 17 } } , 19 } Listing 6.1: Dictionary of settings used as an example Candidate Percentage of Similarity (2d.p.) Thomas Dixon 71.71% Patrick Carter 58.21% Ashley Lawrence 35.37% Christopher Jimenez 28.66% Melvin Fields 20.62% Table 30: Top 5candidates for the job offer presented in Appendix A These results were examined both in terms of calculations and in terms of order and, from there, it was concluded that they are plausible. It should be noted that most of these candidates have experience in the field of Machine Learning, have a degree in Computer Science or similar, demonstrate fluency in English and satisfy some of the necessary technologies, which makes them more suitable for the job offer in question. 6.2. Summary 89 6.2 summary In this sixth chapter, the similarity algorithm engine of the second approach presented in Chapter 3was explained in detail. After testing the algorithm, it was concluded that the results were plausible and, therefore, it was feasible to solve the initial problem. At this point, it is finally possible to summarize all the work and present the conclusions achieved throughout this Master’s Project. 96 Also notice that these results, effectively, make sense, unlike the previous approach where the results not only varied from technique to technique, but also returned candidates who were not suitable for the job offers in question. In particular, for the job offer in Appendix A, were returned candidates who have a background in DevOps, for example, which has nothing to do with the Machine Learning expertise required. These unfortunate results might be explained by the fact that the similarity algorithms used in the first approach deal with the documents as a whole, which means that certain requirements demanded by the job offer for a specific field may be found, but in the wrong field, which conducts to misleading results. So, it is fair to conclude that the Master’s Work here reported proved that it is possible to develop an algorithm capable of automatically matching the most suitable applicants to a certain job offer. However, as future work, there are some details that, in my opinion, could be added to make this algorithm more robust. In the first place, candidates who present more skills than those requested should be valued. Furthermore, it would be interesting to give a negligible value to candidates who, despite not meeting the requirements in terms of programming languages, have experience in similar programming languages (for example, if the job offer asks for Java, but, instead, the candidate has Python, which can also be used for objectoriented programming, they should be valued). Additionally, it may be helpful to provide details on the attributes of the analyzed CV that match the requirements of the job offer. Moreover, it would be useful to develop a Graphical User Interface (GUI), so that users (in this case, the HR team) could interact with the algorithm in a much more intuitive and pleasant way. Lastly, although this Master’s Project has been more directed towards Syone’s CV format, in future iterations, and as said before, it would be interesting to adapt this work to other CV formats, such as Europass CVs, LinkedIn profiles as CVs, etc. and, consequently, being able to handle more and more formats. A SAMPLE JOB OFFER This appendix shows a sample job offer. 97 ML/Data Engineer Job description We're looking for a Data engineer to join a major client! Requirements What do I need to bring?  Proven experience as MLE/DE  BSc in Computer Science or a similar field, with a very solid Spark and Python programming experience; Scala is a plus - for creating data pipelines that will fuel machine learning models in an Azure Databricks based environment  Proven experience with the engineering aspects of popular machine learning practices, libraries, and platforms - such as the ones that allow serving and productionasing machine learning models  Acquaintance with CD4ML and MLOps, software design patterns and RESTful APIs  Main tech stack: Databricks, Spark, Airflow, MLflow, MLeap, Kafka, Delta Lake and Azure ADLS (Parquet, Avro), Azure Devops.  Proficient in code versioning tools: Github  Strong English communicator;  Comfortable with an Agile methologies (Scrum, Kanban) using Jira Agile What can Syone offer me?  Integration in an organization with profound and sustained growth and involvement in pioneering projects with innovative technological solutions;  Strong IT training plans;  Professional evolution with intervention in ambitious technological projects, both national and internationally. B SAMPLE CV This appendix shows a sample CV. 99 • • • • • • • • • • • • • ISCTE-IUL • • • • • • • GS1 Portugal • • KCSIT – Timwe Group • • IT Sector – Millenium BCP • • • D JSON FILE CORRESPONDING TO THE SAMPLE CV This appendix shows the JSON file corresponding to the sample CV presented in Appendix B. 1{ 2"personal_information":{ 3"name":"Thomas Dixon", 4"job":"Machine Learning Engineer", 5"age":"30", 6"gender":"Male" 7}, 8"executive_summary":"The consultant is taking a PhD in Computer Vision in University of Coimbra and hold a master's degree in computer science and Business Management from ISCTE and a bachelor's degree in Computer Science and Engineering from Instituto Polytechnic of Castelo Branco. The consultant started the career in 2014 at IT Sector (Millenium BCP project) as a Java developer. In 2016,changed to GS1,to work with Outsystems,as fullstack developer. Between 2017 and 2019,assumed functions as Data Scientist and Machine Learning Engineer. The consultant was responsible for developing a Document classification system,where was able to obtain 70% of accuracy on a multi-label classification problem using TF-IDF and SVM and worked on an object detection module for automated document parsing ( PDFs) using state of the art Deep Learning techniques. At Celfocus as a Machine Learning Engineer,was responsible for building and developing multi label classification models,building machine learning algorithms, creation of pipeline and a classification model for anomaly detection in Set up Boxes,using Isolation Forest. Regularly works with Graphs to solve problems like costumer churn and client segmentation. Currently,works at Syone managing and leading a team of machine learning composed by 5members ,present the work done to stakeholders other than that he is working on a retail client where was responsible to maintain and create new machine 112 113 learning pipelines. The consultant gives python training and architecture new proposals. The consultant is a team-player and teammate,a good communicator,that enjoys apply to projects that everyone is committed working towards the same goal.", 9"professional_experience":[ 10 { 11 "date_from_professional":"2021-05-01", 12 "date_to_professional":"present", 13 "company_name":"Syone", 14 "occupation":"Lead Machine learning Engineer", 15 "sector":"Consultancy", 16 "activities":"During this experience,the consultant was responsible for the following activities:\n Responsible for leading a team with 5members on 2different projects:customer churn in telecommunications and building an NLP module that classify sentiment and extract Entities from phrases;\n Responsible for Cognitive Board where specific tasks are assigned to each member of the team\n Responsible for the architecture of the previously modules and designing a REST API to interact with the modules and integrate with other services;\n Responsible for mentoring two team members during an internship where was created a scrapping and classification algorithms. The purpose was to create an algorithm that gives the best cv match against job offers;\n Responsible to maintain and add multiple machine learning models in production ( Databricks);\n Development of NLP model to predict whether a review need to answer or not;", 17 "software_used":"Python,Databricks,MLFlow,Azure Service Studio, FastAPI Jupyter notebooks,Jira,Gitlab." 18 }, 19 { 20 "date_from_professional":"2019-10-01", 21 "date_to_professional":"2021-05-31", 22 "company_name":"Celfocus", 23 "occupation":"Machine learning Engineer", 24 "sector":"Telecommunications", 25 "activities":"During this experience,the consultant was responsible for the following activities:\n Responsible for building and developing multi label classification models to classify user requests. with multi hierarchical levels,three levels. The first level had 17 114 categories,we achieve a F-score of 78% on testing set. On the second level we achieve a F-Score of 60%. We use,TF-IDF,SVM,Naive Bayes,Random Forest,and KNN to create our models;\n Responsible for building unsupervised machine learning algorithms (Standard Deviation from average, MAD,Z-score,Grubbs),and the creation of pipeline for anomaly detection;\ n Responsible for mentoring one team member in a time series data science project where we tested unsupervised models MAD,ARIMA,Isolation Forest;\n Responsible for creating new business value proposals and presenting them to stakeholders;\n Development of classification model to detect anomalies in in Set up Boxes using Isolation Forest.", 26 "software_used":"Python,Scala,Apache Spark,Jira,Pyspark,AWS ( S3,EMR),GitLab,Zeppelin,Jupyter Notebooks,Python Bokeh." 27 }, 28 { 29 "date_from_professional":"2019-09-01", 30 "date_to_professional":"2020-02-29", 31 "company_name":"ISCTE-IUL", 32 "occupation":"Instructor", 33 "sector":"Teaching", 34 "activities":"During this experience,the consultant was responsible for the following activities:\n Responsible for create lecture contents and teaching hands-on classes on Big Data Algorithms.", 35 "software_used":"Apache Spark" 36 }, 37 { 38 "date_from_professional":"2017-05-01", 39 "date_to_professional":"2019-10-31", 40 "company_name":"Novabase", 41 "occupation":"Data Scientist and Machine Learning Engineer", 42 "sector":"Finance", 43 "activities":"During this experience,the consultant was responsible for the following activities:\n Responsible for proposal creation,status report,in a python module for categorize,project management documents using Heuristics (specific keywords) and machine learning techniques (Data Extraction,Data cleaning,TF-IDF,SVM,RF,DT);\ n Mentoring in a 1-month internship in a data science project whereas the main purpose is evaluating if the house prices will be increase or decrease ;\n Create a parser library to convert PDF,Image,HTML files into Data 115 Objects,Using Heuristics (Regex,Keywords) and Deep Learning for object detection localization using Faster R-CNN,SSD,YOLO networks;\n Bug fixing and code refactor on a document platform,which classify documents using machine learning techniques NLP.", 44 "software_used":"Java,Python,Deep Learning,Image Processing, Tensorflow." 45 }, 46 { 47 "date_from_professional":"2016-04-01", 48 "date_to_professional":"2017-05-31", 49 "company_name":"GS1Portugal", 50 "occupation":"Full stack developer", 51 "sector":"Retail", 52 "activities":"During this experience,the consultant was responsible for the following activities:\n Developed a tracking platform system for food products that come from different countries and are mixed. e.g.:sausage production.", 53 "software_used":"Outsystems,.Net,JavaScript,CSS,HTML5." 54 }, 55 { 56 "date_from_professional":"2015-04-01", 57 "date_to_professional":"2016-04-30", 58 "company_name":"KCSIT - Timwe Group", 59 "occupation":"Backend developer", 60 "sector":"Telecommunications", 61 "activities":"During this experience,the consultant was responsible for the following activities:\n Developing Billing Api's integrated with Mobile Operators in order to charge clients when they by some content on our platform.", 62 "software_used":"Java,Shell Scripting,REST,SOAP,SVN,JUnit, Tomcat Junit" 63 }, 64 { 65 "date_from_professional":"2014-07-01", 66 "date_to_professional":"2015-04-30", 67 "company_name":"IT Sector - Millenium BCP", 68 "occupation":"Backend developer", 69 "sector":"Finance", 116 70 "activities":"During this experience,the consultant was responsible for the following activities:\n Incident Analysis in Mobile and Web Channels of the bank;\n Developing new features and web services for web and mobile consumption. Ongoing maintenance.", 71 "software_used":"Java,Shell Scripting,SOAP,JUnit,TFS." 72 } 73 ], 74 "education":[ 75 { 76 "date_from_education":"2021-01-01", 77 "date_to_education":"present", 78 "organisation":"University of Coimbra", 79 "subjects":"", 80 "qualification":"PhD in Eletrotechnical Engineering - Specialization in Computer Vision" 81 }, 82 { 83 "date_from_education":"2017-01-01", 84 "date_to_education":"2020-12-31", 85 "organisation":"ISCTE-IUL - University of Lisbon", 86 "subjects":"", 87 "qualification":"Master's Degree in Computer Science and Business management" 88 }, 89 { 90 "date_from_education":"2010-01-01", 91 "date_to_education":"2014-12-31", 92 "organisation":"IPCB Institute Polytechnic of Castelo Branco", 93 "subjects":"", 94 "qualification":"B.S in Computer Science and Engineering" 95 } 96 ], 97 "languages":[ 98 { 99 "language":"Portuguese", 100 "reading":"Native", 101 "writing":"Native", 102 "speaking":"Native" 117 103 }, 104 { 105 "language":"English", 106 "reading":"Very Good", 107 "writing":"Very Good", 108 "speaking":"Very Good" 109 }, 110 { 111 "language":"Spanish", 112 "reading":"Basic", 113 "writing":"Basic", 114 "speaking":"Basic" 115 }, 116 { 117 "language":"German", 118 "reading":"Basic", 119 "writing":"Basic", 120 "speaking":"" 121 } 122 ], 123 "technologies":{ 124 "programming_languages":[ 125 { 126 "name_programming":"Java", 127 "level_programming":"Good", 128 "date_from_programming":"2014-01-01", 129 "date_to_programming":"2016-12-31", 130 "description_programming":"" 131 }, 132 { 133 "name_programming":"Python", 134 "level_programming":"Good", 135 "date_from_programming":"2017-01-01", 136 "date_to_programming":"present", 137 "description_programming":"" 138 }, 139 { 140 "name_programming":"Scala", 118 141 "level_programming":"Average", 142 "date_from_programming":"2021-01-01", 143 "date_to_programming":"2021-12-31", 144 "description_programming":"" 145 }, 146 { 147 "name_programming":"Javascript", 148 "level_programming":"Basic", 149 "date_from_programming":"2017-01-01", 150 "date_to_programming":"2017-12-31", 151 "description_programming":"" 152 }, 153 { 154 "name_programming":".NET", 155 "level_programming":"Basic", 156 "date_from_programming":"2016-01-01", 157 "date_to_programming":"2017-12-31", 158 "description_programming":"" 159 }, 160 { 161 "name_programming":"Outsystems", 162 "level_programming":"Average", 163 "date_from_programming":"2016-01-01", 164 "date_to_programming":"2017-12-31", 165 "description_programming":"" 166 }, 167 { 168 "name_programming":"HTML", 169 "level_programming":"Average", 170 "date_from_programming":"2016-01-01", 171 "date_to_programming":"2017-12-31", 172 "description_programming":"" 173 }, 174 { 175 "name_programming":"CSS", 176 "level_programming":"Average", 177 "date_from_programming":"2016-01-01", 178 "date_to_programming":"2017-12-31", 119 179 "description_programming":"" 180 } 181 ], 182 "databases":[ 183 { 184 "name_database":"PostgreSQL", 185 "level_database":"Good", 186 "date_from_database":"2018-01-01", 187 "date_to_database":"2019-12-31", 188 "description_database":"" 189 }, 190 { 191 "name_database":"Oracle", 192 "level_database":"Good", 193 "date_from_database":"2014-01-01", 194 "date_to_database":"2016-12-31", 195 "description_database":"" 196 }, 197 { 198 "name_database":"MSSQL", 199 "level_database":"Average", 200 "date_from_database":"2021-01-01", 201 "date_to_database":"2021-12-31", 202 "description_database":"" 203 }, 204 { 205 "name_database":"Big data databases", 206 "level_database":"Average", 207 "date_from_database":"2018-01-01", 208 "date_to_database":"present", 209 "description_database":"" 210 } 211 ], 212 "operating_systems":[ 213 { 214 "name_osys":"Linux (Ubuntu,CentOS)", 215 "level_osys":"Good", 216 "date_from_osys":"", 120 217 "date_to_osys":"", 218 "description_osys":"" 219 }, 220 { 221 "name_osys":"Windows", 222 "level_osys":"Very Good", 223 "date_from_osys":"", 224 "date_to_osys":"", 225 "description_osys":"" 226 } 227 ], 228 "tools":[ 229 { 230 "name_tool":"Intellij Idea", 231 "level_tool":"Good", 232 "date_from_tool":"", 233 "date_to_tool":"", 234 "description_tool":"" 235 }, 236 { 237 "name_tool":"VS code", 238 "level_tool":"Good", 239 "date_from_tool":"", 240 "date_to_tool":"", 241 "description_tool":"" 242 } 243 ] 244 } 245 } Listing D.1: JSON file corresponding to the sample CV BIBLIOGRAPHY Academic-careers-cvs-2017.pdf. https://cdn.uconnectlabs.com/wp-content/uploads/ sites/25/2016/06/Academic-Careers-CVs-2017.pdf. Accessed: 2021-11-30. Caa guidelines | standards & guidelines | caa. https://www.collegeart.org/ standards-and-guidelines/guidelines/visual-art-cv. Accessed: 2021-11-30. Ciência vitae. https://cienciavitae.pt/. Accessed: 2021-11-15. Home | europass. https://europa.eu/europass/pt. Accessed: 2021-11-15. Plataforma lattes. https://lattes.cnpq.br/. Accessed: 2021-11-15. Linkedin portugal: entre ou cadastre-se. https://pt.linkedin.com/ . Accessed: 2021-11-22. Ontodl+. https://epl.di.uminho.pt/~gepl/GEPL_DS/OntoDL/index.html . Accessed: 202111-24. Penn treebank ii tag set | clips. https://web.archive.org/web/20190206204307/https: //www.clips.uantwerpen.be/pages/mbsp-tags. Accessed: 2021-12-14. Pretrained models — sentence-transformers documentation. https://www.sbert.net/docs/ pretrained_models.html. Accessed: 2022-06-03. Sentencetransformers documentation — sentence-transformers documentation. https:// www.sbert.net/. Accessed: 2022-05-23. Tensorflow hub. https://tfhub.dev/google/universal-sentence-encoder/4 , a. Accessed: 2022-05-25. Tensorflow hub. https://www.tensorflow.org/hub, b. Accessed: 2022-05-25. Universal pos tags. https://universaldependencies.org/u/pos/. Accessed: 2021-12-20. sentence-transformers/all-mpnet-base-v2· hugging face. https://huggingface.co/ sentence-transformers/all-mpnet-base-v2, a. Accessed: 2022-06-03. sentence-transformers/all-minilm-l6-v2· hugging face. https://huggingface.co/ sentence-transformers/all-MiniLM-L6-v2, b. Accessed: 2022-06-07. 121