Practical Experiences in Web Engineering

Practical Experiences in Web Engineering M.J. Escalona*, J.J. Gutierrez*, D. Villadiego*, A. León** *** *University of Seville, Spain,{escalona,javierj,villadiego}@lsi.us.es ** DMR-Consulting, Seville, Spain,,[email protected] *** State University of Campinas, SP – Brazil, [email protected] Abstract: Web Engineering is defined like a new area to propose models, techniques, processes, architectures, etc. in order to deal correctly with the special characteristics of the web environment. In the last years, new methodological approaches appeared in this environment. However, Web Engineering is not often applied in industries and real projects. This paper presents a general vision of a web approach, named NDT (Navigational Development Techniques) and it is focused on the study of its practical applications. 1 Introduction Since the Net of Nets was born in the 70’s, as a net to spread research material, an amazing change in the use of Internet has taken place. In the last years, Internet has become a popular tool and the number of users who work every day with it has grown crazily. Companies and organizations find in Internet a suitable way to present their businesses, and also, a powerful way to contact with their clients and employees all over the world [14]. This evolution, the high advance of communications and the increase in the benefits of the equipment, networks and routes of transmission have fomented to the fact that most of the actual systems are developed or adapted to internet. Since the development of software systems in Internet appeared, the research community has detected the necessity of proposing new methodologies, techniques and models to offer a suitable reference environment for the new and special characteristics of internet. For this aim, a new research line in the Software Engineering has been developed in the last years: Web Engineering [5]. Web Engineering is the systematic, structured and quantifiable application of methodological approaches to the development, evaluation and , J. Torres process that guarantees the quality of the results. When the Web Engineering appeared as a new research line, several new methodological approaches were proposed and some surveys and comparative studies agreed that it was necessary to offer new methodological environments to deal with the special characteristics of the web [2][3][12][13][24]. Nowadays, the research community accepts all over the world the idea that web projects have special characteristics (critical navigation, hypermedia, customization, etc) that must be treated carefully in the life cycle. Thus, web projects and that need their own models and techniques [18]. Fig. 1. A global vision of the web approaches and its relations After different comparative studies, we can conclude that a big number of web approaches were proposed in the last years. In figure 1, a chronogram with the most famous approaches is presented. Some of these approaches are an evolution of a previous one and some of them are related between them. A continuous arrow between two methodologies expresses that the destination approach is based on the origin one and a dotted line expresses that the younger approach is an evolution of the oldest one. Most of them are focused on the object oriented paradigm, only the shadow ones are structured. It is out of the scope of this paper to present approaches presented in figure 1. They can be depth studied in [12]. However, recent comparative studies [2][3][12][17][18][24] show that Web Engineering has not been enough applied in the enterprise environment. Very few real applications can be found. This is an important gap for the Web Engineering research. In this paper, we present a global vision of NDT[9][11][12]. A web proposal proposal has been widely applied in real companies and projects. In section 3, to deal with requirements capture and analysis in web systems in section 2.This maintenance of web applications [5]. At first, the development of web systems was an ad-hoc process. Applications were developed without following any structured these experiences are presented. In section 4, these experiences are presented analyzing the results and the evolution of the proposal. In the last section, final conclusions and future works are presented. 2 NDT (Navigational Development Techniques) NDT is a methodological process focused on the requirements and the analysis phases. NDT offers a systematic way to deal with the special characteristic of the web environment. NDT is based on the definition of formal metamodels, presented in [14], that allow to create derivation relations between models. NDT takes this theoretic base and enriches it with the necessary elements to define a methodology: techniques, models, methodological process, etc. in order to offer a suitable context to be applied in real projects. neering metamodels and it proposes a methodological environment to drive the team in the capture, definition and validation of requirements following the next ideas: 1. In the elicitation of requirements NDT assumes its own techniques inherited from the requirements engineering environment like interviews, brainstorming or the study of the previous systems [7][21]. 2. In order to describe requirements, NDT uses some standard models, like the use cases, and patterns. A pattern is a special template with predefined fields that the development team must develop with the final user [7]. 3. In the validation of requirements, NDT also proposes a group of techniques like the traceability matrix [7] or the fuzzy thesaurus [19] adapted to NDT patterns in order to propose a more agile requirements validation. In this sense, NDT can be considered like a model driven methodology. NDT also normalizes the structure of the results that must be developed during the requirements engineering. It offers a complete definition of the structure. With the theoretic base of metamodels and relations, the next phase is the analysis phase. In the analysis phase three models are generated: 1. The conceptual model, that defines the static structure of the information and its relations. 2. The navigational model, that defines how users can navigate through the information. 3. The abstract interface model is composed by a group of HTML and XML prototypes that lets validate the conceptual and navigational models [20]. However, the generation of these three models is made in two phases. In the first one, analysis models are generated systematically from the requirements using the theoretic relations defined between models. In this sense, NDT can be de- In this sense, NDT starts with the theoretic definition of the requirements engifined as a model driven approach. These models are named basic analysis models. In these basic models, the analyst can make some changes in order to make these models more suitable, getting the final analysis models. The construction of these final models is not systematic and they depend on the experience of the analyst. NDT offers some guides and processes in order to make easier the analyst’s revision [12]. Besides, NDT controls that the changes proposed by the analyst are agree with the definition of requirements. In this sense, NDT manages that the final analyst models and the requirements definition are consistence. In figure 2, the NDT life cycle is presented with an activity diagram. Final models generated in NDT are compatible with other approaches like UWE or OOHDM. For this reason, from them, the development team can continue the life cycle with other approaches that have been widely accepted by the research community. In conclusion, NDT can be defined as a methodology to the requirements and analysis phases, the rest of the life cycle is dealt with other important approaches. NDT is offered to cover a gap in the treatment of the first phases of the life cycle in the Web Engineering. Finally, it is important to stick out that NDT has an associated tool, named NDT-Tool [9] that supports all the life cycle of NDT. This tool lets automate all the systematic processes of NDT, applies all its techniques and gets the results automatically. Deal with requirements Capture Define Validate correct? Capture Define Validate correct? no yes Analyse Generate basic models Generate final models Generate basic models Generate final models inconsistent? yes no Fig. 2. NDT development process 3 Practical applications of NDT With this basic introduction to NDT, which can be extended with its reference manual available in [9], in this section an historical evolution of NDT is going to be presented. NDT was born from some comparatives and analytic studies of the Web Engineering. From them, we concluded that the Web Engineering was a very young area that was the research environment of several important groups, but there were some gaps stuck out even by these research groups [12]: 1. There are not standard notations. In the first years, each approach offered its own models and they were incompatible with other models in other approaches. Everyday, research groups are looking for standard notations, like extensions of UML, in order to offer compatible models. 2. Classically, the most treated phases in the Web Engineering were the design and the implementation ones. In the last years, the research community has stuck out that it is important to offer suitable ways to go from the user’s requirements the design models. 3. There is not enough CASE support. Although, nowadays some methodologies like WSDM[6] are working in this line, there are very few approaches, like WebML [4], UWE [17] or OOH[3], that cover completely their life cycle with a tool. 4. The Web Engineering has not been enough applied in the enterprise environment. For these reasons, NDT is presented like a solution to cover these gaps in the Web Engineering. NDT has evolved to the use of standards and it offers a open way to work with other approaches. For that, the use of metamodels and UML is fundamental. Besides, NDT works in the first phases of the life cycle that are the less treated in Web Engineering, using the suitable results of other approaches in the next phases. Table 1. The analysed project Name of the project Date of starting Company Systems to manage information about public grants The system to manage information about cultural grants 2000-2001 The system to manage information about international help 2001-2002 Andalusian Government Systems to manage information about historical heritage The system of Movable Heritage in Andalusia 1997-1999 The Thesaurus of Historical Heritage 2003-2004 The system of Historical Heritage Authors 2005-.. Historical Heritage Andalusian Institute System to measure the grade of handicap The system to measure the grade of handicap 2004-2005 Alcer Fundation But, NDT also has worked in points 3 and 4. With NDT-Tool, NDT offers a suitable CASE support in its application. And with its several practical experiences, NDT has shown how the Web Engineering can be applied in real projects. In this section, we are going to present these practical experiences. Projects are presented in three subsections depending on its subject: projects to manage information about public grants, projects to manage information about historical heritage, the system to measure the grade of handicap. For each of one, a global vision of the project objects and the development environment is offered and after the conclusions getting in them are analyzed. Although projects are presented grouped by its thematic, it is important to analyze when they were developed in order to see the evolution of NDT. In the table 1, we presented an abstract with the name of each project, the date of development and the company where they were developed. In the next section, this short schema will be the base to analyze the conclusions in a general way. 3.1. Grants management projects When NDT started to be applied, in 2000, only the requirements metamodels were defined. Patterns and models in the requirements phase were applied in a system to manage the information about grants given by the Andalusia Government for cultural activities. NDT patterns were changed, corrected and modified during this first collaboration. NDT divides requirements into five groups: storage information, actors’, functional, interaction and non-functional requirements. The system developed was critical because it had to manage a high amount of information that depended on a complex administrative process where there was a high number of different roles with different necessities. For that, we detected that it was necessary to offer a way to define new data structures. However, the idea of data structure was not oriented to a programming idea. The data structures that had to be included in NDT must be oriented to the user. They must offer a suitable way to define a set of structured data according to the user’s view. For instance, in the system appeared several times the neces- These data always include the same information: name, address, etc. Despite of defining these data every time, we worked on the idea of offering a way to define this structure in the requirements phase. With this idea, in NDT the concept of new natures was introduced. A nature is a new special kind of requirement, with its own pattern, that allows to define these data structure requirements. Other important idea that was added in NDT in this project was the use of graphics. At the beginning, the requirements phase of NDT was only based on patterns. However, we detected that, at the beginning of the project, was more interesting make meetings based on graphical notations. We decided to enrich the basic definition of NDT with the use cases of UML[25]. Use cases model is a standard notation widely known and very easy to be understood by non-expert people. In any way, the textual description of patterns was kept in NDT. After the first meetings, use cases are very ambiguous to capture all the necessary information [15][26]. This modifications in NDT provoked an additional cost in the project that was several months delayed. However, the final results were quite goods and, some months later, a new similar collaboration was born. sity of storing the identification data of different people: users, managers, etc. Again, the project was a system to manage information about grants in the Andalusian Government. However, in this case, grants were oriented to the international help. Users were not the same and the new ones did not know NDT or the patterns. But, with this new experience, we could test that the introduced changes were very suitable. The double way to work: patterns and diagrams, offered a more agile way to work with users. However, we found in this collaboration an important barrier in the application of NDT. To keep patterns updated was a very difficult task. Patterns are interrelated and a change or a modification in one of them can produce changes in several ones. For this reason, in this point, we detected that it was completely necessary to develop a tool to support NDT. 3.2. Heritage management projects The oldest project where NDT was applied were the system of Movable Heritage in Andalusia (1997-1999). This system lets manage and spread out information about the movable heritage in Andalusia. The initial patterns in NDT were completed and first tested in it. However, the essential contribution of this project to NDT was the special treatment of the different roles. This system had different roles of users and the system changed completely depending of each role. The shown information was different depending on the user was an archeologist, an artist, a tourist or others. But, besides, in this system a same user could navigate in the system playing several roles at the same time. Thus, if a user was an archeologist, an artist or an archeologist-artist, the navigation, functionality or the interface were completely different. The complexity of this role motivated us to find simple but powerful ways to study roles and their relations. From this idea, new models based on heritance and traceability matrixes were added to NDT during this project. These new ideas was again testing in a similar project with the same group of users in 2003. In this case, we applied NDT to develop a Thesaurus of Historical Heritage [10] obtaining very good results. This project demonstrated also the advantages of working with users that knew the NDT patterns. Patterns are very intuitive for users and non-expert people in software engineering. When a user works with them, it is very easy for him/her to use it again. The evolution of NDT and its improvements with the practice were recently value in a project that, nowadays, has just being developed with the same group of users. This new project consists in the development of a system to manage and spread through Internet the information about authors that worked in the Andalusian Historical Heritage. In this project, the development time and also its cost have being lower than in the other projects for several reasons. The first one is because users and the development team know very well the development environment. Users know NDT very well and it was quite easy to apply patterns and requirements techniques. But, also NDT-Tool is completely developed. It is available via web and it make easier the management of the information and the attainment of the results. 3.3. The system to manage the grade of handicap This system is one of the youngest where NDT was applied[27]. In this case, NDT was a consolidated methodology. For this reason, in the application of NDT to this system, patterns, models, techniques and NDT-Tool could be applied without any changes getting very good results. However, the project was essential for another aspect that was forgotten in the research environment but that was essential in some enterprise environment. The system to manage the grade of handicap is a system developed with the Alcer Fundation [1]. This system lets apply the Royal Decree 1971/1999 (23/12) of the Spanish Government and drives how the grade of handicap of an patient must be measured by a medical tribunal. This project offered two important challenges to NDT. The first one was that the environment, the group of users and the terminology was completely new for NDT and the development team and, even, for the group of users because in Spain there is not a similar system. In this sense, the application of NDT was very successfully because it made easier the communication with users. However, the results in the other line wake the restlessness about a very few treated aspect. In other projects, the validation of requirements was made with techniques like reviews, the following of requirements with traceability matrix and the study of the terminology with glossaries. Nevertheless, this system worked in a very specific environment with a very complex argot. The ambiguity of the terminology in this kind of projects could provoke serious disasters. We started to work in more powerful techniques to validate requirements. We made a comparative study and work with the university of Nice in this aspect. This university developed a tool, named fuzzy thesaurus [19], that allows the development team to test ambiguities in software analysis models. After a work together, this tool was adapted to NDT and, nowadays, it is being included in NDT-Tool. The application of the fuzzy thesaurus allows to find errors in the requirements definition provoked by terminology ambiguities in a systematic way. The use of this tool is not always forced in NDT but it is a good technique to validate requirements in complex project with complex environments. 4. Global evaluation During the realization of the projects and with the gotten results, we can present some empirical results. In table 2, we have enumerated each project and we have measured some aspects for each one. We are going to analyze each of these measures and their justification. However, we are going to start for the last ones. The grade of development of NDT and the availability of NDT-Tool are essential in the results of the other aspects. The oldest project was the system to manage the Movable Andalusian Heritage, in this case, NDT was just at the beginning and several changes were made during the life cycle. In the first two projects, NDT was more developed but, also, some changes had to be made. Besides, in these three projects NDT-Tool was not available. These aspects added a new complexity to the project and provoked some delays. The first idea presented on the table is the grade of complexity of the system. We measured this aspect with the number of detected requirements during the requirements phase with the next ranges: xLow complexity: less than 30 requirements xMedium complexity: between 30 to 50 requirements xHigh complexity: more than 50 requirements The grade of knowledge of users is a more subjective measure. We have divided projects into two groups. In the first group, with grade low, users did not know anything about NDT before the project. In the second one, users (o more than the 50 per cent) worked with NDT before. The grade of complexity of the environment for the development team expresses if the group of analysts knew the terminology, the company and the group of users before the project. All analyst were expert using NDT, so this knowledge was not included in the table. The two next ones are only quantitative measures to present a deeper vision of the group of work. But, perhaps, the most interesting rows are the next two. We have presented the estimated time and the real time of the project.The estimation of the project at the beginning was made using the initial method proposed by Gustav Karner [16] and revised in [22] adapted to NDT and its patterns. Table 2. Empirical measures