Learning objects management: theory and practice

Learning Objects Management: Theory and Practice ERLA MORALES, FRANCISCO GARCÍA & ÁNGELA BARRÓN Department of Computer Science, University of Salamanca Plaza de los Caídos s/n, 37008, Salamanca, SPAIN fgarci[email protected] Department of Theory and History of Education, University of Salamanca Pº de Canalejas, 169, 37008, Salamanca, SPAIN erlamoral[email protected], [email protected] Abstract: - Although LO management is an interesting subject to study due to the current interoperability potential, it is not promoted very much because a number of issues remain to be resolved. LOs need to be designed to achieve educational goals, and the metadata schema must have the kind of information to make them reusable in other contexts. This paper presents a pilot project in the design, implementation and evaluation of learning objects in the field of university education, with a specific focus on the development of a metadata Typology and quality evaluate tool, concluding with a summary and analysis of the end results. Key-Words: - Learning Objects, Management, Quality, E-learning, Metadata, Evaluation 1 Introduction Many studies have been done on the concept of learning objects (LOs) but no consensus has been reached on a standard definition or on the technical and pedagogical requirements. Specifications are being developed but have yet to be normalized, and the use of metadata schemas is still under discussion. This has prevented LO creation and management from becoming common practice This paper presents our research on the design, implementation and evaluation of a prototype LO management tool for e-learning systems, containing quality criteria designed to enable LOs to be standardized and attuned to educational needs. The prototype was built on the basis of our own knowledge model, and comprises specific metadata value spaces for classifying LOs into the LOM “5. Educational” metadata category [7]. The paper begins by outlining some issues for learning objects quality and Management (section 2). On this basis, section 3 presents instructional suggestion for leaning object design. In order to support our proposal we presents the development of initial prototype learning object (LO1) and determines what type of metadata should be applied (section 4). It goes on to describe how we implemented and evaluated LO1 using our LO evaluation tool (section 5); then describes how the results of those trials were used to produce a second prototype (LO2), which was also implemented and evaluated (section 6). Finally it presents our conclusions and plans for the next stages of our work (section 7). 2 LOs quality and management The evolution of the Web in regards to semantics supports the idea of giving more significance to content than to syntax. In this way, the machines can realize complex tasks to deliver users the necessary information to meet their needs. Knowledge management and e-learning are closely related because e-learning users need a suitable knowledge management that can help them to obtain the kind of content they need together with as correct and complete information as possible. However the knowledge management concept is not easy to define because it is fairly wide. In a enterprise context [8] this concept is defined as “some actions intended for enterprises to organize and structure process, mechanisms and infrastructures with the proposes to create, save and reuse organization‟s knowledge”. In another way, [21] emphasize people participation in a knowledge management process. According to this he defines this concept as “a process to support creation, storing and sharing of value information as well as experience and perceptiveness inside or through people and organizations communities with similar interest and necessities”. Taking into account both definitions, we think knowledge management for e-learning systems must support all the issues that involve the teaching and learning process, specially content management because it is one of the most important issue in distance learning [12]. The possibility of managing content through elearning systems has an additional value. Knowledge is a source of power that needs to be shared and acquired, e-learning systems mean the possibility of managing information taking into consideration people that can contribute with their experience and enrich the information independent of time and place limits [19]. The challenge of defining the type of information to manage for e-learning systems is a topic that has led to the emergence of new concepts for resource development. One of these concepts is the learning object, which considers resources as independent units that can be re-used for new educational situations. We define a LO as a unit with a learning objective, together with digital and independent capabilities containing one or a few related ideas and accessible through metadata to be reused in different contexts and platforms. On agreement with this, knowledge management for e-learning based on reusable units of learning means the possibility of accessing specific content according to the learners‟ needs. Many organizations have created their own elearning platform solutions for knowledge management. As a consequence they have had interoperability problems at the moment their content is shared with other e-learning platforms or at the moment it is updated. In order for this to be possible, specifications and standards are in development which allow for interoperability of these objects on diverse platforms. However, the ability to interchange learning objects does not mean that the results are of good quality. Some authors [23] define quality in eLearning as the effective acquisition of a suit of skills, knowledge and competences by students, by means of developing appropriate learning contents given with a sum of efficient web tools supported via a net of value-added services, whose process –from content developing to the acquisition of competences and the analysis of the whole interventionis ensured by an exhaustive and personalized evaluation and certification process, and it is monitored by a human team practising a strong and integral tutorial presence through the whole teaching-to-learning process. According to this, in order to promote quality Learning Objects Management it is important to promote quality directed to all the elearning components involved, for example: criteria and instrument for their evaluation [13], suitable metadata typology, methodology and training of online tutoring [23] pedagogical design for the practice community, etc. LOs have some inherent properties according to their characteristics that may be evaluated independent of the context [18]. Principal LOs characteristics are: durability, interoperability, accessibility and reusability. However according to [17] [19] LOs have inherent characteristics that can be use as beforehand quality measure. For this reason, in this section we explain issues related with the LOs characteristics which help to improve their quality for a suitable management.  Reusability. This is the principal characteristic of value for LOs. However, it is not easy to evaluate LOs reusability because it is related with the context of use. In the case of LOs (as well as software engineering) exact measures do not exist, however it is possible to define quality indicators of usability that may be confirmed according to the reusability level. It is an heuristic evaluation according to a context of use. When a LO is reused into a one or different organizations, users may be able to evaluate them in a empirical way, then, it is possible to watch and save results about LOs management and add this information to their metadata. In this way metadata could provide more complete information for LOs reusability.  Suitable format: LOs reusability depends of their content as well as their metadata information. However metadata compatible with some standards like LOM [7] or SCORM [22] is not enough to make them reusable. According to [1] Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation. The idea is that machines can read information to develop complex tasks for users. To make it possible LOs metadata must have a format directed to automated process.  Metadata information: Metadata is the most important thing to know LOs characteristics for this reason according to [25] metadata information must be as correct and complete as possible because it is necessary to know all the information as possible about the LO to reuse it in a suitable way. Metadata are grouped into nine categories, however we have to take special attention to educational category because it contains ten sub-categories with different kind of pedagogical information as: interactivity type, learning resource type, interactivity level, semantic density, intended end user rol, learning context, typical age range, difficulty, typical learning time, and description.  Size or degree of granularity: Other important issue to reuse LOs is their degree of granularity because it is related to their capability to be reused in another contexts and platforms for e-learning systems. However the degree of granularity could to affect LOs reusability depending of their size and metadata information. It is known that too litlle LOs as well as too big LOs have less probabilities to be reused because their possibilities of interchange decrease. In the case of too little LOs like a video without sound or a figure it is not easy to manage for elearning systems because they have a lack of intention and their metadata may result too poor. However if we manage LOs according to our definition, it is means LOs that has a few related ideas is more easy to reuse them than a big content like a software because it is created for a very specific situation. Taking into account our definition teachers are free to decide in which learning context they must to be used. This is because they do not be necessarily related to any time, methodologies, instructional design, etc. 3 Instructional suggestion for LOs Design LOs are individual units of learning or modules which need to be enabled with other ones to build larger units (didactic units, courses, etc.). This means that they are part of the whole, but each LO must be capable of being reused by itself in other didactic units. In order to complete an LO as a quality unit of learning and to compose didactic units (DU) with them, we believe the following issues should be considered. [16], [19]  Overview: According to [2] [3] and [20] a didactic unit needs a general overview to explain general objectives and introduce the LO content. An introduction is an important element for any kind of contents because as well as providing information about the contents, it sets out the purpose of the topics and gives learners an idea of what they are expected to learn. Furthermore, it is a motivational element that aims to engage the students by letting them know why the subject is important for them. An overview must also provide an LO objective. As we explained in the definition of LOs, because of their reusability characteristics, ideally the objective must be simple, with one or several related ideas. We suggest that an objective should be directed to learning one kind of contents because in this way the whole instructional design would be targeted to achieve this specific objective. Other important aspects that must be included in an LO overview are: its title and the title of the learning unit, so that students can know what part of a whole they are working with; the sequenced list of topics; and, finally, keywords to inform students about what related areas are involved with the LO content.  Contents: In general, any kind of content must have some quality characteristics that take into account different issues. From a pedagogical point of view, contents must be logical and psychologically meaningful. That means, on the one hand, a logical view of the discipline (contents sequence, methodology, kind of activities, etc.) and, on the other, user suitability (level of difficulty, user interests, etc.). Other issues related to any kind of contents are information veracity, correct data, good writing and spelling, suitable size, color and font type, etc. However, as regards LO characteristics, it is important that contents should not mention anything about time, for example, “this week” or “this semester,” etc., because this could delay its reusability for other educational situations. The same must be taken into account regarding the audience, then phrases like “dear engineering students…” must also be avoided. Ideally, contents should be presented in multiple formats in order to attend to different cognitive skills and learning styles, e.g. videos, animations, graphics, etc.  Activities: Activities may be addressed to promoting new knowledge acquisition and to preparing users for a final assessment. Activities may be included in any kind of contents during the entire teaching and learning process. They help users to know if they must go on to the next lesson or whether they should seek feedback. An overview must also provide an LO objective. As we explained in the definition of LOs, because of their reusability characteristics, ideally the objective must be simple, with one or several related ideas. We suggest that an objective should be directed to learning one kind of contents because in this way the whole instructional design would be targeted to achieve this specific objective. Other important aspects that must be included in an LO overview are: its title and the title of the learning unit, so that students can know what part of a whole they are working with; the sequenced list of topics; and, finally, keywords to inform students about what related areas are involved with the LO content. Some authors [4] [5] promote constructivist learning environments for Learning Objects. They emphasize that activities must be as diverse as possible to accommodate different kinds of users: case studies, problem solving, teamwork, reflecting on situations, etc. We agree on the need for these kinds of activities, but we feel that deep reflection about them is necessary before they can be applied to LOs. First, activities are highly related to the contents. This issue may affect the kind of activity to use; for example, if LO contents are merely talking about basic concepts, facts or data, the kind of activities may be directed to reinforcing them, by relating the correct concepts, checking true or false, etc. Most likely, an activity such as a case study does not need to be employed at this level of complexity. In accordance with this, in order to support different complexity levels of contents and cognitive domains, we suggest taking into Is based on Import or Create -Data and Concept Contents -Computer Science Ingineering - CONTEXT -Objective -Introduction -Typical Learning Time -Keywords Overview -Schema -Comparative Table Summary -Self-Evaluation Practice Activities LEARNING OBJECT 1 Object-Oriented Programming: General Issues (Aggregation Level 2) Data and Concept LEARNING OBJECT 2 Object-Oriented Programming: Life Cycle (Aggregation Level 2) Data and Concept -Names, facts, characteristics Data -Abstract principles Concept -Cognitive Level Objective -Reflection Assessment Activities -Initiation Type -Individual -Collaborative, etc. Work Modal -Lecture -Forum participation Strategies -Knowledge -Comprehensión -Analysis -Synthesis Cognitive Level and + + LEARNING OBJECT 3 (Aggregation Level 2) + LEARNING OBJECT X (Aggregation Level 2) + + TEACHER Compose Compose -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE 1: Object Oriented Programming Aggregation Level 3 -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE 2 Aggregation Level 3 -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE X Aggregation Level 3 -Overview -Modules -Summary -Practice Activities -Assessment Activities COURSE: Object-Oriented Programming Aggregation Level 4 -Initiation Type -Individual Work Work Modal -Lecture -Self-evaluation activities Strategies + Fig. 1, Knowledge Model of LO1 account three kinds of activities: Initiation, Re-structuring and Application. Initiation activities are designed to teach the basic contents of a specific subject. An example of this is a quiz. Re-structuring activities may be directed to promoting new knowledge acquisition, such as activities that promote questions, research, etc. Finally, application activities may be addressed to fostering students‟ experience in order to strengthen their acquisition of new concepts. An example of this activity is a case study. A Didactic Unit is composed of a group of individual LOs. Because of the reusability characteristic, recommend carrying out some activities at the end of the didactic unit to avoid consistency problems with the adaptation of new LOs [2] [20].  Summary or Conclusions: As with any kind of teaching and learning process, a summary is advisable after a contents review. A good summary should point out the main ideas and the relations between them, making it possible to reinforce the contents. It is also important to relate the contents to other areas of knowledge by means of diagrams, outlines, conceptual maps, etc.  Assessment: An evaluation must take into account each of the learning objectives. It must thus be addressed to any kind of contents and its level of difficulty. Evaluation may be carried out as activities; however, it is very important that students know what activities will be evaluated prior to the assessment. Some authors [2] [16] are in favour of practice activities and evaluation activities. The first has to help students to acquire new knowledge by providing feedback, pointing out the most important information, and to prepare them for a final evaluation. The second type must be a final experience that lets the students know whether they have mastered the objectives or not, i.e., whether they have passed or failed. 4 LO Design and Proposed Metadata Typology The first task to create our initial prototype learning object (LO1) was to chose a context in which to conduct our trials: the Object-Oriented Programming (OOP) option of the Computer Science course at Salamanca University [12] [15]. We then defined a set of specific learning objectives with which we built a knowledge model that served to produce a basic unit of learning which, in turn, served as the basis for designing LO1, entitled “Object-Oriented Programming: General Issues” (see figure 1). One of the key goals here was to enable a knowledge model to be used to standardize LOs, which is crucial for them to be tailored to educational needs, taking into account key elements for learning [17]. Sound LO management requires the incorporation of reliable metadata, but the viability of the only metadata schema currently regarded as a standard [7] has been called into question because it uses vast quantities of ill-defined types of data, and some of its metadata categories do not make it clear what kind of information has to be added, thus further complicating the task of LO management [6]. According to [24] selection and composition of learning objects are two essential activities in automated approaches to Web-based learning. Such activities require high-quality metadata records that are not only conforming to current specifications and standards, but that provide clear system-oriented runtime semantics that support automated decision processes. Although the lack of clarity in the IEEE LOM standard makes its value spaces hard to interpret, most metadata editors today continue to use that standard without seeking to explain the meaning of each space. We set out to address this issue – and, hence, to enable suitable LO management data to be introduced into learning environments – by devising a set of definitions to clarify the content of each value space in the LOM “5. Educational” metadata category:  5.1. Interactivity type: expositive LOs featuring a very low interactivity level, with students receiving information yet remaining unable to interact with the content.  5.2. Learning Resource Type: web pages  5.3 Interactivity Level: low LOs with an expositive interactivity level – minimal student participation (web pages with few links)  5.3 Semantic Density: medium LO content designed to promote smooth learning and application of knowledge  5.5 Intended End User Role: learners  5.6 Context: university level  5.7 Typical Age Range: Unspecified  5.8 Difficulty: easy Information is easily associated with previous knowledge We then incorporated these definitions into our prototype LO1. 5 LO1 Implementation and Evaluation Having designed LO1 based on our knowledge model and incorporating our proposed metadata typology – using Dreamweaver MX – we then set about implementing it with Moodle, introducing the following supplementary elements:  a pdf file: so that our sample students could print out the LO content  a self-assessment section: so that they could see how much they knew about the content, and to repeat the test whenever necessary  a forum: so that learners and teachers could discuss the content  an evaluation tool: for the students to rate the quality of LO1. Current proposals for learning resource evaluation tools include web sites [10] [26] and multimedia tools, [9], and other proposals have been made for assessing the quality of LOs taking into account their instructional use-oriented design [27] and sequencing [28]. We drew on these to design an instrument that would enable learners to assess the value/quality of their LOs (see figure 2). Our sample students were able to access the LO and the evaluation tool via Moodle and to rate them on a scale of 1 to 5: 1=very poor; 2=poor; 3=satisfactory; 4=good; 5=very good. As seen in figure 2 (above), the evaluation tool was designed to gather qualitative and quantitative data about LO1. The qualitative results show a general agreement on its quality. The highest scoring value was the difficulty level (3.87), followed by the objectives and content (3.82). These results reflect our sample students‟ approval of the content in terms of its quantity, consistency, reliability and so on. Navigation was considered well-designed and userfriendly (3.79). The students were slightly less happy with the overall design of LO1 (3.74), and suggested a number of possible improvements. They also made a number of positive comments on the feedback (3.66). „Activities‟ and „interactivity‟ were rated satisfactory (3.51), as was the lowest scoring criterion: „motivation‟ (3.41). The feedback gained from the space provided in LO evaluation tool for students to make comments provided very useful pointers for us to see what needed to be improved when developing our second prototype (LO2). Here is a selection of their comments:  Add a glossary of key concepts and list of acronyms.  Add examples to illustrate/clarify abstract concepts.  Avoid table cells in web page design (as it impeded accessibility for sightless users).  Highlight main points (e.g. in bold).  Avoid too many references in short texts.  Adjust window resolution to avoid too many scroll bars.  Provide more detailed information on what aspects of the criteria the tool is evaluating. To input the quantitative and qualitative data on the quality of LO1 into our metadata typology, we used the LOM “9. Classification” metatada category in combination with our own LO quality rating classification scheme. First we developed the LO quality rating scale shown in Table 1 (below). Table 1, LO Quality Rating Scale Ratings Quality 1.0 – 1.5 very poor: too low. LO to be eliminated 1.6 – 2.5 poor: low, with much room for improvement 2.6 – 3.5 satisfactory: could be better 3.6 – 4.5 high: good but could be better still 4.6 – 5.0 very high: good, with no need for further improvement Table 1 sets out the various LO ratings on the evaluation scale and explains their corresponding quality levels. We believe that quality measurement EVALUATION CRITERIA SCORE DIFFICULTY The level of difficluty was suitable for my prevoius knowledge 3,87 OBJECTIVES I have achieved the learning objectives 3,82 CONTENTS Contents were solid (suitable for objectives, enough information, reliable, supported by references, etc.) 3,82 NAVIGATION Suitable and orientative navigation 3,79 DESIGN Content design was clear and orientative 3,74 FEEDBACK I had feedback possibilities all the time 3,66 ACTIVITIES Activities and self-assessmement were clear and meaningful for learning 3,51 INTERACTIVITY Suitable to achieve the learning objectives 3,51 MOTIVATION I felt motivated during the lesson 3,41 DESCRIPTION My participation was clearly explained 3,28 LO QUALITY FINAL VALUE 3,64 COMMENTS (Describe some examples where this LO can be reused) Fig. 2, LO1 Evaluation Results using a scale like this should be introduced into the “9. Classification” metadata category. Table 2 (below) shows our prototype adaptation using the final quality score taken from the LO1 evaluation results (figure 2). Table 2, LO1 quality rating incorporated into LOM 9. Classification 9.1 Purpose Quality 9.2 Taxon Path 9.2.1 Source Table 1. LO Eval. Rating Scale 9.2.2 Taxon CA*: 3.64 (high) 9.2.2.1 Id CA: 3.64 (high) 9.2.2.2 Entry High 9.3 Description LO considered high quality by sample students. Lowest scoring quantitative items were „motivation‟, „activities‟ and „interactivity‟. Qualitative feedback suggested adding a glossary and examples; avoiding use of table cells in LO design; using fewer references in text; and improving screen resolution 9.4 Keyword quality, value, high, CA_3.64. *CA: CALIDAD (quality) Our thinking was as follows:  Adding a quality value to the LO metadat  category would help locate and retrieve an LO through a search based on keywords (e.g. quality, value, high, etc.) and alphanumeric values (e.g. CA_3.64). An alphanumeric value makes it possible to define a specific vocabulary for running an LO search.  Using specific kinds of values would provide a means of developing more sophisticated search methods, e.g. using an intelligent agent to find and compare LOs according to quality criteria. This would require a multiagent architecture enabling personal retrievals from multiple sources.  LO management would be facilitated by incorporating LO quality ratings into semantic profiles [11],[12].  IEEE LOM metadata categories at present do not consider classifying LOs according to quality ratings and, hence, most metadata editors do not offer the possibility of adding other types of classification criteria.  The sample students‟ comments provided useful pointers for producing an enhanced and more user-friendly design for our second prototype (LO2), with a different font, larger characters and links to further reading (see figure 3). The actual content of LO2 followed on from LO1, taking the learning objectives to a more Fig. 3, Knowledge Model of LO2 Evaluation Results Is based on Import or Create -Data and Concept Contents -Computer Science Ingineering - CONTEXT -Objective -Introduction -Typical Learning Time -Keywords Overview -Schema -Comparative Table Summary -Self-Evaluation Practice Activities LEARNING OBJECT 1 Object-Oriented Programming: General Issues (Aggregation Level 2) Data and Concept LEARNING OBJECT 2 Object-Oriented Programming: Life Cycle (Aggregation Level 2) Data and Concept -Names, characteristics Data -Abstract principles Concept -Cognitive Level Objective -Reflection Assessment Activities -Initiation Type -Individual -Collaborative Work Modal -Lecture -Forum participation Strategies -Knowledge -Comprehensión -Analysis -Synthesis Cognitive Level and + + LEARNING OBJECT 3 (Aggregation Level 2) + LEARNING OBJECT X (Aggregation Level 2) + + TEACHER Compose Compose -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE 1: Object Oriented Programming Aggregation Level 3 -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE 2 Aggregation Level 3 -Overview -Learning Objects -Summary -Practice Activities -Assessment Activities MODULE X Aggregation Level 3 -Overview -Modules -Summary -Practice Activities -Assessment Activities COURSE: Object-Oriented Programming Aggregation Level 4 -Initiation Type -Individual Work Work Modal -Lecture -Self-evaluation activities Strategies + advanced level. 6 LO2 Implemetation and Evaluation LO2 was implemented in the same learning environment as LO1, and was evaluated with an enhanced version of our quality evaluation tool (see figure 4). The final score reflects a similarly high average quality rating on the part of our sample students (3.66). The highest scoring item was „navigation‟ (4.00), followed by „description‟ and „activities‟ (self-assessment) (3.91), both of which figure in the Didactic Curricular Issues category. Content design was considered high quality (3.74), as were three other didactic-curricular issues: – achievement of objectives (3.69), learning time, and LO content (3.63) – and one psycho-pedagogical issue: „difficulty‟ (3.63) . Student comments were even more positive for LO2 than LO1, expressing their approval of the new section with references, links to further reading, a glossary and a list of acronyms. Some, however, considered that the screen resolution was better but needed further improvement: there were still too many scroll bars and accessing table cells remained an impediment to sightless users. Having completed our evaluation, we incorporated the overall LO2 quality rating into the corresponding LOM “9. Classification” metadata category (see table 3), using the LO classification scheme based on our proposed metadata typology [14]. Our proposed adaptation of the LOM “9. Classification” metadata category comprises the key quantitative and qualitative data collected with our LO quality evaluation tool. In presenting a summary of learners‟ comments on LO quality, item “9.3. Description” provides a useful means of further improving that quality. Table 3, LO2 quality rating incorporated into LOM 9.Classification 9.1 Purpose Quality 9.2 Taxon Path 9.2.1 Source Table 1: LO Quality Rating Scale 9.2.2 Taxon CA_3.66* (high) 9.2.2.1 Id CA_3.66 (high) 9.2.2.2 Entry High quality 9.3 Description LO2 was considered high quality by students. Lowest scoring quantitative items were „motivation‟, „interactivity‟ and „feedback‟. Qualitative feedback suggested a need to improve screen resolution and avoid table cells in LO design 9.4 Keyword quality, value, high, CA_3.66. Finally, the “9.4. Keyword” item gives users the search words for finding and retrieving the best possible LOs to suit their needs [5]. 7 Conclusion The research outlined in this paper set out to test a model for enhancing LO management through evaluation of LO quality. Our prototype knowledge model sought to demonstrate how LOs can be established as a basic unit of learning, taking into account key educational needs. It can be used to adapt an LO to a specific type of course at university level. EVALUATION CRITERIA CATEGORY SCORE PSYCHOPEDAGOGICAL ISSUES MOTIVATION I felt motivated during the lesson 3,43 DIFFICULTY The level of difficluty was suitable for my prevoius knowledge 3,63 PARTICIPACIÓN Se ha explicado claramente mi participación en la lección 3,63 DIDACTIC CURRICULAR ISSUES DESCRIPTION My participation was clearly explained 3,91 OBJECTIVES I have achieved the learning objectives 3,69 CONTENTS Contents were solid (suitable for objectives, enough information, reliable, supported by references, etc.) 3,63 ACTIVITIES Activities and self-assessmement were clear and meaningful for learning 3,91 TIME Suitable to achieve LO objectives 3,63 FEEDBACK I had feedback possibitities all the time 3,34 TECHNICAL AND FUNCTIONAL ISSUES INTERACTIVITY Suitable to achieve the learning objectives 3,34 NAVIGATION Suitable and orientative navigation 4,00 DESIGN Content design was clear and orientative 3,74 LO QUALITY FINAL VALUE 3,66 COMMENTS (Describe some examples where this LO can be reused) Fig. 4, LO2 Evaluation Results Our sample students at Salamanca University appreciated the pdf file, the self-assessment component and the forum because these features enabled them to print copies of the content, to assess their knowledge and to exchange views, all of which helped them gain a clearer understanding of the LO content. The LO quality evaluation tool enabled us to collect a wide range of information useful for improving both LO1 and LO2. In attributing a numerical value to LO quality, the rating scale helped specify exactly which data to incorporate into the metadata schema. It is important to remember that metadata editors today only classify LOs according to specific established purposes. We used the LOM “9. Classification” metadata category because we believe it useful for defining and adapting new LO classification schemes that would allow users to acquire and manage LOs suited to their own individual needs. Finally, the results obtained with the LO quality evaluation tool helped highlight exactly what improvements needed to be made. Sorting evaluation criteria into different categories made it possible to evaluate the LOs from both pedagogical and technical points of view. Our future work will focus on developing an LO creation tool based on our knowledge model. We will also seek to improve the quality of LOs by taking into account the accessibility issues that are crucial to LO management. Finally, we are aiming to promote intelligent agent-based automated working methods by developing a prototype multi-agent architecture for quality-based LO management. Acknowledgements This work was co-financed by the Spanish Ministry of Education and Science, the FEDER-KEOPS project (TSI2005-00960) and the Junta de Castilla y León local government project (SA056A07). References: [1] Berners Lee, T., Hendler, J., Ora, L. “The Semantic Web”, Scientific American, 2001, may, 284 (5): 34-43. [2] Clark, R.C., Mayer, R.E. E-learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. San Francisco California: Josey Bass/Pfeiffer 2002. [3] Cysco Systems. Reusable learning object authored guidelines: How to build modules, lessons and topics, White papers. www.cisco.com 2004. [4] Del Moral, M. E., Cernea, D. A. Diseño de objetos como facilitadotes de la construcción del conocimiento. II Simposio Pluridisciplinar sobre Diseño, Evaluación y Descripción de Contenidos Educativos Reutilizables. Barcelona. 2004. [5] García, F. J., Morales, E. M. & Barrón, A. “Learning Objects for e-Activities in Social Web”. WSEAS Transactions on Systems, 6(3):507-513. 2006. [6] Gil, A. B., Morales, E., García, F.J. E-Learning Multi-agent Recommender for Learning Objects. IX Simposio Internacional de Informática Educativa, pp. 163-169, ISBN:978-972-8969-042. 2007. [7] IEEE LOM, IEEE 1484.12.1. Standard for Learning Object Metadata. 2002. Retrieved June, 2007, from http://ltsc.ieee.org/wg12. [8] Kuang-Tsae, H., Lee Yang, W., Wang Richard, " Calidad de la información y gestión del conocimiento", Editorial AENOR, Madrid, 2000. [9] Marquèz, P. Elaboración de materiales formativos multimedia. Criterios de calidad. Disponible en http://dewey.uab.es/pmarques, 2000. [10] Marquèz, P. Criterios de calidad para los espacios Web de interés educativo. Disponible en http://dewey.uab.es/pmarques/caliWeb.htm, 2003. [11] Morales, E. M., Alonso, D., García, F. J. HEODAR: Herramienta de evaluación de Objetos Didácticos de Aprendizaje Reutilizables. X simposio internacional de Informática educativa (SIIE‟08). In press. 1-3 de Octubre del 2008. [12] Morales, E. M., García, F. J., Barrón, A. Morales, E. M., Barrón, A., García, F. J. Research on Learning Object Management. In J. Cardoso, J. Cordeiro, J. Filipe (Eds.) Proceedings of the 10h International Conference on Enterprise Information Systems (ICEIS'08). (June 12-16, 2008, Barcelona, España). pp. 149-154. INSTICC Press. ISBN obra completa 978-972-8865-91-7. http://www.iceis.org 2008. [13] Morales, E. M., García, F. J., Barrón, A. Morales, E. M., Barrón, A., García, F. J. An evaluation Instrument for Learning Object Quality and Management. In J. Cardoso, J. Cordeiro, J. Filipe (Eds.) Proceedings of the 10h International Conference on Enterprise Information Systems (ICEIS'08). (June 12-16, 2008, Barcelona, España). INSTICC Press. ISBN obra completa 978-972-8865-91-7. http://www.iceis.org 2008. [14] Morales, E. M., Gil, A. B., García F. J. Arquitectura para la recuperación de Objetos de Aprendizaje de Calidad en Repositorios Distribuidos. SCHA: Sistemas Hipermedia Colaborativos y Adaptativos. II Congreso