Conducting a library survey using the library package Libra

10 SCONUL Newsletter 29 Summer/Autumn 2003 SCONUL Newsletter 29 Summer/Autumn 2003 11 Conducting a library survey using the Libra package Helen Fallon Deputy Librarian, National University of Ireland, Maynooth, Ireland Tel: + 353 1 7083880 E-mail: [email protected] INTRODUCTION The library at the National University of Ireland Maynooth recently carried out a user survey. While the library had previously carried out surveys, the main impetus behind this particular survey is the quality review exercise the library is currently undertaking. This is part of the quality assurance and quality improvement (QA/QI) process, implemented under Section 35 of the Universities Act 1997, which aims to develop and foster a quality culture across all areas of the university. The University Quality Promotion Offi ce, a body with overall responsibility within the university for the QA/QI process, recommended a broadbased questionnaire survey of both academic staff and students. A steering group, led by the Deputy Librarian, was established to manage the process. THE SURVEY TOOL Along with a number of Irish university libraries the library purchased, in 1996, a 5-year Libra software licence from Priority Search (subsequently Priority Research). The license expired in 2001. The cost of renewing the license was considerably less than taking out a new one, especially when the maintenance cost was borne in mind. The decision was made to renew the licence to Libra, based on a number of factors - NUIM had already used the package, thus there was a level of familiarity and expertise available; other university libraries in Ireland were using the package and could provide a network of local support; from previous experience it was known that the quality of the output was very high and the cost was that of license renewal rather than a fi rst-time purchase. The package is supplied on disc, with print documentation. In addition there are a range of online help screens and personal assistance is available from a London-based help desk. My experience was that telephone calls and e-mail were generally answered immediately, with very clear detailed responses to queries. With the initial licence purchase a digitising tablet and a special digitising stylus was supplied. Loading the software and setting up the digitiser is a relatively straightforward process. However, the help desk is available to talk customers through this process should they experience diffi culties. METHODOLOGY The Libra methodology involves deciding on an open question and focusing, with a sample of the target group, on this question to identify issues and concerns. The open question must be fairly broad and must be articulated in positive terms. Our open question was How can we improve on the service we provide to our users This question was put to students and academic staff through focus group sessions in the library, through meeting with students at key locations such as the student canteen and bar, via an internal postal mail shot to academic staff and through a posting on our library website. A full report on the focusing exercise is available in the Summer 2003 NUIM Library Newsletter at http://www.may.ie/library/publications/pdf/ 2003-summer.pdf The responses to the open question were then grouped or clustered by broad category. These responses formed the basis of the questionnaire, which is made up of three sections: • Datafi elds • Attitude scales • Paired comparisons In the Datafi elds section, information we sought included gender, age, length of service, year of study, faculty, frequency of use of library – both in person and remotely - and preferred method of delivery of information skills training. This section is drawn up in Microsoft Publisher. Sample 12 SCONUL Newsletter 29 Summer/Autumn 2003 SCONUL Newsletter 29 Summer/Autumn 2003 13 templates, which can be customised, are provided as part of the package. The Attitude scales section is also created in Microsoft Publisher. Again, sample templates are provided. These scales give information about how strongly people feel about a specific issue - from strongly agree to strongly disagree. The third part of the questionnaire – the Paired Comparisons – is central to how Libra operates. Based on the responses to the focusing question, from 10 to 42 items are chosen. We selected 30 areas of concern identified through the focusing exercise. These included requests for more textbooks, longer hours of opening, more PCs with internet access and more information skills training. Each suggestion is paired three times and the choice of which suggestion is paired with which is made by the computer on a numerical basis. Paired comparisons allow people to assess the relative importance of items two at a time. It is then possible for the computer to calculate the relative importance of all the items, and rank them in order of importance. While there is a very structured process involved in drawing up a Libra questionnaire, it is up to the individual library to decide what to include. In addition to using the feedback from the focusing exercise, the survey team included areas/services that are relatively new or developing here, such as information skills training and familiarity with subject librarians. Libra allows for one or more open question. However, these must be analysed manually. We added two open questions. With over 800 responses, the manual analysis was time consuming. Also one of our two questions If you could change one thing about the library what would you change? tended to generate information that came out of the other sections of the survey, particularly the paired comparisons. However, it was useful in that it threw light on an ambiguity in the final analysis of the results. A significant number of research postgraduates highlighted the need for more textbooks. On first glance this was puzzling, however from the response to the first open question it became obvious that they were defining textbooks as the core books for their research topic. The second question What one new service would you like to see the Library introduce? generated a number of interesting suggestions including requests for internet connectivity for personal laptops. With the open question included, our survey ran to eight pages, in four sections - datafields, attitude scales, paired comparisons and two open questions. Apart from the open questions, which required free text responses, all other areas involved ticking a response box. CONDUCTING THE SURVEY Planning The survey steering group drew up a project plan to include timescale, staffing resources and finance. The timeframe was influenced by a number of factors the most significant of these being examination schedules. When examinations loom large on the horizon, students do not wish to take even a short time out to complete a questionnaire. The questionnaire had, therefore, to be distributed and returned by the main body of the target population, by 11 of April – the conclusion of the second semester. Because the exercise was part of the quality review process, some funding was available from the university towards the survey. However, the majority of staff time towards the survey had to be met within the normal staffing budget. Piloting After the survey was drawn up it was piloted on a small group of the target population. This helped establish how long it took to complete – 12 minutes – and also assisted in ironing out any ambiguities in terminology. Identifying the sample As with any survey, the idea with Libra is not to survey all your target population, rather a sample of the population. We wished to have a representative sample of our user population and thus used a stratified random sample. A stratified sample is obtained by taking samples from each stratum or sub-group of a population. Within our target population we took a sample number of people in each of the following categories – academics, research postgraduate, taught postgraduate, higher diploma, undergraduate – and further stratified them by subject area – arts, science, theology. We received assistance from a statistician on drawing up the sample. Priority Research Limited recommends that for general user surveys a minimum of 500 correctly completed questionnaires should be the goal. For staff surveys it is suggested that the questionnaire be distributed to all staff and the aim should be to get a 70% response. Bearing these figures in mind and taking into account recommendations from a 12 SCONUL Newsletter 29 Summer/Autumn 2003 SCONUL Newsletter 29 Summer/Autumn 2003 13 statistician, the library produced 1,200 questionnaires. This took some time and involved financial costs of photocopying. Distributing the questionnaire The majority of undergraduates, taught postgraduates and higher diploma students, were surveyed during lecture time. It was thus very necessary that academics be fully informed and supportive of the process. Research postgraduates were surveyed through the external postal system, academic staff though the internal system. Respondents were invited to give their name and contact details – on a separate form - if they wished to be included in a raffle for a one hundred euro book token. Distributing the questionnaire during lecture time meant that it was possible to get a large amount of responses in a fairly short time. It also allowed the survey team to deal, on the spot, with any questions respondents had. This and the fact that very clear instructions were sent out to those answering by post, helped to contribute to the fact that 842 surveys were correctly completed out of 1,159 distributed – a 72.6% response rate - and less than a dozen surveys had to be discarded because of incorrect completion. With a fulltime academic staff population of 260 and over 5,000 students we felt this was a good representative sample of our population. Digitising and processing the results Each completed survey had to be digitised, using the digitising tablet and stylus. This provides for a high degree of accuracy. We found that approximately 25 questionnaires could be digitised per hour. Thus, with a response rate of over 800 questionnaires, digitising took approximately 32 hours, a not inconsiderable length of time. Because of the repetitive nature of the digitising it is not feasible for one person to spend a lengthy time digitising, rather the task has to be shared. Once digitising is completed, the system processes – in a matter of minutes - the information that has been input. Analysis and production of the report Priority Research Limited offers a facility whereby they will take the data at this stage and produce the final report for the client. On average a full survey report costs in the region of £2,500. This was not an option we explored. Instead we used the Microsoft Publisher survey report template which comes as part of the package. Within the template, pages for contents, introduction, methodology, explanation of charts and graphs and overall results –in graphic form with commentary - are provided. These must be customised. Through a variety of output options, including a range of graphs, histograms and pie charts, the Libra software produces two types of analysis. Quantitative analysis provides figures for how many people fall into a particular tick-box category. Using this type of analysis, we were able to see at a glance the breakdown between full-time and part-time students, among our respondents. The qualitative data, which is generated from the paired comparisons section, then allowed us to examine the different priorities of full-time and part-time students. Cross-tabulation and filtering features allow further levels of analysis. From the data generated the library was able to analyse a wide range of information that will be used in the future planning of our service. Very specific information, such as what types of user are accessing the library remotely and how frequently, will have implications for collection management decisions. In the same way student awareness of the subject librarian and their role, can be addressed with the specific groups, where a lack of awareness was recorded. Creating, saving and importing graphs and charts from the Libra package into the Microsoft Publisher template is a relatively straightforward process. While charts have to be resized to fit the template, a detailed knowledge of Microsoft Publisher is not required. For a presentation to library staff, it was possible to insert graphs as pictures into PowerPoint. The same can be done with Microsoft Word. Demographic (quantitative data) can also be exported into Excel and Access should further analysis be required. The sheer range of information Libra is capable of generating, and the extensive range of filtering options available, offer endless possibilities for analysis and commentary. However, thought should be given to the level of analysis that is needed and what type of information the library is most likely to be able to act upon. The commentary to accompany the analysis should illuminate the analysis further and possibly elaborate on the action the library hopes to take on information gained from the survey. CONCLUSION Using the Libra package, the library produced a very comprehensive and attractively presented survey report, which was based on the issues 14 SCONUL Newsletter 29 Summer/Autumn 2003 SCONUL Newsletter 29 Summer/Autumn 2003 15 identified as concerns by students, academic staff and members of the survey team. While the focusing exercise took time, it was a useful opportunity to meet face-to-face with students and to make them more aware of the library and our concern to continue to develop and improve the service we provide. A survey encompassing all our user groups took some time to set up, administer, digitise, analyse and report on. The library would not be in a position to carry out a survey on such a scale each year. However, smaller surveys with specific user groups could be carried out annually using the questionnaire, with some modifications, created for this survey. Customers of Priority Research Limited now have the option of utilising their annual maintenance payment to purchase a free single web based survey each year. The web questionnaire is limited to 2-3 pages of demographics, 4-5 pages of rating statements and 18 paired comparison items. Organisations can also commission Priority Research to carry out a web-based survey on their behalf. The cost of this depends on the size of the survey and the time involved. All in all this was a very useful exercise. The challenge now facing the library is to act on the results in such a way as to further develop library services.