How’s Our Library E-Service? Measuring User Satisfaction in the Virtual Environment

How’s Our Library E-Service? Measuring User Satisfaction in the Virtual Environment Objective. This study aims to determine the satisfaction with the virtual reference service of library users of Nazarbayev University Library during the Spring 2021 to Spring 2022 semesters. Methods. This study employed a descriptive approached to determine the satisfaction of users with the virtual service of Nazarbayev University Library during the Spring 2021 to Spring 2022 semesters. Using the data gathered from LibAnswers’ Quality of Service (QoS), this study examines various metrics in answering the queries received both in synchronous (chat service) or asynchronous (email service) virtual service. Statistical data from were tabulated and analyzed and the feedback from patrons will be interpreted using Voyant Tools, a web-based data visualization software. Results. The study describe how patrons were satisfied with the virtual library service of NU Library as perceived through their feedback and their overall experience of the service. In addition, the study pointed out that the Library considers all virtual transactions urgent and put the satisfaction of their patrons on top priority based on the metrics. Conclusions. The study revealed that patrons were satisfied with the virtual service rendered by NU Library both in synchronous and asynchronous format. With the help of the metrics generated from LibAnswers Quality of Service (QoS) feature, the NU Library was able to provide an excellent service despite the limitations brought by the pandemic. The feedback from patrons was indeed a proof of their satisfaction. As service quality and user satisfaction will always go together, the library, as a service-oriented information arm of any institution, should always put their patrons as their top priority.


Introduction
Measuring user satisfaction even in the traditional or face-to-face setting is quite difficult. With the onset of COVID-19 pandemic, more and more libraries have been offering virtual library services to ensure continuous delivery of services and resources and gauging how satisfied our patrons are might even be more challenging. Although offering library service in a virtual environment has been done even before the pandemic (Vogus, 2020), determining if we satisfy our user's needs is still quite tricky. As the heart of the learning community (Kiran, 2010), university libraries have always been living its mandate to support informational, research, scholarly, and recreational needs of its users and is expected to deliver service whenever, wherever.
Libraries has proven itself relevant despite the changing demands of the time. From a mere repository of print materials to an information hub of various formats including online resources; from just a traditional reference desk to a virtual reference assistant, the library has indeed no doubt repositioned itself in the educational landscape. However, libraries continue to face some challenges as learners resort to get information from various options and competitive sources (Twum et al., 2022). In addition, as learners became even more online and distant, they tend to be self-sufficient (Oliveira & Greenidge, 2020, p. 21), that means that they prefer and resort to getting their own materials without any assistance (Yap & Manabat, 2021, p. 4). With the ever-increasing amount of information available online, libraries need to assert itself in the information landscape by assessing the quality of the service being rendered to their clients alongside improving user satisfaction (Kiran, 2010) as this will also ensure "increase in their user base and remain to be responsive on meeting their users' expectations" (Adeniran, 2011, p. 209).
Gauging user satisfaction has indeed imperative for any service-oriented institutions and organizations from the get-go and users or customers' voices always matter (Adeniran, 2011). Libraries as service-oriented informational arm of any organizations, should also seek ways on how to assess their service as perceived by the users and improve it based on their feedback. With majority of library services going virtual especially with the onset of pandemic, it became even more challenging and tricky for libraries to determine if they meet user expectations. To further help libraries on this, various models and tools were created and introduced to provide a deep dive on analyzing user satisfaction to improve service quality and to make informed decisions.

Service Quality in Libraries
Service quality is defined as 'the overall evaluation of a specific service firm that results from comparing that firm's performance with the customer's general expectations of how firms in that industry should perform' (Parasuraman et al., 1988, p. 15) and with the intensified competition, having a superb "service quality seems to be a prerequisite to success, if not survival" (p. 15). To provide a more structured approach in measuring the service quality, Zeithaml, Parasuraman, and Berry (1988) introduced SERVQUAL, a set of 22 pairs of statements used to measure user expectations in the marketing field. These statements focuses on five interrelated dimensions that customers most value when they assess the quality of service regardless of providers (Kiran, 2010;Nitecki & Hernon, 2000;Smith, 2010;Wang & Shieh, 2006):  Tangibles (physical facilities, equipment, and the appearance of personnel);  Reliability (ability to perform the promised service dependably and accurately);  Responsiveness (willingness to help customers and provide prompt service);  Assurance (knowledge and courtesy of employees and their ability to inspire trust and confidence); and  Empathy (caring, individualized attention the firm provides its customers) In the library context, service quality is "the overall excellence of library services that satisfy users' expectation" (Wang & Sheih, 2006, p. 195). Over the years, there were studies conducted focusing on service quality in libraries. With regards to library service quality, there are several determinants to look out for such as electronic resources, collections of printed publications, other library services, technical facilities, library environment, and human side of user service based on the study conducted by Martensen & Gronholdt (2003). The application of SERVQUAL model and its ability of monitor library service quality were highlighted in the studies conducted by Nitecki (1996), Nitecki & Hernon (2000), Kiran (2010), and Gathoni & Van der Walt (2019). Aside from SERVQUAL, the LIBQUAL model introduced by Association of Research Libraries in 1999 was also evident on the studies conducted by Cristobal (2018), Dahan et al. (2016), and Pedramnia (2012). It offers a standard measure of services dedicated to libraries which consists of 25 items that focuses on gauging user perception on service quality in the library setting aligned in these four dimensions and has further determined the relationship of service quality, user satisfaction, and loyalty in libraries (Twum et al., 2022;Cook et al., 2003): 1. Service affectrelates to the human side of libraries consisting of traits of empathy, personal competence, and accessibility. 2. Library as placeconsisting of various issues from the users' perspective on the space of study and collaboration, a sanctuary for contemplation and reflection.
3. Information accesslibrary users' assessment of the adequacy of library collections, and their ability to have access to needed information conveniently or regardless of the medium of the resource in question. 4. Personal controlthe extent to which library users are able to navigate and have control of the information provided by the library on the internet.

Customer Satisfaction
Law J. (2011) defined customer satisfaction as the degree to which customer expectations of a product or service are met or exceeded. In addition, it is also a "consumer's fulfillment response, which act as an evaluation based on an emotion-based response to a product or service received" (Ahmad & Allan, 2014, p. 17). This further denotes that customer satisfaction is an evaluation of quality of products and services received that adheres to their needs and wants (p. 17). In the simplest sense, customer satisfaction stresses on perceptions, expectations, and evaluation (p. 17). Putting this on the context of libraries, customer satisfaction is defined as "the levels of service quality performances that meets users' expectations" (Wang & Shieh, 2006, p. 197).
Organizations should understand the perceptions and expectations of customers towards a certain product or service (Ahmad & Allan, 2014). These comes from their experiences and feelings as well belief, opinion, and predictions about the products and services they received (p. 20). Therefore, to measure customer satisfaction, it is also essential to know your customers even in libraries.

Service Quality and Customer Satisfaction
Service quality and customer satisfaction are very much related concepts. Service quality should be treated as an antecedent of customer satisfaction (Twum et al., 2022;Wang & Shieh, 2006;). The measure of service quality differs from one person to another as people comes from various backgrounds, beliefs, experiences, perceptions, and expectations, thus, customers, given these factors, are the determinants of service quality (Ahmad & Allan (2014). Having said that, there is a need for the organization to know the customer's perceptions and expectation before and after using the product or service. This is primarily the reason why some organizations conduct a user needs assessment as well as a user evaluations to manage both the user perceptions and expectations. As noted by Ahmad & Allan (2014), satisfaction is defined to be "the gap between the perception and expectation of the end-user", thus, "customer satisfaction should be measured only by performance" (p. 20) as service "quality is also a pride of workmanship" (p. 45). This also applies in organization such as libraries which are primarily service-oriented and a quality service is also expected. Satisfaction of library users is result of their perception of the quality of library services, thus, service quality will give way to user satisfaction resulting to user loyalty and patronage in libraries (Twum et al., 2022).

SpringShare LibAnswers' Quality of Service (QoS)
One of the tools used nowadays to measure user satisfaction with regards to library services is the LibAnswers' Quality of Service (QoS) feature of SpringShare. Launched in January 2021, this new feature aims to gather statistics dedicated to assessing the quality of service libraries provided through LibAnswers' chat and tickets (SpringShare, 2021). This also gathers user feedback and analyze metrics relevant to virtual reference and library assistance "to reduce dissatisfied tickets, enhance patron satisfaction, and capture staff output" (Breeding, 2021). To measure the user satisfaction on library virtual services, the NU Library decided to introduce Quality of Service (QoS) feature in March 2021. This new feature of the SpringShare's LibAnswers is designed to collect patron feedback on their overall satisfaction with the library's virtual service. Metrics such as checking the turnaround time in answering tickets and chats and the number of interactions, replies, as well as ratings were also generated from this feature. To get the user feedback, the Library sends a customizable follow-up survey to at least 25% of the closed tickets everyday. With this new feature, the Library was able determine not just the patrons' feedback but also the promptness of the response or service. Given the metrics, the Library was also able to further improve the service rendered to the academic community.

Objectives
Given that gauging user satisfactions is quite challenging especially in a virtual library service, this study aims to determine the satisfaction of users with the virtual service of Nazarbayev University Library during the Spring 2021 to Spring 2022 semesters. Using the data gathered from LibAnswers' Quality of Service (QoS) from March 2021 to June 2022, this study examines various metrics such as the turnaround time and the number of interactions in answering the queries received both in synchronous (chat service) or asynchronous (email service) reference service. In addition, feedback from users were also explored and considered to determine the overall effectiveness virtual reference service as well as the areas that need to be improved.

Methodology
This study employed a descriptive approached to determine the satisfaction of users with the virtual service of Nazarbayev University Library during the Spring 2021 to Spring 2022 semesters. Using the data gathered from LibAnswers' Quality of Service (QoS) from March 2021 to June 2022, this study examines various metrics in answering the queries received both in synchronous (chat service) or asynchronous (email service) virtual service such as:  the turnaround time;  the number of interactions;  the number of replies;  the duration of each transaction; and  the ratings. In addition, feedback from users were also explored and considered to determine the overall effectiveness of the virtual library service as well as the areas that need to be improved. Statistical data from the LibAnswers' QoS were tabulated and analyzed and the feedback from patrons will be interpreted using Voyant Tools, a web-based data visualization software. For data visualization, the tools used were Cirrus, a word cloud that visualizes the top frequent words used; a Collocates Graph which represents keywords and terms that occur in proximity as a force directed network graph; and Bubbles, a playful visualization of term frequencies of a corpus , or in the context of this study, the feedback from the users gathered from QoS.

Results and Discussions
To measure the user satisfaction on library virtual services, the NU Library decided to introduce Quality of Service (QoS) feature in March 2021. This new feature of the SpringShare's LibAnswers is designed to collect patron feedback on their overall satisfaction with the library's virtual reference service. This includes some metrics such as checking the turnaround time in answering tickets and chats and the number of interactions before closing the tickets. From March 2021 to June 2022, the NU Library received 3,534 tickets and 1,732 chat transactions.

Ticket Turnaround
One of the determinants of service quality is the responsiveness in providing the service. This includes the willingness to assist the clients and provide prompt service in no time. The QoS has the ability to analyze metrics with regards to how long it takes for the operator to reply to a patron's question and the time duration from the moment the question is asked up to the time the ticket has been closed.
For NU Library, all tickets were received through the dashboard. These tickets came from the official email address of the Library. From March 2021 to June 2022, out of 3,534 ticket transactions, 2,045 tickets or 58% were answered in less than an hour from the time the transaction was received as noted on the Table 1 below. This means that the operators assigned to monitor the tickets on the dashboard ensures that all tickets were handled as quick as possible. To ensure that all tickets were handled, dashboard monitoring and scheduling has been implemented for all responsible departments assigned. In addition, it was agreed among the operators that tickets should be handled at least within 10 minutes from the time it was received from the dashboard. When it comes to closing a ticket transaction, it has been noted that 1,513 tickets or 43% were closed in less than an hour as shown on Table 2. It was also noted that there were some tickets that took some time to answer. Possible reason for this was that answers to some questions were not as easy as it seems and would need some consultation or checking with other library staff or resources. To guide the operators in handling, answering, and closing a ticket, a set of guidelines were created. On the guidelines, it was noted and agreed that a ticket should be closed as soon as possible or if operators will need some time to answer, the operator should provide an initial response and the ticket should be closed within the day or at least first thing in the morning of the next working day for rare cases. Through the QoS, the Library can also check how many interactions happened to close a transaction. Interactions means the number of times the operator interacted with the ticket before it was closed which includes responding to the patrons, leaving an internal note, transferring the ticket to another user, or changing the status without sending a reply to the patron. Table 3 shows that 2,065 transactions or 58% were closed in a single interaction from the operator. This further means that most of the transactions were handled and closed with a single interaction from the LibAnswer user. Another metric that is being analyzed through the QoS is the number of responses the operator sent to the patron. Table 4 below shows that there were 4,148 replies that were sent to the patrons from March 2021 to June 2022. Out of 3,534 tickets, 2,713 tickets or 77% were closed with just a single response from the operator. This further signifies that, in one way or another, those one-time responses seem to be sufficient to satisfy the patrons' query.

Ticket Ratings
Aside from getting feedback, patrons were also asked to rate the library virtual service. Using the QoS, the Library can send an automated follow-up evaluation email to at least 25% of the tickets closed everyday. There were patrons who have generously provided their honest feedback to improve the service. In rating the service, the Library used an available template with a simple Likert scale from 1-5 as shown below: 5 Excellent 4 Good 3 Satisfactory

Poor 1 Needs Improvement
From March 2021 -June 2022, QoS received 644 ticket ratings. Out of 644 tickets, 557 or 86% received an excellent rating from the patrons as shown in Figure 1. This denotes that the patrons were satisfied with the virtual responses and the overall service they received from the library.

Chat Turnaround
Time is of the essence with regards to library chat service in a virtual environment. When handling the chat service, the library needs to make sure that operators are online within the working hours. There are times that multiple chat transactions were received, thus, it is highly encouraged to have at least 1-2 operators online to ensure that all chat transactions were handled as quick as possible. Since the operators are reference librarians who are also subject librarians, they have other things to attend to such as conducting library instructions, information literacy sessions, and even liaise with their respective schools. Thus, scheduling the chat duties of reference librarians has been practiced from the get-go.
Table 5 below revealed that out of 1,732 chat transactions, 1,629 or 94% were claimed by the operator within 15 seconds or less. This further signifies that operators treat each chat transactions urgent and ensure that the chat has been handled as quick as possible.  Chat duration is also being analyzed through the QoS. This is also fundamental in assessing the quality of service as well as the user satisfaction during the chat transaction as it determines the duration until the patron is satisfied with the response from the operator. There are times that the patron has other questions and concerns that he can ask during the chat. Given the nature of the chat as instant and more personalized since the operator is also conducting a reference interview in a virtual setting, chances are the patron will feel comfortable chatting to the operator, thus, asking more questions. Depending on how the chat went through, most of the chat transactions (540 or 31%) lasted for 5-10 minutes as shown in Table 6 below.

Chat Ratings
The same with ticket ratings, QoS also collects chat ratings to measure the synchronous service provided via instant messaging. Through chat, patrons can get quick on-the-spot answer to their questions. From March 2021 to June 2022, the library received 1,732 chat transactions. Despite more than half of the total transactions did not receive any ratings, 663 chats transaction or 38% rated the transactions excellent which further signifies that patrons were satisfied with the synchronous service given by the chat operators. Given the metrics presented, it can be noted that patrons were generally satisfied with the virtual service rendered by NU Library. Moreover, creating a set of guidelines to ensure that all transactions were handled also plays an essential role in keeping all operators guided especially during the crisis.

User Feedback
Aside from the metrics from QoS, it is indeed interesting to know how satisfied the users based on their honest feedback. Through this feedback, the library can understand how the users perceived the service. Since the service rendered were virtual in nature, it is quite challenging and tricky to know if the patron is satisfied with the service he received as well as his overall experience.

Ticket and Chat Feedback
To provide a visual representation of the patrons' feedback on tickets handled by the NU Library, the researcher made use of the Voyant Tools, a web-based application used for text analysis. This tool provides various ways to create a visual representation and analysis of the text. For this study, the researcher made use of Cirrus, a word cloud that visualizes the top frequent words used; a Collocates Graph which represents keywords and terms that occur in close proximity as a force directed network graph; and Bubbles, a playful visualization of term frequencies of a corpus , or in the context of this study, the feedback from the users gathered from QoS. Figure 3 below provides a Cirrus word cloud representation of the most frequent words used by patrons in the ticket feedback. This particular word cloud provides a more convenient and simplified overview of the context of feedback from patrons with the words in bigger font sizes as the most frequent words used. In an hindsight, the most frequent words used were thank (39), service (34), library (30), books (25), and time (19). In the same context, Figure 4 below provides a Cirrus word cloud representation of the most frequent words used by patrons in the chat feedback. It was observed from the word cloud that patrons were satisfied with the chat service given the most frequent words used. Surprisingly, patrons even mentioned the name of the specific operators whom they found very helpful and responsive during their chat experience. The most frequent words used were helpful (25), thank/s (20), quick (9), april (7), and excellent (6). Another way to visualize the feedback of patrons is through the Collocates Graph which is a network graph visualizing the proximity of terms. The most frequent keywords were in blue box and collocate keywords (terms in proximity) were in orange boxes. This further denotes that every time the keyword (in blue) appears, chances are, that the collocate keywords will be found, too. Figures 5 and 6 show the collocates graphs for ticket and chat feedback, respectively. Voyant Tool's Bubbles is a playful visualization of frequent terms in the feedback received. A bubble will be created in the canvas for each keyword in the feedback. As the same keyword is being read and repeated all throughout the document, the number of count for that keyword increases alongside with the size of the bubble. At the same time, high frequency terms in descending order were also found at the right side of the canvas. Figures 7 and 8 show the bubbles for ticket and chat feedback, respectively. In all three data visualization tools used in the study, the same keywords were found to describe how patrons felt satisfied with the virtual library service whether in synchronous or asynchronous format. In a hindsight, the patrons seem to be satisfied with their overall experience with regards to virtual services provided by the NU Library.

Conclusions
With the emergence of technology and the changing demands of users, having a virtual library service is not really a new thing. However, the pandemic has made the world realized that libraries should remain flexible and innovative without compromising the quality of service they rendered to their clients. Having said that, this study revealed that patrons were satisfied with the virtual service rendered by NU Library both in synchronous and asynchronous format. With the help of the metrics generated from LibAnswers Quality of Service (QoS) feature, the NU Library was able to provide an excellent service despite the limitations brought by the pandemic. The feedback from patrons was indeed a proof of their satisfaction. As service quality and user satisfaction will always go together, the library, as a service-oriented information arm of any institution, should always put their patrons as their top priority.