IS OUR COLLECTION ENOUGH OR UNDERUTILIZED? A CITATION ANALYSIS OF MASTER’S THESES IN THE FIELD OF EDUCATION

Objective. The quality of library collection depends on how the librarians select materials for purchasing based on existing collection development policy. In the absence of approved policy, working guidelines take effect so there would be a selection guidelines criteria. The use of available selection tools, online book reviews, faculty recommendations, and standard lists will help the collection building. This paper aims to provide an overview of the type of materials Kazakh students utilize to complete their master’s thesis. Specifically, it will have the following objectives: 1) to categorize the types of sources graduate students use in their thesis; 2) to compare the actual count of e-journal titles available in our collection versus the number of e-journal titles used in the reference list; 3) to identify the common journal titles used by the students and its overall journal ranking credibility based on prestige; 4) To further develop the e-resources collection based on the citation analysis. Methods. In 2019, 20 e-theses were uploaded in the repository coming from one school (may also be called college or faculty). It was decided to select from one school only to determine the relevance of the collection coming from one discipline. A sample of 10 theses (50%) was analyzed in this study. Citation analysis was used to measure which materials are currently available in the library. It will also give an impression of whether the material is being used or not. Collection of data was done in an organized manner. A table was created to list the number of referenced items found per thesis. The researchers identified the number of print books, e-journals, conference proceedings, theses and other types of materials. To determine the specific journal titles, they were carefully extracted. Duplicate titles were combined and were counted as one unique entry. Using the SJR online, the researchers identified the quartiles of each title and determined the impact of the journals based on the ranking. Journal titles were searched and availability was verified using the online journal finder A-Z journal title list provided by the EBSCOHost. Non-quartile journal titles were also verified using Scopus. Results. An average of 59 references was listed by a graduate student in his/her master’s thesis. Each student may have used 17.6 print books to cite in their thesis and 32.4 journal articles in which some of them may have referenced a suspected predatory journal. There are 199 unique journal titles cited in 10 master’s theses. 19.6% of which is unavailable in the subscription but needs to be verified if they are indexed in leading citation databases. At least 50% of the theses had more journal titles categorized as Q1 journals. Another 50% the theses included more Q2 journals than Q1. One thesis cited more journal publications with no quartile but was able to reference 68% of journal titles coming from Q1-Q4 journals. In terms of journal availability, the library holds 80% of the journal titles cited in the reference list that are available whether by subscription or Open Access. Conclusions. It is apparent that master’s students from the graduate school of education will likely cite more journal articles than books. At least 50% of the students get their references from Q1-ranked journals based from the SJR ranking. The library has a promising and huge collection of e-journal titles as it has almost 80% of the titles available in the library subscription. The sample revealed that the library is capable of providing a good quality of journal collection that can be used by the students and faculty members as well. We can proudly say that our collection is above the average. We need to still dig deeper how much of our collection is used extensively by the other members of the academic community.


Introduction
The Graduate School of Education (GSE) of Nazarbayev University has produced about 56 master's theses since its inception in 2012. More than 300 master's and PhD students have graduated from the GSE but due to a competitive environment, only some of the best thesis are uploaded in the NU Repository. With this in mind, only 20 master's theses were qualified to be uploaded and for the purpose of this study, only 10 were examined as sample to generate some partial results. This paper deals with citations analysis to determine the types of sources used by the students in their thesis and to identify common journal titles extensively used by the graduate students.
Citation analysis scholars would argue that as early as 1984, librarians have already used the method of citation analysis as their basis for collection development and retain journals that are important in a specific field of research (Beile, Boote, & Killingsworth, 2003). Citation analysis is also about reputation. Gaining some expertise in the field, authors and journals cited in a research gives them prestige. A citation analysis study for the journal "International Journal of Educational Leadership Preparation" revealed that several authoritative scholars are always being cited because of their professional background and increase in publication outputs (Borgemenke & Brown, 2013). By this way, they were able to identify the well-known experts in the field. Aliyu (2018), conducted a citation analysis study of doctoral theses in Nigeria and he was able to identify top twenty core journals in the field of education. He also revealed that most post graduate students cited more books and monographs compared to journal articles.
Objectives of the Study. The quality of library collection depends on how the librarians select materials for purchasing based on existing collection development policy. In the absence of approved policy, working guidelines take effect so there would be a selection guidelines criteria. The use of available selection tools, online book reviews, faculty recommendations, and standard lists will help the collection building. Results of citation analysis would also be a good method to follow when evaluating the existing collection. This paper aims to provide an overview of the type of materials Kazakh graduate students utilize to complete their master's thesis. Specifically, it will have the following objectives: 1. To categorize the types of sources graduate students use in their thesis. 2. To compare the actual count of e-journal titles available in our collection versus the number of e-journal titles used in the reference list. 3. To identify the common journal titles used by the students and its overall journal ranking credibility based on prestige. 4. To further develop the e-resources collection based on the citation analysis.
Review of Related Literature. Different fields of discipline conducted their own citation analysis study. Brush (2015), a subject librarian for physical sciences and engineering, shared the results of her study on the citation analysis of engineering master's theses. She was able to identify the oldest book cited by an engineering student and that six is the average number of books cited. In terms of analyzing the journal publications cited in thesis, she gave a good report on the number of unique journal titles cited and the journals that the university subscribe to. Students also cited conference papers and other sources such as websites, newsletters, and personal communications. It was interesting that less than half of the books cited were not owned by their library. Timms (2018) studied 72 marine biology master's theses and extracted a huge number of journal citations. He was able to extract 5, 747 references coming from journal articles. He also tabulated more than a thousand unique journal titles. His study gave him the opportunity to provide an evidence of how the library collection is being used by the graduate students. Other similar citation analysis studies mostly have the same objectives. It is to determine the types of materials used by each student and to identify if the library was able to cater those materials based on existing holdings (Becker & Chiware, (n.d.) ;Miller, 2011;Sherriff, 2010). It is worth noting that each discipline have unique citation preferences. Historical researchers use more monographs than journals (Sheriff, 2010).

Methods
Most citation analysis encompass a number of years to compare the citation trend per year. The sample given in this paper is just on overview of the current citation situation in the graduate school of education since we are a young university. With only 56 master's theses as of 2019, only 10 were studied (17.86%) from the total number of available theses in the repository.
In 2019, 20 e-theses were uploaded in the repository coming from one school (may also be called college or faculty). It was decided to select from one school only to determine the relevance of the collection coming from one discipline. A sample of 10 theses (50%) was analyzed in this study. Citation analysis was used to measure which materials are currently available in the library. It will also give an impression of whether the material is being used or not. Collection of data was done in an organized manner. A table was created to list the number of referenced items found per thesis. The researchers identified the number of print books, ejournals, conference proceedings, theses and other types of materials. To determine the specific journal titles, they were carefully extracted. Duplicate titles were combined and were counted as one unique entry. Using the SJR online, the researchers identified the quartiles of each title and determined the impact of the journals based on the ranking. Journal titles were searched and availability was verified using the online journal finder A-Z journal title list provided by the EBSCOHost. Non-quartile journal titles were also verified using Scopus.

Results and Discussion
A total of 590 references were extracted and examined from 10 master's theses in education. 30% of the theses came from MA Multilingual Education and 70% of the theses came from MS Educational Leadership. Figure 1 shows the distribution of references by type of source.

Figure 1. Pie chart representing the kind of reference sources the student used in their theses
Students cite more journal articles (55%) compared to print books (30%). Only a few students cited thesis and conference proceedings. The smallest number of references used is 45 and the highest number of citations is 76. The cited reference mean is 59, median is 55.5 and mode is 55. The range is 31. From this amount, 324 (55%) are citations coming from journals. There are 199 unique journal titles identified. 29 of these journal titles (Table 2) appeared in several theses and 39 are unavailable in the collection. Table 2 shows that the journal Gifted Child Quarterly appeared in two student theses but was cited 14 times. Teaching and Teacher Education which appeared in three studies received 11 citations. The Journal of Multilingual and Multicultural Development appeared in three student theses and was cited 7 times. The Asia Pacific Education Review and Procedia-Social and Behavioral Sciences which appeared in four different studies received four and six citations respectively. It is noticeable that journal titles appear in two or three studies but they were only cited either once in each study. Students also cite journal articles that are not existing in our subscription. They were able to access these sources from their own referral systems or from Inter-Library Loan service. It is recommended to add these titles in the future acquisition plan of the library.
Scholars are critical on what sources to cite. Given the ease of access for online resources, it is advisable to consider those items with high impact and just use the electronic databases of the library. Five theses cited more Q1 ranking journal titles. These are theses 2, 3, 4, 5 and 10. Theses 1, 6, 7, 8, and 9 cited more Q2 journals. In any case, they were able to choose good quality resources rather than choosing the predatory ones.

Conclusions
Timms (2018) believed that the best evidence to identify if the serial or journal is being used is by the numbers. Citation analysis will provide us evidence-based record that our journal subscriptions are being used and accessed by our patrons. The high cost of e-journals will prevent us from renewing any database or journal title but if there is usage, then we will not be scared to ask for budget. Meanwhile, although each journal is being used, there is still low usage as one journal title entry in the reference list equates to one journal article only. The students should also increase their use of relevant articles coming from expert and discipline-oriented journals. It is apparent that master's students from the graduate school of education cite more journal articles than books. At least 50% of the students get their references from Q1-ranked journals based from the SJR ranking. With 199 unique journal titles identified, only 29 were used in more than one thesis. With two kinds of graduate programs, it is noteworthy that students have a diverse selection of journal titles that the library has to offer. The library has a promising and huge collection of e-journal titles as it has almost 80% of the titles available in the library subscription. The sample revealed that the library is capable of providing a good quality of journal collection that can be used by the students and faculty members as well. We can proudly say that our collection is above the average. We need to still dig deeper how much of our collection is used extensively by the other members of the academic community.
Further Directions. This study needs to be expanded and add explore the theses submitted prior to 2019. It will provide a more comprehensive view of the types of sources used by the students and the quality of journals they select as part of their thesis.