Evaluation of E-Learning Implementation During the Covid-19 with the DeLone and McLean Models

During the Covid pandemic the learning process was carried out using e-learning media. SMK TI Airlangga Samarinda is one of the schools that has used e-learning in its learning process both before the Covid pandemic. Furthermore, an evaluation of the success of e-learning implementation will be carried out using the DeLone and McLean information system success model (D&M IS Success Model). In this model, there are six measurement variables, namely system quality, information quality, service quality, usage, user satisfaction, and net benefits. The purpose of this study is to provide an overview of the use of e-learning at SMK TI Airlangga Samarinda to the school and to find out the factors that influence the successful implementation of e-learning at SMK TI Airlangga Samarinda. To achieve this goal, SEM-PLS analysis (structural equation model partial least square) is used. The subjects used were 336 students of SMK TI Airlangga Samarinda. The sampling technique used in this study was purposive sampling, with consideration of the availability of data obtained a sample of 192. Data collection techniques in this study using questionnaires, interviews, and literature study. The results of this study indicate that the quality of information and system quality in e-learning has a positive effect on the satisfaction of using e-learning which in turn has an effect on the benefits felt by students while using e-learning. In addition, the quality of the e-learning system also has a positive effect on the intensity of e-learning use, which means that the better the quality of the system will increase the intensity of e-learning use.


Introduction
Technological progress is currently increasing rapidly, one of which is in the field of education, especially in the teaching and learning process, namely the application of e-learning technology [1]. Sekolah Menengah Kejuruan Teknologi Informasi (SMK TI) is one of the private vocational schools located in Samarinda. SMK TI Airlangga has several expertise programs or majors, namely, multimedia (MM), software engineering (RPL), and computer network engineering (TKJ). SMK TI Airlangga has internet facilities that can be accessed by teachers and students. The learning process combines conventional methods and e-learning since 2012 [2]. Since the covid-19 pandemic, SMK TI Airlangga has fully used e-learning. It is not known to what extent the successful implementation of e-learning that has been implemented at SMK TI Airlangga Samarinda, therefore it is necessary to measure the success of e-learning implementation at SMK TI Airlangga Samarinda, so this is what underlies researchers to evaluate the success of e-learning implementation in SMK Airlangga Samarinda as a support for the learning process. The purpose of this study is to provide an overview of the implementation of e-learning at SMK TI Airlangga Samarinda and to find out the factors that influence the success of elearning implementation at SMK TI Airlangga Samarinda. To measure the success of e-learning implementation at SMK TI Airlangga Samarinda, researchers used a model developed by DeLone and McLean (2003) or what is called the De Lone and McLean information system success model (D&M IS Success Model) [3].

Research Methodology
This research is a case study with a non-experimental research design in the form of a survey. This research was conducted at SMK TI Airlangga which is located on Pahlawan street Number 2A, Dadi Mulya Village, Samarinda Ulu District, Samarinda City, East Kalimantan from October to November 2020. The subjects in this study were all students of SMK Airlangga Samarinda, each of which totaled 336 people. The sampling technique used is a non-probability sampling method, namely purposive sampling, taking samples with certain considerations or criteria [4]. In this study, the consideration is the availability of data. Data collection conducted in this study using several techniques, namely questionnaires, interviews, and document studies.
The variables used in evaluating the successful implementation of e-learning based on the DeLone and McLean methods in Figure 1 are information quality, system quality, service quality, use, user satisfaction, and net benefit [3] [5]. The system quality variable consists of three indicators, namely response time, ease of use, and ease of access [6][7] [8]. The information quality variable consists of four indicators, namely completeness, relevance, accuracy, and comprehensibility [9] [10]. Service quality variable consists of three indicators, namely assuranc e, responsiveness, tangibles [6][8] [9]. The use intention variable consists of two indicators, namely daily use and frequency [6] [9]. User satisfaction variable consists of four indicators, namely effectiveness, information satisfaction, system satisfaction, and overall satisfaction [7]. The net benefit variable consists of five indicators, namely productivity, being able to do more learning, usability in the learning process, facilitating the learning process and user prestige [6][7].

Figure 1. DeLone and McLean Model 2003
Data analysis was carried out through several stages, namely processing (tidying up) the data from the questionnaire so that it could be analyzed, describing the questionnaire result data in the form of characteristics from students, drawing path diagrams, evaluating measurement models or the outer model, evaluating structural models or inner models, and drawing decisions and conclusions. The null hypotheses tested to determine the success of e-learning implementation at SMK TI Airlangga Samarinda are: a) System quality does not significantly influence the use of e-learning.
b) The quality of information does not significantly influence the use of elearning. c) Service quality has no significant effect on the use of e-learning. d) System quality has no significant effect on e-learning user satisfaction. e) Information quality has no significant effect on e-learning user satisfaction. f) Service quality does not have a significant effect on e-learning user satisfaction. g) Use does not have a significant effect on e-learning user satisfaction. h) Use does not significantly influence the benef its of e-learning. i) User satisfaction does not have a significant effect on the benefits of e -learning.

Overview of The Learning Use
Some of the e-learning platforms used at SMK TI Airlangga Samarinda are Moodle, Schoology, Classroom, learn.smkti.net, and Edmodo. However, since the existence of Covid-19, SMK TI Airlangga Samarinda students have carried out online learning at their respective homes using the Classroom the most. Some of the obstacles faced in implementing learning with e-learning, namely the presence of attendance has not been integrated into the curriculum, the unpreparedness of teachers in preparing learning content has resulted in decreased student enthusiasm for learning and a lack of soft skills possessed by these students. These results are known from the results of interviews with principal of SMK TI Airlangga Samarinda.

Data Description
Of the 336 students of SMK TI Airlangga who had been given questionnaires in this study, 192 students or 57% had filled and returned the questionnaire. With the consideration of data availability, the number of samples in this study was 192 people. Figure 3 shows the majors and classes of students who have filled out a questionnaire which is then used to evaluate the successful implementation of e-learning using the Delone and McLean model. This data is taken six months after students are required to study at home online.

Figure 2. Graph of the Number of Students Filling the Questionnaire by Class and Department
Based on Figure 2, it is known that the students who filled out the questionnaire the most were from the office automation and management majors, multimedia, computer and network engineering, software engineering, and the least were the online business and marketing majors.

Model Evaluation
Model evaluation consists of evaluating the measurement model or what is called the outer model and evaluating the structural model or called the inner model [11]. Evaluate the measurement model or the outer model by assessing the validity and reliability of the model. The validity to be tested is convergent validity and discriminant validity. For convergent validity, the loading factor and Average Variance Extracted (AVE) parameters are used, the loading factor value > 0.6 and AVE > 0.5 to meet the convergent validity [12].  Table 1 shows the loading factor value for all variables and their respective indicators, it can be seen that all indicators have a loading factor value of more than 0.6. The AVE value is shown in Table 3 for each variable, it is obtained that the AVE is greater than 0.5, so that the convergent validity has been fulfilled. Furthermore, for discriminant validity, cross loading is a parameter where the value must be greater than 0.7 for each variable [11]. Table 2 shows the cross loading value for each latent variable and its indicators, it can be seen that all variables have a cross loading value greater than 0.7, so the discriminant validity measurement has been fulfilled. In testing the reliability, one of the parameters used is composite reliability, the value must be greater than 0.7. Table 3 shows the results of composite reliability greater than 0.7 for all variables, so that all variables are reliable. Evaluation of the structural model in this study consists of exposure to the coefficient of determination or R Square and the significance of the path coefficient. Table 4 shows the R Square results for the user satisfaction, net benefits, and usage variables.  Table 4 it can be seen that the R Square value for the user satisfaction variable is 0.459, including the moderate category. Furthermore, the R Square value for the net benefit variable of 0.500 is in the moderate category. Finally, the R Square value for the use variable is 0.119, including the weak category.
The results of the value and significance of the path coefficient for testing the hypothesis in chapter 2, the hypothesis is rejected if the T statistical value is > 1.96 (5% significance level) or P value < 0.05 [11], shown in Table 5 below. The results of hypothesis testing based on Table 5 are as follows: 1. User satisfaction to net benefits. In Table 5, the T statistic is 13,165 > 1.96 and the P. value is 0.000 < 0.05, then the hypothesis is rejected, it is concluded that user satisfaction has a significant effect on the net benefits of e-learning. The value of the coefficient of user satisfaction on the net benefits is 0.665. This means that the direction of the relationship between user satisfaction and net benefits is positive. Satisfaction with the use of e-learning increases the benefits of using e-learning. This is in line with research [6] and [13] also supporting the result of research [3].

Quality of information to user satisfaction.
In Table 5, it is obtained that the T statistic is 3,836> 1.96 and the P. value is 0.000 <0.05, then the hypothesis is rejected, it is concluded that the quality of information has a significant effect on e-learning user satisfaction. The value of the information quality coefficient on user satisfaction is 0.386, which means that the direction of the relationship between information quality and user satisfaction is positive [6]. It can be said that the better the quality of information provided on e-learning will increase student satisfaction with the use of e-learning. These results support research [3] and are in line with studies [6] [13] [14]. 3. Quality of information to use.
In Table 5, it is obtained that the T statistic is 0.800 < 1.96 and the P. value is 0.424 > 0.05, so the hypothesis fails to be rejected, so it is concluded that the quality of information does not have a significant effect on the use of e-learning. 4. Quality of service to user satisfaction.
In Table 5, it is obtained that the T statistic is 1.822 < 1.96 and the P. value is 0.069 > 0.05, then the failed hypothesis is rejected, it is concluded that service quality has no significant effect on e-learning user satisfaction. 5. Quality of service to use.
In Table 5, the T statistic is 1.877 < 1.96 and the P. value is 0.06 1> 0.05, so the hypothesis fails to be rejected, so it is concluded that service quality does not significantly influence the use of e-learning. 6. System quality to user satisfaction.
In Table 5, it is obtained that the T statistic is 2.079> 1.96 and the P. value is 0.038 <0.05, then the hypothesis is rejected, it is concluded that the system quality has a significant effect on e-learning user satisfaction. The value of the system quality coefficient on user satisfaction is 0.143, which means that the direction of the relationship between system quality and user satisfaction is positive. It can be said that the better the quality of the system given to e-learning, the better the student's satisfaction with the use of e-learning, and this is in accordance with the model proposed [3]. 7. The quality of the system to use.
In Table 5, it is obtained that the T statistic is 2.084> 1.96 and the P. value is 0.038 <0.05, then the hypothesis is rejected, it is concluded that the quality of the system has a significant effect on the use of e-learning. The value of the system quality coefficient of use is 0.229, meaning that the direction of the relationship between system quality and usage is positive. It can be said that the better the quality of the system in e-learning, the better the use of e-learning. 8. Use to user satisfaction.
In Table 5, it is obtained that the T statistic is 1.057 <1.96 and the P. value is 0.291> 0.05, so the hypothesis fails to be rejected, so it is concluded that use does not significantly affect e-learning user satisfaction. 9. Use to net benefits.
In Table 5, it is obtained that the T statistic is 1.805 < 1.96 and the P. value is 0.072> 0.05, so the hypothesis fails to be rejected, so it is concluded that the use does not significantly influence the benefits of e-learning.

Conclusion
Several things have come to the conclusion of the research that has been done. First, the e-learning platform that is most widely used by teachers at SMK Airlangga is Google Classroom. Where in its implementation there are obstacles to the presence of attendance and learning content. Second, the factors that influence the successful implementation of e-learning at SMK TI Airlangga Samarinda are the quality of the system and the quality of information. Where the quality of information and system quality in e-learning has a positive effect on the satisfaction of using e-learning which in turn affects the benefits felt by students while using e-learning. Besides the quality of e-learning system was also positive effect on the intensity of use of e-learning, which means that the better the quality of the system will increase the intensity of use of e-learning.