Factors Contributing to Success in an Introductory Mechanical Engineering Course: A Data-Driven Case Study

Recent studies have paid attention to the success and performance of students in higher education, concerned finding significant contributing factors. This paper attempts to examine and investigate the effect of a number of factors on students’ success in engineering courses. The data was collected from an introductory course, the Engineering Graphics and Practice, in the mechanical engineering department at the University of Texas at San Antonio (UTSA) during fall 2018 semester. The level of students in the class is mixed. Although this course is primarily designed for freshmen, there are upper-division students (juniors and seniors). The study finds that attendance and homework have the most significant effects on the overall performance of students in the course. Surprisingly, a counterintuitive finding is that the effect of a student’s class level is insignificant. The upper-division students did not outperform those in a lower division. Ultimately, the finding of this paper proves that active learning has a greater impact on a student’s overall performance than a number of earned credits.


Introduction
Improving students' understanding and learning procedure is the main goal of any instructor [1,2].There are factors that influence students to join an engineering program [3].However, the factors related to students' success in an engineering course is varied, and their level of effectiveness is in question [4].For an instructor, knowing a golden contributing factor into student success, especially for a big class with over a hundred students, is an essential challenge.This investigation is done on an introductory course, the Mechanical Engineering Graphics and Practice, in the mechanical engineering department at the University of Texas at San Antonio during the fall semester in 2018.This course consists of two main parts, the lecture part, and the lab part.
The motivation for such an investigation in the first place begins by observing results that are far from the expectations.Scrutiny on students' performance demonstrates unaccommodating distributions of students' grades.Regarding the grades of midterm exams, final exams and the final grade of the course, the first thing that got the attention of instructors of the course was the distributions of the grades.For example, Figure 1 demonstrates the histogram of the second midterm exam in the lecture class, and it can be seen that the distribution of the grades is bimodal ME 1403, the Engineering Graphics and Practice, is primarily designed to be a freshmen course and it is recommended to be taken by the students at the first semester of their study (Figure 2) [5].However, after a few semesters, the instructors of the course came to understand that the majority of the students are not freshmen, especially in the spring semesters.As Figure 3 shows, in fall 2018 only about one third of the students were freshmen and the remaining two thirds were mostly sophomores and juniors.It was hypothesized that this could give those students in the upper division leverage over the students who follow their mechanical engineering recommended program of the study.The reason for this thought was because the course is to teach the basic concepts of engineering design and drawing that students will learn with further details later in their program of study.Besides, some of the students in the upper division are transferred students who passed a similar course at another college or university but it was not accepted at UTSA.Moreover, there are cases of students with years of industrial experiences in design taking the course.They seem to have an advantage over the inexperienced freshmen.With the aforementioned doubts, the objectives of this study are defined in the next section.Following that, the research method including the sampling, data collection, variables, and analytical methods will be explained.The following section discusses the results of the study with the analysis of variance and regression analysis for all involved factors and later significant factors will be discussed in more details separately.Finally, the last section concludes this paper by summarizing its main contributions and highlights possible future work to be undertaken to further improve our understanding of the significant factors to enhance students learning procedure and outcomes.

Research problem
The purpose of the study is to examine factors related to students' study success in university-level engineering education.More specifically, the research attempts to investigate the effects of different variables in student success in the course.The objective is to understand the correlations between these variables and their final grades and find the most important one to support better teaching strategies.

Method Sample
Data for the study were collected from ME 1403, engineering graphics and practice, in the mechanical engineering department at the University of Texas at San Antonio which represents a large class introductory course designed to be taken by the freshmen on the first semester of their study.This course consists of two main parts, lecture part and lab part, including two sections for lecture classes and eight sections for lab portions which were handled by a group of two instructors, four teaching assistant and two graders.The study sample population was collected from overall performance of students in one of the lecture classes during the fall semester of 2018, excluding ones with a special condition such as students in the lists that never showed up and did not take any of the exams (n=105).However, two students with prior industrial experience, admitting having competent knowledge of the course materials that wanted to have the challenge exam are considered.

Data collection
The data for the research were collected from two different sources that later integrated into one dataset.The first source of data is a virtual learning environment and course management system, Blackboard Learn [6], utilized by the whole campus of UTSA.The Blackboard Learn provides a web-based environment to interact with students for different purposes.The managing of the course including the collection the homework, projects and online quizzes were done by this tool.The second data collection tool is the iClicker Cloud.The iClicker is a student and audience response system designed for higher education [7].The system was primarily used to take attendance in such a big class.It also provided an easy-to-use mean to have the in-class activities.Quizzes and polls were created on the iClicker cloud which could be seen by students and answered by them through the iClicker app.

Variables
The variables for this study were considered based on the syllabus of the course.As Table 1 demonstrates, the final grade of the course is calculated based on 40% of lecture portion which provides the basic theory and 60% from the lab portion which students learn how to use the theory with a CAD software.The software used in this course is SolidWorks 2018.However, factors with the obvious effect such as midterm and final exams were excluded from the study.Also, though the sixty percent of the total grade comes from the lab portion of the class main purpose of the study is to measure the effectiveness of the lecture factors since it provides the engineering basic understanding for the lab portion of the class.
The four factors considered for this study are attendance, homework, in-class activities for the lecture portion (iClicker Questions), and the class level of students.The first three factors demonstrate the active learning procedure of each student during the semester while the last one shows the effect of seniorship for success in such an introductory course.

Proceedings of the 2019 ASEE Gulf-Southwest Annual Conference
The University of Texas at Tyler Copyright © 2019, American Society for Engineering Education

Analytical methods
To analyze the dataset, analysis of variance (ANOVA), regression and pivot technique have been utilized.Firstly, the ANOVA is utilized to find significant factors on the final grade of the lecture portion and also their effect on the total grade of the course for each student.Later, having known the significant contributors, we find the correlation between each factor and the final outcome of each student.Lastly, to summarize and demonstrate some of these correlations the pivot technique was used.These analyses are done by using the Minitab to enhance the process.

Results and discussion
In this section, the results of the analysis will be presented followed by the discussion about the reasons behind each result.At first, we will use the analysis of variance considering all the factors explained in the previous section to find the significant contributors.Then to realize the impact of each factor on the final grade of students, the regression analysis will be utilized.Lastly, significant variables will be discussed separately to have a better interpretation of the influential factor in the students' success.

Analysis of Variance
To analyze the factors, we need to know if each factor contributes significantly to the final grade of students in the course.To achieve that, the analysis of variance (ANOVA) is used considering that factors which are analyzed do not have a high percentage in the calculation of the final grade.
In other words, we like to understand the indirect impact of these factors into students' success.

Proceedings of the 2019 ASEE Gulf-Southwest Annual Conference
The University of Texas at Tyler Copyright © 2019, American Society for Engineering Education Factors considered for this ANOVA are average homework grade, attendance percentage, in-class activities (iClicker questions), and students class level.As the Table 2 is demonstrating the results of the ANOVA, three of the factors, average homework grade, attendance percentage, in-class activities (iClicker questions) comes as significant since their P-value is under 0.05.However, interestingly, the class level of the students that we expected to be the most effective factors comes as the nonsignificant.Our expectancy was to see students with upper-class level have a better average final score which was proved wrong.The ANOVA's result could be interpreted that active method of learning, meaning when a student learns the material through the semester and not at once, is the main element of student success in a course similar to the one in this study.However, even though students with upper-class level normally should be familiar with the basic concepts because they are being thought with more details in other courses, still it does not give any extra leverage over the freshmen students.

Regression Analysis
Now after knowing the significant factors into students' success, we use the regression analysis to find the most significant factor with the highest impact.The results of the regression analysis are shown in Table 3.As can be seen, the attendance comes as the most significant effect on the final grade of students with the coefficient of 0.3668.Then the homework is the second most important factor with coefficient 0.2666, and lastly, iClicker questions are the least important factor among the considered factors effective on the final grade of the course.
As had seen in the ANOVA analysis the class level is not significant as some level of them comes with the negative sign that compensates the effect of positive ones, and also all levels come with p-value higher than 0.05.
Same as the ANOVA analysis, the regression analysis shows the importance of the active learning of students during the semester and demonstrates how effective is being present in the lecture class even though it does not have the direct effect on students' final grade.

Proceedings of the 2019 ASEE Gulf-Southwest Annual Conference
The University of Texas at Tyler Copyright © 2019, American Society for Engineering Education

Effect of Attendance
To have a better understanding of the importance of the attendance in student's success, we use a pivot table to summarize its effect by each group of students.As can be seen in Table 4 the majority of students (70) was attended most of the classes during the semester, had less than 3 absences, and as the result, they got the highest total grade and also lecture grade.Also, it can be seen as the number of absences increases the average of both grades decrease which is compatible with the results of ANOVA and regression model analysis.
We believe that the correlation between attendance and total grade is mostly due to the mentality of each student.Normally, students with a high level of commitment and motivation for their program study and more specifically for the course have better attendance and show the highest activeness which results in higher achievement.
The only exception for this analysis is the student with a special condition which leads not to attend any of the session during the semester.However, even though the score of the student with more than 28 absences is higher than the ones with 24 to 27 absences, it is not good enough grade.The mentioned student was a student with ten years of experiences in industry related to mechanical design who was familiar with all concept.However, it shows even for a student with such a great background prior knowledge was not good enough.

Effect of In-Class Activities
As shown in Table 3, despite that the in-class activities done by the iClicker questions has a lower impact on student overall performance, it still has a meaningful impact on the total grade.As Table 5 shows the same positive correlation can be seen between active student for in-class activities and their total grades.It should be said that since most of these in-class questions were designed and asked as a multi-choice question, similar to their final exam, students with better performance on them performed better on their final exam as well.

Effect of Homework
As mentioned in the ANOVA section, homework contributes significantly to students' success for this course.To have a better understanding of its effect we compare the other element regarding the homework that is the number of homework submitted by any student.We wanted to check if a student makes mistake on his/her homework could have a better or worse overall performance in the course.
ANOVA and regression analysis were utilized considered two factors namely, average homework grade for the whole semester versus the number of homework submitted (in percent).As Table 6 and Table 7 illustrate the number of submitted homework comes as insignificant while the average grade of homework has a high correlation with the overall performance of each student.This result also could be interpreted similarly to one regarding the attendance that students with higher motivation and firm purpose for their program study do the homework with higher accuracy (Table 8).

Proceedings of the 2019 ASEE Gulf-Southwest Annual Conference
The University of Texas at Tyler Copyright © 2019, American Society for Engineering Education

Conclusions and Future Work
The results of this study indicate that a student's success in a course is highly related to their active learning style during the semester and persistent participation in the class.Among the factor we investigated, attendance and homework performance have the highest impact on the students' overall performance.Also, in-class activities are important for their success despite that it has a lower impact on the students overall performance.
Surprisingly, the class level of students proved to be insignificant in students' success in this course.In other words, even though the course was designed for freshmen and meant to be taken in the freshman year, it does not give any leverage to those students who postponed taking the course in later years of their study.It again proves that active learning style is more important than

Proceedings of the 2019 ASEE Gulf-Southwest Annual Conference
The University of Texas at Tyler Copyright © 2019, American Society for Engineering Education having prior knowledge and familiarity of the course materials.This is an important finding for the instructor team to consider revising the teaching strategies and tactics to better engage students in an active learning style for better performance.
Knowing that factors related to active learning have a high impact on students' success, a few action items have been identified for future work.First, a survey should be carried out to measure their learning activeness perception during the semester.This could be done through the same data collection tools that we used for this research.
Moreover, to have more accurate and reliable results, it is recommended to have a larger data sample.This data set may be collected for a few academic years by different instructors so that the instructor's perception and biases on the data set can be compensated.
Lastly, the same data could be collected from different courses preferably at freshmen and sophomore levels so it could be analyzed for courses requiring higher student attendance and active learning.Furthermore, the impact of the students' class level should be investigated to check if it possibly is significant on the students' performance for other courses.

Figure 1 :
Figure 1: Bimodal distribution of the second midterm exam Proceedings of the 2019 ASEE Gulf-Southwest Annual ConferenceThe University of Texas at Tyler Copyright © 2019, American Society for Engineering Education

Table 1 :
Grading calculation criteria

Table 2 :
ANOVA of all factors

Table 3 :
Regression of all factors

Table 4 :
pivot of absences vs. final grade and lecture grade Proceedings of the 2019 ASEE Gulf-Southwest Annual ConferenceThe University of Texas at Tyler Copyright © 2019, American Society for Engineering Education

Table 5 :
in-class activities vs. final grade and lecture grade

Table 6 :
ANOVA of number of homework submitted vs. homework grade

Table 7 :
regression of the number of homework submitted vs. homework grade

Table 8 :
homework grade vs. final grade and lecture grade