Exploring the Relationships of Grading Assessment Learning Perceptions Scales to Perceived Added Education Value Required Course Business Scales

This study’s purposes were to: develop added education value required business course scales; and investigate their relationships to four grading assessment learning perception (GALP) scales. Using a sample of spring, 2018, n = 944, graduating business undergraduates, three reliable (coefficient alpha) added education value required business course scales were identified: Lower-level Foundation (.92), Business Administration (.88), and Quantitative (.84). The Quantitative scale had a higher perceived added education value (Mean, M = 4.38 out of 6), versus Lower-level Foundation (M = 4.22), and Business Administration (M = 3.97). However, the relationships for three of the four GALP scales, i.e., exam, individual engagement, and team, were significantly stronger to the Business Administration scale, and the average correlation (r) across all GALP scales was higher to the Business Administration scale (r = .36) versus the Lower-level Foundation (r = .28) and Quantitative (r = .20) scales. Part of curriculum assessment should involve measuring the perceived added education value of each required course, in any school or college, not just business. This can represent a significant portion of a student’s investment in a degree. It is hoped that this study will stimulate continued research on the development of added education value scales. (198 words)


Introduction
Generally business students must take required core courses, regardless of their major, to complete their degree (Martell, 2007). There were two purposes of this exploratory study: (1) to measure business student perceived added education value of twenty-one required business courses, to then test if a smaller number of required course business scales could be successfully created, and (2) to test the relationships of four previously established grading assessment learning perception (GALP) scales to these added education value required course business scales. Prior relevant literature is reviewed below.

Required Business Core Curriculum
In the United States, business schools must create a business core curriculum to adequately prepare undergraduates for more specialized study in their major, as well as meet the standards of the Association to Advance Collegiate Schools of Business (AACSB), including providing a common body of business knowledge (Pharr, 2003). A list of required skill areas has been provided by the AACSB (AACSB, 2013, p.34), including general skill (e.g., written and oral communication, multicultural environments), technological agility (e.g., application of statistical tools, data management), and general business knowledge. The broadest of the three areas, general business knowledge (p.35), includes: (1) economic, political, regulatory, legal, technological, and social contexts of organizations, (2) social responsibility, including sustainability, diversity and ethical behavior, (3) financial theories, (4) production/operations, supply chains and marketing, and (5) group and individual behaviors within organizations. Across many business schools, including those AASCB-accredited, a number of required courses are typically needed to address the knowledge requirements for these skill areas. These required courses cut across different disciplines, including accounting, business administration, finance, human resources, management information systems, marketing, operations management, and statistics (Pharr, 2003). knowledge content across the above-noted areas. The relationships of other variables to the required business curriculum have been studied, including: stakeholder attitudes towards a required business curriculum (Pharr, 2003); student teaching evaluations (Yunker & Yunker, 2003); and student comprehensive business examination scores (Hahn, 2018). Working with a sample of only six of 21 required business due to sample size limitations, Blau (2019) asked for student perceptions of the "added value to their education" for these six required business courses. Two scales were found, labeled Business Administration (BA) Unique (4 items) and BA Generic (2 items). The BA Unique courses, such as "Professional Development Strategies" and "Excel for Business Applications" were based on more distinctive business school factors, including its large size which allowed for a school-specific career center. The BA Generic items included "Business Ethics" and "Global Business Policies" courses, which were more consistent with other business school required curricula. The BA Unique scale had stronger relationships to program degree satisfaction and business school reputation than the BA Generic scale, supporting the development of more unique core courses when relevant. However, being able to only work with six of the 21 required business courses was an important study limitation. The current study was able to utilize all 21 required business courses.

Developing Distinct Added Education Value Business Scales
Student teaching course evaluations filled out by students generally focus more on the instructor (e.g., grading fairness, teaching enthusiasm, subject mastery), versus a course adding educational value (Miles & House, 2015). Beyond this to meet demand, different instructors may be needed to teach multiple sections of a business core course. Standardizing the required course content (e.g., common texts) across different course sections is necessary to then build on this with more specialized courses within a student's major. It is important, within a business school to measure the perceived added educational value of each business core course to enhance the core course curriculum. However, it is not efficient to work with each require courses as an item for research purposes. Multicollinearity (Stevens, 1996) or redundancy between individual courses makes it more challenging to separate out different individual required course perceived added education value relationships with other variables. Representing the added education value of different required courses as "items," to then be aggregated, allows for creating a smaller set of distinct scales for research. However, it must be acknowledged that aggregating individual required courses into a scale can lead to more missing data, if different students waive out of different required courses (e.g., when they transfer in). As part of their required curriculum in the business school at this research setting, undergraduates were required to take twenty-one business courses. The limited prior research suggests presenting research questions for testing. Thus, the first research question (RQ) was: RQ1 -can distinct added education value required business scales be developed from a larger set of required course items, such that these scales are valid, reliable and sufficiently distinct?
A second research question to test would be, assuming distinct scales, are they perceived by students as equal in added education value? This suggests: RQ2 -are distinct added education value required business scales perceived as equal in added education value when compared?

Testing the Relationships of GALP Scales to Added Education Value Required Business Scales
Both components, grading assessment and learning outcomes, are typically found in a course syllabus (Smith & Razzouk, 1993). There are usually different grading assessment items detailed in a course syllabus, for example: quizzes, examinations, individual papers, presentations, participation, attendance, as well as group projects (Flores, Veiga Simao, Barros & Pereira, 2015;Smith & Razzouk, 1993). Grading fairness assessment has been more extensively investigated than grading assessment learning perceptions (GALP). Past research on grading fairness (Bacdayan & Geddes, 2009;Pepper & Pathak, 2008) focused more on students understanding how their grades were calculated for individual assignments as well as for the course, i.e., transparency. Flores et al. (2015) found that students perceived more actively-involved grading assessment methods (e.g., individual and group portfolios) as fairer and more effective than more passive/traditional methods (e.g., examinations, written tests). Although no formal grading assessment method scales were created by Flores et al. (2015), their results suggested that aggregating grading techniques into broader scales could be done. GALP's goal is to successfully aggregate course grading techniques into broader scales for research.

Measuring GALP
Blau, Blessley, Kunkle, Schirmer and Keen (2017a) used two independent samples of graduating senior business undergraduates to initially develop two three-item GALP scales, one scale was labeled closed GALP (e.g., multiple-choice exams/quizzes) and the second scale was labeled open GALP (e.g., written assignments).
Regression analyses found that across both samples perceived satisfaction/reputation of the business school program was the strongest positive correlate to both GALP scales. However, Blau et al. (2017a) noted that additional GALP items were needed measuring other grading assessment learning perception items such as team assignments, and distinguishing between class participation versus class attendance. Other research (Flores et al., 2015) has worked with additional grading assessment methods, such as individual written reflections and portfolios. A second, more recent study by Blau, Gaffney, Kim and Jarrell (2017b) using a broader set of 13-items, found support for four distinct and reliable GALP scales (number of items/scale): individual engagement (4), exam-based (3), individual creative (3), and team-based (2) across two samples of senior business undergraduates. One-item was deleted. Logistic regression analyses showed that beyond internship experiences, only the individual creative GALP scale was significantly positively related to two expected post-graduation employment measures, securing a full-time job and securing a full-time job in one's major.

Testing for Scale Differences in Added Value to Education
Gathering the student perspective is important, and undergraduates' overall perceived value of education has been positively linked to general student satisfaction across multiple studies (Alves & Raposo, 2007;Ledden, Kalafatis & Samouel, 2007). However, limited prior research (Blau, 2019) has investigated forming added education value scales. This suggests a third research question to be investigated:

Samples and Procedure
Spring of 2018 graduating undergraduate business students constituted the sample. The Association to Advance Collegiate Schools of Business (AACSB) accredited business school research site is one of 11 schools within a large United States Mid-Atlantic urban state-supported university. Using general email blasts and individual advising sessions, graduating seniors were encouraged to fill out a voluntary online Senior Student Satisfaction Survey (SSSS) prior to graduation. All measures used were contained in the SSSS. A Qualtrics data base was used for data storage. Nine hundred and forty-four graduating seniors filled out at least part of the SSSS. The research was approved by the university institutional review board (IRB). Full-time students, i.e., those taking at least 12 credits/semester, comprised 96% of the sample.

Measures
Student background variables. Four variables were measured as single-item, two-category variables: gender, in-state resident, currently working, and lived at least one semester on or near main campus. Gender was measured as: 1 = male or 2 = female. In-state resident was measured as: 1 = yes, 2 = no. Student records were used for these two items. Currently working was measured as: 1 = no, 2 = yes. Lived at least one semester on or near main campus was measured as: 1 = no, 2 = yes.
Grading assessment learning perception (GALP) items. The referent: "please indicate your level of agreement with the following statement: I find the following grading methods best reflect my course knowledge and skills," was used for 13 items. A 6-point response scale was used, where 1 = strongly disagree to 6 = strongly agree. Prior research (Blau et al., 2017b) found that 12 of the 13-items loaded onto one of four GALP scales. Using the graded components within quantitative and qualitative (non-quantitative) Bachelor of Business Administration (BBA) core (required) course syllabi, 13 GALP items were measured. The four GALP scales (number of items) were: exam-based (3), e.g., multiple choice exams/quizzes, and online exams (exams, quizzes); individual creative (3), e.g., open-ended question exams/quizzes, and individual written assignments (case analyses, essays, etc.); individual engagement (4), e.g., online message boards and class participation; and team-based (2) -team written assignments (case analyses, essays, etc.), and team presentations (oral/visual communication, Power Point, etc.). One-item was deleted due to poor loading. Prior reliabilities for these scales (Blau et al., 2017b) were: exam-based -.76 and .74; individual creative -.72 and .68; individual engagement -.77 and .74; and team-based -.82 and .81.

Added Education Value Required Business Courses as Items.
The business school is large (over 6,800 undergraduate business students) and offers sixteen different majors taught by over 200 full-time faculty and many adjuncts. There are 21 required business core courses that a business student must take as part of their Bachelors of Business Administration (BBA) degree. Students were asked, "please indicate your level of agreement with the follow statements. The following BBA core course added value to my education." A 6-point response scale was used, where 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = slightly agree, 5 = agree, 6 = strongly agree. In addition, a "not applicable" response option was also given, and coded as missing.

Data Analyses
All data were analyzed using SPSS (2015). To test RQ1, exploratory factor analysis (EFA) was done using the 21 required business courses as items. The goal of RQ1 was to create reliable, valid and distinct added education value required business scales to then use for RQ2 and RQ3. Assuming that RQ1 was sufficiently supported, paired-sample t-tests were used to test RQ2. To test RQ3, correlations between the GALP scales and added education value required business scales were tested for significant differences. Two-tailed tests at p < .01 were used for all analyses for determining significance.

Required Business Course Perceived Added Value Descriptive Statistics
Each required business course represented an item, and course item means, standard deviations, and missing cases for each course are reported in Table 1. Items asked "please indicate your level of agreement with the follow statements. The following BBA core course added value to my education." A 6-point response scale was used, where 1 = strongly disagree, 2 = disagree, 3 = slightly disagree, 4 = slightly agree, 5 = agree, 6 = strongly agree. In addition a "not applicable" response option was also given, and coded as Missing.
The missing cases represents a combination of students: waiving out of a course due to transferring in; not answering the item (still finishing up the course/waiting for final grade, no opinion), or perhaps clicking on the wrong response option. Looking at the 21 items, it must be acknowledged that some required courses are unique to the business school compared to a typical required business course curriculum (AACSB, 2013). These include: "Professional Development Strategies" (BA 2101), "Excel for Business Applications" (BA2104), "Business Communications (BA 2196), "Integrative Business Applications" (BA 3103), "Introduction to Risk Management" (RMI 2101), and "Operations Management" (MSOM 3101). BA 2101 and BA 2104 are one-credit courses, while BA 2196, BA 3103, RMI 2101, and MSOM 3101 are each three-credit courses. These more unique courses were developed based on student, faculty, employer or other stakeholder needs. Collectively, these 21 courses address the previously-noted three required skill areas of the AACSB: general skill, technological agility and general business knowledge.

Complete Versus Missing Data
There was a complete data sample of 509 out of 944 (54%) respondents. Given the large amount of missing data (n = 435) it was important to check for missing data bias (Roth, 1994) (Roth, 1994) to prevent further data analyses using the complete data sample.

Added Education Value Required Business Scale Development
The EFA results for the 21 added education value core course items are reported in Table 2. Using a principal components analysis, along with a screen test (Stevens, 1996) three factors was indicated. All three factors had an eigenvalue over one. Using varimax rotation (to maximize factor independence) and the criterion of at least a .60 item loading on a factor, along with no double loading complications, seven items cleanly loaded on the first factor (items #1, 2, 9, 10, 12, 14, 15). The course item loadings, i.e., ACCT 2101, ACCT 2102, ECON 1101, ECON 1102, HRM 1101, LAW 1101, and MKT 2101, suggested naming this factor "Lower-level Foundation" for several reasons. All lower-division foundation required courses are numbered at either the 1000-level (typically taken the freshman year) or 2000-level (typically taken sophomore year). The courses loading on this first factor cut across different disciplines, including accounting, economics, human resource management, legal studies and marketing. This factor accounted for 48% of the total variance.
For the second factor there were also seven items (# 3, 5, 6, 7, 8, 16 and 17) which loaded strongly (above .60) with minimal double loading complications, i.e., BA 2101, BA 2196, BA 3102, BA 3103, BA 4101, MIS 2101, and MSOM 3101. Five of the seven items represented business administration (BA) courses, so this factor was named "Business Administration." This second factor accounted for 9% of the total variance. Finally, for the third factor, there were three items (#11, 20 and 21) that loaded strongly, i.e., FIN 3101, STAT 1001, and STAT 1102. This factor was named "Quantitative." This third factor accounted for 6% of the total variance. Overall 63% of the total variance was accounted for by these three factors. Four items (# 4, 13, 18, and 21) were dropped because of poor loadings (less than .60) as well as double loading complications, i.e., BA 2104, IB 3101, RMI 2101, and STAT 2103. These four items were not included in further analyses. Note. N = 509. Factor loadings above .60 bolded a General referent for all items: "The following BBA core course added value to my education". Responses from 1 = strongly disagree to 6 = strongly agree b Factor 1 = Lower-level Foundation; Factor 2 = Business Administration; Factor 3 = Quantitative Based on these EFA results, further reliability analyses showed the following coefficient alphas: Lower-level foundation -.92; Business Administration -.88; and Quantitative -.84. Coefficient alphas of at least .70 suggest a reliable scale (Nunnally, 1978). Correlations between these three scales were .67 or less. By squaring the correlation, we can determine the amount of overlap, so (.67) 2 = 45%, which indicates the scales are sufficiently distinct from each other (Stevens, 1996). Overall, these results provide support for RQ1, i.e., distinct added education value required business scales were developed.

Testing the Second and Third Research Questions
Means, standard deviations and correlations for all scales are shown in Table 3. The following means (M) were found for each added education value business scale: Lower-level Foundation, M = 4.22; Business Administration, M = 3.97; and Quantitative, M = 4.38. Paired sample t-tests showed that the Quantitative scale mean was significantly higher than the Lower-level Foundation scale mean, t(508) = 3.98, p < .01; and the Lower-level Foundation scale mean was significantly higher than the Business Administration mean, t(508) = 7.25, p < .01. Thus, the Quantitative scale had the highest perceived added education value to the students. These results provide support for RQ2. In order to analyze RQ3, i.e., testing for differences in the relationships between GALP scales to added education value business scales, significant differences between correlations from the same sample, using the r to z transformation formula (Stevens, 1996) was applied. The results are also shown in Table 3. To summarize, as seen from Table 3, the relationships for three of the four GALP scales to the Business Administration added education value scale were significantly higher than for the other two added education value scales. More specifically, the correlation of GALP exam-based to Business Administration, r(507) = .33, was significantly higher than the correlation of GALP exam-based to Lower-level foundation, r(507) = .24, t(505) = 2.41, p < .01, and also higher than the correlation of GALP exam-based to Quantitative, r(507) = .21, t(505) = 2.92, p < .01.

Discussion
This study builds on prior work (Blau, 2019) by working with a full complement of 21 required business course items to create added education value required business course scales. As such the results of this study are best regarded as promising. The three added education value required business course scales were labeled: Lower-level Foundation, Business Administration, and Quantitative. These scales demonstrated sufficient reliabilities and discriminant validity from each other based on correlation analyses. The Quantitative scale had the highest added education value, followed by Lower-level foundation, which was significantly higher than Business Administration.
The general item referent for all required core courses was "added value to my education." This phrase could be differentially interpreted students. Prior research on education value students' perceptions indicate that some type of "price versus quality" or functional value, e.g., getting a good job, assessment is typically made (Alves & Raposo, 2007). Related to this are perceptions of "skills developed" or "preparation for future" (Gibson, 2010). However, other interpretations of perceived value can be made. Dziewanowska (2017) suggested that added value to one's education could also fit into a student-related factor having relational value, or a self-development factor having intrinsic value. More research is necessary assessing student interpretations of "added value to my education." Although lower on perceived added education value than the other two scales, the Business Administration scale had significantly higher positive correlations to three of the four GALP scales, i.e., exam-based, individual engagement, and team-based; versus the Lower-level foundation and Quantitative scales. The correlations of the Lower-level foundation and Business Administration scale to the GALP individual creative scale were each higher versus the Quantitative scale. Another way of summarizing these relationships would be that the average correlation of the added education value Business Administration scale to the four GALP scales was r = .36; the average correlation of the added education value Lower-level Foundation scale to the four GALP scales was r = .28; and the average correlation of the added education value Quantitative scale to the four GALP scales was r = .20.

Study Limitations and Implications for Future Research
There was a significant loss in complete-data responses when aggregating the 21 courses as items for factor and correlational analyses. This was inevitable since approximately 50% of the business students' transfer into the business school research site in a given year, and these transfer students can often waive at least some of the required courses. Despite this loss in data, there was still a large complete-data sample size to work with. As such, this greater size increased the power to find significant differences when statistically testing RQ2 and RQ3 (Stevens, 1996), even when the magnitude of a difference was not big. All data were cross-sectional and retrospective, e.g., students were asked to remember the perceived added education value of required business courses they may have taken several years earlier. Unfortunately, since this was an exit survey for graduating seniors, longitudinal data collection was not possible. To assess common method bias, since only self-report data was collected, a one-factor test (Podsakoff, MacKenzie, Lee & Podsakoff, 2003) was performed. There were nine factors with eigenvalues over one, and 30% of the variance was explained by the first factor. If this first factor represents common method bias, it is not a severe restriction.
Three added education value scales were found with good reliabilities and discriminant validity between each other: Lower-level Foundation, Business Administration, and Quantitative. However, four course items were "dropped" as a result of poor loadings on the factor analyses. Also, two non-business administration courses, MIS 2101 (Management Information Systems in Organizations) and MSOM 3101 (Operations Management) cleanly loaded onto the Business Administration scale. It must be acknowledged when looking at Table 1 that there was a wide range (from 3.60 to 4.70) of perceived added education value across the 21 required courses. Future research on validating these added education value scales is needed.

Practical Implications
These relationships found indicate that students do perceive significant positive relationships between grading assessment learning and added education value. It is concerning that although the quantitative scale had the highest perceived added education value, the relationship of the quantitative added education value scale was lower, compared to the Business Administration and Lower-level Foundation scales, across all four different types of grading assessment learning perception scales. Although stronger, the correlations of the GALP scales to both Lower-level Foundation and Business Administration added education value scales should be higher, e.g., at least .50.
For the closed-response format of a course teaching evaluation (Miles & House, 2015), students are typically asked about the instructor, e.g., providing quick and useful feedback, as well as other course-related items, e.g., meets educational objectives. However, asking if the grading methods used in the course best reflected the student's course knowledge and skills is not measured. Using this type of closed-response item, followed by an open item asking for elaboration, allows the instructor to consider student input suggesting other potential grading assessments. Testing the linkages between how students are taught and what they learn is important (Weisler, 2015). Measuring GALP may help to increase an instructor's teaching evaluation scores (Flores et al., 2015), and also improve the course's perceived added education value. Stronger, i.e., more valid, reliable and distinct, added education value scale formation may result from improving each course's perceived education value.

Conclusion
Although limited to a business school context, the results of this study show promise for developing perceived added education value required course scales. Even if scales cannot be developed, the perceived added education value of each required course, in any school or college, needs to be assessed (Floyd, Harrington & Santiago, 2009). Required courses often comprise a large portion of a student's degree, and represent a significant cost investment. For example, in this study, the 21 required business courses studied here total 61 credits which represents 49% (61/124) of a student's BBA degree. When students are asked to submit their teaching evaluations for a course, the perceived added education value of that course needs to be measured. Such added education value could include measuring: "Did this core course facilitate development in the students' major?"; "Can the students apply/transfer this required core course learning to their professional career?" These closed items should each be followed by an open item asking "How?" Such questions need to be investigated in future research, to improve the quality of a student's educational investment.