Examining Dimensionality and Validity of the Academic Integrity Survey Instrument

Dimensional reduction is one of the methods to ensure the quality of a questionnaire. This study examined two methods to reduce the dimension of the questionnaire: multidimensional scaling (MDS) and exploratory factor analysis (EFA). The questionnaire, Awareness of Academic Dishonesty consists of 30 questions. Participants included 110 college students. Multidimensional scaling analysis reduced the multidimensions to essentially two dimensions. The exploratory factor analysis reduced the multidimensions to three dimensions. MDS allowed the researchers to evaluate the questionnaire items by looking at the similarities of these data points. EFA provided an alternative thought about the construct of the questionnaire


Validity of a Questionnaire
Validity is the extent to which an instrument such as a questionnaire measures what is supposed to measure. There are several different ways to examine the validity (Agarwal, 2012). Face and content validity are subjective opinions of the researchers. Face validity is often seen as the weakest form of validity. Therefore, the researchers usually seek other forms to validate the instruments. The other two concepts of validity are often used in questionnaire designs, criterion validity, and current validity. Criterion validity is the extent to which the measures derived from the survey relate to other external criteria. concurrent validity is measured at the same time as the survey, either with questions embedded within the survey, or measures obtained from other sources (Deniz & Alsaffar, 2013).
Another validity is construct validity, which is considered a higher-level concept because the "constructs are higher level concepts which are not directly observable or measurable (nature) while variables (sometimes used interchangeably with indicators or measures) seek to measure the underlying construct (nature exposed to our method of reasoning)" (Agarwal, 2012, p.2). Thus, the construct is latent because it is not directly observed. The hypothesis is that an individual's responses to each of the survey questions are influenced by the underlying latent construct, which can be measured through responses to questions related to the construct (Heisenberg, 1958;Morrison, n.d.).
Another concept associated with construct validity is dimension, which is related to the construct. In a survey study, the questionnaire scales consist of a series of questionnaire items, which are organized in different groups, which may be the external representation of the construct. Therefore, assessing questionnaire dimensionality is one aspect of validating the internal structure of the questionnaire scale (Rios, & Wells, 2014).

Dimensionality of Questionnaire Scales
When they initiate engaging in a survey research project, the researchers may not consider how to organize these questionnaire items in different groups and the meaning of grouping these items. Thus, examining the dimensions ensures the survey quality and the construct validity (ER Services (n.d.).

Modes of Inquiry
The issues of dimensionality are related to several aspects of the questionnaire quality such as reliability and validity. The dimensionality is also related to convergence. When working with multiple constructs in a survey study, it is important to satisfy convergent and discriminant validities in order to satisfy construct validity. If researchers can demonstrate that they have evidence for both convergent and discriminant validity, then the researchers demonstrate that they have evidence for construct validity (Trochim, Donnelly, & Arora, 2006).
One method to strengthen the convergent and discriminant validity is the reduction of the dimensions of the multidimensional scales (Sarveniazi, 2014;Weng, & Young, 2017;Zhang & Takane, 2010).
This study is to examine how to strengthen the convergent validity by reducing the dimensions of the multivariate constructs. The authors compared two reduction methods, multidimensional scaling and factor analysis, to evaluate different findings and results.

Data Resources and Evidence
In this study, there were 110 participants participated in the survey questionnaire. The focus of the survey had questions about the frequency of academic dishonesty.
There are 23 items in the questionnaire scales. These items can be classified into 4 dimensions: Cheating, Plagiarism, Obtaining Unfair Advantage and Falsification of records and Official Documents. These items were associated with 9-point Likert scales where 1 stood for "strongly disagree" and 9 for "strongly agree". One hundred and 8 college students responded to the questionnaire.

Multidimensional Scaling for Dimensionality Reduction
Since real-world data is usually highly unstructured, the extraction of features of the data for analysis challenged researchers. Therefore, dimension reduction is the one of effective strategies to better represent the structure and construct of a set of data. Wijaya (2020) summarized several reasons why we have to reduce the dimensions of a survey instrument: A higher number of features increase variance in data, which could cause overfitting -Especially where the number of observations is less than the amount of the features present. The density and distance between data become less meaningful, which means the distance between data is equidistant or equally similar/different. This affects clustering and outlier detection as critical information from the data is undervalued. Combinatorial Explosion or a large number of values would lead to a computationally intractable problem where the process just takes too long to finish. (p. 2) There are 30 questions in the questionnaire of Awareness of Academic Dishonesty. As shown in Figure 1, the questionnaire items fall into two groups roughly Group One consists of A24, A1, A18, A8, A6, A29, A15, A2, and A25. Group Two consists of A7, A10, A20, A30, A5, A9, A14, A19, A4, A17, A11, A13, A16, A12, A3, A22, A23, A26, and A28. "The data for Multidimensional Scaling (MDS) analyses is usually coined as proximities, which indicate the overall similarity of the elements in the data" (Imperial, 2019). MDS look for a special configuration of the elements so that the distance between the elements matches their proximities as closely as possible. The outputs showed that there are two "piles" of data points. Each pile of data points indicates there is a similarity among these data points.

Dimensionality Reduction With Factor Analysis
Exploratory factor analysis (FA) is a dimensionality reduction technique that attempts to group intercorrelated variables together and to produce interpretable outputs (Henrique, 2021). Exploratory factor analysis assumes that there are several latent variables in a model. These variables are unobserved but they consist of a construct and also these variables explain a significant proportion of the variation common among the manifest variables. This study is to examine the latent variables of the questionnaire. This dimensional reduction is a different practice from a latent construct perspective. Again, there are 30 questions in the questionnaire on Awareness of Academic Dishonesty.
As shown in Figure 2, there are three dimensions in this study. Items A7, A16, and A22 consist of dimension 1. Items A1, A13, and A24 consist of dimension 2, and the other 30 items are in dimension 3. Exploratory factor analysis reported three dimensions. The result is slightly different from the one of MDS.

Conclusions
Two dimensional reduction methods were introduced for the analysis of the questionnaire on Awareness of Academic Dishonesty. Multidimensional Scaling reduced the dimensions of the questionnaire to two dimensions. Exploratory factor analysis reduced the dimensions of the questionnaire to three dimensions.
The assumptions and philosophies are different between these two-dimensional reduction methods. MDS is a non-linear dimensionality reduction technique that tries to preserve the distances between instances while reducing the dimensionality of non-linear data. Thus, the new researchers examined the questionnaire dimensions and visualized the distributions of all of these data points. They know the questionnaire and the data better via evaluating the data point pattern in the reduced data space.
This analysis aids beginning researchers to guarantee the quality of the survey. This analysis provides a good foundation for the next step of analyzing latent variables and constructing them with reduced dimensions. Factor analysis is used to develop a new set of uncorrelated variables, called latent variables, with the hope that these new variables better represent the construct of the data being analyzed. The factor analysis model assumes there is a smaller set of uncorrelated latent variables driving the value of the variables that are actually being measured. This is an advanced step to know the construct of a set of measuring structures.

Scholarly Significance of the Study
This study examined the dimensionality of the questionnaire on Awareness of Academic Dishonesty. Multidimensional Scaling (MDS) and Exploratory Factor Analysis (EFA) were applied to the data. MDS recognized two dimensions of the 30 questionnaire items. The factor analysis reported a 3-dimension construct. This study recommends an effective strategy. When beginning researchers develop their questionnaire as an instrument of the survey study, they can use MDS to examine the patterns and distribution of the data points and variables. Typically, the factor analysis provides an advanced tool for inspecting the construct, so as to achieve the purpose of ensuring the quality of the questionnaire.

Limitations
Multidimensional scaling and factor analysis are two different methods to reduce the dimensionality of the scale such as questionnaires. However, the theories and assumptions are different. Beginning researchers should carefully think about the dimensions and constructs and then make a choice between them.