The Influence of Affect in Help-Seeking Behaviors and Performance in a Math Intelligent Tutoring System

Abstract

This study investigates the association between help-seeking behaviors (hints, hints per step, hints with steps requests, and hint to error), affect (boredom, confusion, frustration, happiness, and engagement), and performance in seventh and eighth-grade students using the Cognitive Tutor Bridge to Algebra as a self-regulated learning environment. Analyses focused on correlations between students’ help-seeking behaviors and their affect in units 4 and 14. Affect was also used to predict help-seeking behaviors in the next units (e.g., 5 and 15). Moreover, we examined how associations between help-seeking behaviors and performance differed as a function of affect. The results showed that a pattern emerged in which students showed more executive help-seeking behaviors rather than instrumental ones. Students feeling bored, confused, and frustrated tended to use more hints, and they were less likely to switch to external help-seeking sources. Also, those feeling happy or engaged were less likely to use hints.

1. Introduction

Help-seeking is a self-regulatory strategy in self-regulated learning (SRL) in which learners use help-seeking to sustain cognition, behavior, affect, and motivational factors in order to achieve goals in a domain-specific activity [1, 2]. According to Aleven [1], help-seeking is defined “[…] as episodes in which a learner, in the context of a specific learning activity […] takes the initiative to seek assistance from a source within or outside of the learning environment, as opposed to persisting at trying to make progress independently” (p. 312). Help-seeking can be an instrumental (adaptive) strategy, in which learners seek to learn with understanding, or executive (maladaptive), when learners seek prompted answers without developing understanding [1, 3].

Previous research shows that help-seeking has been studied within the math domain [4, 5, 6, 7, 8, 9, 10, 11]. Du and colleagues [12] showed that help-seeking was positively associated with math homework interest at the individual and class levels in eighth graders. The role of peers is a significant predictor of help-seeking in middle and high school students [11, 13, 14, 15, 16]. Perfectionism and feedback were also positively related to help-seeking behaviors [4, 10, 17, 18]. Adaptative help-seeking behaviors significantly predicted mastery-approach goals in middle and high school students [13, 19]. In addition, perceived parent achievement goals predicted students’ help-seeking and help-avoidance behaviors in middle school students’ achievement goals [6]. Regarding gender, female students are more likely to demonstrate adaptative help-seeking behaviors, whereas boys are more likely to exhibit executive help-seeking behaviors [8, 15, 18, 20].

Help is a typical feature in intelligent tutoring systems (ITSs), and it is usually provided at students’ request through the form of hints, internal help-seeking sources, or switches, and external help-seeking sources such as glossaries or hyperlinks. Previous research showed that high school students in a math help forum adopted executive help-seeking behaviors [9]. Best and colleagues [21] demonstrated that social support was increased and stigma was reduced by adolescent males seeking help online. Roll and colleagues [7] showed that high school students asking for help in online Geometry Cognitive Tutor had more productive learning, whereas those who overused it had poor learning. Contrary to previous research, avoiding help was associated with better learning. Otieno and colleagues [5] showed that hints and glossaries in the Cognitive Tutor Geometry differed from the answers in the questionnaire they gave to high school students, yet with a high prediction of learning. Roll and colleagues [4] showed that high school students were able to transfer their learning from a Geometry Cognitive Tutor to a new domain-level content.

The decision to seek help is influenced by metacognitive, affective, and social competencies. Affect can be understood “[…] as a generic term that includes emotions, mood, feelings, attitudes, etc.” (p. 65) [22]. Students are likely to experience boredom, confusion, frustration, happiness, and engagement in intelligent learning environments (ILE) because they are affectively charged experiences [23]. Previous research shows that affect has been studied within the math domain. Positive academic emotions such as engagement, enjoyment, joy, contentment, and hope have been associated with adaptative behavior and lower math difficulties. However, negative academic emotions such as anxiety, confusion, frustration, shame, and hopelessness have been associated with executive behaviors and higher math difficulties [24, 25, 26, 27, 28, 29, 30]. Academic emotions and learning motivation were also studied in which students were more motivated when they exhibited positive academic emotions such as hope, pride, and enjoyment [31, 32, 33]. In addition, students exhibiting higher enjoyment and lower boredom had higher subsequent achievement, but pride was negatively correlated with achievement [29, 34].

Affect influences learning outcomes in face-to-face and mediated learning environments. Previous research has studied the effect of ITS on affect [33, 35, 36, 37, 38, 39, 40, 41, 42, 43]. In addition, an ITS provided transition from negative to positive affective states in high school students, and it improved their frustration and confidence by using online learning companions [24]. San Pedro and colleagues [25] showed that high school students who felt more engaged made more careless mistakes and those who felt confused or bored made fewer careless mistakes. Pre-school children demonstrated engagement and valence behaviors to nonverbal behaviors from the social robot tutor [44]. Padrón-Rivera and colleagues [28] showed in a math action units study that high school students experiencing confusion had better learning than those experiencing frustration.

However, to the best of our knowledge, no research was found about the influence of affect in students’ help-seeking behaviors in math ITSs until the present date. Thus, more research is needed on how students’ affect and motivation may influence learners’ decisions to seek or not to seek help in ITSs [45]. According to D’Mello [23], there is still the need to understand “[…] how affective states arise, morph, decay, and impact learning outcomes […]” (p. 2).

The purpose of this study was to examine affect and its implications for help-seeking as an SRL process. According to D’Mello [23], “Affective impacts of technologically infused learning environments are not very well understood” (p. 2). In addition, Karabenick and Gonida [3] acknowledged the need for additional research to better understand help-seeking as an SRL process. We aimed to investigate how students’ academic emotions correlated with help-seeking within one task and if affect was a significant predictor of help-seeking on a subsequent task. Thereafter, we examined the effects of help-seeking on performance and the ways affect moderates this relationship. We attempted to answer the following research questions: (1) does students’ affect after learning correlate with their help-seeking behavior within a unit of math problem-solving? (2) Is students’ affect reported in one unit significant predictor of help-seeking in the next? (3) How do students’ help-seeking behaviors and affect relate to their performance in a consecutive unit of math problem-solving tasks? We also hypothesized that, first, students presenting academic emotions such as happiness and engagement are less likely to switch to external help-seeking sources and make errors, and they are more likely to use hints as an internal help-seeking source and respond correctly to the questions. Secondly, students presenting academic emotions such as boredom, confusion, and frustration are less likely to use hints and respond correctly to the questions, and they are more likely to switch to external help-seeking sources and make errors.

2. Method

2.1 Participants

One hundred and ten students from math courses at a suburban, public middle school participated in the study. Students were 50% male and 50% female, and 96% were Caucasian. 71.8% reported low socioeconomic status (SES), and they were eligible for a free or reduced lunch from the school. Students were primarily enrolled in the seventh grade, but there were students from the eighth grade. Participation occurred in two math periods per week (45–50 minutes each) across the whole academic year as part of regular classroom activities through online questionnaires about math exercises, their academic emotions, and motivation using the Cognitive Tutor Bridge to Algebra [46].

2.2 Materials

The Cognitive Tutor Bridge to Algebra [47] collected fine-grained traced data about students’ help-seeking behaviors in pre-algebra exercises in 54 units being units 4, 5, 14, and 15 used for this study, and it presented online questionnaires with a Likert scale ranging from 1 to 5 for students to report their academic emotions starting with boredom, confusion, frustration, happiness, and engagement, respectively [2, 48]. The unit topics comprised “Lowest Common Multiple” in unit 4, “Greatest Common Factor” in unit 5, “Converting Fractions to Decimals” in unit 14, and “Decimal Operations” in unit 15. Units 4, 5, 14, and 15 were chosen because they represented the most challenging ones for students in which the difficulty was operationalized by time per unit and time per problem.

2.3 Procedures

This study focused on unit and problem level assessments employing a microgenetic and longitudinal approach. In the microgenetic approach, transactions were examined at unit and problem levels to understand the relationship between help-seeking, affect, and performance [2]. Researching at the transaction level allows to correlate help-seeking tools to affect and how the latter may predict the former in consecutive units. With longitudinal and microgenetic data, it is possible to determine if there is a correlation between help-seeking and academic emotions and how this correlation changes when it is investigated at unit and problem levels [2, 48].

In order to investigate the unit and problem level assessments, the Cognitive Tutor Bridge to Algebra uses the model of desired help-seeking behavior developed by Aleven and colleagues [49]. Briefly, students should start by thinking about the problem. If the problem is familiar and they know what to do, students make an attempt. If the problem is familiar but they do not know what to do, students switch to use the glossary to learn more about the problem. If they know what to do after consulting the glossary, students make an attempt. If not, they may either use the glossary again or ask for a hint. If the problem is not familiar, students ask for a hint, spend time reading it, and decide if the hint is helpful. If yes, they make an attempt. If not, students ask for another hint. If students make a correct attempt, the problem is complete. If not, students may require another hint and/or make a new attempt. In this study, hints reflect the total number of hints requested in raw terms. Hints per step normalize these requests to the amount of problem steps students had to complete. Steps with hints normalize the hint requests to account for steps in which students ask for one or more hints. Hint to error ratio reflects the proportion of times students opted to use a hint, rather than to make an attempt at a step that ultimately yielded an incorrect answer. Switch reflects the total number of glossary requests in raw terms [45, 49].

3. Results

Analyses focused on correlations between students’ help-seeking behaviors and their affect in units 4 and 14. Affect was also used to predict help-seeking behaviors in the next units (e.g., 5 and 15). Moreover, we examined how associations between help-seeking behaviors and performance differed as a function of affect. The alpha level was set at .05 for simple correlations and multiple regressions.

Missing data were observed on 28 of the 34 variables examined. In unit 4, 20.9% of the data were missing on the boredom variable, 20% of data were missing on confusion, 19.1% of data were missing on frustration, happiness, engagement, hints, hints per step, steps with hint requests, and hint to error. In unit 5, 19.1% of data were missing on performance, hints, hints per step, steps with hint requests, and hint to error. In unit 14, 18.2% of data were missing on boredom, confusion, and frustration, 20% of data were missing on happiness and engagement, 12.7% of data were missing on hints, hints per step, steps with hint requests, and hint to error. In unit 15, 11.8% of data were missing on performance, hints, hints per step, steps with hint requests, and hint to error.

Rubin [50] proposed a taxonomy of missing data mechanisms to explain why the choice of missing data method impacts model parameters. The missing data were found to be consistent with a missing completely at random (MCAR) mechanism as determined by Little’s [51] omnibus MCAR test, d2(199) = 213.907, p = .223. Therefore, listwise deletion (N = 97) was used to handle missingness for the present research problems because it is in accordance with the MCAR mechanism and it does not bias the parameter estimates generated by the correlations and multiple regression analyses. Table 1 shows the descriptive statistics for affect, help-seeking behaviors, and performance in units 4, 5, 14, and 15. Happiness and engagement had the highest coefficients in units 4 and 14. Hints also had the highest coefficients with the exception of unit 5 in which switch exceeded hints. This may be due to the highest level of difficulty of the unit. Moreover, it demonstrated that students favored the usage of hints as an adaptative help-seeking behavior.

	Unit 4		Unit 5		Unit 14		Unit 15
	M	SD	M	SD	M	SD	M	SD
Affect
Boredom	4.70	1.84	—	—	4.94	1.76	—	—
Confusion	4.08	1.97	—	—	4.80	1.97	—	—
Frustration	3.81	2.19	—	—	4.54	2.01	—	—
Happiness	5.83	1.19	—	—	5.84	1.25	—	—
Engagement	5.43	1.48	—	—	5.45	1.47	—	—
help-seeking behaviors
Switch	0.54	1.94	19.25	14.35	3.72	5.69	0.88	1.96
Hints	6.34	11.45	6.34	11.45	39.12	77.94	11.03	13.02
Hints per step	0.22	0.04	0.22	0.04	0.13	0.14	0.03	0.03
Steps with hints requested	274.82	115.77	274.82	115.77	227.8	110.8	332.62	113.11
Hint to error	0.18	0.24	0.18	0.24	0.77	0.77	0.26	0.23
Performance			0.94	0.03			0.93	0.03

Table 1.

Descriptive statistics for help-seeking behaviors and affect in units.

Note. Listwise N = 110.

Table 2 answers the first research question if students’ affect after learning correlates with their help-seeking behavior within a unit of math problem-solving. It shows correlations between affect components and help-seeking behaviors in units 4 and 14. In contrast to hypotheses one and two and the assumptions by Karabenick [52], a positive moderate correlation of .213 between hints and boredom was found in unit 4. Students feeling boredom were more likely to use hints in unit 4 as an instrumental help-seeking behavior. However, this correlation was nonsignificant in unit 14. Negative moderate correlations of −.218 and − .209 between happiness and hints per step and between happiness and steps with hints requests, respectively, were found in unit 14. They are also in contrast with hypotheses one and two. Students feeling happy were less likely to use hints per step and steps with hints requested in unit 14. It might indicate an executive type of help-seeking behavior. No other correlations were significant in units 4 and 14.

Variable	Switch	Hints	Hints per step	Steps with hints requested	Hint to error
Unit 4
Boredom	0.097	.213*	0.189	0.143	0.094
Confusion	0.034	0.186	0.164	0.102	0.017
Frustration	0.107	0.065	0.109	0.021	−0.051
Happiness	−0.182	−0.064	−0.089	0.012	−0.038
Engagement	−0.110	−0.022	−0.008	−0.039	−0.029
Unit 14
Boredom	−0.062	−0.025	−0.001	−0.031	−0.046
Confusion	−0.044	−0.043	0.000	0.027	−0.042
Frustration	−0.001	0.033	0.072	0.09	0.053
Happiness	−0.146	−0.197	−.218*	−.209*	−0.186
Engagement	−0.039	−0.108	−0.137	−0.09	−0.128

Table 2.

Correlations between affect components and help-seeking behaviors in units.

^*

p = 0.05.

Note. Listwise N = 87.

Multiple linear regression analyses with ordinary least squares estimations were conducted to answer the second research question. It was investigated if students’ affect (boredom, confusion, frustration, happiness, and engagement) reported in units 4 and 14 were significant predictors of help-seeking behaviors (hints, hints per step, steps with hints requested, hint to error ratio, and switch) in the next units 5 and 15, respectively (Table 3).

Variable	F	Df	p	R^2	b	SE	β	p
Unit 5 from unit 4
Switch	1.301	5,80	0.272	0.075
From boredom					1.194	0.952	0.181	0.214
From confusion					1.425	0.970	0.228	0.146
From frustration					−1.462	0.878	−0.260	0.100
From happiness					0.318	1.238	0.031	0.798
From engagement					−1.086	1.012	−0.132	0.287
Hints	2.768	5,80	0.023	0.148
From boredom					3.344	1.139	0.406	0.004*
From confusion					0.598	1.161	0.077	0.608
From frustration					−2.134,	1.050	−0.304	0.045*
From happiness					1.622	1.48	0.127	0.276
From engagement					−2.253	1.211	−0.22	0.067
Hints per step	5.101	1.85	0.026	0.057
From boredom					0.002	0.001	0.238	0.026*
From confusion					−0.001	0.001	−0.077	0.618
From frustration					0.000	0.001	−0.053	0.730
From happiness					0.002	0.002	0.096	0.425
From engagement					−0.002	0.002	−0.174	0.157
Steps with hints requested	0.955	5,80	0.450	0.056
From boredom					11.984	24.336	0.072	0.624
From confusion					36.745	24.803	0.232	0.142
From frustration					−40.521	22.436	−0.285	0.075
From happiness					−11.455	31.631	−0.044	0.718
From engagement					−14.937	25.879	−0.072	0.565
Hints to error	1.274	5,80	0.283	0.074
From boredom					0.039	0.019	0.300	0.041
From confusion					−0.021	0.019	−0.171	0.273
From frustration					0.006	0.017	0.052	0.738
From happiness					0.010	0.024	0.052	0.670
From engagement					−0.024	0.020	−0.150	0.227
Unit 15 from unit 14
Switch	0.087	5,81	0.994	0.005
From boredom					−0.016	0.205	−0.013	0.937
From confusion					−0.125	0.252	−0.116	0.621
From frustration					0.103	0.239	0.095	0.667
From happiness					0.072	0.239	0.042	0.764
From engagement					−0.032	0.206	−0.022	0.877
Hints	2.639	5,81	0.029	0.140
From boredom					−1.735	1.133	−0.239	0.13
From confusion					0.135	1.392	0.021	0.23
From frustration					2.679	1.324	0.414	0.046*
From happiness					−1.326	1.321	−0.13	0.319
From engagement					−0.282	1.139	−0.033	0.805
Hints per step	3.018	5,81	0.015	0.157
From boredom					−0.006	0.003	−0.32	0.041*
From confusion					0.001	0.003	0.078	0.720
From frustration					0.007	0.003	0.430	0.037*
From happiness					−0.002	0.003	−0.098	0.447
From engagement					−0.001	0.003	−0.039	0.764
Steps with hints requested	0.855	5,81	0.515	0.05
From boredom					−5.017	10.788	−0.076	0.643
From confusion					3.996	13.255	0.069	0.764
From frustration					9.503	12.612	0.162	0.453
From happiness					−11.066	12.581	−0.12	0.382
From engagement					1.459	10.844	0.019	0.893
Hints to error	4.483	5,81	0.001	0.217
From boredom					−0.075	0.019	−0.576	<.001*
From confusion					0.013	0.024	0.11	0.598
From frustration					0.059	0.023	0.506	0.011*
From happiness					−0.001	0.023	−0.005	0.965
From engagement					−0.004	0.02	−0.026	0.836

Table 3.

Multiple linear regression predicting help-seeking behaviors from affect.

*

p < or = .05.

The first significant regression equation was found when predicting hints in unit 5 from affect components in unit 4 (F(5, 80) = 2.768, p = .023, r2 = .148). Boredom reported in unit 4 was a statistically significant predictor of the total number of hints requested in unit 5 (b = 3.344, SE = 1.139, p = .004, 95% CI = 1.077, 5.610; β = .406), such that, for each unit increase in boredom, total hint requests were predicted to increase by 3.344. Frustration reported in unit 4 was also found to be a statistically significant predictor of hints in unit 5 (b = −2.134, SE = 1.050, p = .045, 95% CI = −4.224, −.044; β = −.304). For each unit increase in frustration, students were predicted to request 2.134 fewer hints. Predicting hints per step in unit 5 from boredom in unit 4 yielded another significant regression equation (F(1,85) = 5.101, p = .026, r2 = .057). Boredom reported in unit 4 was a statistically significant predictor of hints per step in unit 5 (b = .002, SE = .001, p = .026, 95% CI = .000, .005; β = .238), such that, for each unit increase in boredom, hints per step were predicted to increase by .002 points. Models regressing steps with hints, hint to error ratio, and switch on reported affect components were not statistically significant, F(5,80) = 1.301, p = .272, r2 = .075. Collectively, a pattern emerged where boredom increased tendency toward help-seeking, while frustration led to help avoidance. It demonstrates that frustration was the only significant predictor that corroborates with hypothesis two but not with hypothesis one, in which students feeling frustrated predicted to be less likely to use hints and respond correctly to questions in the subsequent unit. We next examined the stability of relations in units 14 and 15.

When predicting hints in unit 15 from frustration in unit 14, a significant regression equation was found (F(5,81) = 2.639, p = .029, r2 = .140). Frustration reported in unit 14 was a statistically significant predictor of hints in unit 15 (b = 2.679, SE = 1.324, p = .046, 95% CI = .044, 5.314; β = .414), such that, for each unit increase in frustration, hints were predicted to increase by 2.679 points. Another significant regression equation was found (F(5,81) = 3.018, p = .015, r2 = .157) when predicting hints per step in unit 15 from affect components in unit 14. Boredom reported in unit 14 was a statistically significant predictor of hints per step in unit 15 (b = −.006, SE = .003, p = .041, 95% CI = −.011, .000; β = −.320). For each unit increase in boredom, hints per step were predicted to decrease by .006 points. Frustration reported in unit 14 was also found to be a statistically significant predictor of hints per step in unit 15 (b = .007, SE = .003, p = .037, 95% CI = .000, .013; β = .430), such that, for each unit increase in frustration, hints per step were predicted to increase by .007 points. The final significant regression equation was found (F(5,81) = 4.483, p = .001, r2 = .217) when predicting hint to error in unit 15 from affect components in unit 14. Boredom reported in unit 14 was a statistically significant predictor of hint to error in unit 15 (b = −.075, SE = .019, p < .001, 95% CI = −.114, −.037; β = −.576), such that, for each unit increase in boredom, hint to error were predicted to decrease by .075 points. Frustration reported in unit 14 was also found to be a statistically significant predictor of hint to error in unit 15 (b = .059, SE = .023, p = .011, 95% CI = .014, .104; β = .506), such that, for each unit increase in frustration, students’ use of hints rather than to make an attempt at a step that resulted in an error was predicted to increase by .059 points. The pattern that emerged was flipped in units 14 and 15 in which frustration increased the tendency toward help-seeking, while boredom led to help avoidance. Contradicting hypotheses one and two, students feeling frustrated were predicted to be more likely to use hints and respond correctly to questions in the subsequent unit. On the contrary and in accordance with hypothesis two, students feeling bored were slightly less likely to use hints and respond correctly to questions in the subsequent unit. Models regressing steps with hints and switch on reported affect components were not statistically significant (F(5,81) = .855, p = .515, r2 = .05). All models appear in Table 3.

We next examined the third research question on whether the association between help-seeking behaviors and academic performance differs as a function of the affect. Prior to conducting substantive analyses, help-seeking behaviors were centered, and the help-seeking behaviors-by-affect components product term was generated via multiplication of centered help-seeking behaviors with the affect components. The continuous predictors, as well as their product terms, were entered into simultaneous regression models. Significant and nonsignificant regression equations were found, but only significant ones are presented in Table 4. There were no significant associations between switch as a help-seeking behavior and academic performance differing as a function of affect. It suggests that the interaction between the switch and affect was not a significant predictor of performance in subsequent units.

Variable	b	SE	β	t	p	F	df	p	R^2
Unit 4 predicting performance in unit 5
Constant	.946	.006	−	170.610	<.001	3.862	(3,84)	.012	121
Hints	−.001	.000	−.560	−2.783	.007
Frustration	−.002	.001	−.181	−1.766	.081
Hints × frustration	.000	.000	.420	2.086	.040
Constant	.945	.006	−	167.640	<.001	2.969	(3,84)	.036	096
Hints per step	−.403	.172	−.553	−2.349	.021
Frustration	−.002	.001	−.178	−1.709	.091
Hints per step; X frustration	.064	.031	.484	2.056	.043
Unit 14 predicting performance in unit 15
Constant	.947	.009	−	109.715	<.001	5.652	(3,86)	.001	165
Hints per step	−.225	.073	−1.076	−3.080	.003
Confusion	−.003	.002	−.156	−1.549	.125
Hints per step; X boredom	.036	.014	.869	2.485	.015
Constant	.945	.008	−	124.354	<.001	5.961	(3,86)	.001	172
Hints per step	−.186	.055	−.887	−3.380	.001
Confusion	−.002	.001	−.150	−1.489	.140
Hints per step; X confusion	.030	.011	.693	2.633	.010
Constant	.946	.007	−	136.648	<.001	7.732	(3,86)	<.001	212
Hints per step	−.223	.059	−1.073	−3.772	<.001
Frustration	−.003	.001	−.181	−1.838	.070
Hints per step; X frustration	.039	.012	.906	3.189	.002
Constant	.950	.009	−	110.651	<.001	5.242	(3,86)	.002	155
Hints to error	−.041	.016	−1.113	−2.569	.012
Boredom	−.003	.002	−.189	−1.882	.030
Hints to error; X boredom	.007	.004	.875	2.016	.047
Constant	.946	.007	−	135.838	<.001	7.213	(3,86)	<.001	201
Hints to error	−.046	.014	−1.270	−3.385	.001
Frustration	−.003	.001	−.192	−1.933	.056
Hints to error; X frustration	.009	.003	1.074	2.866	.005

Table 4.

Interaction effect between help-seeking behaviors and affect in subsequent performance unit.

Significant interaction effects were found in multiple units. In unit 4, a hint × frustration interaction effect indicates that every unit increase in frustration led to a slight but statistically significant additional effect where hint use conducted while frustrated predicted an even poorer subsequent performance (e.g., as measured by the percent of first attempts at problem steps) over and above the negative effect of hint use as a help-seeking method. A hints per step × frustration interaction effect indicates that every unit increase in frustration led to a slight but statistically significant additional effect, where hints per step predicted an increase of .064 units on subsequent performance after help-seeking.

In unit 14, a hints per step × boredom interaction effect indicates that every unit increase in boredom led to a slight but statistically significant additional effect, where hints per step predicted an increase of .036 units on subsequent performance. A hints per step × confusion interaction effect indicates that every unit increase in confusion led to a slight but statistically significant additional effect, where hints per step predicted an increase of .030 units on subsequent performance. A hints per step × frustration interaction effect indicates that every unit increase in frustration led to a slight but statistically significant additional effect, where hints per step predicted an increase of .039 units on subsequent performance. A hint to error × boredom interaction effect indicates that every unit increase in boredom led to a slight but statistically significant additional effect, where greater use of hints compared to making errors predicted an increase of .007 units on subsequent performance. A hint to error × frustration interaction effect indicates that every unit increase in frustration led to a slight but statistically significant additional effect, where hint to error predicted an increase of .009 units on subsequent performance. These patterns contradicted hypotheses one and two in which students feeling bored, confused, and frustrated combined with the usage of hints predicted additional performance in subsequent units.

Subsequently, we examined how many students used switch as a help-seeking behavior in units 4, 5, 14, and 15. Table 5 shows that students used more switch in unit 4. However, there were more students who overall preferred not to use switch across the four units analyzed in this study. It corroborates with descriptive findings from Table 1 in which hints had the highest coefficients yielding students’ preference.

Variable	Unit 4	Unit 5	Unit 14	Unit 15	Total
Switch users	17	88	52	28	185
Non-switch users	93	22	58	82	255

Table 5.

Use of switch as a help-seeking behavior in units 4, 5, 14, and 15.

Eventually, a series of independent t-tests were conducted in order to compare affect components including boredom, confusion, frustration, happiness, and engagement between students who used switch and those who did not use it as a help-seeking behavior. There were no systematic differences between cases with switch users and nonswitch users, according to Table 6. The group of students using switch did not have a significantly higher number of affect scores than the group of nonswitch users. These results suggest that switch does not have an effect on affect components such as boredom, confusion, frustration, happiness, and engagement. It contradicts both hypotheses stated in this article. Students presenting academic emotions such as happiness and engagement are less likely to switch to external help-seeking sources, whereas students presenting academic emotions such as boredom, confusion, and frustration are more likely to switch to external help-seeking sources.

Variables	N	M	SD	t	p
Unit 4
Boredom				t(85)= −1.192	.236
Switch users	17	5.18	2.128
Non-switch users	70	4.59	1.757
Confusion				t(87) = −.637	.526
Switch users	17	4.35	2.290
Non-switch users	72	4.01	1.895
Frustration				t(86) = −.773	.442
Switch users	17	4.18	2.128
Non-switch users	71	3.72	2.212
Happiness				t(87) = .256	.799
Switch users	17	5.76	1.147
Non-switch users	72	5.85	1.206
Engagement				t(87) = −.134	.894
Switch users	17	5.47	1.663
Non-switch users	72	5.42	1.451
Unit 14
Boredom				t(88) = .046	.963
Switch users	47	4.94	1.686
Non-switch users	43	4.95	1.851
Confusion				t(88) = .384	.702
Switch users	47	4.72	1.942
Non-switch users	43	4.88	2.014
Frustration				t(88) = .537	.593
Switch users	48	4.44	1.945
Non-switch users	42	4.67	2.103
Happiness				t(86) = .600	.550
Switch users	47	5.77	1.255
Non-switch users	41	5.93	1.253
Engagement				t(86) = .131	.896
Switch users	46	5.43	1.241
Non-switch users	42	5.48	1.700

Table 6.

T-tests comparing switch as a help-seeking behavior with affect in units 4 and 14.

*

p < or = .05.

This statistically significant result means that no effect was observed.

4. Discussion

This study was a response to Karabenick and Gonida’s [3] call for additional research on help-seeking as a self-regulatory process, and it also examined the ways affect influenced their help-seeking intentions, behaviors, and performances within an ITS context. Students’ affect—boredom, confusion, frustration, happiness, and engagement—related to students’ concurrent problem-solving and help-seeking behaviors, predicted future help-seeking and moderated performance.

Students were more likely to use hints in unit 4 when they felt bored. This may indicate a trace of disengagement from the task and executive help-seeking behavior [52]. It contradicted hypothesis two that students feeling bored are less likely to use hints. On the other hand, students who reported they felt more positive were less likely to select multiple hints per step and rely on hints across more steps in unit 14. It does not corroborate with hypothesis one in which students feeling happy were thought to be more likely to use hints and respond correctly to the questions. According to Karabenick’s [52] description, this may instead be construed to reflect engagement in the task and trace executive help-seeking behaviors.

When predicting future help-seeking behaviors from students’ affect reported in previous units, results showed that students feeling bored in unit 4 were more likely to use hints and hints per step in unit 5 contradicting hypothesis two. On the other hand, students feeling frustrated were less likely to use hints in the subsequent unit which corroborates with hypothesis two. In unit 14, this relationship flipped and those who felt frustrated were more likely to use hints (overall, per step, and compared to making errors) in unit 15. Those who felt bored were less likely to use hints in the subsequent unit also corroborating with hypothesis two. This may indicate executive help-seeking behaviors because students were trying to keep engaged in the task by seeking help through hints while feeling bored or frustrated which contradicted the prior hypothesis. There is a need for future research to investigate the reasons why this relationship flipped from one unit to the other. Researchers should investigate more units and more students across time which was one of the limitations of this study. We hypothesized that it may be due to the level of difficulty of the unit in which Decimal Operations in unit 14 may have led to a greater frustration than Greatest Common Factor in unit 4.

Aleven and colleagues [45] acknowledged the need to investigate whether help-seeking as an SRL component could lead to improved students’ performance in later units on ITSs. Thus, we examined whether the association between help-seeking behaviors and academic performance differs as a function of affect. In unit 4, the usage of hints when students felt greater frustration predicted an even poorer subsequent performance in unit 5 than hint usage alone. This may indicate an instrumental help-seeking behavior in unit 4 which slightly undermined students’ performance in unit 5 due to the effect of frustration. On the contrary, the interaction between hints per step and frustration led to a slight increase in unit 5 performance. This may indicate an executive help-seeking behavior in unit 4 which slightly increased students’ performance in unit 5 due to the effect of frustration. In unit 14, the association between hints per step and affect (boredom, confusion, and frustration) and between hint to error and affect (boredom and confusion) led to a slight increase in students’ performance in unit 15. These results contradicted hypotheses one and two in which students feeling happy or engaged should be more likely to use hints, but no significant interactions were found. Additionally, students feeling bored, confused, and frustrated were more likely to use hints, and it led to subsequent performance. This may also indicate an executive help-seeking behavior in which students are trying to keep engaged in the task perhaps without developing much understanding in the present unit so they can progress to the next one.

We also examined how many students used switch as a help-seeking behavior in units 4, 5, 14, and 15, and we investigated if switch had an effect on affect components such as boredom, confusion, frustration, happiness, and engagement. The results showed that students used more switch only in unit 4, Greatest Common Factor. It also demonstrated that switch did not yield a significant effect on affect components. We hypothesized that it may be because students may consult other sources even if they know the answer to reach full certainty. It could also be because they were not aware of how helpful this resource could be in an ITS. There is also the need to investigate switches, as an external help-seeking source, in relation to other internal help-seeking sources and affect components.

5. Conclusion

These findings contribute to the growing body of SRL literature because it demonstrates new ways students are regulating their actions and seeking for help in ITSs in relation to affect. The results from the present article showed that a pattern emerged in which students showed more executive help-seeking behaviors rather than instrumental ones. Students feeling bored, confused, and frustrated tended to use more hints, and they were less likely to switch to external help-seeking sources. Also, those feeling happy or engaged were less likely to use hints. It may be because students were feeling disengaged from the task or because of the level of difficulty of the task in the ITS. There is a need to research more units across time and investigate if these patterns hold.

Future research may assess students’ affective states which could shed light on the affective conditions that lead students to use hints and to profit from hint use. These results suggest a complex relationship between affect and hint use where affect explains variance in the tendency to use hints, but the directions of affect sometimes contradict the prior theory [52].

Moreover, future research may also focus on students’ help-seeking behaviors and affect in relation to their self-efficacy beliefs in ITSs. Efficacy beliefs influence peoples’ thinking process, goals, effort, commitment, outcomes, resilience, and how much stress or depression they tend to feel when performing a task [53]. Students’ usage of hints may be related to their sense of efficacy, and it may be a product of hint usage across the span of different units.

Acknowledgments

The authors gratefully acknowledge the support of the National Science Foundation (NSF) that funded this research. The collaboration of Steve Ritter and Tristan Nixon at Carnegie Learning is also especially appreciated regarding their assistance in adapting the Cognitive Tutor to include the instruments used in this study.

Declaration of interest

The authors declare no conflict of interest.