Student Success: Redefining Early Alert and Batch-Based Interventions
Originally published on www.sightlinedata.com
Why do college students drop out and how can we use data to proactively prevent poor long-term outcomes?
The truth is that students drop out for a wide variety of reasons, some of which can be preemptively addressed, and others cannot. At Sightline, we believe that by looking at student success with a holistic lens, from enrollment through graduation, we can achieve the best outcomes. This means we must redefine the timeframe for “Early Alert” as identifying students who are at-risk, out one to two semesters before they drop out. As a result, the data-informed strategies we use to provide support to students must evolve as well. The following whitepaper provides a framework for timing and methods for early and impactful at-risk student interventions through predictive modeling, data segmentation, and batch-based student intervention strategies.
Current Retention Platforms
Most current analytics-based retention solutions are learning management systems (LMS) or university-wide student success software. Both solutions typically identify at-risk students on a day-to-day or weekly basis. These at-risk student predictions are dependent on data points such as missing classes, not logging into the LMS, or performing poorly on homework and midterms. Though these are important data points, we view this information as retroactive, rather than proactive. Additionally, professors and academic advisors should already be aware of poor academic performance or habitual absenteeism. An analytics platform most likely is not needed for these kinds of metrics.
Some student retention platforms even collect data that is granular down to the level of internet usage and geo-tracking on campus. We view these data points as intrusive and unnecessary to meet the primary goals of student retention.
Ineffective Intervention Strategies
Popular early intervention strategies have recently been centered around “nudges”. A nudge is generally defined as an intervention directed toward an individual to take a particular action. Many higher-ed retention platforms deliver these nudges as automated notifications if students don’t meet particular criteria, for example, if they miss class too many times or have not logged into their student LMS recently. These intervention systems can work if done properly but, can have a strong negative impact if not thoughtfully deployed. Many stumbling blocks can occur while using automated nudges that can cause students to ignore them- sending too many, not using student-centric language, unclear actions, etc.
Other intervention strategies that fail to yield the intended results are siloed and do not involve the faculty and staff who are most familiar with the student. There must be a collaborative effort between the student success teams who are using predictive analytics software to identify at-risk students and professors and advisors who may have an existing relationship with the students. This is another reason getting email or text nudges for missing class is not an effective universal approach for all types of students.
How We Are Different — The Earliest At-Risk Student Identification
At SightLine, we proactively identify students who are at-risk of dropping out with a one to two semester lead time. On average, 86% of the continuing undergraduate students that we identify as being at-risk, drop out within one to two semesters if no intervention is applied. This means that if our predictive algorithms identify 300 students who are most likely to drop out, at an institution, on average 258 of them will drop out if they do not receive additional strategic support from the institution. Our goal is to provide our clients a precise list of the students who are at-risk and provide data-backed long term intervention strategies that leverage existing resources on campus.
We prepare students for the year ahead rather than reacting to lagging indicators such as missing classes and loss of academic engagement.
The following figure is an example timeline for identifying at-risk students and applying outreach strategies. Once the spring semester is complete, we apply our predictive models that are tailored to each individual university client. We predict which students are most likely to drop out before the following fall semester, and again which students are likely to drop out by the end of the fall or beginning of the following spring semester. All annual forecasts, intervention strategies and recommendations are provided during early summer. We are the only student retention consultants facilitating such long-term intervention planning and interactions with students.
We have eliminated the need for student retention data dashboards. When we work with a university to set strategic goals and student intervention plans, we are developing the strategy for the entire year. This reduces the time and complexity of working with the data, so that the institution can focus on supporting the right students.
Our predictive algorithms assess multi-faceted retention factors, some of which are listed below.
Optimizing Resources — Batch-Based Interventions
We recognize that your time and resources for student interventions are limited. By using predictive analytics, we can guide your institution on which students to focus your resources on. But the question remains, what are the best resources to make available to which student?
At SightLine, our team of data scientists and student retention specialists create at-risk student segmentation and intervention matching algorithms. This process starts with student segmentation algorithms where we identify segments, or clusters, of at-risk students who are most similar to each other.
The following figure is called a dendrogram, it mathematically determines how close students are to each other. Each vertical line on the horizontal axis represents a single student so you can see how some students are quite close, or similar in certain qualities. Our customers liken this to a visual of a city, where students who are similar live close to each other, and students who are dissimilar live far apart.
We use this dendrogram to determine how many segments or clusters of at-risk students the university should be working with, and which students belong to those segments. The number of segments we select may depend on intervention resource availability and the makeup of each segment. In this example, we identified 12 unique student segments.
After identifying these unique student segments, we create various statistical summaries and visualizations to understand the makeup of each at-risk student segment. For example, in the scatterplot below there are two different at-risk student segments. The green circles represent cluster 1 which consists of students primarily at the sophomore level, indicated with below average academic progress, average amounts of need base aid awarded, and below average academic performance. Cluster 4 consists of students with high academic progress, so primarily seniors, with above average academic performance. With further analysis of these clusters, it was clear that neither are engaged in on-campus activities such as intramurals or clubs and they are not employed on-campus.
We go through the process of describing each student segment to begin to understand the different groups of at-risk students at the university. Our student retention and behavioral intelligence specialists then work with the university to create an intervention plan for each student segment, rather than an individual intervention for every individual at-risk student.
Batch-based student interventions make the best use of limited university resources and time. This method is used to target resources where they will have the greatest impact.
Some at-risk student segments may still need individualized attention. An example of this is shown in cluster 3 of the above proportion stacked bar plot, where there was truly a mixture of student types from all grade levels, and all had poor academic performance. This unhomogenized at-risk student cluster indicates the need for individualized outreach, which the university now has the resources for because all other students are already assigned to a specific intervention plan.
Leveraging long-term student retention forecasts in combination with quality batch-based intervention strategies is the best way to retain students long-term without the use of complex student tracking software. This solution was designed to be simple for the university leaders to apply, because SightLine’s data analytics process is doing the heavy lifting.
To learn more about SightLine’s proactive student success solutions, contact the SightLine team or visit our solutions page.
About SightLine: SightLine is a boutique analytics firm, providing easy to interpret, student-centric predictive analytics solutions without the need for complex software. We support colleges and universities throughout the entire student lifecycle, using data to answer the complex questions that institutions have about how to support students.
