4 ms·

Do they not? "At scale" is somehow often synonymous with "with less effort and care per instance", but that doesn't have to be the case. Things work worse if yo

Do they not? "At scale" is somehow often synonymous with "with less effort and care per instance", but that doesn't have to be the case. Things work worse if you scale only one part of the equation (the "customers"), but do not scale the other side appropriately (the "service provider"). That should be obvious. > Text campaigns are most likely to be effective when they: Are sent by a school and even staff member who students know and trust Add meaningful and novel value above and beyond existing outreach and support efforts like giving students real-time information on the status of their financial aid applications Provide content that is personalized to the student and where they are in the college application and financial aid process Having a new org sending out mass messages to all students is not scaling The described. It is something different. Scaling it would be convincing more school staff and schools to make a similar effort. The positive effect was not caused by "getting a message", it was caused by "getting personal advice from someone you know". They scaled the wrong thing.

Spooky23 5y ago
Some problems can be improved with a simple contact point. I have a colleague who works in this space in a research capacity. They were looking at use of different systems to measure behavior and identify behaviors that are correlated with poor performance. In one scenario, they surveyed a cohort who was leaving and entering the dorm after 1AM, resulting in poor AM class attendance. 40% of the responses were essentially: “I went to meet my [girl/boy]friend and have sex.” Getting a text message to tell that kid to get his beauty rest isn’t going to work. Just treat people like adults, imo.
- ineedasername 5y ago
  Some problems can be improved with a simple contact point Absolutely. Most schools don't employ enough advisors to have sit-down appointments with every student every semester. However, I did see a school implement a policy where 1st semester students-- a higher risk population-- were required to have a sit down like that before they were allowed to register for the next semester. That single effort resulted in a 15% decrease in 1st-year dropouts.
  - Spooky23 5y ago
    Makes sense. I went to a big public university where they had two freshman tracks, a "college within the university" where, if you knew to sign up, you'd be in a cohort of students to meet some general education requirements and would get access to faculty motivated to help you. The problem was that the kids who needed the least help tended to discover this program. The "normal" path was being dumped into the University bureaucracy blind, and you had to learn to sink or swim. It was tough -- the registrar in those days was an experience like DMV. I'm pretty sure the system was devised as a funnel to disqualify people for popular majors like Business, Computer Science, etc.
    - ineedasername 5y ago
      the kids who needed the least help tended to discover this program Heck yeah did you hit the nail on the head. I've analyzed the data over a period of years for which students utilize tutoring services: They very much tend to be students that were already doing alright before they started getting tutoring. Conscientious self-motivated students are precisely the ones most likely to seek out help on their own. As a result, for students that go on academic probation we now require them to attend a certain amount of tutoring hours each semester, and the software we use to manage advising caseloads allows faculty to raise a flag for students performing poorly so that advisors can pro-actively reach out and direct them to tutoring resources. This has led to a huge increase in tutoring resource utilization, but unfortunately they are now at the limits of their capacity. Our next hurdle is to analyze the academic outcomes of students using those resources, which will hopefully show increased performance & allow us to petition for more support services. Fortunately our leadership is not completely short-sighted on these issues, but as obvious as "more one-on-one advising & support" would seem, they (not too unreasonably) still demand very concrete evidence to justify additional expenditures. However it would be nice if they viewed customer service of this sort as the profit center it is, rather than a cost center. The problem is that things like the benefits of quality tutoring of freshmen may not yield tangible results for a few years, so things move slowly.
- alisonkisk 5y ago
  It might help to send a text at 9pm reminding them to go attend to their affairs earlier so they can get beauty rest.
dkarl 5y ago
Things also "work" if you are able to scale down a cost (employment of human labor such as teachers and counselors) while maintaining or scaling up a source of revenue (student enrollment.) I worked for a company that offered this service (among other data-y things.) It employed lots of bright-eyed young people who were extremely happy they had found a tech job where they could do good, helping struggling students succeed at college. It was a dream job. However, the information we heard trickling in from the customers (public universities and community colleges) was pretty demotivational. They ones that were shrinking were panicked about their future existence, and the ones that were growing were basically like, yeah, hiring a proportional number of front-line staff to interact with students like in the past would just be silly. Nobody is going to do that. Run along, idealistic tech people, save these poor kids with analytics and text messages, because we're not paying to do it the old-fashioned way.
- newnamenewface 5y ago
  This really gets at the heart of why I have very little allegiance and essentially no desire to donate to my alma mater: Why would I care about or donate to a business that really doesn't care about me individually besides $$$ signs? Sure, I patronized them for four years of my life, but I've done the same with In & Out for longer and you don't see me sending them a check for nothing. Much like In & Out, the positive memories of my alma mater were more so about the people I was with than the location or business proposition itself. Ok, I'm ending this metaphor before it becomes too contrived...
  - toomuchtodo 5y ago
    Better to donate to Khan Academy. Can even enjoy an In N Out burger while doing so.
    - ineedasername 5y ago
      Khan academy is absolutely fantastic. However, it's only a step above giving students free textbooks or other self-learning resources: It mostly work for students that are already self-motivating. This tends to be a minority of students, and most require some degree of a human support network to succeed.
splithalf 5y ago
Excellent point. It’s a problem with operational definitions which plagues a lot of research. But “scale” in this context implies economic efficiency. You can’t give every kid a full time counselor. You can write a script to send sms messages. One question for the tech community, to what extent can advances in ai and other technology impact this question of scale/efficiency. If the student doesn’t know his counselor is a bot, maybe the nudges become more impactful.

Spooky23 5y ago

Some problems can be improved with a simple contact point. I have a colleague who works in this space in a research capacity. They were looking at use of different systems to measure behavior and identify behaviors that are correlated with poor performance. In one scenario, they surveyed a cohort who was leaving and entering the dorm after 1AM, resulting in poor AM class attendance. 40% of the responses were essentially: “I went to meet my [girl/boy]friend and have sex.” Getting a text message to tell that kid to get his beauty rest isn’t going to work. Just treat people like adults, imo.

ineedasername 5y ago
Some problems can be improved with a simple contact point Absolutely. Most schools don't employ enough advisors to have sit-down appointments with every student every semester. However, I did see a school implement a policy where 1st semester students-- a higher risk population-- were required to have a sit down like that before they were allowed to register for the next semester. That single effort resulted in a 15% decrease in 1st-year dropouts.
- Spooky23 5y ago
  Makes sense. I went to a big public university where they had two freshman tracks, a "college within the university" where, if you knew to sign up, you'd be in a cohort of students to meet some general education requirements and would get access to faculty motivated to help you. The problem was that the kids who needed the least help tended to discover this program. The "normal" path was being dumped into the University bureaucracy blind, and you had to learn to sink or swim. It was tough -- the registrar in those days was an experience like DMV. I'm pretty sure the system was devised as a funnel to disqualify people for popular majors like Business, Computer Science, etc.
  - ineedasername 5y ago
    the kids who needed the least help tended to discover this program Heck yeah did you hit the nail on the head. I've analyzed the data over a period of years for which students utilize tutoring services: They very much tend to be students that were already doing alright before they started getting tutoring. Conscientious self-motivated students are precisely the ones most likely to seek out help on their own. As a result, for students that go on academic probation we now require them to attend a certain amount of tutoring hours each semester, and the software we use to manage advising caseloads allows faculty to raise a flag for students performing poorly so that advisors can pro-actively reach out and direct them to tutoring resources. This has led to a huge increase in tutoring resource utilization, but unfortunately they are now at the limits of their capacity. Our next hurdle is to analyze the academic outcomes of students using those resources, which will hopefully show increased performance & allow us to petition for more support services. Fortunately our leadership is not completely short-sighted on these issues, but as obvious as "more one-on-one advising & support" would seem, they (not too unreasonably) still demand very concrete evidence to justify additional expenditures. However it would be nice if they viewed customer service of this sort as the profit center it is, rather than a cost center. The problem is that things like the benefits of quality tutoring of freshmen may not yield tangible results for a few years, so things move slowly.
alisonkisk 5y ago
It might help to send a text at 9pm reminding them to go attend to their affairs earlier so they can get beauty rest.

dkarl 5y ago

Things also "work" if you are able to scale down a cost (employment of human labor such as teachers and counselors) while maintaining or scaling up a source of revenue (student enrollment.) I worked for a company that offered this service (among other data-y things.) It employed lots of bright-eyed young people who were extremely happy they had found a tech job where they could do good, helping struggling students succeed at college. It was a dream job. However, the information we heard trickling in from the customers (public universities and community colleges) was pretty demotivational. They ones that were shrinking were panicked about their future existence, and the ones that were growing were basically like, yeah, hiring a proportional number of front-line staff to interact with students like in the past would just be silly. Nobody is going to do that. Run along, idealistic tech people, save these poor kids with analytics and text messages, because we're not paying to do it the old-fashioned way.

newnamenewface 5y ago
This really gets at the heart of why I have very little allegiance and essentially no desire to donate to my alma mater: Why would I care about or donate to a business that really doesn't care about me individually besides $$$ signs? Sure, I patronized them for four years of my life, but I've done the same with In & Out for longer and you don't see me sending them a check for nothing. Much like In & Out, the positive memories of my alma mater were more so about the people I was with than the location or business proposition itself. Ok, I'm ending this metaphor before it becomes too contrived...
- toomuchtodo 5y ago
  Better to donate to Khan Academy. Can even enjoy an In N Out burger while doing so.
  - ineedasername 5y ago
    Khan academy is absolutely fantastic. However, it's only a step above giving students free textbooks or other self-learning resources: It mostly work for students that are already self-motivating. This tends to be a minority of students, and most require some degree of a human support network to succeed.

splithalf 5y ago

Excellent point. It’s a problem with operational definitions which plagues a lot of research. But “scale” in this context implies economic efficiency. You can’t give every kid a full time counselor. You can write a script to send sms messages. One question for the tech community, to what extent can advances in ai and other technology impact this question of scale/efficiency. If the student doesn’t know his counselor is a bot, maybe the nudges become more impactful.