The authors, who are decidedly non bot-like, are exploring how HCI methods can improve the process of programming language design. They offer a method "PLIERS:

The authors, who are decidedly non bot-like, are exploring how HCI methods can improve the process of programming language design. They offer a method "PLIERS: Programming Language Iterative Evaluation and Refinement System.", and they "evaluated PLIERS by using it to create two different languages". Funders for the work include IBM and Ripple, a blockchain company. Table 1 in the paper is the clearest summary of the work in terms of challenges and approaches. The underlying problem is that it is expensive to get to the point where you can iterate well with users in this design space. It's interesting to think of a programming language as a user interface. My takeaways here were that academic programming language design has a data problem, which is finding the right population of test users, and the idea of "Wizard of Oz" to simulate tool interaction. Wizard of Oz is a dev "behind the console" who is simulating appropriate responses to user actions. That's much more like the person switching the pages around as you walk through a paper prototype than an endpoint returning mocks. If I were an academic in this space I would want to see how lab approaches to design and eval compare against the sort of proposal and review cycles that I see python and golang use for language changes - the latter seems like the status quo, for better or worse.