5 ms·
So here's some more balanced feedback, if you will (playing with a language I know quite well already): On the positive side: the bot itself has excellent voic
by lisasays 3y ago
So here's some more balanced feedback, if you will (playing with a language I know quite well already):
On the positive side: the bot itself has excellent voice quality (it still sounds bot-like -- but people expect that) and makes very few pronunciation mistakes (I only noticed one). Its vocal recognition is, I would say ... not great, but good enough (it recognizes well enough if I say simple, standard things, but it easily confused if I use less common -- but also perfectly correct -- formulations). At least, vocal recognition as such is not its weakest spot.
You can ask it questions about fairly detailed topics and get reasonably correct answers most of the time (as with GPT, which I'm guessing is its backend). It can sometimes give you very nuanced feedback as to your diction and usage (it will notice when you're using a formulation that is correct but not standard, and give you a detailed explanation as to why).
However, what I don't like -- and what made me ultimately decide not to invest further time with the app at present:
(1) You can easily feel lost (in "Teacher" mode) because all it can do is answer questions -- it isn't guiding you anywhere, and you have to already know what questions to ask.
(2) Even so, I thought to myself -- "well I can see how a good Q+A bot can be useful -- let's see what we can do with it". And here's where I got most disappointed.
I told it I wanted to practice my pronunciation -- and first asked for it to come up with some sample sentences, and then thought of one of my own -- one of those sentences you would say to someone at the family dinner table to try to wow them with your language skills.
In both drills, I played a little trick on the poor bot -- I deliberately mispronounced the sentences, the way a stereotypical Ugly American would -- basically mangling every single syllable in some obvious way, hoping to get a long list of corrections from the bot.
And it totally failed at this task. It kept saying "You said that perfectly! Well done!" or some variant. When I knew for a fact that my pronunciation was an epic fail, and I would have greatly embarrassed myself at that family holiday dinner.
(3) But things got even weirder when I tried to tell the bot "Wait you're wrong, I actually made lots of mistakes." More than half the time it would get horribly confused, sometimes telling me -again- that I said that sentence perfectly, sometimes "correcting" its own sentence.
So it's quite scatterbrained (just like GPT-4 is, when you try to make it backtrack and acknowledge some kind of mistake). It also misunderstands questions about grammar if I ask it to explain something it just said, or which I said in a sentence previously.
Bottom line -- it's a very mixed bag. Partially quite impressive -- but far too often, extremely frustrating to work with. Once one gets past the "wow!" aspect of working with a bot that seems well-spoken and (quantitatively) seems to have a lot of knowledge about grammar and usage -- the frustration arising from its scatterbrained-ness, and more fundamentally, just never knowing if I can really trust its answers takes over.
On the whole it just grinds my gears ... like pretty much any other chat bot I've attempted to try to get useful information out of.
That and again -- it isn't guiding you anywhere. If it can't reliably keep track of what it just said (or what I just said), it's definitely not going to be able to assess my progress, or tell me how to improve (beyond the random piece of generic advice it doles when it doesn't have anything better to say).
Of course it won't. It's just a transformer bot, after all. And the technology is categorically nowhere capable of doing that.
- fabiensnauwaert 3y agoHey! Thanks for the balanced feedback, all totally valid. (1) I'm self-directed in my learning and always knew what I wanted my teachers to do, but everyone's different. Definitely going to add more choice of content (and not just roleplay.) I have a very structured and successful program to learn English and will try to adapt what works well in it to Gliglish. (2) You're right! The speech recognition and LLM layers are separate. I actually do have data about the quality of pronunciation, I'm just not using it yet. It's a matter of adding a layer to handle it. Basically, all the building blocks are here and I need to get them to play nicely together. (3) It's the same issue as with ChatGPT on which it's built. Part of the progress will come from OpenAI or similar endeavors themselves, part from further checks on my end as in (2). I just don't share your pessimistic view about the future of such tech though (well, if I did I probably wouldn't have started the whole thing! ) Once you put ChatGPT into a corner, there's usually no hope of it getting back on track. BUT with clear context and prompts on things it does well it's just incredibly useful! Again, is the glass half empty or half full? As for this use case, there's tremendous value in being able to practice. Just talking to oneself in the mirror or journaling in a foreign language is already super productive and this is it… on steroids.
- lisasays 3y agoJust talking to oneself in the mirror or journaling in a foreign language is already super productive and this is it… on steroids. I agree that a working pronunciation "coach" could be highly useful. And a great distinguishing feature for the app, if done right. However right now the app objectively fails in that category. It is simply unable to tell me when I'm not saying things properly. (Which I hope you will take positively. The main thing is -- you're doing it, putting it out there. Best of luck).
- fabiensnauwaert 3y agoYes, one step at a time :) Definitely high on the list, pronunciation is very dear to my heart.