The exact meaning of a certain setting in Calendly (app I use w/students for office hours) was really unclear in the UI. I checked the docs, which also were not clear. So I gritted my teeth and clicked the dreaded AI chatbot button. And guess what!
…it was completely unhelpful. (It did find a link to the correct unhelpful documentation, at least.) But lo and behold, it connected me to a human!
…who was also unhelpful for ~5 minutes before they finally understood my question.
1/2
This entire process took ~12 minutes.
Calendly had to pay both some energy-munching LLM and a live human for that time. Multiply that by their userbase, and I find it hard to imagine that this was cost-effective for them vs giving the UX and doc writing, say, 20% more breathing room from the start.
Cost-trimming is so expensive.
2/2
@inthehands Discount that by the fact that only a tiny percent of their user base is willing to spend even 1 minute figuring this out never mind 12. (Obviously people not understanding how that feature works has its own cost or downside or whatever, but it's pretty hard to account for)
@aubilenon
I mean, that’s just the thing: vague dissatisfaction’s impact materializes slowly, is hard to measure, and is hard to trace to a root cause. Crappiness always looks cost-effective at first.
@inthehands my best chatbot experiences are measured by how quickly they get me to a human who can actually help me