Having problems with message search?
I know! Thus my surprise.
I'm not "pro bad systems", but I am very much against "bad testing".
I know you love to turn people's arguments back on themselves, but I am not the evil people hater you are trying to make out I am.
I am the one talking about making the system work for real users.
You are wanting to see it work for fake, unrealistic test data.
That will not help real users. Quite the contrary.
If we were back at the farm, with the naive crowd laughing at the car stuck in the mud and taking their concerns seriously, we would all still be either a) riding horses or b) driving tractors, because the original users, who heard the car is proposed to replace the horse, all agree a car must be able to cut it in the field to do that.
If you produced your real voice-mail box, with your weird and whacky voice-mails, and it didn't do an OK job on most of them, then we'd have something serious to talk about.
Until then these sort of tests are pointless tyre-kicking.
From: vuids@yahoogroups.com [mailto:vuids@yahoogroups.com] On Behalf Of Phillip Hunter
Sent: Friday, 3 July 2009 11:42 AM
To: vuids@yahoogroups.com
Subject: Re: [vuids] Google Voice
Never mind, Peter. You go on standing up for those systems. They need all the help they can get.
Meanwhile, some of will design and build for people.
ph
On Thu, Jul 2, 2009 at 6:54 PM, Peter Nann <peter.nann@...> wrote:
"made complete sense" is completely irrelevant.The "design parameters" are simple - It's designed to work on real voice-mails.These tests were not real voice-mails.Phil, I am surprised at your apparent lack of understanding of how the SLM technology behind this works, and the concept of real test data.
Sent: Friday, 3 July 2009 1:23 AMPN,
You're right that I should have included the "owner" of the voice mailbox. So, my question corrected: "How would someone receiving or leaving a voice-mail have this kind of knowledge of the "design parameters" of the system?" Which is rhetorical, btw. Google markets it as a voicemail system that delivers transcriptions in email.
Those excerpts were not random. Though perhaps not typical, the audio content made complete sense to me, and I bet to all who sampled the article. As a husband/father/son/brother/friend/manager/employee/co-worker/acquaintance/business owner/... I have received voice mails of such varied content that I can assert that nearly anything goes in voice mail content, not to mention delivery aspects such as audio quality, speaker characteristics, etc. Either that long tail needs to be covered fairly well, or the system is nearly unusable.
If you want to stand by your system focus, so be it. Meanwhile, real people will look at the results in that article and say with conviction and validity, "That's why I hate that automated voice recognition crap. It can't even recognize the President's name!".
ph
On Thu, Jul 2, 2009 at 6:31 AM, Peter Nann <peter.nann@...> wrote:
The "leaver" of the message is not the user here. I place no assumption of anything on them.The person who signed up to get their received voice messages translated to text is the user.If that user expects this is going to work on any random excerpt of words thrown at it, then their expectations have not been properly managed.Do some REALISTIC voice-mail messages, from first word to last word, complete with a couple of appropriately contextualised Mills and Boon features if desired, then we can pull it apart.This was a bunch of "Top Gear challenge" style tests - Not indicative of real world requirements, and designed to have catastrophic failures.Amusing - yes, but the annoying thing is most seemed to hope for something better.I completely stand by my family car plus plough analogy.(No, that wasn't a Top Gear challenge that I know of...)
Sent: Thursday, 2 July 2009 1:17 PMCouldn't disagree with you more. Your analogy is completely off. How would someone leaving a voice-mail have the remotest idea of the "design parameters" of a system they don't even know about? That's old system-centered thinking. If a lover decided to read one of Shakespeare's sonnets to a sweetheart, why is the expectation of correct delivery by either party any less valid than someone leaving business meeting details?
ph
On Wed, Jul 1, 2009 at 9:36 PM, Peter Nann <peter.nann@...> wrote:
Guilty. I didn't take much notice of that part. My bad.I still get the feeling the article, and most people who responded, expected it to work on that stuff.And/or that one-day it should work well on it.In my opinion none of that showed the system was "broken".It was being used outside of its design parameters.It's kind of like taking the family car, finding that it doesn't pull a plough in the field very well, so saying "Geeze, they still have some development to do on that car idea..."My apologies again, I am stuck on a rant this week about bad test data...This touched a nerve...
PN,
Maybe there was a text reco problem on your end. ;-) The second paragraph pretty much puts the entire article into the proper light.
"The aim here was not to see how well the service, called Google Voice, handled transcriptions of the day-to-day mundanities of voice mail. Google itself acknowleges that the feature... is still sort of experimental. The company is pushing the limits of technology with this, so the goal was to see how far they could be pushed before they broke."
ph
--
www.design-outloud.com
www.twitter.com/designoutloud
vzn 469-853-9016
--
www.design-outloud.com
www.twitter.com/designoutloud
vzn 469-853-9016
Offline 
Send Email