Wednesday, September 4, 2013

My SpeechShim for Desktop Development of Speech Recognition and TTS Apps

So I've been working on some plugins for PhoneGap to enable people to develop their apps with Speech Recognition and Text to Speech functionality. I've been following this specification and while it isn't on track to be adopted by the W3C anytime soon it does have two big benefits:

1) The speech recognition bit is already available in Chrome.
2) The specification is "sane". I guess it helps when only a couple of people are credited as authors instead of a committee.

One of the things that has been bugging me for awhile is the inability to develop these type of apps on the desktop using the same API. Yes, as I said above the speech rec bit is available in Chrome but the objects have the "webkit" prefix and there is not TTS support. So during one of the Ottawa Ruby project nights I set out to write a shim that would give everyone the ability to use the same API that will be available from the PhoneGap plugins on their desktop.

Basically that is what SpeechShim is in a nutshell. When you add speechshim.js into your web app you will be able to access the "SpeechRecogntion" object instead of needing to prefix it like "webkitSpeechRecognition". As well if you combine it with the speak.js project the speechshim.js code will add the methods necessary so you can use the SpeechSynthesis interface.

To get up and running you will need:
  1. A copy of Google Chrome version 29 or higher.
  2. A copy of the speak.js project from github.
  3. A copy of my speechshim.js from github.
Here's an example HTML page:


and you can run the live demo here. It's nothing fancy. Just click the big button then say something. It should get updated under the button and you will hear it spoken out by the TTS. More on plugin availability for PhoneGap is coming soon.


at

6 comments:

  1. jcesarSeptember 4, 2013 at 1:39 PM

    I released a TTS and ASR iOS plugin the other day, did you saw it?

    ReplyDelete
  2. Simon MacDonaldSeptember 4, 2013 at 1:49 PM

    @jcesar

    I did see that and starred it to get back to you. I should have my Android Plugins out before PG Day EU and they I wanted to get back to the iOS stuff. I was going to talk to you about collaborating on it to use your code but adopt the W3C API.

    ReplyDelete
  3. jcesarSeptember 4, 2013 at 4:59 PM

    Sure, let me know when you finish the js and I'll use the same.

    ReplyDelete
  4. Simon MacDonaldSeptember 5, 2013 at 9:31 AM

    @jcesar

    Will follow up with an email.

    ReplyDelete
  5. JuNinhOOctober 29, 2013 at 11:23 AM

    Hi, Simon! Thanks for all your posts. It has helped me many times. I am facing a problem with phonegap. I have a app that works offline. Once a day, my client sends all orders to my online database using ajax. The problem is that sometimes the devices can not connect. I open Google Chrome and access other sites, but the app is not able to connect. The status code is 0 and response text is empty. I think the problem is not access cross domain because sometimes it works, sometimes it doesnt. If I uninstall it and install it again, it works. Do you have any idea to give me? Thanks!

    ReplyDelete
  6. Simon MacDonaldOctober 30, 2013 at 8:15 PM

    @JuNinhO

    Well that is kinda hard to figure out without seeing some code. The status code response of 0 is not uncommon when you are running from the file protocol on mobile.

    Based on the fact that you have to uninstall-reinstall I'd say there is some issue in your resend code that gets cleaned up when you uninstall.

    ReplyDelete