/** * @class ListenBundle * @memberof jibo.mim * @description Internal bundle of listen specific data. * @private */ /** * Timeout duration in milliseconds. If NaN, do not timeout. * @name jibo.mim.ListenBundle#timeout * @type {Number} */ /** * Beginning of timeout in epoch time. Timeout is triggered when the difference between this * time and current time is greater than timeout. * @name jibo.mim.ListenBundle#startTime * @type {Number} */ /** * Listener from jibo.mim#listenDelegate. * @name jibo.mim.ListenBundle#listener * @type {any} */ /** * If we had failed to get a listener previously. Skill will be exited on another failure. * @name jibo.mim.ListenBundle#failedToGetListener * @type {boolean} */ /** * @class MimStates * @memberof jibo.mim * @description Bundle all the Mim's state machine states, with strong typing. Each * property of MimStates is a State from jibo-state-machine. All actions are performed * by functions of the Mim class. * @private */ /** * Return prompt data for MIM. Properties are used as variables * for MIM condition evaluation and prompt text insertion. * @callback jibo.bt.behaviors.Mim~GetPromptData * @returns {any} */ /** * Called on listen complete during Optional Response or Question, before the MIM analyzes the * result. Allows skills to implement their own check on the result to tell the MIM how to handle it. * @callback jibo.bt.behaviors.Mim~CheckResult * @param {jibo.jetstream.types.ListenResult} result Data from NLU service. */ /** * Called when an Optional Response or Question MIM completes successfully. * Optional Response MIMs always complete; * check `options.state.lastResultState` to see if user responded. * @callback jibo.bt.behaviors.Mim~OnSuccess * @param {any} results Bundles data from MIM. * @param {jibo.jetstream.types.ListenResult} results.asrResults Data from NLU service. * @param {jibo.mim.SpeakerIds} results.speakerIds If available, the ID of the speaker. * @param {jibo.mim.MimState} results.state Current state of the MIM. * @param {String} results.firstGrammarTag (Flow use only) The value of the first rule slot (results.asrResults.entities[results.asrResults.slotActions[0]]). The default transition from a MIM. If a listen has multiple potential slots, this value should not be considered reliable. */ /** * (Flow use only) Called when a Question MIM fails entirely. Either the user did not * respond to Jibo or Jibo could not understand the user and has run out of error prompts. * Return value finds an error handler within the flow. Unhandled * errors use Jibo's standard MIM exception handling. * @callback jibo.bt.behaviors.Mim~OnFailure * @param {any} results Bundles data from MIM. * @param {jibo.jetstream.types.ListenResult} results.asrResults Data from NLU service. * @param {jibo.mim.SpeakerIds} results.speakerIds If available, the ID of the speaker. * @param {jibo.mim.MimState} results.state Current state of the MIM. * @param {String} results.exception Exception for this failure; either `~InteractionError.noMatch` or `~InteractionError.noInput`. */ /** * (Internal to Menu behavior only) Called to confirm that a positional selection makes sense, allowing custom handling to select certain types of items, if desired. * @callback jibo.bt.behaviors.Mim~OnPositionalSelect * @param {jibo.jetstream.types.ListenResult} commandASR Data from NLU service. * @param {Number} intendedIndex Index that the menu plans to use for selection. * @param {jibo.face.views.MenuView} menu Created menu. Use to read button data, if you didn't keep track of it yourself. * @returns {number} Return an item index to change what will be selected, or null to select nothing. Returning undefined will use the default behavior (selecting intendedIndex). * @private */ /** * @class Mim * @extends jibo.bt.Behavior * @memberof jibo.bt.behaviors * @description Perform a dialogue interaction. Can be an announcement, an * announcement with optional user response, or a * question and attempt to get answer from the user. * @param {Object} options See {@link jibo.bt.Behavior|Behavior} for all options. * @param {String} options.mimPath Path to the MIM file to use. * @param {jibo.bt.behaviors.Mim~GetPromptData} options.getPromptData Returns the prompt variable data. * @param {jibo.bt.behaviors.Mim~CheckResult} options.checkResult Called and passed data about what the Mim is hearing when a listen is complete. Optional Response and Question MIMs only. * @param {jibo.bt.behaviors.Mim~OnSuccess} options.onSuccess Called and passed data about what the Mim heard when it completes. Optional Response and Question MIMs only. * @param {jibo.bt.behaviors.Mim~OnFailure} options.onFailure Called and passed data about what the Mim heard or did not hear when it completes. Question MIMs only. */ /** * Stops the MIM. The optional parameter here is considered private - this should be used * like the stop() method it inherits from Behavior, with no parameters. * @method jibo.bt.Mim#stop * @param {boolean} [isMenuTap] If the stop() was due to tapping a menu button. * @private */ /** * If this Mim has an active view, pops the top view from the list and destroys this Mim's * active view. * @method jibo.bt.Mim#removeView * @private * @param {string} intent Reason for removing the view - 'success', 'failure', or null if view should never be left blank. */ /** * Remove combinedRuleHandle and ruleHandle from NLU memory. * @method jibo.bt.Mim#unloadRules * @private */ /** * Cleans up global event listeners that the MIM added. * @method jibo.bt.Mim#cleanUpEvents * @private */ /** * Opens the MIM's GUI if it is not already open. * @method jibo.bt.Mim#openGUI * @private */ /** * Callback for events from `jibo.mim.openGUI`. Opens the GUI, if it is not already open. * @method jibo.bt.Mim#onOpenGUIEvent * @private */ /** * Callback for events from `jibo.mim.handleSpeech`. Stops active speech or listen to parse a * spoofed utterance. * @method jibo.bt.Mim#onSpeechEvent * @param {string} utterance The spoofed utterance in need of parsing. * @private */ /** * Callback for events from `jibo.mim.end`. Ends the MIM immediately. * @method jibo.bt.Mim#onEndEvent * @param {any} data Not currently used. In the future, data about how the MIM should end. * @private */ /** * Callback for events from `jibo.mim.heyJibo`. Handles barge in via "Hey Jibo". * @method jibo.bt.Mim#onHeyJiboHeard * @private */ /** * State entry for states.loadConfig. Load .mim file(s) and then go to states.loadRule for OR/Q * mims or states.choosePrompt for AN mims. * @method jibo.bt.Mim#loadMim * @private */ /** * State entry for states.loadRule. Load .mim file(s) and then go to states.loadRule for OR/Q * mims or states.choosePrompt for AN mims. Sets up loading of rule for mim and unionizing it * with mim global rule as a Promise. Immediately go to states.choosePrompt. * @method jibo.bt.Mim#loadRule * @private */ /** * State entry for states.choosePrompt. Choose prompt text to be played. Go to states.listen * if no prompt or silent prompt chosen. Go to states.speak to speak the prompt. * @method jibo.bt.Mim#choosePrompt * @private */ /** * State entry for states.speak. Speaks the chosen prompt through the TTS system. When finished, * goes to states.reportResults for an AN mim or states.listen for an OR/Q mim. * @method jibo.bt.Mim#speak * @private */ /** * State interuption for states.speak. Stops jibo.mim#speakDelegate playback. * @method jibo.bt.Mim#stopSpeaking * @private */ /** * State entry for states.listen. Starts a listener from jibo.mim#listenDelegate. * If appropriate, starts timeout. If appropriate, displays the mim GUI. When listen completes, * transitions to states.analyzeResults for mim results or states.analyzeMimGlobal for mim * global commands. * @method jibo.bt.Mim#startListen * @private */ /** * Callback for screen interaction. Resets timeouts during listen. * @method jibo.bt.Mim#resetTimeout * @private */ /** * State interuption for states.listen. Stops listener and removes mim GUI if present. * @method jibo.bt.Mim#stopListen * @private */ /** * State exit for states.listen. Stops listener and removes screen touch listeners. * @method jibo.bt.Mim#listenExit * @private */ /** * State update for states.listen. Tracks timeout. If timeout triggered, change state to * states.analyzeResults. * @method jibo.bt.Mim#updateListen * @private */ /** * State entry for states.analyzeMimGlobal. If global command is 'repeat', go to * states.choosePrompt. Otherwise, redirect to states.analyzeResults. * @method jibo.bt.Mim#analyzeMimGlobal * @private */ /** * State entry for states.analyzeMenuGlobal. If global command is 'left' or right' and GUI is * active, scroll list and return to states.listen. Otherwise, redirect to * states.analyzeResults. * @method jibo.bt.Mim#analyzeMenuGlobal * @private */ /** * State entry for states.parseSpoof. Perform NLU parse on utterance text from GUI, followed * by going to states.reportResults. * @method jibo.bt.Mim#parseSpoofedUtterance * @private */ /** * State entry for states.analyzeResults. Analyze results to determine if result of listen is * 'noMatch', 'noInput', or 'match'. If result is 'match' or mim has run out of error prompts, * go to states.reportResults. Otherwise, go to states.choosePrompt. * @method jibo.bt.Mim#analyzeResults * @private */ /** * State entry for states.reportResults. If mim has failed due to noInput or noMatch errors, * call onFailure to allow flow to handle exception. If exception not handled, go to * states.handleException. If mim successful, call onSuccess and end Mim behavior. * @method jibo.bt.Mim#reportResults * @private */ /** * State entry for states.handleException. Begins global mim exception flow for either 'noInput' * or 'noMatch' failures. * @method jibo.bt.Mim#handleException * @private */ /** * State update for states.handleException. Update exception flow. If flow is complete act upon * flow result. If flow result is 'exit', exit skill and return to be/idle. If flow result is * 'restart', go to states.choosePrompt. * @method jibo.bt.Mim#updateException * @private */ /** * State interuption for states.handleException. Stop and clean up exception flow. Remove any * flow created view. * @method jibo.bt.Mim#stopException * @private */ /** * Display text that mim rules could not match. Provided to noMatch exception flow as a * callback. * @method jibo.bt.Mim#showInputText * @param {string} notMatchedText Text from speech to text that NLU could not match. * @private */ /** * Remove text display created during noMatch exception flow. * @method jibo.bt.Mim#hideInputText * @private */ /** * State entry for states.waitForHJEnd. Begins waiting for the Hey Jibo listen to end in a way * that the MIM should recover from. * @method jibo.bt.Mim#waitForHJEnd * @private */ /** * Standard method to tell the waitForHJEnd state to go back to prompt selection. * @method jibo.bt.Mim#recoverFromHJ * @private */ /** * State interuption and exiting for states.waitForHJEnd. Removesevent listeners added in the * state entry. * @method jibo.bt.Mim#hjEnded * @private */