DateTime Handler

Being able to handle date and time expressions is crucial for many applications. Think of bots for travel booking or insurance claims for example. Teneo provides a DateTime Handler which can be used to catch date and time expressions from the user input. The DateTime Handler consists of three components:

1. a DateTime library to map the user input to language-independent representations,
2. a global listener to map collated date expressions of the user input to language-independent representations and
3. an interpretation script (.JAR file) to map these representations to actual dates and times.

Many date and time expressions have to be interpreted with respect to an anchor date and time. For example "tomorrow" denotes different dates, depending on when it has been said. The following table contains some examples returned by the DateTime Handler. DateTime expression is what the user said, DateTime representation¹ reflects what the DateTime library caught and DateTime interpretation is a map containing Java LocalDates and Java LocalTimes returned by the interpretation script. Note that the assumed anchor date for the interpretations below was November 9th 2018:

As you can see from the examples, there are cases where it is not clear which exact time the user meant. In such cases, the interpretation simply contains both alternatives, as "03:00" and "15:00" for "3 o'clock". The ambiguity can then be resolved in the solution, if needed.

Add the DateTime Handler to your solution

In the following, we will show you how to add the DateTime Handler to your solution. Basically, there are three things you need to do:

• assign the DateTime library, which is needed to map the user input to a language-independent representation.
• setup a global listener, which is required to annotate collated date expressions.
• import the .JAR file needed for the interpretation step.

Assign the DateTime library

You can assign the DateTime library in the same way as you assign Lexical Resource. Visual instructions of how to do that can be found here. The first part of the DateTime Handler is now assigned. However, before you can use it, you still have to setup the global listener and add the .JAR file for the interpretation part.

Setup the global listener

The DateTime library assigned above recognizes the vast majority of all DateTime expressions. However, for collated DateTime expressions like '11h30' that consist of one word but bear several units (here: hours and minutes), we use a global listener instead. This is how you set it up:

1. Open the 'SOLUTION' tab in the solution's window.
2. Select 'Globals' in the purple bar on the left hand side.
3. Select 'Listeners' at the top.
4. Add a 'Pre-Listener' and call it DateTime Handler: identify merged date/time expressions.
5. Use the 'Back' arrow at the top to come to the Listener specification.
6. Paste %TRUE.SCRIPT in the 'Condition' field. This way, the pre-listener is always active.
7. In the 'Execute Script' field paste

   // The DateTime library recognizes the vast majority of all DateTime expressions.
   // However, for collated DateTime expressions like '11h30' that consist of one word
   // but bear several units (here: hours and minutes), we use this global listener instead.
   datetime.Handler.preProcess(_)

8. Hit 'Save'.

This step concludes the recognition part of the DateTime Handler. Now the only thing that remains to be done is to add the interpretation script, which we will address in the next section.

Import the .JAR file

Now it's time to download the .JAR file which is required for the interpretation of the DateTime representation. Note that you have to unzip it before you can import it to Teneo:

1. Open the 'SOLUTION' tab in the solution's window.
2. Select 'Resources' in the purple bar on the left hand side.
3. Select 'File' at the top.
4. Use 'add' on the upper right to add the .JAR file. Alternatively, you may 'drag and drop' it.
5. Change the path to /script_lib.
6. Hit 'Save'.
7. After returning to the main solution window click on the 'Reload now' message that has now appeared. This will reload Teneo and make the .JAR file usable in your solution.

Visual instructions of how to import a .JAR file to your solution can be found here.

That's it! You can now use the DateTime Handler in your solution!

Use the DateTime Handler in your flow

In the following, we will build a simple example flow that makes use of the DateTime Handler to book tables.

We will first set-up the basic flow structure:

1. Create a new flow and call it Book a table.
2. Paste the following learning examples:

   I want to book a table
   I would like to book a table
   I'd like to book a table
   Can I book a table
   Can I make a reservation for a table
   Could I book a table

3. Name the trigger I want to book a table.
4. Select the output node and paste When do you want to book it? into the 'Answer text field'. Name the node When?.
5. Add an empty script node after this output node and call it Interpret date and time.
6. Add an empty output node after the script node and call it Booking confirmation.

This is how your flow should now look like:

In the following, we will add content to the empty nodes and add conditions to the transitions.

Catch Date and Time using language objects

Teneo's DateTime library features numerous different language objects that can be used to catch date and time expressions in the user input. Some of which are illustrated in the table below. Depending on which kind of DateTime expressions you are expecting from the user, i.e only date or only time or only weekdays, you may use different language objects. All language objects in the English DateTime library start with the prefix 'DT'. You can thus simply search for DT* in Teneo's search interface (prefixes for languages other than English can be found in the grey box below), to see all language objects that belong to the DateTime library. We recommend to start from the main object, DT_DATE_TIME.PHR and then cllick your way through the language objects used there and continue doing so until you found the level of granularity that you were looking for.

In our example, we will use the most general one, DT_DATE_TIME.PHR:

1. Add the flow variable dateTimeRepresentation to your flow and assign it the default value [:].
2. Select the transition between the output node 'When' and the script node 'Interpret date and time' and call it Get date and time
3. Click the 'Examples' panel on the right and activate the toggles for 'Gets input before continuing' and 'Conditional'.
4. Paste %DT_DATE_TIME.PHR^{dateTimeRepresentation = lob.datetime} into the transition's condition field.

While not fully functional yet, you can now use the output node 'Booking confirmation' to print the intermediate DateTime representation:

1. Select the 'Booking confirmation' node.
2. Paste Great! I booked a table for ${dateTimeRepresentation} in the 'Answer text' field.
3. Make sure that the transition from the empty script node 'Interpret date and time' to the 'Booking confirmation' node is dashed, which indicates that it is not getting any input before continuing.
4. Hit 'Save'.
5. Give it a try!

You should now get the following response:

User: I want to book a table
Bot: When do you want to book it?
User: Tomorrow, 6:30.
Bot: Great! I booked a table for [ date: [ named_relative: tomorrow ], time: [ hour: 6, meridiem: ampm, minute: 30 ]]

Note that this is the intermediate DateTime representation which simply reflects what the user said. This representation is not really meant to be printed to the user. We just wanted to show you what happens internally here. For example, you can see that the time expression '6:30' is ambiguous with respect to am/pm. We will show you below how you can resolve such ambiguities. In order to get an actual date and time, this representation needs to be handed over to the interpreter. In the next section, we will show you how.

Interpret Date and Time via script call

Go back to the 'Book a table' flow in edit mode. Then:

1. Add a flow variable called dateTime and assign it the default value [:].
2. Select the script node 'Interpret date and time'.
3. Paste dateTime = datetime.Handler.interpret(dateTimeRepresentation) into the 'Script Action' field.
4. Select the Booking confirmation node, delet the existing answer and replace it with: Great! I booked a table for ${dateTime.time.start[0]} on ${dateTime.date.start[0]}. This selects the first time point ('time.start[0]') and the first date point ('date.start[0]') from the date time map which the interpreter returned and prints it to the user in a more readable format.
5. Hit 'Save'.

Now go ahead and give it a try in try out! For the dialog above, you should now get the following answer (assuming you spoke to the bot on November 9th):

User: I want to book a table.
Bot: When do you want to book it?
User: Tomorrow, 6:30.
Bot: Great! I booked a table for 6:30 on 2018-11-10.

Disambiguation strategies

Many time expressions like "6:30" or "3 o'clock" are ambiguous as to whether they denote 'am' or 'pm'. The DateTimeInterpreter returns both possible times for such expressions: "06:30" and "18:30". However, for many applications it is crucial to correctly disambiguate such time expressions. We will in the following illustrate two disambiguation strategies that you may use in your solution. One is to disambiguate the representation before sending it to the interpretation script, and the other one is to disambiguate what the interpretation script returned.

Ambiguous representation

The safest way to disambiguate a time expression is to let the user disambiguate it. The fact that time expressions are ambiguous with respect to 'am'/'pm' is visible from the intermediate DateTime representation. Thus, it can be resolved there before it is handed over to the DateTime interpretation script:

1. Open the 'Book a table' flow in edit mode.
2. Select the transition 'Get date and time'. Right-click on it and select 'Insert after' → 'Junction' from the drop-down menu.
3. Add an Output node after the Junction.
4. Name this new output node Resolve ambiguity.
5. Select the transition between the junction and 'Resolve ambiguity' and call it Ambiguous Time.
6. Click the 'Examples' panel on the right and assign this transition order number '1' using the toggle on the bottom.
7. Draw a transition between 'Resolve ambiguity' and 'Interpret date and time'.
8. Name this transition Resolve am/pm.

The basic structure is now in place. Time to fill it with content!

1. Select the 'Ambiguous Time' transition.
2. Open the 'Examples' panel to the right and make sure that the 'Conditional' toggle is active but the 'Gets input before continuing' toggle is not.
3. Paste {dateTimeRepresentation.containsKey('time') && dateTimeRepresentation.time.meridiem=='ampm'} to the condition field. This condition is fulfilled if the DateTime expression holds a time and if this times does not hold a determined 'am' or 'pm'.
4. Select the 'Resolve Ambiguity' node.
5. Paste Do you mean ${dateTimeRepresentation.time.hour} am or pm?. For our example this will return 'Do you mean 6 am or pm?'.
6. Select the transition 'Resolve am/pm'.
7. Open the 'Examples' panel to the right and make sure that both, the 'Conditional' toggle and the 'Gets input before continuing' toggles are active.
8. Paste the following to the condition field. Depending on what the user said, this condition sets the meridiem to 'am' or 'pm'.

   (am)^{dateTimeRepresentation.time.meridiem='am'}
   /
   (pm)^{dateTimeRepresentation.time.meridiem='pm'}

9. Hit 'Save'.

That's it. You should now get the following dialog:

User: I want to book a table.
Bot: When do you want to book it?
User: Tomorrow, 6:30.
Bot Do you mean 6 am or pm?
User: pm
Bot: Great! I booked a table for 18:30 on 2018-11-10.

Ambiguous interpretation

In the previous section we have shown how the decision about 'am' vs 'pm' can be returned to the user to request clarification. Sometimes however, the context only allows one interpretation and in such cases we should avoid asking the user. Let's say that we still want to book a table. As the restaurant is open from 5pm to 11pm, all table bookings will denote 'pm'. In cases where the DateTime interpreter returns more than one possible time, we will thus always select 'pm'.

This is how we will go about: before returning the DateTime interpreter's answer to the user, we will select the 'pm' time, if applicable. Whenever two times are returned, the second one (at index 1) is always 'pm'. If this is the case, we copy the 'pm' value from index 1 to index 0. This way, we do not have to modify the output node:

1. We start from the 'Book a table' flow that we created above (without disambiguation of the representation) in edit mode.
2. Select the 'Interpret Date and Time' script node. and paste the following after its current content:

   if (dateTime.time.start[1]) {
     dateTime.time.start[0] = dateTime.time.start[1]
   }

3. Hit 'Save'.

That's it. When talking to your bot, you should now always get the 'pm' interpretation for ambiguous time expressions like 6:30:

User: I want to book a table.
Bot: When do you want to book it?
User: Tomorrow, 6:30.
Bot: Great! I booked a table for 18:30 on 2018-11-10.

Ambiguous interpretation direction

For some date and time expressions the interpretation depends on the conversation context. For example, the expression 'on Wednesday' may be interpreted as a date in the past or in the future. The following example dialoge happened on Friday, November 9th 2018:

User: I arrived in Barcelona on Wednesday.
Bot: You mean 2018-11-07, right?

User: I will arrive in Barcelona on Wednesday.
Bot: You mean 2018-11-14, right?

By default, the interpretation direction is 'forward', so 'on Wednesday' would be interpreted as the following Wednesday. It is possible to change this interpretation direction to 'back' to point to the previous Wednesday instead. We show how to edit the default interpretation direction in the next section. Here, we will show you how you can take advantage of the verb tense to make an appropriate decision. One component of the Teneo Input Processing Chain assignes part-of-speech tags to all words of the user input. We will make use of one of these tags, %$PAST.POS, which indicates that a verb occurred in past tense in order to set the interpretation direction to 'back'. In the following, we build a small example flow in order to illustrate the functionality:

1. In the main solution window, click on the little black triangle under the 'Flow' icon and select 'Flow - with Syntax Trigger' from the drop down menu.
2. Name the flow Disambiguate Interpretation Direction.
3. Add a flow variable called dateTimeRepresentation and assign it the default value [:]. It will store the representation of the DateTime expression uttered by the user.
4. Add a flow variable called pastTense and assign it the default value null. This one will store whether or not a past tense tag was found in the user input.
5. Add a flow variable called dateTime and assign it the default value null. It will store the DateTime interpretation.

Visual instructions on how to create a flow with a syntax trigger can be found here. For instructions on how to add flow variables see here.

1. Paste the following into the Trigger condition and call the Trigger DateTime + Tense.

   %DT_DATE_TIME.PHR^{dateTimeRepresentation=lob.datetime} 
   &^ 
   (%$PAST.POS^{pastTense = true}):o

2. Add a script node and make it the Start node by clicking 'Set Start Node' in the top ribbon.
3. Paste the following into the script node:

   dateTime = pastTense?datetime.Handler.interpret(dateTimeRepresentation, "back"): datetime.Handler.interpret(dateTimeRepresentation)

   This will call the interpreter with the interpretation direction 'back' if the pastTense variable has been set in the Trigger condition. Otherwise, the interpreter will be called with the default interpretation direction "forward" (which does not need to be explicitly specified).

4. Name the script node Call Interpreter.
5. Connect 'Call Interpreter' to the existing output node.
6. Paste You mean ${DateTime.date.start[0]}, right? into the Output node and name it You mean ... right?.
7. Hit 'Save'.

Edit default settings

Anchor date
By default, the DateTime interpretation script assumes the current day to be the anchor date for relative date expressions like 'tomorrow' or 'Monday next week'. Should it be necessary for you to change this anchor date, you can do so by adding the desired anchor date to the DateTime interpreter calll. Say you want "14th of August 2019" to be your anchor date. Then, the DateTime interpreter call in the script node of your flow will look like this: dateTime = datetime.Handler.interpret(dateTimeRepresentation, "2019-08-14"). Please note that the new anchor date must be passed in ISO-8601 format and put into quotes (""). If you want to change the anchor date globally for the whole solution, go to the global pre-processing script and paste datetime.Handler.setAnchor("2019-08-14"). Note that you have to restart the conversation before changes in the global script will apply.

Anchor time
You may not only modifiy the anchor date, but also the anchor time, if neccessary. In order to do so, simply add the time to the anchor date by keeping the ISO-8601 format, like so: dateTime = datetime.Handler.interpret(dateTimeRepresentation, "2018-11-21T1630"). The first two digits after the 'T' denote the hour and the last two digits the minutes of the anchor time.

Interpretation direction
For some expressions like 'on Monday' or 'in September' the interpretation depends on the context. They may denote a date in the past or in the future. Depending on the domain for which you created your bot, different interpretation directions might be appropriate. For example, in our table booking domain, one can assume that all dates occuring in the conversation denote future events. However, in an insurance claim domain a backwards interpretation direction is more suitable. By default the interpretation direction is set to 'forward', but you can change it to 'back', by adding it to the DateTime interpreter call: dateTime = datetime.Handler.interpret(dateTimeRepresentation, "back").

Both
You may also change both, the anchor date and the interpretation direction, at once: dateTime = datetime.Handler.interpret(dateTimeRepresentation, "2018-11-21T1630", "back").

Advanced Usage

The DateTime Handler covers a wide range of different date and time expression and it always returns the most plausible interpretation. However, if neccessary, you may adapt the DateTime Handler for example to recognize named days or to adjust the interpretation of existing dates and times. For more information, read this page on Advanced Usage.

 

---

1. See DateTime Handler Reference for more details on the language-independent DateTime representation.