The state of information overload in enterprises today

Posted by Carylyne Chan on 19 October 2017

Challenges in making decisions due to too much available information — And what to do about it

Information overload, also known as infobesity, infoxication, information glut, data smog, and information tsunami, among other trendy terms, basically means reduced understanding and decision making capacity due to having too much information than can be processed. First discovered in psychology, information overload has demonstrated that even the most intelligent and upstanding people can have problems with acting appropriately (or even acting at all) when they are faced with too many choices or situational issues that overwhelm them.

Dealing with VUCA: Harvard Business Review

As you can imagine, this is a huge problem for us today. In a VUCA world (borrowing a term from the military parlance) — a volatile, uncertain, complex and ambiguous world a.k.a “It’s a crazy world” — like ours today means that decision making needs to be responsive and on point in order to drive businesses forward.

Another problem compounding work and decision making is the issue of “information anxiety”, where there is a gap between what we believe we should understand and what we actually do know. There is massive pressure, as we are told “better more information than less” and spend hours try to load up on more information from emails, blogs, books, podcasts, book summaries, news sites, social media and other sources, even if it may not be directly relevant to what we’re doing. Not only is this practice distracting, it is a form of cognitive overloading that affects thinking and productivity.

5 main causes of information overload today

There are many causes contributing to the dire state of information overload, though there are five causes causing this shift. From macro changes to micro changes in the business environment, these causes are pervasive through every level of management and across industries.

  1. The changing nature of work

The world is increasingly becoming a global economy, with global customers and cross-border communication and travel 24/7/365 across the team. Even small and medium businesses today are finding that their trade is cross-country and they frequently have to be aware of customers’ and partners’ time zones when working with them; the larger the corporation, the more this would hold true.

This means that there is greater complexity and more requirements to deal with, in order to fulfill our objectives. Not only do we have to deal with temporal differences, there are also cultural differences to read up on, more laws and taxes to understand, specific requests to cater to, and customer support tickets to respond to round the clock.

In addition, teams are adapting to cope with the changes in this globalized economy, taking on remote workers, demanding greater flexibility in work arrangements such as telecommuting and hotdesking (among others), and lobbying for a bigger push towards work-life integration. With more remote collaboration, more documents are being created and stored in large drives that are often complicated and intricately organized, leading to overlap in work that has already been done, and making it harder to find information in general.

All of these changing aspects of work are causing anxiety in people who feel there is always something more they need to know, and more to do, even in their downtime.

2. Big data and computing power

It’s no longer a secret that “big data is the future” or some similarly grandiose declaration about the necessity and importance of data in running businesses. With the rise in computing power, right down to the devices we hold in our palms today, more data is generated and more data can be analyzed, quicker.

However, having the potential to do it quicker doesn’t always mean analysis is getting better. Unstructured data in text is exploding across increasing numbers of customer-facing channels, on top of more operational data being collected from connected devices. This is causing a glut of data that can be hard to sift through and categorize for any disambiguation to happen at any level meaningful for the business.

There is now a stronger need for data-driven decision making, since this smorgasbord of data sources can reveal unprecedented patterns that otherwise might have been missed by making decisions by gut alone. This is definitely a welcome trend, though it is amplifying the effect of information overload by conditioning knowledge workers to think “all data is good data” when this is, in fact, not the case in most situations. Usually, there are vast amounts of misleading and non-essential data that may pollute decisions rather than promote them.

3. Consumer expectations and workload

Consumers today are spoilt, many of us included; not only are we spoilt for choice due to the globalizing economy, we are also spoilt in the sense of expecting higher levels of service in quick responses, fast resolutions, and even immediate refunds or replacements upon complaining.

Majority of people expect responses in under 10 minutes — 48 hours turnaround no longer works

Therefore, there actually is an increase in the workload for most businesses as they have to deal with more customer requests. Higher expectations by customers also signify bigger outlays required to staff support teams, and margin squeeze due to costs related to keeping customers happy so they continue to purchase.

Also, customers are less likely to be loyal to specific products, hence there needs to be more creative (or expensive) ways to retain and continually attract customers back to the business. More planning and decision making is needed, whether based on past customer data or financial data within the business to continue running profitably.

4. Compliance and transparency

To make matters even more dire for most knowledge workers today, there is also a stricter bar for compliance and controls within companies today (imposed by auditors or regulators). On top of the information that passes through their hands day in and day out, they have to find a way to log all these information comprehensively. An essential part of risk management practices today, compliance can lead to additional stockpiles of information while reducing the attention that people pay to similar information.

With more employees clamoring for transparency and openness, businesses today are also under pressure to share more information and openly communicate about decisions, policies, changes, improvements and other areas of the business. Not only do employees receive more and more memos, executives also feel the need to listen to more feedback and communicate about their reactions to them. These are worthy steps towards better workplaces and cultures, though the downside is being barraged with even more information to pay attention to.

5. More communication mediums

To help businesses cope with this barrage of information, companies such as Slack, and other chat products, have emerged. These new communication mediums have been proven to come with issues of their own, as employees now feel obliged to tune in more often, around the clock. They may feel pressured to respond more, and respond quickly due to the more synchronous nature of chat conversations.

Slack Daily Active Users: VentureBeat

There are three main problems with this mode of communication: With so much noise, it’s hard to determine what went on in the conversation before they dropped in; they may have to spend time deciding if messages are important enough to be responded to immediately; and they may not be sufficiently processing information before replying, leading to lower response quality across the board.

The impacts of information overload

Research has shown that 25% of a knowledge worker’s day is spent just finding information — Just think about this for a minute. Across the business, people are spending an entire quarter of their time just trawling for information hidden somewhere in their emails, files, chat conversations, and other sources. This is a quarter of a day that could have been spent working on something higher value or more productive.

Related to looking for information is the very real problem of distractions and multitasking at work. There are studies on distractions that shows that people are distracted almost every 3 minutes and 5 seconds by the buzzing of their phones of a new message or email, people walking around the office, and similar phenomena in the office. If they are not distracted by others, workers are so used to the constant distraction that they end up distracting themselves too. Each distraction takes another 23 minutes and 15 seconds on average to refocus on the task at hand, until they are distracted by another stimuli again.

Since employees are looking for information across a wide variety of sources, they are more likely to be distracted and brought off-track by something they see, and are more likely to have multiple tabs open. Multitasking has also been cited to damage the brain and lower overall productivity by 40%.

Employee engagement also tends to suffer due to frustrations over duplicate work, gaps in knowledge, higher stress levels potentially resulting in aggression, and contradictory decision making that work against teams internally. Work-life balance may also take a hit with the ongoing need to tune in constantly, and may affect sleep quality, reducing work performance.

The most grievous affliction comes in the form of reduced decision making capacity. Information anxiety is inflicting stress on knowledge workers today who believe they need to have a handle on all these sources of information despite the sheer volume and copious amount of analysis to turn information into knowledge. Analysis paralysis (also the paradox of choice) often ensues, causing many to make subpar decisions while in a condition of cognitive overloading.

Limits in the brain while multitasking: http://vivian2848th.wordpress.com/

Let’s say you are choosing a car. You either get a fact sheet with 6 main features that you care about, like the fuel efficiency, color and a sunroof; or another with 24 features chock full of information about the transmission, emission values, chassis weight and so on and so on. With the 24 features list, you are probably more likely to give up on choosing, or simply only look at the features you cared about anyway.

This is the same for deciding on plans in business: With more moving parts, the more decisions need to be made, with more information input, and more confusion — and these decisions may just end up being made like the car choice you just (fictionally) made. A related problem is the fact that workers are also less likely to have time to sit back and really think about things, which means less strategically-sound actions and more reactive work.

Some solutions to information overload

With so much at stake at the whims of information overload today, there needs to be a more tailored and manageable approach to working with information. While there are no “definitive” solutions yet, there are many products out there that can relieve some of these issues now.

Knowledge management systems, as you may have already realized, aren’t necessarily effective by themselves. Besides investing a good knowledge management system, it’s important to train and monitor the information being added to the system periodically. While many files should be saved for compliance and posterity, these should be encouraged to go into a “data dump” and working files should go into a central area for easy access. It is also important to establish proper naming and cataloguing practices, so as to take the fullest advantage of the search functionalities that come with most systems.

See any real insights? A familiar custom data puke: Avinash Kaushik

Data dashboards are also another important source of distilled information. Since most dashboards we have seen in our lives are what have been dubbed “custom data pukes”, where information have simply been dumped on a page and made to look impressive, data analysts and associates need to be held to higher standards when creating dashboards. By always asking how this will quantitatively affect the business bottomline and strategy first and foremost, effort can be expended on analyzing and turning over only the right knowledge to key decision makers in the business.

Finding the right data analysts and data-driven team members in this case, then, should also be a key focus. There are trained analysts who can handle big data, and the challenge becomes imbuing them with business logic and obsessive doggedness about using these vast amounts of data to extract only important insights and actionable recommendations for decision makers.

Apart from humans, you may also consider virtual data assistants. Even the CEOs of Slack and Box feel like things will change with more intelligent bots automating questions from employees, so there is an always-on assistant that can respond with information needed based on the query asked of it. They believe that this artificial intelligence will ease information overload in the long run, and it’s easy to imagine how this could seamlessly fit into our daily workflows eventually. Arria, Narrative Science and Automated Insights, are products that help to turn data sets into readable text explanations.

Products like KeyReply and other customer support companies also help to automate customer engagement using bots, so workers can focus more on other non-repetitive tasks instead. Increasingly, this technology is also more democratized such that products like Botdesk can help you to set up a bot to answer repetitive FAQ questions in just 5 minutes.

Slack Digest / Digest.ai are products that prioritize the conversations that go on in your team daily, so you can get the most relevant summary of what happened in your conversations, even if you’re not actively monitoring them. Many teams have jumped on these products as a way to get insightful updates to their Slack channels, and it’s not a stretch to see them personalizing your summaries even more based on your role and interests in the near future.

The future of the world lies in being able to better capture and summarize information for better decision making. Start taking a step to better strategy and bottomline today by managing information overload better in your team!

Read More

The Chatbot Landscape, 2017 Edition

Posted by Carylyne Chan on 26 June 2017

To help decision makers and users wade around the vast landscape of bots, this landscape gives a high-level overview of providers and tools.

Why this landscape, now?

Since we started building bots more than 2 years ago, the landscape has seen massive interest and change. This makes it hard for companies and customers to figure out what’s really happening and what they should do if they really want to build a chatbot for their business.

Through this exercise, we deeply explored various bot platforms, bot use cases, and bot frameworks — and we’ve arrived at some interesting observations and insights that may be useful to you (hopefully ).

Obviously, there’s no way to squeeze everyone into the landscape, hence we selected those which fulfill these objectives:

  1. Give readers an overview of the industry, such as the industry’s structure, notable examples, dominant providers, and tools widely used to develop chatbots.
  2. Help decision makers understand and choose the most appropriate chatbot development strategies by crossing business needs and capabilities in the market.

The basics

The global market for chatbots reached US$88.3 million in 2016. The market will grow to 36% CAGR, to more than $1 billion, through 2023[1]. Other than this growth rate, the current state and structure of the industry is clearly useful to know better — it’s gotten pretty crowded since we started building bots more than 2 years ago!

Ready? Get ready for a long post

Methodology

We began the mapping process by conducting secondary research, collating data from hundreds of sources, including published studies, companies’ websites, online articles, and personal interviews. In order to give a fair representation of dominant players in the landscape, the study spanned diverse geographical regions and 6 industry verticals, namely:

  1. Commerce
  2. Fashion & Beauty
  3. Travel & Hospitality
  4. Education
  5. News & Entertainment
  6. Finance & Insurance

Additionally, we validated the information presented on the landscape by seeking the opinion of established industry players.

Classification

To put everything into a coherent structure, we define the parameters and the terms as such:

Horizontal axis: The “marketing” function refers to a bot’s ability to drive exposure, reach, and interaction with the brand or product for potential and current customers. The “support” function refers to a bot’s ability to assist current customers with problems, and to resolve those problems for them.

Vertical axis: “Managed” refers to companies outsourcing the development of bots to external vendors, whereas “self-serve” refers to them building their bots in-house or with an 0ff-the-shelf tool.

From the inside out, the concentric circles represent:

  • Platforms: The messaging platforms that enable the existence bots through robust send-and-receive APIs, frameworks and ecosystems.
  • Brands: Companies which have launched and experimented with bots, in that particular quadrant (for example, Managed x Support).
  • Providers: Companies who have the capabilities to deliver exceptional work in that quadrant.
  • Tools: The supporting tools used by providers, brands, or developers of bots in delivering bot experiences.

Details & Explanation

Here are some of our observations about each of the concentric circles: platforms, brands, providers and tools.

Platforms

In this study, text is the main interaction mode we explore (we might explore voice bots in another study).

As Facebook Messenger is one of the leading chat platforms (with over 1 billion daily active users worldwide), with a strong push internally for Messenger bots, it’s understandable that companies and developers alike have been heavily investing in Facebook Messenger bots.

Clearly, everyone is waiting with baited breath for WhatsApp to open up a bot platform, but it’s still conjecture at this stage. SMS remains a baseline option for communication, and companies continue to use it to send automated reminders and information. As more messaging apps gain a foothold among their intended audience, such as Line and Telegram, their bot platforms will become more attractive for companies to invest in.

Another point to note is that platform adoption and use vary greatly among geographic regions, and certain platforms may work better than others. WeChat, for example, has a naturally huge user base, but may or may not work for your intended target audience.

By platforms, the type of bot content and interaction paradigm also differs. Line and Kik bots tend to be more brand engagement focused, and are more likely to be “loudhailer type” bots (i.e. Announcements and promotions mostly) than SMS or Messenger bots, which tend to be more varied across support and brand engagement.

Brands

Brands are companies who have launched their own bots, split by bot type in specific quadrants as defined above. Here we highlight some interesting bots and their functions, and their respective providers, if applicable.

Marketing x Managed

  • Universal Studio promoted their horror film ‘Unfriended’ with a Facebook Messenger bot speaking in the character of Laura Barnes[2], which was developed by Imperson — a conversational AI startup from Disney’s 2015 accelerator class.
  • Maroon 5 Bot[3] on Facebook Messenger, developed by Octane AI, improves the personal touch and interactions between the band and their fans.
  • The Gov.sg bot[4] on Facebook Messenger, built with KeyReply, helps the government of Singapore provide timely and accurate news and information to their citizens, and broadcast emergency news in the event they are needed.

Support x Managed

  • Citibank India’s virtual assistant, developed by Creative Virtual, is located in the Customer Service Center of their website and is designed to provide information to customer queries about Citibank products and services[5].
  • The Bosch service assistant, developed by Artificial Solutions, makes it easy for customers to troubleshoot an issue with an appliance or arrange a service call via its website[6].
  • NinjaBot [7](for Ninja Van, a fast-growing logistics company in Asia) developed with KeyReply, helps customers track and update their parcels before they reach them, deflecting queries to the support team about parcel status.

Marketing x Self-serve

  • Cheapflights, Kayak, Expedia, and more have launched bots of their own to give recommendations on travel products, help customers book flights or hotels, and send booking confirmations on Messenger[8].

Support x Self-serve

  • Marriott International launched Mobile Requests service (via its App) that includes an ‘Ask Anything’ concierge service and ‘Anything Else’ function that allows guests to chat directly with hotel staff, 72 hours before their stay[9].

Providers

Providers are companies who have the capabilities to deliver exceptional work in that specific quadrant.

Marketing x Managed

  • Conversable AI is a software-as-a-service (SaaS) platform for messaging and voice experiences across multiple platforms, including Facebook Messenger, Twitter, SMS, Amazon Echo, Google Home, and many others[10]. Some of the notable use cases are the Marvel Interative Story Bot, Pizza Hut Ecommerce Bot, and CES Twitter Guide Event Bot.
  • Assist started out as a local services chat assistant and developed into a chat provider that deploys bots on Facebook Messenger, Twitter DMs, Kik, iMessage and Telegram, among others [11]. They have served Fandango, 1800Flowers, Hyatt, and more.
  • Msg.ai develops chatbots for multiple channels and provides a dashboard to centrally manage the experiences[12]. They have deployed bots for Sony, CLEAR, and Signal, among others.

Support x Managed

  • KeyReply is one of the top AI bot providers in Asia for enterprises and governments. KeyReply’s AI capabilities aim to completely replace humans for some specific support tasks. KeyReply’s customers include the government of Singapore, Fortune 500 companies and Asian brands across multiple verticals[13].
  • Servicefriend provides hybrid bot technology for consistent messaging experiences at scale. They offer an “Interactive Text Response” (the text equivalent of IVRs, interactive voice responders over the phone) for companies such as Globe Telecom[14].
  • Kasisto enables banks to offer services through an automated “Virtual Specialist” so that consumers can use voice or text to assess financial information and perform transactions on their mobile banking application. It requires no coding by banking implementers and is designed for banks to private label their own brand[15].

Marketing x Self-serve

  • Chatfuel lets anyone build bots for Facebook Messenger. Their users include NFL and NBA teams, publishers like TechCrunch and Forbes, and millions of others[16]. Chatfuel is great for simple and focused bots, but may not work for complex chatbots requiring lots of customization.
  • Octane AI, focuses on allowing celebrity content creators to create tree-like stories that others can engage with on Facebook Messenger. Celebrities such as 50 Cents, Maroon 5, Lindsay Lohan, Jason Derulo, have used Octane AI to build their chatbots[17].
  • Gupshup is a bot-building platform that offers businesses pre-made bot templates. It offers both a “no coding required” version and an IDE for developers to build bots[18].

Support x Self-serve

  • Flow XO’s focus is to create an easy chatbot building experience[19]. Flow XO enables the deployment of bots to different platforms, such as web, Messenger and Slack.
  • Morph.ai is comes with built-in training and understanding modules to give a more accurate understanding of the business’s target audience. However, the platform only allows limited targeting for broadcasting[20].
  • Motion AI offers turn-key templates for many chatbot use cases including customer service, meeting scheduling and surveys[21]. However, it only operates on a multiple choice or single bot statement basis. It services global consumer brands including Kia, Fiverr, Sony, and Wix.
Read More

How to make sure your new chatbot gets a head start with machine learning

Posted by Carylyne Chan on 7 March 2017

Avoid a “clueless chatbot” situation by equipping your chatbot with past data from your business before it goes live

For most of us, building a chatbot serves a real business purpose and isn’t just an amusing project. Since a chatbot is meant to serve or interact with customers in some way, it should be given as much information and intelligence as possible to properly interface with customers. The sooner this data is learnt by the chatbot, the better — and it’s best when it is done before the chatbot even goes live.

There are a few reasons why this makes absolute sense:

  1. You already have data sitting around that could be used to avoid a “cold start” that would put friction on the user experience and cause big drop-offs in use due to the chatbot’s inefficacy in replying.
  2. You are investing a lot of time and effort into building a chatbot, and expect it to perform a strategic purpose in the grand scheme of things.
  3. Nobody likes a chatbot that keeps saying “Sorry, I didn’t understand your question. Can you rephrase?”. Nobody.

Raiding your support archives

A customer-facing chatbot naturally lends itself to using information from customer support tickets. Support tickets are a treasure trove illuminating the types of concerns and issues that customers already have, and can inform your decisions on what features to build into the chatbot, and anticipate what kind of questions your chatbot should be able to answer. Since these are actual, concrete data points, you don’t run the risk of imagining problems and solutions unrelated to your customers. From our experience, this is the best way.

Frequently-asked questions are another great resource for provisioning your chatbot with the first cut of replies. Since these are distilled from repetitive queries (hopefully, and not just imagined), they can be the baseline from which your chatbot can draw responses. However, you should be aware that not all FAQs are updated regularly or relevant to customers, and so you have to be sure to vet these questions before haphazardly deciding to pop them into your chatbot.

Training the chatbot

While it is ideal to use past tickets to train your chatbot, it can also be challenging if you or your chatbot provider don’t have the prerequisite expertise to analyze and extract the important information needed at scale. If you have a big archive that you believe will be important to aggregate, then it will require some knowledge of text mining and natural language processing to adequately start classifying, topic modeling and summarizing these data into usable nuggets for your chatbot development process.

You can of course take the time to manually go through the tickets and categorize them yourself, or get someone on your team to do so. Regardless, be sure to select a representative sample of tickets across these parameters:

  • Products — to represent all product types that you carry
  • Time frame — to account for events or seasonal fluctuations
  • Account type — to capture the variety of customer accounts
  • Priority — to understand differences in urgent and normal issues
  • Resolution time — to sample how easily/quickly different issues are resolved (the easier or faster, the more likely your chatbot can handle them automatically)
  • Tags or categories — to make use of the human-classified data for training

There will be many other factors that you can sample for representativeness of your customer support tickets, depending on your business and product structures.

Now, the hard part to do without machine learning and NLP expertise is classifying and counting the results of your analysis by hand for a large number of tickets. Just classifying about 1,500 to 2,000 tickets would be overwhelming for the average person, and could take between 3 days to 2 weeks, depending on the complexity of issues and time availability of your “human labor”.

Assuming you do have some expertise at hand, you can consider using semi-supervised training and start off with these labeled instances for classification, run some topic modeling through a hybrid vector-based LDA, and have some fun experimenting with different models and hyperparameters that do well on your dataset.

At the end of your training phase, you should have a relatively stable and production-ready model that can classify incoming queries that your chatbot receives from users. To maximize the effort that you put into training this model, you can choose to package it into an API and letting your chatbot call this API when it needs to classify new queries from customers.

Testing the results of training

It’s really important to test and retest the results of your trained model with real users. Sometimes (actually oftentimes) your models will overfit on the data that you trained it on, and will turn out to be useless in the face of new data. Hence, other than the split test/dev sets you will use in evaluating the model during training, you should give the first iteration of the chatbot equipped with this model to users.

This is usually achieved during the trial phase or pilot/beta launch of your chatbot, when you are testing the experience with a group of customers or users. Since you probably won’t have that many users at this stage, take the time to look through each response and figure out if there are systematic samples or examples which were ignored or misclassified in the model.

Also, take advantage of the way that the chatbot can ask for feedback from users, such as “Did I answer your question? [Yes/No] that you can pop into the end of every response, as a way to add structured data to continue training the supervised classification model you have set up.

Launch and optimize

After the trials, it’s time to launch your chatbot — and really see what happens to your chatbot’s response-abilities in the field. If all goes well, you may be able to answer 50–80% of questions properly at launch. In this launch period, continue to observe and note what kind of questions people are asking the chatbot, and factor that into your next version for optimization.

Realistically, you’ll definitely have a lot more tuning to do in order for it to work at the level you expect it to. Be patient: Every 2–4 weeks, expect to update your model with more information if you can afford to. The more data you have amassed, the better; there is almost no exception to this rule (maybe except when the data is of low quality or improperly handled).

Want to go deeper?

So you have heard about the wonders of advanced techniques in deep learning and you feel that you can beat the performance of your vanilla machine learning model with it. Yes, you’re probably right — especially if you want your chatbot to be able to interpret the context of sentences or turns. Deep learning in the context of chatbots is probably most useful with LSTMs (GRUs), and can improve the experience over time.

However, don’t feel the need to jump straight into this option until you have tested the other machine learning models first; it’s as important to understand your baseline performance, and it’s also hard to find vendors or engineers who know how to tune the models enough for them to be useful in production. Many of the techniques (at the time of writing) are either still in, or fresh out of, research labs, and may not yet be robust enough to be used in mass production, facing your customers. Additionally, you’ll need a significantly bigger dataset, likely in orders of magnitude bigger, that you may not have in order to build a meaningful deep learning model.

Hopefully, this was a useful primer for you to start thinking about and planning your use of data in your chatbot’s dialog capabilities. We’re open to chatting about the newest research or ideas that you have, since you can always find us at KeyReply digging into and testing new research models for our own chatbots!

This piece was first posted on VentureBeat on March 2, 2017.

Read More

The one thing your bot will fail without

Posted by Carylyne Chan on 3 March 2017

If you don’t have this, it won’t work…

Having built several bots ourselves, there is one thing that we’ve noticed that is markedly missing that prevents bots from being particularly useful: Bot-ready data.

We’re not talking about little hacky experiments where we run fun bots stringing some open APIs together in a GUI. We’re talking enterprise level bots, expected to solve some customer problems and drive business results.

There are 5 roadblocks in getting to bot ready data:

  1. Data availability — Does it exist?
  2. Data comprehensiveness — Does it include everything?
  3. Data cleanliness —Is it readable by machines?
  4. Data compatibility — Is it useful for bot workflow tasks?
  5. Data openness — Can it actually be used?

Let’s define data in the bot sense. Data for bots is intended to lead to some sort of resolution, such as answering a question or completing a transaction. Hence, data cuts across types like dynamic updates (parcel tracking statuses for example), generally static content (FAQs), system information (inventory stock levels), and temporal data (campaign launches or limited promotions), among others.

What makes these difficult to be used in today’s context?

1. Data availability: Is the data available in the first place?

As an example, sometimes people ask why a particular staff cannot be found in a staff directory. Often, the main reason is that the person in question simply wasn’t added to the directory in the first place and hence this data was never available.

Another would be information on new products, or perhaps an offline guerilla campaign. Data on where users may find roving trucks, for example, may not be available anywhere, so it would be impossible for the bot to query it. Being able to cater to data availability issues across the organisation requires strict data input processes to be put in place, which doesn’t exist yet in many organisations.

2. Data comprehensiveness: Is data only updated for some instances and not others?

Related to availability is completeness, which is what happens when some data are kept updated while others are not.

For example, sometimes people ask why bots show websites instead of giving a short answer; the main reason is almost always that web pages are often updated, even when FAQs or blurbs are not.

3. Data cleanliness: Is the data actually machine readable and usable?

In reference to websites, which are mostly maintained so that humans can read them, there are significant technical challenges in accurate informational retrieval (IR) from sources that machines can structure and respond to queries with.

Notwithstanding these IR tasks, machine comprehension in knowing which of these chunks of answers to select as the right answer, and displaying them in a palatable or dialog-driven format, is harder than most people realise.

4. Data compatibility: Is data in a format that can be used for bot workflows?

Having data sitting around in various forms doesn’t necessary mean that they can be used for meaningful resolution. For example, having access to CRM+ERP+Support+Call Centre data can be useful for analysis, but they tend to be static or stuck in their respective systems.

Some of these data are not structured to be compatible with bots. For example, having data displayed in obscure code with lots of data tags that need to be stripped away before they can be displayed, will require significant processing before they can be manipulated and shown in natural language or structured cards to end customers, and vice versa.

5. Data openness: Can the data be used even if it exists in the right format?

Hence, the final question in our examination of internal data for chat automation is, can all of these data actually lead to a resolution of conversations? Since many of these data sources are meant for capturing information as systems of record, they require data pipelines that can enable data to flow both in and out, as connectors between the data and the workflow.

At the very least, there needs to be a send and receive API layer that can ask your systems for their data, and update the systems with new information that come in from conversations. The next step would be to set up actual actions that can be taken to resolve customer issues — “How do I change my details or plans in the system”, for example — and allow conversation states to take actions and trigger workflows.

Sometimes, this is a matter of time (things are too complicated to open up quickly); sometimes, it’s a matter of reluctance (let’s not change the status quo); other times, it could be a multitude of organisational or industry factors that prevent the open flow of data from one system to the next.

The metaphor for data & AI adoption and bots today is like, back in the day, people asking why they need a website when the real issue was why they needed to go digital. Now, it’s all about how to turn current data sources into machine-readable and -usable data.

Building a bot hence becomes less of building a bot, but rather requires a larger effort to catalog and enable the flow of data through organisations. With the view of how important techniques such as machine learning will be to organisations going forward, this becomes a necessity that needs to be discussed in parallel with conversations around bots and what they can do for customers.

Since we work extremely closely with our clients and these problems at KeyReply, we are seeing patterns around how to augment and resolve some of these issues around data and its applicability beyond bots. If you’d like to have a chat, we’re always open to talking about hard problems

Read More

Developing the future of government–citizen interaction in Singapore

Posted by Carylyne Chan on 3 February 2017

Fulfilling our purpose of making government information accessible to everyone

In November 2016, we were fortunate enough to be chosen as the partner for the first-ever Gov.sg Messenger bot. As fellow Singaporeans, we personally believed it could be a great way to bring information to peers and foreign friends alike, making it easy and accessible for them to stay connected and be heard.

Give it a shot! Use this QR code with any scanner, or go to https://m.me/gov.sg

In this post, we want to share some of the details of how we brainstormed, created, optimised and launched this Facebook Messenger bot with the wonderful team at Ministry of Communications and Information (MCI). We hope that you’ll like the bot as much as we do, so this is the behind-the-scenes cut of how it came about! Also, if you like it a lot, we’re hiring :)

The planning process

1. Start with objectives

As with any chatbot, the most important first step is to determine the objectives for building the bot. For MCI, the reason for the bot’s existence was to make information easier to consume and access for more citizens.

As such, the overarching user experience direction was to include sources of information that could be useful on a daily or weekly basis, or would be extremely useful to have at a snap.

2. Working out the features

Based on this original set of requirements from MCI, we got to work defining some of the main features and functionalities that the bot should have in order to fulfill the objectives. We sketched out a few ideas and thought of how everything fit together.

It’s important to note that these tend to change over time, so the first draft is just a quick overview to get started on working on the bot. MCI was really flexible with ideas too, and that gave us a lot of space to experiment and dream up ideas.

3. Determining the tone

As the Facebook page that the bot would be housed in interacts with citizens and foreign friends alike, it has to portray our friendly Singaporean image (and sassiness) while still being professional and courteous. We developed a range of different tones and personalities, and culled them down until we found the one that could best fit the bot.

4. Drawing the dialog flow

One of our standard signature ways of building dialog is in a dialog flow map. We plot out all the functionality and dialog branches in a map and write out the initial cut of what the bot would say. This way, we got a feel for how the bot would function and what it might possibly say to the user.

We also discussed this map with the MCI team, and debated on the functionality and design of the bot until everyone was happy with the vision of the bot.

5. Set up technical infrastructure

A detailed technical discussion on building and managing bot infrastructure would probably take another few blog posts, but at a high level, the bot runs on Azure in a secure and scalable hosting environment. Each of our clients’ bots has its own separate code base, servers, networking gears, and so on. This is to keep everything private to each client, as part of our enterprise-grade promise to them.

By the end of this planning process, we already had a working demo of the bot with the main bulk of the functions in place.

Building the bot

As you’re probably used to hearing by now, the devil is in the details. Anyone could build a scrappy little bot if they wanted, but to build a robust bot with well-thought-out details based on tried-and-tested methods, natural language processing, information retrieval, and machine learning, coupled with enterprise-grade infrastructure and scaling, definitely requires more time and experience. We put everything that we had learnt in designing, writing, and building, bots into the gov.sg bot, and here are the features you’ll see:

Onboarding flow

While we started with a more generic welcome message, we decided to move it towards a purpose-driven onboarding flow in the middle of development. The reason is that some of the users who talk to the gov.sg Facebook Page are actually looking for information only, and would focus on asking questions.

As such, we segmented incoming users by those who were looking to ask questions (see Ask a question below), and those who were actually here to engage with the bot and browse around for interesting information.

Menu cards

Bot menu cards need to have interesting images, and logically ordered content in each. It is a classic case of information architecture in user experience design, where you have to figure out what makes each piece coherent in the overall story.

For the gov.sg bot, largely an informational bot, each of the cards serve a different shade of this purpose: Consuming content (news and videos), information seeking (look for people in Government), focused exploring (current affairs and happenings), news verifying (official sources) and providing information (feedback and preferences).

Latest news

To help citizens access latest content relevant to them on the bot, the Latest News section is updated daily with content from gov.sg news, Facebook posts, and YouTube videos. Users can subscribe to it, with finer-grained preferences that allow them to control how often they get content — daily, only on weekdays, or once a week.

An interesting take can be found in the “Read it here” function, where users can listen to a dynamically generated voice clip that reads the news by the page aloud. It’s great for commuting, and for people who prefer hearing (rather than reading) the news. We think this could be really useful for parents teaching younger children to read, listen and speak formal content, too.

Video channel

The Singapore government produces lots of video content regularly, and most of them are very interesting for lots of citizens, if they had the chance to see them. Hence, increasing discoverability of this content and making them more visible to users is a key reason for putting this video content upfront, together with other news content that users may be interested in.

To streamline the experience of watching these videos, they are shown in a half-screen slide-up that improve the native experience of watching videos without bringing the user out of the Messenger context.

Ask a question

As one of the goals is to help users find content easier, we integrated sites across the government so that one search on the gov.sg bot can surface information from multiple sources.

We used a list design, so that users can choose the most relevant result/site to them when asking a question. Since each government organisation keeps its own site as updated as possible, the bot is able to help citizens and visitors find the best source of information across all of them effectively.

Responding to questions comes down to the indexing, organising and retrieval of information from these large corpora of sites and links and matching it to the query. This will be a part of the bot where ongoing optimisation will improve results over time. This function was made to help users search for information before they have to ask someone for help, and to save them the trouble of having to navigate out of Messenger when looking for said information.

Gov Directory

As it’s imperative that people can find ministries, government organisations and officials who serve them, the Directory function enables users to connect to them more quickly. There are a few ways to get to the information, all of which were designed to help users find what they need efficiently.

First, there are quick replies with the most-searched organisations that users already look for, including MHA, MOE, MOM, MCI, and so on, based on past data. These put the top information at the users’ fingertips. Following that, users can either find organisations by their acronyms, such as NRF, or by searching for a related term, for example “library” for National Library Board.

From there, you can search for related eServices that you need. For example, maybe you’re looking for ICA so that you can renew your passport online. By choosing this eServices option for ICA, you’ll find the APPLES service right there.

Then there is the search for officers, which requires finer interpretation to disambiguate between similar names from different departments. The bot tries to clarify the name of the person you’re trying to reach, and suggests based on the name you type.

Not only can you pull up the website information, in almost all cases you can simply hit a button to call, or find the address and navigate to it directly. Now you can find your MP even faster. Talk about convenience

Current affairs

To highlight current issues in Singapore that people may be most concerned about, 5 microsites are brought up prominently so that people who are interested can go to the microsite to explore and understand these issues in more detail.

An interesting use for this in unusual situations, for example, would be providing haze information from NEA at a tap within Messenger — it would be a breeze to keep going to the gov.sg bot and clicking on that button instead of searching for it and having to go through the NEA site.

Experiences

To add an additional element to the bot, we decided to pull in upcoming events and features that people would find interesting around Singapore. This would be useful for visitors to Singapore, and for Singaporeans to rediscover interesting things happening around them in the country.

It’s easy to find events and happenings based on your interest, and you can load more events by simply navigating through attractive images that also aim to give you more descriptions and the event venue when you choose to see more details.

Press centre

Often, it is necessary to find accurate information on things happening in a country, and gov.sg happens to have a very well-updated page with press releases and speeches — an affirmation of just how organised our government is, really. (Trust us — we’ve lived overseas for some time )

However, the issue was that most of these also tend to be PDFs or sound clips. Hence, the interface is simple and brings up a short description of the press releases, bringing a user to the web page to download these media if that’s something that they are truly interested in. Otherwise, just reading the headlines gives you a pretty good sense of what’s going on and serves as a secondary source of formal news on top of the news articles being served to the user daily.

Terms translator

Further, to bring another excellent resource more easily to users, we bring up the Government Terms Translator page up within the window so that users can type quickly into the big search box on the page.

This is one of the unique points of Messenger bots: sometimes, the best way to show information is in the bot, but sometimes, it simply directs the user to the best possible way to interact with an interface. The balancing act between too much of either is one of the other things that experience and previous deployments can help inform, much like how a designer knows when an UI element might work and when it might not.

Feedback

Of course, no proper bot that takes its mission seriously should be without a feedback mechanism. Especially as one that aims to help its citizens and visitors, the bot tries to collect the most salient information for feedback. Here, effort has also been put in to suggest the top feedback topics, so that users may choose those feedback topics quickly.

Building a feedback form in a bot can be hard, because you don’t want to impose hard coded words too much (like for example “You have to type ‘Stop Feedback’ when you are done typing”) or collect feedback from only one response (people usually type more than one message at a time in a chat conversation), though sadly many bots do this.

Preferences (Subscription)

It’s also really easy to manage subscription preferences, and users have various options in frequency that they can choose from. Over time, these may be extended to include other preferences (maybe even what kind of voice you want narrating your latest content).

Behind the scenes

Management dashboard

For the administrators of bots, there are often many things that they would like to control and manage on the back end to make everything run smoothly for users. Hence, KeyReply also has a backend dashboard that allows for fine-grained management of content, broadcasting, feedback and analytics, and other proprietary elements to help improve and upkeep the bot over time.

Machine learning

Across the bot, there are various areas where machine learning plays a role: personalisation of content, increasing accuracy of general search and directory search, and suggested quick options. As the bot ramps up in adoption, there will be more data that will be useful in training and further improving the performance of these functions. These improvements will create a better and more optimised experience for users.

Natural language processing

Other interesting backend processes include parsing, stemming, information retrieval tasks, and even handling localised words. As it’s not exactly a chatty bot, there is less of a need to make the bot say chatty things, but rather to improve the accuracy of its suggestions, focusing the scope a lot more.

There are also some easter eggs hidden here and there… But it wouldn’t be fun if we told you what and where, right? Maybe weather… or haze?

Testing procedures

There are various things that we test for as part of our Quality Assurance processes at KeyReply (we detail some of these in our ebook, if you’re interested in learning more). For bots, we start with testing functionalities, dialog flows, sequential steps, buttons, and information sources.

Following that, there are other parts which most likely require tuning, including edge cases and fallback messages. Here, you want to humanise the bot and make it more palatable when it is unable to comprehend or execute what the user expects.

As a well-built bot is essentially a big project in itself, it also requires rigorous procedures and a lot of discussion and back-and-forth with all stakeholders involved. The MCI team was extremely responsive and that helped to make the process more efficient, meaning the bot could pass through testing and get to a production-ready bot much faster.

After all internal testers are happy, it’s important to get another round of organisational testers to ensure that everything is clear, self-explanatory and good to go.

Planning and executing a bot launch

Beyond building the bot, having a robust strategy behind launching and getting adoption of the bot is imperative to the success of the bot. There are a few aspects of planning that can help in the launch: current channels, advertising formats, and dedicated campaign optimisation based on bot performance.

Putting the bot on current channels (like the website or social media) and promoting it on existing channels like email lists or press releases can be a simple but effective way to get early adoption.

Following that, choosing the right ad format, such as the new Messenger “Message Us” ad format can be useful in getting the next bump in adoption.

The ongoing process of optimisation will then determine the ultimate success of the bot; by observing the adoption and usage of the bot over time, improvements to the bot functions and onboarding can be planned. This will be a huge focus for us next, by pairing quantitative bot analytics with unstructured text data and the machine learning data to continue pushing the bot to improve and become even more useful for citizens and visitors alike.

Try it out yourself

Now that you know the ins and outs of the bot building process, and the list of features and why they were made that way, go try the bot out! We’d love to hear your feedback and suggestions, and promise to take them seriously in the next few bot iterations.

If you work at a government department you feel could benefit greatly from better government-citizen relationships too, manage an enterprise marketing team, or are part of a creative agency, let’s talk about how we can work together to deliver more exceptional experiences.

Read More