In this article, I’m using several AI-related terms interchangeably. The text relates to Seam - a chat-based SaaS unlocking the power of data insights and operations for business users. Seam was built on top of GPT-4, a Large Language Model. Large language models fall under generative AI (models capable of generating different types of content based on their training). Generative AI falls under a much broader category of Machine Learning, which refers to a subset of artificial intelligence algorithms capable of analyzing patterns in data and performing tasks without explicit instructions. All terms are encompassed by an umbrella of “Artificial intelligence,” referring to any computer or machine aiming to simulate a human-like thought process. While this language makes sense in the context of Seam, it’s worth noting that the terms are not interchangeable in general, and each refers to a specific, distinct software category.

Introduction

Artificial intelligence (AI) has been one of the most, if not the most, transformative technologies that emerged in the 21st century. Analyzing Google Trends, we can see the phrase “AI” surge in popularity in late 2022-early 2023. This spike correlated with the release of massively successful generative technologies like ChatGPT from OpenAI. Their popularity can undoubtedly be attributed to how accessible they were. For the first time at this scale, anyone could try communicating with a Large Language Model and get a result in seconds. This ease of use has translated into AI being rapidly adopted across niches, helping non-technical users automate tasks like content creation. The developments in Natural Language Processing (NLP) and AI applications in healthcare, finance, defense, and transportation have shown the world what’s possible and how humanity can be propelled forward with technology. Every month since then, the biggest companies in the field have made significant announcements, and the technology hasn’t shown signs of slowing down. 

As AI continues to evolve, it is crucial to confront challenges related to bias, data security, privacy, and transparency. By prioritizing trust and explainability in AI software design, we can foster the development of innovative products that drive market adoption and uphold ethical standards and societal well-being.

‍

“The hot topic in AI is… How do we do this responsibly?”

Betsy Greytok

Vice Presidents, Ethics and Policy, IBM

No, but really - what is AI?

Artificial intelligence, or AI for short, is the most colloquially known term for computers that simulate human-like intelligence in critical analysis, problem-solving, and decision-making. AI refers to any “intelligent” machine. It’s important to note that different algorithms might be better suited for various use cases. That’s because, unlike traditional software, AI is not programmed but rather trained on vast amounts of data. The type and quality of that information determines the patterns AI will learn to recognize and, in turn, the type and quality of outputs it will return.

What is AI?
‍Artificial intelligence (AI) is a field of computer science that aims to create systems capable of performing tasks that typically require human intelligence. It encompasses a wide range of techniques and approaches, including but not limited to pattern recognition, problem-solving, data analysis, natural language understanding, and robotics. AI algorithms learn from data to make predictions and decisions and automate tasks across various domains, including healthcare, finance, transportation, and more.

The fact that artificial intelligence is capable of recognizing patterns and making decisions poses a significant challenge for product designers and developers. They must ensure that end users know the capabilities and limitations of AI-based products and trust them enough to engage continuously with them.

What’s the difference between AI, ML, Generative AI, and LLMs

There are many terms surrounding artificial intelligence that are often used interchangeably. Even though they all broadly relate to the same topic, each has a slightly different meaning and describes a broader or narrower area of AI studies and computing. For example - Seam utilizes a Large Language Model in its conversational interface. An LLM is a type of Generative AI that falls under the ‘Machine Learning’ subcategory of Artificial Intelligence. Every LLM is a Generative AI, but not every Generative AI algorithm is an LLM. Any Machine Learning algorithm belongs under the ‘Artificial Intelligence’ umbrella; conversely, not every AI algorithm utilizes Machine Learning. It’s no wonder most people get these confused, so I will explain the key differences below. 

‍

Artificial intelligence

This umbrella term refers to the entire field of computer science, which aims to replicate what is typically perceived as human intelligence. AI has multiple applications, far exceeding user-facing web applications like Seam. Systems utilizing artificial intelligence can identify fraud in financial systems, verify human identity through facial recognition, help develop new drugs, operate autonomous delivery drones, and much more.

Machine learning

This is a sub-field of artificial intelligence that deals with pattern recognition. Machine learning models are trained on vast datasets to analyze information and make predictions about new or future data. While AI can include both - programmed and trained algorithms, Machine Learning (ML) can only be trained as it needs to make nuanced decisions based on multiple factors and data inputs that don’t follow an explicitly programmed logic. Machine learning is applied across various industries, but some common real-life examples of ML algorithms include recommendation systems, fraud detection algorithms, and Natural Language Processing (NLP).

Generative AI

As the name suggests, Generative AI (or GenAI for short) is an AI model that can generate content that hasn’t existed before. These models utilize machine learning to learn patterns in existing content and then create unique output. For example, to create an image of a house, the model must be taught to recognize houses in existing content in different scales, perspectives, forms, and angles. As these models learn to recognize and generate multiple individual objects, they can also combine them to create more multi-faceted content. That’s why AI can generate a realistic image of a house in a canyon on another planet, even though it hasn’t encountered an image like that before. Generative AI isn’t limited to images - it can also generate articles, create videos, write code, and more. 

Large Language Models

Large Language Models (LLMs) are a subset of Generative AI that can generate coherent and contextually relevant text responses based on user prompts. They achieve this by predicting the statistical likelihood of the next word in a given string of text, drawing upon extensive training on human-made texts. Through continuous analysis of patterns and relationships within this data, LLMs develop a nuanced understanding of language structure and semantics, allowing them to produce responses that are virtually indistinguishable from those generated by humans.

Live demo

Interested in trying out Seam for yourself? The live demo is waiting.

Try live demo

Differences between designing for traditional computing and generative AI

Humans instinctively tend to attribute human-like qualities to intelligent systems they interact with. Since we can’t predict the answer they’ll give us, we think of them as conscious or personified, even though they’re not. Typing “Thank you” into a Large Language Model is a common phenomenon. In the case of products like ChatGPT or Seam (the subject of this article), it only seems natural, as the primary way of interacting with them is through natural language. It’s essentially no different than chatting with a human, or… is it? How do we effectively draw the line between human and machine interaction? As computers become more intelligent and human-like, the way we approach designing user interactions needs to change. As systems and algorithms move towards “assistants” and “companions” instead of executors of tasks, we move from creating wireframes to designing entire relationships people build with the products.

Fundamental differences between AI and conventional computing

Conventional computing returns predictable outputs based on user input. An action or prompt is explicitly programmed always to return the same result. In other words, how they work is deterministic - a specific action determines an outcome. AI models (more specifically, LLMs), on the other hand, work from a completely different premise. Instead of simply returning information based on pre-programmed rules, they interpret the user’s action (typically a text prompt) and return output with the highest probability of being correct (aka closest to the pattern learned by the model in the training phase). The way a large language model returns output is probabilistic, meaning that the exact outputs can’t be predicted with total certainty and that they can vary depending on context.  

Implications for designing digital products

The rise in popularity of AI in software has exposed us as product designers to new problems. While traditional software is predictable, it’s also easier to trust. Applications like Slack help businesses perform specific tasks like communicating with their team, so users naturally learn to expect the same output every time. Interacting with LLMs is different as it typically happens through a chat interface. While this way of communicating user intent to the software is natural, it leaves plenty of room for friction if the model misunderstands the prompt or generates inaccurate results. Users need to be aware of the capabilities and limitations of the model to allow room for mistakes without compromising their confidence in the product. 

Relationship with the software

With the introduction of AI into everyday software, the way people think about computers continues to evolve. Humans naturally tend to attribute human-like qualities to objects, including machines. This inclination grows stronger as interactions with computers become more similar to interactions with other humans.

Since AI learns based on the data it interacts with, it has the capability to improve continuously. This is a fundamental departure from a traditional model, where the human indicates tasks for the software to execute. Modern software utilizing AI technology aims to augment human life more consciously. Many AI companies use terminology like “co-pilot” or “assistant” to describe their products. This semantic reverses the traditional dynamic between user and software, elevating software to the position of a partner, or the “second brain,” able to understand user pains and offer contextualized solutions. There are many tasks that LLMs can perform better and faster than humans, so it’s natural that the way we think about software and its place in our everyday life changes. As a result, this also impacts our perception of the user interface. The more feedback the model receives, the more accurate it becomes. Therefore, it's essential to embed a way for the users to give spontaneous feedback so the model can provide more accurate responses in the future.

Importance of trust in AI

Creating trustworthy AI products is imperative, especially when considering their potential impact on users. Establishing trust is paramount regardless of the target audience. Users must feel confident that the AI products they engage with can effectively address their needs and concerns from the very first interaction. Trustworthy products instill confidence and foster long-term engagement and satisfaction among users, ultimately leading to the product's success.

The business perspective

Ensuring that AI inspires trust is not just a matter of good practice but a strategic necessity for businesses looking to maximize the profitability of their software products. Inspiring customer trust and maintaining a positive reputation are foundational elements for long-term success. Customers are more likely to engage with and remain loyal to products they trust, leading to increased retention and more word-of-mouth referrals. Secondly, by prioritizing ethical and transparent AI practices, businesses can ensure compliance with data privacy regulations and industry standards, safeguard operations, and appeal to more AI-sceptical customers. Finally, prioritizing customer experience and satisfaction through trustworthy AI enhances user engagement and loyalty, driving revenue growth and market reach. Designing your product to inspire trust is not just about ethical responsibility but also a strategic investment in long-term profitability and business success.

The ethical perspective

AI-powered products can unlock new levels of productivity and improve everyone's decision-making. They’re undoubtedly powerful tools, but with great power comes great responsibility. Since artificial intelligence is used for sensitive use cases like facial recognition, identity verification, or data processing, ensuring it’s being designed and developed responsibly is essential. The emergence of AI has implications in all sectors that directly or indirectly affect all people worldwide; that's why it’s vital that we, as humanity, approach its growth with a common understanding of how it can benefit us or create potential risk. Most companies developing AI solutions have published their version of a code of conduct to guide their explorations into the topic in an ethical way. 

Some examples of general guidelines for developing AI products responsibly are:

-list-normal-

• Google’s AI Principle (Link)
• Meta (Link)
• IBM (Link)

-list-normal-

How Seam makes decisions

Seam leverages GPT-4 as the underlying Large Lanugage model. While this technology has undergone extensive training on large data sets, it (similarly to other LLMs) has two significant issues. Since most of the information on the internet is written in a confident tone, the model learns to sound confident even when it shouldn’t be. LLMs are known to “hallucinate” - fill in the gaps in their knowledge with inaccurate information. On top of that, the information they’d been trained on might be outdated, which can result in misleading responses. These issues are even more jarring when combined with a particular use case, requiring expert domain knowledge. That’s why most companies building products with LLMs go through an additional phase of optimization called Retrieval-Augmented Generation. While a language model is really good at generating responses, these responses aren’t grounded in a source by default. That’s why in RAG, the LLM is aided by an external source from which it retrieves data before returning an answer. Now that the answer is more grounded in reality, the chances of it being correct increase. "Instead of returning the response based on data used in training, the model will first “consult” a vector database and retrieve information relevant to the question."

Seam’s co-founder Nick calls this approach “Crawl, walk, run.” Other early-stage startups typically follow a similar path. With the goal of continuously improving the responses' accuracy in user-specific contexts, RAG is just the first step. Because of its relative ease of setup and lower cost compared to other techniques of adapting LLMs (like finetuning or independent training), it’s the most logical approach at this stage. As Seam grows, the founders anticipate implementing more advanced optimization methods to further improve their model.

How we’ve designed Seam to inspire trust

When designing Seam’s interface, we paid close attention to incorporating subtle cues and messages that would help build and then reinforce the user’s trust in the product and enable a way to improve the results users get from it over time.

Differentiating between AI and non-AI capabilities

Transparency is one of the main pillars of designing responsible AI interfaces. Identifying features using artificial intelligence also helps manage users' expectations about interacting with the products. Users often anticipate varied outcomes from AI features compared to regular ones. When designing Seam’s interface, we highlighted to the user whenever AI was involved in processing data or decision-making. To do it subtly, we’ve used a well-adopted magic wand iconography. The usage of the magic symbol is somewhat controversial as it implies that AI is somehow supernatural when it’s not. I’ve gone into more detail about my thoughts on symbolism surrounding AI depictions and why, in most cases, I believe that refraining from invoking “sparkles” or “magic” is the better approach in this article. At the same time, I recognize that this symbol has gained significant adoption. Since our goal was to immediately communicate the presence of an AI algorithm with no confusion, we relied on a well-recognized, universal symbol (even though not entirely accurate). 

We’ve labeled each component that utilizes the Large Language Model with the same wand icon to build an association between the symbol and AI capabilities in users' minds. By relying on a common symbol, we’ve ensured that AI usage is transparent from the first interaction with Seam.  

Providing a way to verify answers

Building on another core tenet of responsible AI - explainability we’ve included an easy way for humans to trace the process Seam’s LLM used to generate the response. Explanability is the notion that AI systems should include a way for humans to understand and verify the process used to generate the response. The way we’ve achieved this is twofold. On the one hand, each response generated by the model includes a subtle button that reveals a high-level reasoning path used by the model.

Additionally, users can drill down deeper to understand the full context. A “Details” button always present in the top right corner of the chat interface reveals a contextual side panel that lists all data sources, filters, definitions, functions, and table joins used in the viewed response.

Helping users communicate feedback to the model

Large Language Models can evolve over time; the more data they’re exposed to, the better their future responses will get. That’s why it’s essential to allow users an easy way of interacting with the LLM to provide feedback about the accuracy of generated responses. In Seam’s case, users can save responses as “verified” in their team’s library directly from the chat interface. This way, the model learns the significance of specific queries in the user context, making it possible to generate more accurate responses in the future. 

The value of trust in AI

In conclusion, designing trustworthy AI products requires a multifaceted approach that considers technical aspects, ethical considerations, and user experience. As AI technologies evolve, it becomes increasingly significant to prioritize transparency, explainability, and user empowerment in the design process. 

By incorporating subtle cues, such as clear differentiation between AI and non-AI capabilities, providing avenues for users to verify answers, and facilitating seamless feedback mechanisms, products like Seam exemplify how trust can be built and maintained in AI-driven interfaces. By inspiring customer trust and prioritizing user satisfaction, businesses can drive long-term engagement, loyalty, and, ultimately, profitability. From an ethical standpoint, it is imperative for companies developing AI solutions to adhere to responsible practices guided by principles of fairness, accountability, and transparency. 

In essence, by designing AI products that inspire trust, we can capture artificial intelligence's transformative potential while ensuring that all humans can equally benefit from its achievements. As we navigate the evolving landscape of AI technology, we are committed to designing a future where AI serves as a force for good, enhancing human lives and our progress as a species.