Enterprise AI Needs Architecture, Not Wrappers

Machine-generated transcript from the episode video, cleaned for punctuation and names. Speaker labels are not included.

[0:05] Welcome to another episode of the GTM Vault Podcast. I'm your host Rick Koleta, and today we're thrilled to have Yuhong Sun, co-founder of Danswer. Danswer is an innovative startup backed by Y Combinator that is revolutionizing the way enterprises approach generative AI. With their open-source platform, Danswer focuses on providing a secure, self-hosted AI solution that keeps all data under the control of the organization, making it ideal for companies with stringent data security requirements. Today we'll dive into how Yuhong and his team are pioneering this space, the unique challenges they face, and the future they envision for enterprise AI. Yuhong, thanks so much for joining us today. Yeah, thanks for having me. What motivated you to co-found Danswer and develop an open-source enterprise generative AI platform? Yeah, so I'd say there's a bit of a gap in the space. A lot of these gen models have been

[1:07] extremely popular and very useful for consumers. So I think everyone's heard the story now, like ChatGPT, fastest app ever to reach a million users. A lot of use cases replacing even web search, challenging Google, all of that. But one thing that they're not really doing is bringing additional value to teams in the workspace. So OpenAI, they've pioneered the space, but I think where the value really lies is being able to ask questions like, for example, what are some common features our five biggest customers are asking for? So if you ask that to ChatGPT or ChatGPT Enterprise, it doesn't have any idea. It doesn't know who you are. It doesn't have any of the knowledge of your organization. So our goal is to make gen AI a lot more useful for teams across the world. The basic premise is, anytime someone wants to ask a question or they want to explore a due topic, or maybe just use gen AI in general to be more efficient, then they should think of Danswer. Yeah, and how does Danswer's self-hostable architecture set it apart from other gen solutions available on the market today? Yeah, so I mean, first and

[2:10] foremost, we're open source. We're actually the biggest open-source project right now in enterprise search. And just for a little bit of context for some of the listeners, so enterprise search is this idea of having a single place to access any of the knowledge across the organization, so connecting up to all of your customer conversations, all of your internal documentation, all of your internal communications, and all of the knowledge essentially that you can think of at work. Yeah, we're also self-hostable, and so what that means is teams can use Danswer without needing to send any of this sensitive information back to us. So because of this, the teams that we work with, we don't need to go through a heavy NDA process and security back and forth and then talking contract. A lot of our users are actually, they start with our MIT version, and actually a lot of them stay on the MIT version. Even we have extraordinary teams like zenes, like gab, other teams using our open source repo. But yeah, the idea is that they don't need to talk to us, they don't need to trust us, all of their data, it stays within their

[3:12] control. All of the processing happens within their deployment, and so this gives people a lot of peace of mind to use us, and oftentimes they see value in a single afternoon, whereas for other cloud hosted solutions it could be months before you even get the clearance to get a pilot going. All right, can you discuss them importance of data privacy and control and enterprise AI applications, and how Danswer addresses these concerns? Yeah, so data privacy in our space is extremely important. So for example, we connect up to all of your customer conversations, and you can imagine that's really bad if that leaks, right? Like you might have quoted them some particular per seat cost, and you don't want that leaked externally. So I mean, I think that part is very straightforward, but the way I think some of the details that people don't realize is, when you're using a cloud solution and you're sending all this data over to a third party, you're not just trusting that third party. You don't really know what all of their additional providers are. So for example, they might have some other team that's

[4:14] providing an embedding model for them, or another service that's doing reranking, or OCR, or there's just potentially a million other services that you actually sending this information to. And by the time you get back an answer from this gen AI system, who knows how many people have had access to it. A lot of these kind of new age gen solutions, some of the early ones, they rely on OpenAI, and you don't really have the choice of choosing which third party you want to work with. So being open source, being transparent, that makes it a lot better, right? You can choose any LLM provider of your choice, you don't have to use OpenAI. You also know exactly who we're working with, and outside of the LLM provider everything else is local, so there are not a dozen hands on your data in order. Some of the key technical challenges you've encountered while building Danswer, and have you navigated them? Yeah, so I think in the vein of trying to be secure, we've done a lot of work to make Danswer be deployable fully locally on prem. So for teams that don't have GPUs, we have a separate set of models

[5:18] that we run, and a separate process at inference as well. So the idea there is that it doesn't matter if you're a small team or you're a large enterprise, we want to respect the privacy of your data, and we also want to make sure that give you the best possible answer. So having the ability to support kind of this really wide range of use cases, that's been not necessarily challenging, we have the expertise to do it in house, but it's certainly more investment on our side to make sure that it works really well across the board. I like to talk a little bit about the pricing strategy you guys are enforcing. Can you tell me about the community plan alongside paid cloud and enterprise plans? What was the strategy behind offering these different pricing tiers, and how do they cater to various customer needs? Yeah, for sure. So our kind of core mission is that we want to bring enterprise gen AI to all of the teams in the world. We want to make it easier for people to access information, and we're just a startup at this point, and we figured, even if we're like Google for example,

[6:21] to be able to work very closely with every team in the world just becomes infeasible. And so the idea is that we can let the vast majority of teams just self-serve. They don't need to talk to us, they can see value immediately, they can use the open source version, and small teams can have all the value from the open source version. They don't need to convert to a cloud user or purchase the enterprise plan. As team scale, we hope to grow with them, and as their needs evolve, potentially they find out that they want more fine grain access controls, right? For example, a lot of startups are extremely transparent, everyone has access to everything, and that's one of the best things about being a startup, right? But as teams grow and they find out that, hey, maybe engineers should only have access to engineering documentation, and sales folks should be the only person, only people to access transcripts, etc., then they realize they need some of these kind of more premium features. Our cloud offering is another way for people to have non-technical teams to be able to use Danswer. For the teams that really want, like, all my data fully in control, in my own control, all of the processes running locally, then often

[7:24] times they'll opt for self-hosted enterprise. Got it. And with the premium community plan providing access to a range of features, have you found that this frictionless entry point has been effective and driving user acquisition and growth, especially at the top of the funnel? Yeah, absolutely. A lot of teams that come to talk to us have actually been using the Community Edition already for a long time. All of our current customers have found us actually, not the other way around. We've never done any cold calling. Our goal for users, they should be able to quickly self-serve, find value within a day, and so this all plays into that. Again, other solutions, again oftentimes it's months before you can actually see value, so all of this plays into kind of that frictionless experience. The community definitely resonates with that, so people come in a lot for that reason. Can you share some examples of how enterprises are currently using Danswer to improve their workflows or operations? Yeah, so some typical use cases are, I'll just list a couple, there's customer

[8:26] support, asking product questions, general self help, and cross team communication use cases. So I'll talk a little bit about customer support. So for these roles, oftentimes the folks have to handle a wide variety of issues across the entire stack, across the entire product, and it's very unclear where they should go to find information. A lot of times the resolution is really technical, so having an AI also is very helpful there. And of course getting issues resolved for customers, it directly affects the top line of retention, so this is a clear win. We also have a lot of teams that have IT support or ask HR type of channel in Slack, and so they'll have our AI assistant connected to a relevant documentation, and it'll sit in these channels, and when someone goes in that channel to ask a question, then instead of waiting a half a day for human to respond, the AI with all of the context of the org just gives the correct answer right away. So yeah, those are, I would say, are kind of the most popular use cases. What role do you see for generative AI in trans

[9:29] transforming enterprise operations, and how is Danswer positioned to lead this transformation? Yeah, so we believe every enterprise within five years or so, and actually even smaller teams too, they'll all have some sort of AI assistant to help with essentially every work related task. So we're definitely on that trajectory, I would say, in terms of the gen AI development and also the adoption that we're seeing with Danswer. So yeah, I think things either accelerate, or even if they say stay at the same pace, that future is coming extremely quickly. Yeah, what are some of the trends you see emerging in that future in the next few years? Yeah, so I think the space is really promising for us. The cost of inference is dropping, and yeah, cost of inference for gen models specifically. So what that means is we'll be able to pass in so much more context for every user, a lot more information about them and the relevant information for the query. Yeah, every question, every user flow, I think is going to, the way that people interact with data is going

[10:31] to completely change. So we internally, we are dogfooding our own product, and for the vast majority of questions now I just go and I ask Danswer. I don't really go to Google or Google Drive anymore, unless I actually need to make some edits to a doc. If I'm just trying to find some information, I just go to Danswer. It's ingrained in my head as a default way, and I think more and more people will become this way. What advice would you give to other entrepreneurs looking to enter the open source or gen AI space? Yeah, I think personally it's really important to have an understanding of the space and kind of the capabilities of the current state of the art models before doing Danswer. I was working in deep learning, and so also more specifically in NLP actually for several years, so when ChatGPT came out I had a good sense of what these models were good at. I believe around that same time there was a lot of hype around agents and AutoGPT, but if we look back I think it's pretty clear to everyone that trying to do kind of like Devin type projects, AutoGPT, these types of

[11:34] agents with a GPT-3.5 model, well, we just weren't really quite there yet, right? So for us specifically, we're building within the capabilities of the models as they are today, but as the models improve, the capabilities of what we're doing gets better as well. So I think for the kind of individuals out there looking to do a startup, it's much better to be selling something that's really useful rather than selling a dream, and I think for us we've really nailed that one on the head. So yeah, both being on the cutting edge, but also working on something that is currently solvable. You guys seem to be on a great success trajectory with the recent sizable funding round. Is there any advice you can give to startup founders looking to navigate the fundraising round? Yeah, I guess I'll give the standard YC quote, but I believe this one very deeply, and it's also what they have on all their t-shirts and merch, but just build something people want. It's much better to raise based on traction rather than raising based on, like, you

[12:38] know, oh, you should believe in me, I have this background and this degree or whatever. It's much better to say, hey, I have a product and people absolutely love it, here are some examples of people saying they can't live without it. Yeah, I think that's probably the best way to approach it. Got it. Yuhong, thanks so much for joining the GTM Vault Podcast, it's been a pleasure. Yeah, thanks for having me. Yeah, really enjoyed the conversation.